Tackling “Gaming” and ‘Unwanted Behaviours’ in Implementing Payment by Results

Below is a short essay I submitted in October 2012 as part of my uni course, as my next post will be on Social Impact Bonds (effectively a form of PbR for investors) and will refer to the outcomes setting section of this essay I figured I’d put this up too for any interested readers. It’s a bit longer and far more academic in style than my usual posts but I hope you enjoy it all the same 🙂

PbR

Payment by results (PbR) is a method of payment whereby providers of public services are largely paid on ‘the basis of the outcomes their service achieves, rather than the inputs or outputs the provider delivers’ (Fox, Albertson, 2011, p.395). It is currently being used or piloted in the delivery of a number of services in the UK. This essay refers to works discussing the Department of Work and Pensions’ Work Programme (WP), PbR in the NHS and tackling re-offending in the criminal justice sector. It compares modern criticisms to a historic example of PbR; its use in English and Welsh elementary education in 1862-1897 (Rapple, 1994, p.1), in order to test the potential validity of the claims against modern systems that are still too new to draw conclusive evidence from.  Each of these initiatives, despite utilising different PbR frameworks, highlight a common set of challenges in implementing PbR effectively. Widely discussed is the potential for providers to attempt to “game the system” and exhibit ‘unwanted behaviours’ (Miraldo et al, 2006, p.13). To help counter these issues commissioners need to reduce the effect of ‘perverse incentives’ (ibid, p.4) inherent in PbR structures by working closely with providers to ensure that outcomes are measured appropriately and payments are set at optimal levels. Where “gaming” persists access to a range of high quality data (ibid, p.7) can allow commissioners to track and therefore challenge anomalies potentially linked to the practice. It appears that, for now at least, PbR is here to stay. The challenge for those commissioning public services will be how to reduce ‘information asymmetry’ (ibid, p.6) and control these behaviours in order to implement it successfully.

PbR, arguably rightly, leaves providers under pressure to meet targets and to do so in such a way that makes their organisation a profit. Whilst potentially leading to an increase in innovation, efficiency and value for money (Ministry of Justice, 2010) this can also lead to negative consequences. As with other target driven systems PbR has the potential to ‘turn the knight into a knave’ (Le Grand, 2003, p.168) and encourage the variety of gaming methods noted by Bevan and Hood (2006). In the 1860s-1890s PbR in education model, gaming behaviours were endemic to the point that they were regularly referred to in Annual Reports of the Committee of the Council on Education (Rapple, 1994). In modern debate about PbR in the NHS such behaviours include “up-coding”, where providers assign patients to unnecessarily high Healthcare Resource Groups in order to claim an associated higher payment for their care (Miraldo et al, 2006, p.9) and ‘Dumping (explicit avoidance of high severity patients) and deliberate attempts to attract relatively low severity patients’ (ibid, p.6). Likewise in other sectors ‘the potential exists for PbR providers to ‘game the system’, for example by intervening only in easier cases’ (Fox, Albertson, 2012, p.48) and for large contractors to ‘pass the toughest cases on to…sub-contractors’ (Corry, 2012, p.47).  It is unclear how common these behaviours currently are, however, as Mannion et al suggest with regards to the NHS, as practitioners’ knowledge of these relatively new systems increases so too might the number who manipulate it ‘for their own gain’ (2008, p.85).

In order to reduce the ‘perverse incentives’ (Miraldo et al, 2006, p.4) built into PbR which may encourage providers to behave in this way, particular care must be taken over setting which outcomes are measured and levels of pay. As seen in the use of PbR in education in 1862-1897, where the outcomes measured are too narrow, they can produce effects contrary to the overall aim. Here the aim was to make grants dependent on the “quality” of a child’s education but the end result, arguably, was that the overall “quality” dropped whilst results in the measured areas appeared to improve (Rapple, 1994). This issue has been raised in debate surrounding modern PbR programmes, as Fox and Albertson state ‘there is evidence that the use of simple ‘headline’ targets can be counter-productive’ (2012, p.46).  Outcomes must be chosen in such a way that they incentivise high quality service provision for which there is public demand, not a facile attitude of merely meeting superficial targets in order to receive payment.

Likewise, where payments are too low these behaviours may occur as the pressure on providers to achieve a profit increases. Where one provider is clear he believes that ‘incentives are very cleverly aligned’ (Sean Williams, 2012, 4:08) in the WP, others disagree. In his analysis of the WP Mulheirn states that the outcome payments, relative to attainable minimum requirements, are too low, potentially ‘too low for the programme to be financially viable’ (2011, p.11). Commissioners, then, are faced with the challenge of pricing outcomes at a level that offers value for money and encourages providers to strive and innovate, without unintentionally encouraging them to manipulate results to make a profit. In order to improve the likelihood of successfully setting outcome measures and payments correctly, commissioners need to maintain constant communication and consultation with providers whilst defining them, and as they deliver contracts. In this way where providers raise reasonable issues with outcome measures or payment or challenges out of provider control (e.g. the economy) threaten the viability of their contract, commissioners can respond accordingly and thus reduce the likelihood of providers resorting to ‘unwanted behaviours’ and “gaming”.

The final means by which public service commissioners can challenge gaming is through ‘information systems’ (Mannion et al, 2008, p.87) utilising ‘high quality data’ (Miraldo, 2006, p.8). Where prevention fails, commissioners can increase their chances of catching providers who are “gaming the system” by gathering and analysing comparable information from all providers in a sector in order to locate anomalous results and investigate them. This, combined with the increased communication with providers surrounding outcomes measured and payments, would help tackle the issue of ‘information asymmetry’ (ibid, p.6) which both allows providers to “game the system” undetected and results in commissioners producing unviable contracts that encourage unwanted behaviours. Each of these proposals undoubtedly comes at an economic cost at a time when cuts are required. However as seen from the mistakes of the 1860s-1890s PbR scheme if these challenges are not adequately met the negative effects can outweigh the benefits of PbR (Rapple, 1994, p.18). Implementing PbR in public service delivery is going to be a challenge to public services in the years to come. However, by ensuring due care is taken over monitoring and evaluating information, responding to the concerns of providers and learning from past mistakes, PbR has the potential to be a highly useful tool in commissioning public services.

 

Bibliography

Bevan, G., Hood, C. (2006) ‘What’s Measured is what Matters: Targets and Gaming in the English Public Health Care System’, Public Administration 84 (3):517-538

Fox, C. and Albertson, K. (2011) ‘Payment by results and social impact bonds in the criminal justice sector: New challenges for the concept of evidence-based policy?’, Criminology and Criminal Justice, 11 (5): 395-413

LeGrand, J. (2003) Motivation, Agency and Public Policy: Of Knights and Knaves, Pawns and Queens, Oxford: Oxford University Press

Mannion, R., Marini, G., Street, A., (2008) Implementing payment by results in the English NHS: Changing incentives and the role of information, Journal of Health Organization and Management, 22 (1):79 – 88

Ministry of Justice (2010) ‘Payment by Results’ in Breaking the Cycle: Effective Punishment, Rehabilitation and Sentencing of Offenders, London: HMSO, pp.38-48

Miraldo, M, Goddard, M., Smith, P, C. (2006) The Incentive Effects of Payment by Results, Working Papers, Centre for Health Economics, University of York.

Mulheirn, I., (2011) Will the Work Programme Work?, London: Social Market Foundation

Puddicombe, B., Corry, D., Fox, C., and Albertson, K. (2012), Payment by Results, Criminal Justice Matters, 89 (1): 46-48

Rapple,B. A. (1994), Payment by Results: An Example of Assessment in Elementary Education from Nineteenth Century Britain, Education Policy Analysis Archives, 2(1):1-21

Youtube (2012) Sean Williams, Head of G4S Work Programme, gives his views on payment by results [online] Available from: http://www.youtube.com/watch?v=HGBJkR0EEHA [Accessed 18 October 2012]

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s