Achieving outcomes in complex environments Toby Lowe
Summary Outcomes-Based contracting/Performance • Management makes it harder to produce good outcomes Systems produce outcomes – take • responsibility for the health of your systems Embrace complexity – people’s lives are • complex
Overview What happens when people implement • Outcomes-Based Performance Management? What can you do differently? • Working in complexity •
What happens when people implement OBPM? What’s the evidence?
A quick summary of the evidence…. “The overall conclusion from international experience of implementing an outcomes approach is that the journey is long and the results are disappointing .” Wimbush , Erica (2011): ‘ Implementing an outcomes approach to public management and accountability in the UK — are we learning the lessons?’ , Public Money & Management
What happens? Gaming OBPM turns everyone’s job into the production of data, not meeting client need.
Targets for results “frequently distort the direction of programs, diverting attention away from, rather than towards, what the program should be doing. ” Burt Perrin, “ Effective Use and Misuse of Performance Measurement”, American Journal of Evaluation, 1998
“Ossification, a lack of innovation, tunnel vision and suboptimization ” S van Thiel and F. L. Leeuw “The Performance Paradox in the Public Sector”, Public Performance and Management Review, 2002
The Work Programme: “the available evidence to date suggests that providers are engaging in creaming and parking , despite the differential payment regime.” Work Programme evaluation: Findings from the first phase of qualitative research on programme delivery, DWP, 2012
The breadth of evidence Young People – Australia (Keevers et al 2012) • Healthcare – UK (Bevan and Hood 2006), • Employment programmes (Perrin 1998 – USA • and Canada, Soss et al 2011 USA, Newton 2012 – UK) Public services - Europe and USA, (Van Thiel • and Leeuw 2002) Education - USA (Rothstein 2008) •
Latest evidence Systematic review of Outcomes-Based Contracts by Australian Government: Emma Tomkinson 2016 Outcomes-contracts delivered the contracted • outcomes But contracted outcomes weren’t what • people wanted Other outcomes got worse or stagnated •
Measurement Problem: OBPM doesn’t measure genuine impact, it measures proxies
Proxy measures “ If we had a thousand measures, we could still not fully capture the health and readiness of young children. We use data to approximate these conditions and to stand as proxies for them.” Mark Friedman, Results Based Accountability Implementation Guide #KittensAreEvil
Proxy = Measure = distortion effect of Proxy when used as a Experience Performance Measure of outcome #LittleHeresies
“Outcome” measurement
The attribution problem: Interventions don’t create outcomes
Programme Logic Model Robert Schalock & Gordon Bonham “Measuring outcomes and managing for results”, Evaluation and Program Planning , 2003
Effects of proxy measures + Lack of control = Gaming
What else can we do? Embrace the complexity of the work
Findings - Headline Complexity-friendly commissioning is based on a different attitude to: Motivation Learning System health – quality of relationships
Motivation Motivation is intrinsic, not extrinsic.
Learning Learning drives improvement: Positive error culture • Reflection on practice • Measurement •
System Health Taking responsibility for the health of the system as a whole: Networks • Building positive relationships, • nurturing trust
Principles Context matters “X would be completely ineffective if was done in [another city], but they might benefit from this model of infrastructure.” (Charitable Funder) “Humility is needed – recognising we don’t know the answer as individual organisations” (Charitable Funder) Outcomes matter – as motivation and a focus for learning “We do not, we never want to get into the measuring impact thing, and the attribution thing. It’s all nonsense I think…. I think for us it’s about saying ‘Did it work, did it do what we thought it would? If not why not?... It’s about learning rather than measuring.” (Charitable Funder).
Cultures Developing trust “The two things that are most important that I would say are, the first is trust and the second is respect. Without those two things, you can’t do anything.” (Public Sector Commissioner) It’s not cosy – it’s challenging “He [funded organisation] might also be very critical of me face to face. So he can say very critical things…. That’s okay” (Public Sector Commissioner) Learning and listening “I think creating a culture that models generosity and listening leads to critical thinking and asking questions.” (Charitable Funder)
Findings: practice
Trust Trust provides the confidence to let go of the illusion of control. I trust this organisation to do the right thing when the world changes . Question: what is a good set of reasons for a commissioners and providers to trust one another?
Trust Some potential answers: • Because they collaborate well • Because they know their role(s) within the system(s) in which they operate • Because they share and reflect on their practice with others • Because they use data intelligently to learn
Complex systems produce outcomes How can you make your places work better, as systems?
Examples
Plymouth Council: adults with complex needs • Realised outcomes-based commissioning wasn’t working • Appreciative enquiry – whole system approach • Built trusting relationships & co-production • Single alliance contract to 26 organisations • New network infrastructure – e.g. wicked problems forum • Saved over £500k • Award winning!
Buurtzorg Letting go of illusion of control Devolved power to the frontline = • Reduced home care costs • Increased client and worker satisfaction
I want to work like this… • Read the report: https://blogs.ncl.ac.uk/tobylowe/ • Check out the case studies • Join our Learning Group 13 th November - Newcastle • Help us to experiment
Thanks for listening Toby Lowe E: toby.lowe@newcastle.ac.uk Twitter: @tobyjlowe
Recommend
More recommend
