Measurement to improve well-being of children and families Moira Inkelas, PhD, MPH Associate Professor UCLA Fielding School of Public Health, Department of Health Policy and Management 7 December 2016 Best Start Victoria 1
Percent of children with cystic fibrosis who are below 5th percentile for weight and are receiving supplemental feedings Rates for 120 Centers of Excellence (ranked low to high) 100% Guideline/goal 80% 60% Actual for 120 centers 40% 20% 0% Source: Schechter MS & Margolis P. 2005. Improving subspecialty healthcare: Lessons 3 from cystic fibrosis. Journal of Pediatrics.
Median Predicted Survival Age, 1994-2006 40 First reports reveal significant variability Quality Improvement Quality Improvement starts 741 35 Lives 30 Predicted survival Predicted survival improves from 29 improves from 28 years to 37 years years to 29 years 25 '94 '95 '96 '97 '98 '99 '00 '01 '02 '03 '04 '05 '06 Year Source: GT O ’ Connor/Cystic Fibrosis Foundation
The change works in The change works in 100% of the 50% of the innovation group implementation group Source: Parry, Carson-Stevens, Luff, McPherson, Goldmann. Recommendations for evaluation of health care improvement initiatives. Academic Pediatrics . 2013;13:S23-S30.
The change works in The change works in 100% of the 90% of an innovation group implementation group Source: Parry, Carson-Stevens, Luff, McPherson, Goldmann. Recommendations for evaluation of health care improvement initiatives. Academic Pediatrics . 2013;13:S23-S30.
Impact of Improvement Improve multiple Improve one Improve one type Improve one processes, process of organization condition, across organizations, and many organizations child conditions Preventing central Primary care for Cystic fibrosis Child well-being in line infections in asthma a geographic hospitals population (2 years) (2 years) (6 years) Sources: Billett AL, Colletti RB, Mandel KE, Miller M, Muething SE, Sharek PJ, Lannon CM. Exemplar pediatric collaborative improvement networks: achieving results. Pediatrics . 2013 Jun;131 Suppl 4:S196-203. Mandel KE, Kotagal UR. Pay for performance alone cannot drive quality. Arch Pediatr Adolesc Med . 2007 Jul;161(7):650-5.
To achieve an outcome for a population, we are seeking solutions that… …work at scale (do not break down when we try it for everyone) …will spread to others (all organizations implement the change, not just the most “enlightened” organization) …are sustained over time (do not degrade as attention turns to other topics)
as� a� System� I ’ m� sure� glad� the� hole� is� not� in� our� end!� Every system is perfectly designed to achieve exactly the results it gets.
“A system is an interdependent group of items, people, or processes working together toward a common purpose.” Associates in Process Improvement, Quality as a Business Strategy , 1987 10
How is improving a system different from improving a program? • Programs can be planned, implemented and evaluated. • It is not possible to plan and specify each of the detailed actions necessary for a system to produce better results. • Optimizing one part of a system does not optimize the overall system. • Meddling with one part of a system often sets off other problems • Community systems are complex and are never permanently “fixed”. To change outcomes for a population, we need an approach that sets a heading but allows for adaptation and adjustment, using testing to learn its way forward.
The Model for Improvement Model for Improvement What are we trying to Aim (stretch goal) accomplish? By when? How will we know that a Measures change is an improvement? What change can we make that Changes will result in improvement? Act Plan Study Do Source: Provost L. Model for improvement: Aims, measures, changes. Associates in Process Improvement.
The Model for Improvement Model for Improvement What are we trying to 95% families have a “good” credit score accomplish? By when? How will we know that a % with “good” score change is an improvement? • Increased banking What change can we make that • Risk-based coaching on will result in improvement? income/credit/savings/debt • Risk-based behavioural health supports • Trusted relationships with families Act Plan Study Do Source: Provost L. Model for improvement: Aims, measures, changes. Associates in Process Improvement.
The Model for Improvement Model for Improvement Global aim: Increase family self-sufficiency score What are we trying to Specific aim: Meet all prioritized basic needs of accomplish? By when? 25 families How will we know that a change is an improvement? % with all basic needs met What change can we make that • Trusted relationships will result in improvement? • Families with advocacy skills • Agencies willing to change systems • “Whatever it takes” service mentality • Extraordinary customer orientation • Sustainable solutions Act Plan Study Do Source: Provost L. Model for improvement: Aims, measures, changes. Associates in Process Improvement.
“OK people, listen up! The people upstairs have handed us this one, and we’ve gotta come through. We gotta find a way to make this – fit into a hole for this – using nothing but that .”
Focus / Inputs / Strategies / Short-term outcome Primary Best Start target group resources high-level actions areas outcomes (medium term) Service accessibility Children engage and participate in early Set locally, but Service continuity and childhood education focused on all collaboration Aboriginal children and families and Set locally Set locally Relationship-based practices children and families Children and families Active outreach and experiencing actively engage with MCH engagement vulnerability services at key ages and stages visits Family awareness and beliefs about early learning Local Core How are we measuring PDSAs indicators indicators progress and impact?
Using plan-do-study-act (PDSA) cycles for sequential building of knowledge Include a range of conditions in the sequence of tests, before implementing the change Changes that result in improvement A P S D A P S D Implement A P the change S D Test new conditions A P Theories, S D More Testing hunches, & best practices Small Scale Source: Associates in Process Improvement
Deciding the Scale of Testing Test small at the outset, when we know less, which make it easier to see cause and effect Readiness to Make the Change Current Situation Not Ready Indifferent Ready Cost of failure is Low large Confidence that current Cost of change idea will failure is lead to small Improvement Cost of failure is High large Confidence that current Cost of change idea will failure is Implement lead to small Improvement Source: The Improvement Guide , Langley et al. 2009
“I need a flashlight.” “That’s not what they have up there. Don’t give me anything that they don’t have on board.”
Indicators versus PDSA Data I NDICATORS Indicators: Achieving Overall results related to the project stretch Aim goals - are we getting better on the goals of the initiative? Dashboard Indicators T EST PDSA Data: Adapting Specific to the idea that is being tested Changes Quantitative or qualitative data on the impact of a particular test/change Example: How much time did it take? Simple tally of “liked” and “didn’t like” . How many people completed the process?
http://www.youtube.com/watch?feature=player_embedded&v=jsp-19o_5vU
Successful Cycles to Test and Adapt Change Ideas • Scale down the size of test (# of people who try it, who receive it) “Cycle of 1” - conduct the test at one meeting, with one caller, with one potential participant. Think of the smallest possible test that would be useful to you. Then reduce it by half, and by half again! • Conduct the test over a short time period • Test with volunteers • Do not try to get buy-in or consensus for the test • Collect useful data during each test • Think a couple of cycles ahead • Plan multiple cycles to test and adapt change • Share results and discuss with the full team (don’t keep results secret) • Create space for the team to test the idea In later cycles, test over a wide range of conditions Source: The Improvement Guide , Langley et al. 2009
How We Collaborate to Innovate Extraordinary family orientation Put families at the center of care Clarity of purpose Produce a coherent vision out of many problems Solutions that scale Create solutions that customize to work for all, spread, and sustain Bias toward action More “creating and doing”, than “meeting and planning” Embrace experimentation and use of data for learning Build to think and learn Embrace ambiguity Expect fog and take small steps to get unstuck All contribute and take ownership Bring together partners with diverse roles and viewpoints Sources: StartStrong Co-Creation Session, February 25, 2014 (Business Innovation Factory ), and IDEO
“There’s 1,000 things that have to happen, in order. We’re on number 8. You’re talking about number 692.” Embrace ambiguity Expect fog and take small steps to get unstuck
Recommend
More recommend
