Scuola Superiore Sant’Anna, Management & Health Laboratory (MeS Lab) Effectiveness of the Performance Evaluation System in the Public Health Sector Sabina Nuti s.nuti@sssup.it PUHMA Seminar, Lugano, November 27°, 2012 1
The experience of MeS Lab on Performance Evaluation Systems (PES) Ministry of Health Ministry of Health Network of Regions Network of Regions Tuscany Region Tuscany Region SIVeAS Project SIVeAS Project The performance The performance The performance The performance The performance The performance evaluation system of evaluation system of evaluation system at evaluation system at evaluation system at evaluation system at regional healthcare regional healthcare regional level regional level regional level regional level systems systems Transparency and It aims to support It aims to support accountability to ensure governance system at governance system at essential levels of care regional level regional level (LEA) at national level 34 indicators, of which 23 130 indicators, of which 80 250 indicators, of which 130 concern performance concern performance concern performance evaluation. evaluation. evaluation. Data is available on the Italian Data is available since 2008 Data is available since 2006 Ministry of Health’s website at the following website: at the following website: since 2010 in the SIVeAS www.performance.sssup.it/ www.performance.sssup.it/ section: www.salute.gov.it network toscana 2
Structure of the evaluation system at regional level 250 indicators in total 130 evaluation indicators 50 index indicators 6 areas
The multidimensional reporting system In order to describe the performance evaluation system, six areas have been identified to highlight the core results of the regional healthcare system. Capacity to pursue Population health regional strategies status Clinical Efficiency and 6 performance performance financial performance reporting areas Patients Employees Satisfaction Satisfaction 4
The five assessment bands Scores and colors: Excellent Performance (Strength) 4 - 5 GREEN 3 - 4 Good Performance LIGHT GREEN 2 - 3 Average Performance YELLOW 1 - 2 Poor Performance ORANGE Very poor Performance (Weakness) 0 - 1 RED 5
The reference criteria for assessment bands 1. I nternational standards , if existing (i.e.: Caesarean rate by WHO); set out by the Regional 2. Regional standards Government; 3. The regional mean , standardized by several factors to allow comparisons across Health Authorities. 6
How to build up the indicators “I ndicator tree” I ndex indicator I ndicator I ndicator I ndicator I ndicator 1.1 1.2 1.3 1.4 7
I ndicator C5: Clinical Quality Level: Healthcare C5.2 Percentage of femoral fractures operated Provider within 2 days of admission Percentage of interventions for femoral fracture with length of stay between admission and intervention ≤ Definition: 2 days No. of interventions for femoral fracture with length of stay between admission and intervention ≤ 2 days Numerator: No. of interventions for femoral fracture Denominator: No. of femoral fracture interventions with length of stay between admission and intervention ≤ 2 days Mathematical x 100 formula: No. of interventions for femoral fracture Only inpatients admissions are considered. ICD9-CM Codes for principal diagnosis: Fracture of the femur neck 820.xx AND ICD9-CM codes for principal or secondary intervention: Notes: 79.15 Closed reduction of femur fracture, with internal fixation 79.35 Open reduction of femur fracture, with internal fixation 81.51 Total hip replacement 81.52 Partial hip replacement 78.55 Internal fixation of the femur without fracture reduction Regional Reporting System – SDO Source: Reference Regional objective: 80% parameter: 8
C5.2 % of femoral fracture operated within 2 days from admission - 2010
C5.2 of femoral fracture operated within 2 days from admission 2008-2009-2010 10
To visually represent the results of the six areas, each Health Authority has a personal “target” diagram, divided in five assessment bands. The more the Health Authority is able to reach objectives and obtain good results in each of the six areas, the nearer the performance indicator is to the centre. 11
Lugano, 27 novembre 2012
Lugano, 27 novembre 2012
Lugano, 27 novembre 2012
Lugano, 27 novembre 2012
Lugano, 27 novembre 2012
Lugano, 27 novembre 2012
Lugano, 27 novembre 2012
Lugano, 27 novembre 2012
Lugano, 27 novembre 2012
Lugano, 27 novembre 2012
Lugano, 27 novembre 2012
Lugano, 27 novembre 2012
Dissemination of results Each year MeS-Lab issues an annual Report with the performance of the Tuscan Health Authorities and delivers it to the Regional Board, the Regional Council, the management and all interested citizens. From 2007 the evaluation system is available on web after registration: http://performance.sssup.it/toscana
Linking measurement to performance management in public health care systems Performance evaluation allows policy to be linked to management and to orient output results in order to achieve outcomes. Some evidences from the Performance Evaluation System (PES) adopted in the Tuscan health care system Nuti S., Seghieri C, Vainieri M. Assessing the effectiveness of a performance evaluation system in the public health care sector: some novel evidence from the Tuscany Region experience. Journal of Management and Governance forthcoming January 2012 25
The percentage of femur fractures operated within 2 days of admission– National Trend 2007-2008-2009 90.00 80.00 70.00 60.00 50.00 40.00 2007 2008 30.00 2009 20.00 10.00 0.00 26
Strategies and results… Pinnarelli L., Nuti S,Sorge C, Davoli M.Fusco D,Agabiti N, Vainieri M, Perucci CA, 2012 What drives hospital performance? The impact of comparative outcome evaluation of patients 27 admitted for hip fracture in two Italian regions.BMJ Quality and Safety Vol.2
Plot per capita cost vs % overall performances 2007, confirmed in all the following years Plot per capita cost and % overall performances 60% 8 % overall performances 50% 11 5 10 4 12 7 40% 3 6 2 30% 1 20% 9 10% 0% 1450 1500 1550 1600 1650 1700 1750 1800 Per capita cost Significance level p<0.05 The reference lines correspond to regional average Each number represents a LHA as follows: 1. Massa Carrara; 2. Lucca; 3. Pistoia; 4. Prato; 5. Pisa; 6. Livorno; 7. Siena; 8. Arezzo; 9. Grosseto; 10. Firenze; 11. Empoli; 12. Viareggio Now management and professionals are aware that high costs do not mean high quality 28
Governance through the PES By working on variability of quality and appropriateness indicators, Tuscan health system could re- allocate about 7% of its financial budget
How is the Tuscan Health system managed? The Performance Evaluation System • Indicators of the PES are selected since 2004 by Tuscan Health Authorities, Regional Administration professionals, health professionals • Continuous improvement and better results than other italian regions ( Pinnarelli, Nuti et al. What moves hospital performance? Impact of outcome evaluation for patients admitted for hip fracture in two Italian Regions. BMJ Quality &Safety,2012) PUBLIC DISCLOSURE OF RESULTS VISUAL REPORTING SYSTEM PROFESSIONALS AND MANAGERS LARGE INVOLVEMENT STRONG POLITICAL COMMITTMENT PES LINKAGE TO CEO’S REWARDING SYSTEM Nuti S, Seghieri C, Vainieri M. Assessing the effectiveness of a performance evaluation system in the public health care sector: 30 some novel evidence from the Tuscany Region experience. Journal of Management and Governance 2012
The Tuscan rewarding system Definition of the targets based on baseline room for improvement (for PES indicators where there is a clear standard to achieve) Exception: Overall performance indicators that follows the standard evaluation bands 31
Reward system in the Tuscany Region INTRINSIC EXTRINSIC HEALTH PROFESSIONALS CEO (managers) Professional reputation Financial incentives that can achieve the 20% of public disclosure of results the salary Enabling peer review mechanism CEO’s rewarding system added emphasis on the Tuscan PES: incentivized indicators improve 2.7 times than other PES indicators. Moreover the results of a second model on 2008-2010 data show that incentivized indicators that keep into account the baseline performance improve more than the others (OR 1.5) . Due to this empirical evidence, in 2011 every Health Authority receives 32 personalized target for each indicator of the Tuscan PES in order to gather the financial reward related to the overall indicator. S.Nuti, M.Vainieri: Do CEO reward system drive performance in the public health sector?Evidence from Italy., 2012, Under Review
The performance evaluation system is able to drive improvement… but is it also a tool to achieve equity? The Pes shows that there is large variation not only between north and south but also in each region. Variation management across and within the regions should be included in the governance system as a strategic tool at each level. 33
Linking measurement to performance management in public health care systems FROM… …TO Snapshot of the Improvement and performance variation map thorugh the target 34
Recommend
More recommend
