Measur suring ing Result lts s and nd Evalu luating ating Impa pact: t: Turn rnin ing g Promise ises s in into Evid idence ence RWANDA RUSA Louis HUMUZA James LUDWIG DE NAEYER NSHIMIRUMUREMYI Christophe JENIFER STURDY BASINGA PAULIN Cape Town, South Africa December 2009 Human Development Development Impact Africa Spanish Impact Network Evaluation Initiative Region Evaluation Fund
1. Background Results Based Financing on raise in the development agenda, specially in the health sector Importance of linking payment to performance How to measure performance routinely so that payment can be linked to accurate performance Scaling of RBF 2
Background : RWANDA PBF at the health facilities nationally implemented Concerns about the accuracy of the data being reported by health facilities Measurement of the performance rewarded by PBF High temptation of cheating Experimentation setting for an evaluation experiment on data verification Results to be used : Improve reporting system in Rwanda Behavior change by implementing penalties Generate knowledge on data verification strategy in context of RBF (what cost effective strategy) Inform other RBF projects
Unanswered questions What is the “right” indicators to monitor for cheating? Amount of Incentive ? Frequency ? What is the proportion of planned and unplanned cheating What are the right methods to detect cheating: Record verification at facility level? Patient tracing at the community level? Combination of methods? How can we use the results of the data verification to inform the system What are the appropriate penalties that can be applied to correct perverse behavior What intensity and frequency of data check and validation is sufficient to ensure quality?
3. Study objectives General objective : Strengthen and Improve accuracy of the information used to reward performance Specifics objectives: Improve data quality Ensure accuracy of the data Pay for true performance Promote culture of accountability 5
5. Intervention Checking the consistency of the data system at health facility level: Data reliability check Data accuracy check : tracing of the patient across different services (reception, consultation, laboratory, pharmacy). Check the date of payment in the registry book (health insurance if appropriate) Tracing patients: ghosts clients Trace patients at the community 6
5. Identification Strategy/Method Random assignment of facilities into 5 arms: Control group : Data reliability check: monthly report check with registry T1: Data accuracy check. Patient tracing across different services : Once a year T2: Data accuracy check. Patient tracing across different services : Twice a year T3: Patient tracing in the community : once a year T4: Patient tracing in the community : twice a year 7
6. Sample and data Selection of the indicators to be checked Total possible : 24 (13 PMA and 11 HIV) Choice based on level of incentive? Tracer indicators (maternity indicators: prenatal & deliveries?) Ethical consideration (HIV services, modern contraceptive) Power calculation to determine the number of patients / facility to be sampled Patient record Community Study tools development 8
Methodology Data monitoring program: Matching of facilities based on the quality score of data registry Block randomization of health facilities in 5 study arms Baseline in all arms on data quality Public announcement of the data quality check to everybody in the coming 3 years Intervention : auditing Follow up study 9
Identification Strategy/Method CONTROL DATA ACCURACY PATIENTS TRACING CHECK ACROSS IN THE COMMUNITY RECORDS Low Low High High frequency frequency frequency frequency (1/year) (1/year) (2x/ year) (2x/ year) 10
7. Time Frame/Work Plan 2010 -2013? 11
8. Sources of Financing HRBF ? 12
Recommend
More recommend
