Ev Evaluating aluating Im Impact: act: Tu Turning ning Promises omises in into to Ev Evid idence ence Third Pakistan Poverty Alleviation Fund Project Tanvir Hussain/PPAF, Champak Prasad Pokharel/NES, Syed Hassan Akbar/PPAF Christel Vermeersch/World Bank Kevin Crockford, Imtiaz Alvi/World Bank Kathmandu, Nepal February 2010
1. Context Nine years of Pakistan Poverty Alleviation Fund (PPAF) operations (total US$900 mill). Third 5- year project started in 2009 Strategy and approaches continue to evolve over time Strong demand from Government of Pakistan, PPAF management and WB to show results not just project outputs Increasing number of donors and sources of financing 2
2. Background to the Project Objective : To empower the poor with increased incomes, improved productive capacity and access to services to achieve sustainable livelihoods Components : Social Mobilization and Institutional Building Livelihood Enhancement and Protection Micro-credit Access Basic Services and Infrastructure Outreach : 126 districts, 51,000+ villages Partners : 82 partner organizations 3
3. Results Chain Inputs Activities Outputs Outcomes Longterm Outcomes • Project Identification • Revitalized and new Community Poor funds and appraisal community institutions empowered of partner organizations empowered with • Technical organization increased assistance • Capacity Following incomes, Evaluation of development at livelihood • 200 improved proposals community and assets for PPAF Staff productive from partners individual levels the poor capacity and • 82 and financing increased: • No. of loans available access partners Technical Financial to services to • • Productive asset capacity achieve transfers to poor Social • building of sustainable partners • Community physical Physical livelihoods • infrastructure built Monitoring Human • and • Community health Natural • evaluation centers and schools opened and renovated 4
4. Primary Research Questions Has the project been able to affect household income, consumption and production? Has the project succeeded in enhancing the following assets of its target beneficiaries: Financial, Physical, Human, Social, Natural Has the project been able to increase access to basic services and infrastructure? Has the project been able to improve community level institutions in terms of their social capital, empowerment, and governance? Has the project been able to reduce vulnerability and improve food security and resilience to external shocks? 5
5. Outcome Indicators Household income, consumption, food consumption and household assets Agricultural production and non-farm enterprises (gender disaggregated) Access to drinking water, roads, sanitation, irrigation schemes, and education and health facilities Linkages and access to other public and private service providers Participation of women in community institutions Participation of poor in community institutions This exercise will focus on Research Question 1 and the first two clusters of outcome indicators Existing evaluation looking at infrastructure and social mobilisation 6
6. Identification Strategy/Method Basic sampling unit: Settlement Preferred Option: Randomly selected treatment and control settlements, based on phasing. Option 2: Treatment settlements pre-determined and matching control villages to be identified (propensity score matching). Several baseline and impact evaluations staggered over 8 cohorts, households interviewed at 3 points of time. Difference-in-difference method for analysis, use of IV for any possible contamination. Results augmented with monitoring reports and data. Poverty scorecard administered in each settlement. 7
7. Sample and data A representative sample taking care of geographic spread, partner diversity, financial weight of interventions, and multitude of components and interventions is a challenge. Sample to be stratified to ensure adequate representation of each cateogry. Sample is expected to provide statistically valid results at the project level. Based on final stratificaton, 95% confidence level, and ± 3% error margin, a representative sample of settlements and households to be selected. Based on simulation, an estimated 4,000 treatment (and 4,000 control) households will be required. 8
8. Time Frame/Work Plan Timeline of evaluation surveys Treatment and control groups for each cohort of baseline, mid- term and end-of-project evaluation, cohort fixed effects. Case studies to be conducted at various points of time. User/beneficiary surveys to be conducted at the end of each financial year. 9
9. Sources of Financing A portion of the financing from project funds Seek remaining funds from national, regional, and global sources Existing MoU with premier academic institutions. Similar linkages to be established with international institutions seeking partnership 10
Recommend
More recommend
