Employment Policy Course Executive Evaluation Course Youth Employment
Why M&E? Internal 1 4 (agency, organization, Policy Credibility and policy) management sustainability External 2 3 (donors, partners) Knowledge Accountability generation Learning Legitimacy Executive Evaluation Course Employment Policy Course 2 2 A906137, Turin, 16 – 27 September 2013 Turin, 22 – 26 June 2015
… however, while it is clear that we need M&E for adequate planning, we face challenges…. • Face difficulties knowing about all the different tools and how to apply them in their policy planning • Struggle to respond to the increasing demand for impact evaluations (capacity, costs, etc.) • Even when M&E concepts are well understood, they face difficulties in moving towards implementation • Have a different “language” vis -à-vis M&E guys • Need concrete tools (e.g. indicators, survey instruments, funding sources, etc.) … then let’s get to work Executive Evaluation Course Employment Policy Course 3 3 A906137, Turin, 16 – 27 September 2013 Turin, 22 – 26 June 2015
M & E Employment Policy Course Executive Evaluation Course 4 4 A906137, Turin, 16 – 27 September 2013 Turin, 22 – 26 June 2015
It depends on what Type of evaluation questions you want to know to be answered? Descriptive Cause-and- Normative (only) effect Has the policy been Learning What is going on What is the impact of implemented as objectives with the policy? the policy on outcomes intended / performed of interest? as expected? Impact Type of Monitoring Evaluation evaluation evaluation Executive Evaluation Course Employment Policy Course 5 5 A906137, Turin, 16 – 27 September 2013 Turin, 22 – 26 June 2015
A continuous process of collecting Descriptive and analyzing information to see how (only) well a project, program, or policy is being executed and performing What is going on against expected results with the policy? Type of Monitoring evaluation Executive Evaluation Course Employment Policy Course 6 6 A906137, Turin, 16 – 27 September 2013 Turin, 22 – 26 June 2015
A systematic, objective assessment of an ongoing or completed project or Normative policy design, implementation, and result to determine its relevance, efficiency, effectiveness, impact, and Has the policy been implemented as sustainability intended / performed as expected? Type of Evaluation evaluation Executive Evaluation Course Employment Policy Course 7 7 A906137, Turin, 16 – 27 September 2013 Turin, 22 – 26 June 2015
A special type of evaluation that Cause-and- assesses the changes in the wellbeing of effect individuals, households, or communities that can be attributed to a particular intervention What is the impact of the policy on outcomes of interest? Impact Type of evaluation evaluation Executive Evaluation Course Employment Policy Course 8 8 A906137, Turin, 16 – 27 September 2013 Turin, 22 – 26 June 2015
What’s the difference between: Monitoring and Evaluation A. Nothing. They are 67% different words to describe the same activity B. Monitoring is conducted internally, Evaluation is conducted externally C. Monitoring is for management, Evaluation is for accountability 17% 17% D. Don’t know E. Other 0% 0% Employment Policy Course Executive Evaluation Course A. B. C. D. E. 9 9 Turin, 22 – 26 June 2015 A906137, Turin, 16 – 27 September 2013
PROGRAMME THEORY Employment Policy Course Executive Evaluation Course 10 10 A906137, Turin, 16 – 27 September 2013 Turin, 22 – 26 June 2015
Results Chain Causal chain Causal model Cause map Logical Framework Impact pathways Intervention theory (Log Frame) Intervention framework Intervention logic Theory of Change Investment logic Logic model Outcomes chain Results Framework Outcomes hierarchy Outcome line Outcome Mapping Program logic Program theory Programme theory Results chain Theory-based evaluation Theory-driven evaluation Theory-of-action Employment Policy Course Executive Evaluation Course 11 11 A906137, Turin, 16 – 27 September 2013 Turin, 22 – 26 June 2015 Source: Patricia Rogers
Building a RESULTS CHAIN A results chain answers 3 main questions: • What are the intended results of the program? • How will we achieve the intended results? • How will we know IF and HOW we have achieved the intended results? Executive Evaluation Course Employment Policy Course 12 12 Turin, 22 – 26 June 2015 A906137, Turin, 16 – 27 September 2013
The RESULTS CHAIN HIGHER ORDER INPUTS OUTPUT OUTCOMES ACTIVITIES GOALS Use of outputs by Financial, Actions taken Project Changes in beneficiaries and human, and or work deliverables outcomes that stakeholders other performed to within the have multiple outside the resources convert control of drivers. control of mobilized to inputs into implementing implementing support specific agency agency activities. outputs. SUPPLY DEMAND SIDE. SIDE. • Budget • Training • Training plan • New practices • Poverty reduced • Staffing • Studies • Income inequality completed adopted • Construction • Cash transfer • Use of the road reduced • School • Labor productivity delivered • Road attendance up increased • Health service constructed • School built use up Implementation Results Executive Evaluation Course Employment Policy Course Results-based management 13 13 Turin, 22 – 26 June 2015 A906137, Turin, 16 – 27 September 2013
Few Examples: Longer-term Activities Outputs Outcomes Outcomes • Teachers trained in • New methods used • Teacher training • Increased • Increased labor new methods • Textbooks Education • Textbooks completion rates/ productivity developed delivered lower dropout rates • Attendants • Birth attendants • Doctors hired applying methods • Improved maternal trained • Birth attendants • Increased use of Health • New doctors mortality training health clinics for practicing deliveries • CCTs delivered to • Increased food • CCTs delivered • Decreased poverty Social target households consumption • Targeting system • Lower child Protection and • Increased child in accordance with • MIS Labor mortality conditions health visits Employment Policy Course Executive Evaluation Course 14 14 Turin, 22 – 26 June 2015 A906137, Turin, 16 – 27 September 2013
EXERCISE: What type of question? Time : 15 minutes Instructions : Your boss wants to set up an M&E system for the national employment policy. You are asked to advise your boss of what type of evaluation he will need to set up depending on the questions he needs to answer. Read the questions in the next slide For each question, indicate on the flipchart if you think it is a “Monitoring”, “Evaluation” or “Impact Evaluation” question by sticking the round dots under the M, the E, or the IE column Executive Evaluation Course Employment Policy Course 15 15 A906137, Turin, 16 – 27 September 2013 Turin, 22 – 26 June 2015
RESULTS: Distinguishing “M”,“E” &“IE” M E IE 1. Were resources made available to the Ministry in the quantity and at the times specified by the work plan 2. Did the target population of the program “Training for the promotion of employability and economic autonomy” benefit from it and at what cost? 3. Did the expected changes in employability of vulnerable groups occur? 4. Were the policy activities carried out as planned 5. Can improved competencies at the local level be attributed to policy efforts? 6. How much did financial flows to promote self- employment increased? Executive Evaluation Course Employment Policy Course 16 16 A906137, Turin, 16 – 27 September 2013 Turin, 22 – 26 June 2015
RESULTS: Distinguishing “M”,“E” &“IE” M E IE x 1. Were resources made available to the Ministry in the quantity and at the times specified by the work plan 2. Did the target population of the program “Training X for the promotion of employability and economic autonomy” benefit from it and at what cost? 3. Did the expected changes in employability of X vulnerable groups occur? 4. Were the policy activities carried out as planned x 5. Can improved competencies at the local level be X attributed to policy efforts? 6. How much did financial flows to promote self- X employment increased? Executive Evaluation Course Employment Policy Course 17 17 A906137, Turin, 16 – 27 September 2013 Turin, 22 – 26 June 2015
SMART INDICATORS Employment Policy Course Executive Evaluation Course 18 18 A906137, Turin, 16 – 27 September 2013 Turin, 22 – 26 June 2015
What is an indicator? a basic concept that, when measured over time, Indicator: documents change in issues of interest Can you think of an example? Which of the following is NOT an indicator? • Number of legislations to promote decent work improved • Level of revenues • Number of studies carried out • Improved business performance Executive Evaluation Course Employment Policy Course 19 19 A906137, Turin, 16 – 27 September 2013 Turin, 22 – 26 June 2015
Recommend
More recommend
