Joint Scandinavian Evaluation of support to Capacity Development - PowerPoint PPT Presentation

Joint Scandinavian Evaluation of support to Capacity Development Øyvind Eggen Peter Jul Larssen Lennart Peck

Scandinavian Joint Mission

Utvärderingens design och tidsplan Initial Joint Planning 2013 Danida Norad Sida Pre-study Pre-study Pre-study ToR 2014 Main Main Main evaluation evaluation evaluation Synthesis report 2015 Dissemination

Organisation • A simple MoU • Each agency responsible for its own parts • No mixing of funds • A joint ”management group” • Series of meetings at crucial points • External Advisor on CD

As compared to single donor evaluations + Stronger evidence base + Efficiency gains through division of labour + Additional perspectives – richer process + Increased attention and status within agencies + Relations between evaluation units strenghened - Requires compromise - More coordination

As compared to ”traditional” joint evaluation + A minimum of formalities + Work and resposibilities more equally divided + Flexibility to include agency specific concerns + 3 stand-alone reports of relevance for each agency - Not full use of the ”scale” - Less admin in some respects – more in other - Potentially difficult to syntesize

Reflections

Joint Scandinavian Evaluation of support to Capacity Development Conceptual approach and methodology Øyvind Eggen Peter Jul Larsen Lennart Peck

Capacity is understood as the ability of people, organizations and society as a whole to manage their affairs successfully. ‘Capacity development’ is understood as the process whereby people, organizations and society as a whole unleash, strengthen, create, adapt and maintain capacity over time. DAC (2006). The Challenge of Capacity Development: Working Towards Good Practice. Paris, OECD.

Key questions • Are we talking about the same when we talk about support to capacity development? • How to select cases/samples from an enormous heterogeneous universe with porous boundaries (Norway: 1625 database entries)? • How to compare across institutions? • Is it at all possible to measure effectiveness? 11

Pre-studies • Literature review (Sida) • Methodology study (Norad) • Conceptual framework (Danida)  Concept paper  Specification of methodology  Joint ToR Available at www.sida.se/joint-cd-evaluation 12

The Theory of Change hypothesis • Donor support to capacity development is (more) effective when it i. fits the drivers for and constraints to change (“builds on what is there”) ii. donor support is accepted as legitimate and appropriate; iii. uses results sensibly to measure progress, correct course and learn; and iv. looks beyond “supply-side”, to broader accountability relations 13

Selection • Limitation to public sector • Joint selection criteria, different processes • Three sets of samples: 1.Portfolio screening of 100 interventions (Danida 30, Sida 29, Norad 41). 2.Desk based reviews of 46 interventions (Danida 15, Sida 14, Norad 17) 3.Country visits to nine countries (Danida: Uganda, Tanzania and Nepal; Norad: Malawi, Mozambique, Vietnam; Sida: Kenya, Cambodia, Bosnia-Hercegovina) 14

Data collection • ‘Score cards’: approx. 20+ standardised variables (quantitative + qualitative). • Desk based reviews: documents + interviews • In-depth studies during country visits

Lessons • Joint conceptual and analytical framework - a challenge with different evaluation teams • Standardized variables across interventions and teams – could it be done? • Desk based reviews need more time and efforts • How to make it possible and meaningful to compare and synthesize experiences from CD across institutions? 16