ecologic.de Example of an Evaluation Tool: Ex-Post Cost-Effectiveness Analyses of Environmental Policies EEA expert workshop on ex-post policy effectiveness evaluations Copenhagen, 6 / 7 November 2006 Benjamin Görlach, Ecologic (with contributions by eftec and IVM) EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06
ecologic.de ecologic.de Contents • Background on the original research • What is cost-effectiveness, why is it relevant? • Ex-post cost-effectiveness requirements in EU environmental legislation • Example: Directive on Renewable Energy • Existing guidelines and manuals for assessing cost-effectiveness • Some lessons from the practice • Conclusions and way forward Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 2
ecologic.de ecologic.de Outline of the project • Carried out for the EEA by Ecologic, eftec & IVM, September 04 - April 05 • Task: Survey of ex-post cost-effectiveness analysis of European environmental policies • Legal requirements in EC law • Guidelines and manuals for ex-post CEA • Examples of applied ex-post CEA • Final reports and literature database at: http://www.ecologic.de/projekte/3ea/panacea/ Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 3
ecologic.de ecologic.de Cost-effectiveness analysis explained • A CEA seeks to identify the activity, process or intervention that minimises resource use to achieve a given, desired result. • The cost-effectiveness of a policy option is calculated as annualised costs divided by a quantified measure of the physical effect. • Interpretation of effectiveness depends on policy target - can be pressures or impacts • Cost concepts: narrower or broader Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 4
ecologic.de ecologic.de Cost-effectiveness analysis explained • Cost-effectiveness can be assessed both ex- ante and ex-post • CEA is distinct from CBA: • CEA = achieve a given objective at least cost • CEA does not ask whether objective is worth achieving (as a CBA would) • Drawback: CEA most applicable for single- dimension objectives • With multiple side-effects / co-benefits / spillovers, CEA is less suited Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 5
ecologic.de ecologic.de C-E requirements in EU legislation 18 legislative items, 4 types of requirements: • Non-environmental legislation (with environmental impact) requiring ex-post CEA: 4 items • Environmental legislation requiring ex-post CEA (or evaluation with cost-effectiveness aspects): 4 items • Environmental legislation requiring ex-ante CEA (or evaluation with cost-effectiveness aspects): 6 items • Environmental legislation requiring ex-post evaluation, but not (necessarily) cost-effectiveness: 4 items Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 6
ecologic.de ecologic.de C-E requirements in EU legislation II Environmental legislation 4 6 4 Ex-post evaluation Cost-effectiveness (4)* required considerations required * only most relevant for the environment Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 7
ecologic.de ecologic.de Example: Evaluation of RES Directive • Article 4 of the Directive 2001/77/EC on Electricity from Renewable Energy Sources: “not later than 27 October 2005, the Commission shall present a well documented report ... [that] shall assess the success, including cost effectiveness, of the support systems [for] ... electricity from renewable energy sources [established in the Member States].” • Commission Communication & Impact Assessment published in Dec 2006 • Annex III: ex-post CEA for 1998 - 2004 Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 8
ecologic.de ecologic.de Evaluation of the RES Directive (II) Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 9
ecologic.de ecologic.de Evaluation of the RES Directive (III) Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 10
ecologic.de ecologic.de Evaluation of the RES Directive (IV) • Information presented in an accessible way • Cost ranges for generation cost and support levels (= incentive for producers, cost for consumers) • Comparison of costs & effectiveness across countries and across support instruments • No C/E ratios calculated, but graphical comparison of cost & effectiveness • Verbal explanations, e.g. co-benefits Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 11
ecologic.de ecologic.de Existing guidelines and manuals • Academic literature tends to focus on CBA: CEA is mentioned in passing, little attention • Some tailored guidance on CEA for specific policies, e.g. WFD - but: all ex-ante • Some general guidance on policy evaluation, not necessarily focused on economic tools • Commission: DG Budget, DG Regions, SecGen guidance on Impact Assessment • UK: Green Book • NL: Wegwijzer Evaluatieonderzoek Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 12
ecologic.de ecologic.de Existing guidelines and manuals • Many useful elements in different documents, together provide good insights on ex-post CEA - but none that has it all. • General focus on ex-ante assessment • Work on ex-post evaluation does not address cost-effectiveness in detail • Few documents specific on environmental policy • Little emphasis on real-life difficulties • Good examples found esp. in NL, UK, US Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 13
ecologic.de ecologic.de Case studies of applied ex-post CEA • Non-exhaustive selection of (mainly) European (mainly) ex-post CEA • 88 studies surveyed, 17 analysed in-depth • e.g. biodiversity management in Scotland • US evidence on CFC phase-out • CO2-based taxes in European countries • IMPOL studies on Large Combustion Plant Dir., Municipal Waste Incineration Dir., EMAS • Dutch Manure and Fertiliser Policy Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 14
ecologic.de ecologic.de Lessons learnt from Case Studies • Diversity of topics covered • Diversity of methods and shortcuts applied: • Few studies have applied discounting • Importance & difficulty of defining a baseline • Sensitivity testing only in some cases • Marginal abatement cost taken as proxy for compliance cost • Effects on national economy blended out • Results presented in qualitative form Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 15
ecologic.de ecologic.de Summary CEA in environmental policy • Ex-post evaluation is not a new thing - but assessment of the cost-effectiveness of environmental policy is fairly novel in EU • Some experiences in UK, NL • Little on EU level so far • Few cases where ex-post CEA is an institutionalised part of the analysis Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 16
ecologic.de ecologic.de Challenges and issues in ex-post CEA • Causality between policy measures and observed effects / blend out other influences • Data gathering - much more difficult ex-post unless monitoring existed up front • Moving up to higher (spatial/temporal) scale of analysis: dealing with greater uncertainty • Other issues: • Which cost types to consider (private / economic) • Effectiveness as intermediate or final goals (pressures / impacts) Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 17
ecologic.de ecologic.de Conclusions • Ex-post CEAs differ in scope, level of detail and methodological rigour • No single “common approach” to ex-post CEA across countries or policy issues • Ex-post evaluation requires clear targets and good data - both difficult to find ex-post • Guidance required for “real-life solutions”: methodological shortcuts to deal with time pressure, political desires, data gaps etc. Datum EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06 18
ecologic.de Thank you for your attention. Benjamin Görlach Ecologic, Pfalzburger Str. 43-44, D-10717 Berlin ! ! ! ! +49-30-86880-0, " " " " +49-30-86880-100 goerlach@ecologic.de, www.ecologic.de EEA expert workshop on ex-post policy effectiveness evaluations - Copenhagen, 6 / 7 Nov 06
Recommend
More recommend
