Evaluating the effect of policy intervention w ith a continuous - PowerPoint PPT Presentation

Departm ent of Statistics P. Fortunati University of Bologna Evaluating the effect of policy intervention w ith a continuous treatm ent: the case of subsidies to capital accum ulation Valentina Adorno, Cristina Bernini, Guido Pellegrini Firenze, 18 gennaio 2007 1

Motivations Econom ic problem : � Does the law 488 affect performances of subsidized firms? � Are the effects different with respect to the amount of public subsides? Standard program evaluation m ethods: � Most of the relevant literature on the program evaluation deals with the estimation of causal effects of a binary treatment Solution: � Extension of the methodological tools from a binary to a continuous treatment regime 2

Objectives � Program evaluation of the effects in a continuous treatment case � Development of a new methodological approach: a matching estimator � Analysis of the impact of differences in treatment level on policy outcomes � Empirical application: the case of the Law 488/ 1992 for the manufacturing sector in Italy 3

Overview on the literature: � Despite the interest on generalization of the program evaluation framework has been recognized only few studies on continuous treatment have been carried out. � The aim is to remove any bias associated with differences in observable and unobservable characteristics among units. � The previous studies lead to generalized the propensity vious studies lead to generalized the propensity score pproach might be classified on the basis of the estimated ers of interest al outcomes Y(t) treatment effect 4

Our starting point The main drawbacks of these approaches are: � no comparison between treated and non-treated units � no concern on the selection process � The parameters of interest: � Underestimation of the continuity dimension � Estimation of the potential outcomes Y(t) Why not focus on: � Expected value of treatment causal effects at each level � Relation between effects and treatment levels 5

Our proposal We propose to develop a matching estimator to estimate the causal treatment effects as a function of the doses. The innovations deal with: 1. Parameters of interest (average treatment level effects and relation between effects and levels) 2. Splitting of the selection process (the participation decision and the treatment level assignment) 3. A matching estimator to compare treated and non-treated units that are similar in terms of their observable characteristics in the two selection processes. 6

1 . New param eter of interest � What is the effect on the response variable whit respect to the treatment level? Average treatment level effect (ATLE): α = − = ( ) [ ( ) ( 0 ) | ] T E Y T Y T t Heterogeneity of the effects: � among units (transition from individual to mean) � among treatment levels (average effects at different doses) 7

1 . New param eter of interest � How does the treatment level differently affects the effects on the response variable? Relation of treatment effects on treatment levels: α T = ε ˆ ( ) ( , ) f T � Homogeneous effects: � Proportional effects: OLS estimator � Heterogeneous effects: � quantile regression � local non parametric regression 8

2 . Splitting of the selection process � Potential outcome approach ( Y i (t) ) = + − � General observed outcome: ( ) ( 1 ) ( 0 ) y d y t d y i i i i i i � Splitting of the selection process: if d i = 1 g(Z i )+ u i t i = 0 otherwise 1 if I i > 0 d i = 0 otherwise where I i = h(w i )+ v i 9

How to deal w ith the tw o processes Two alternative solutions: A. Single equation models: � Model for the participation process: propensity score approach: P(D= 1| X) � Model for the treatment level given P(D= 1| X): Linear regression model given a specified range of P B. Sample selection models: � Model for the participation process: propensity score approach: P(D= 1| X) � Model for the treatment level given P(D= 1| X): censored data model 10

3 . The m atching estim ator � Solution A : Two step matching estimator: ⎡ ⎤ 1 2 w w ∑ where α = − = ˆ ij ij ⎢ ⎥ ( ) ( ) ( 0 ) t E y t m y m ∑ ∈ i i ij j ij 1 2 ⎣ ⎦ i w w ∈ C j ij ij j C no substitutability between the two processes � Solution B : One step matching estimator (only on the level of treatment): ⎡ ⎤ ∑ α = − ˆ ⎢ ⎥ ( ) ( ) ( 0 ) t E y t w y i i ij j ⎣ i ⎦ ∈ C j 11

Em pirical application: the Law 4 8 8 / 1 9 9 2 in I taly � The most important policy intervention to subsidise private capital accumulation in the poorest Italian regions in the last decade. � Few studies concerning ex-post evaluation of the impact of 488 (Bronzini and De Blasio ,2005; Carlucci and Pellegrini , 2005; Pellegrini and Centra, 2006) � All the papers consider the 488 as a case of a binary treatment 12

Law 4 8 8 / 1 9 9 2 : how does it w ork? � Allocation of incentives: the five indicators � quota of owner capital invested in the project � number of new employees per unit of investment � ratio of the requested subsidy on the highest subsidy applicable (auction mechanism) � a score related to the priorities of the region (location, project type and sector) � a score related to the environmental impact of the project � Identification of the ranking by region for each auction � The subsidies are allocated to projects until funding granted to each region is exhausted 13

W hy not a RDD approach? � Presence of multiple ranking by region and auctions: firms with the same level of the selection variables can be subsides or rejected, depending on regions, size and auction. � If all the firm are considered, an overlapping area of firms with the same propensity to be subsidized that are in the treated group and in the control group is available. � Then, we can contrast firms with the same characteristics but with different selection results, as the matching evaluation technique requires. 14

The data Identification of the eligible projects: � treated group: funded projects in Mezzogiorno and in manufacturing sectors (665 financed projects) � control group: projects in Mezzogiorno and in manufacturing sectors, admitted to the rank but non financed (1228 non financed projects) 15

Single equation m odels: Eq1: The participation decision process � Logit specification of the binary treatment variable � Balancing hypothesis satisfied Std. Std. Std. variables Coef. Err. P>|z| variables Coef. Err. P>|z| variables Coef. Err. P>|z| ind1 20,14 5,87 0,0010 dum_g1 -5,16 1,41 0,0000 ateco_DJ 0,31 0,23 0,1730 ind2 225,96 31,53 0,0000 dum_g2 31,84 45,14 0,4810 ateco_DK 0,40 0,32 0,2130 ind3 3,64 0,39 0,0000 area1 1,73 0,50 0,0010 ateco_DL 0,56 0,33 0,0890 ind1_2 -33,28 15,14 0,0280 area2 1,54 0,43 0,0000 ateco_DM 0,08 0,37 0,8280 ind1_3 28,64 12,35 0,0200 ateco_DB 0,24 0,29 0,4020 ateco_CB -0,24 0,46 0,6080 ind2_2 -2844,27 885,92 0,0010 ateco_DC 0,03 0,38 0,9280 ammod 1,22 1,05 0,2470 dum_g 2,56 0,79 0,0010 ateco_DE 0,31 0,34 0,3530 ampliam 1,70 1,03 0,1000 dum_ban1 5,93 0,58 0,0000 ateco_DF 0,62 0,37 0,0930 nuovoimp 1,85 1,03 0,0740 dum_ban2 3,27 0,23 0,0000 ateco_DG 0,84 0,34 0,0140 ristrut 1,59 1,09 0,1450 dum_ban3 -0,98 0,17 0,0000 ateco_DH 0,96 0,29 0,0010 _cons -16,21 1,62 0,0000 dum_m1 0,70 0,33 0,0350 ateco_DI 0,65 0,24 0,0070 LR chi2(31) 1065,59 Prob > chi2 0,000 Pseudo R2 0,4349 Number of obs 1888 16

Single equation m odels: Eq2: Treatment level assignment given PS � Linear specification for the share of subsidies on investment for each PS block obs. mean std min max quota 665 0,4548 0,1313 0,0860 0,9464 quota coef Std.Err P>|t| Number of obs 85 dum_g -0,10 0,03 0,000 F( 6,78) 9,68 ateco DF -0,21 0,07 0,003 Prob > F 0,000 ban3 -0,10 0,03 0,003 R-squared 0,427 area2 0,12 0,03 0,000 Adj R-squared 0,383 area1 0,19 0,05 0,000 ateco DK 0,09 0,06 0,099 cons 0,35 0,03 0,000 � Balancing hypothesis satisfied � Matching on the level of treatment 17

I m pact of the law 4 8 8 Effects on what? Outcome variables: � Firm growth: � Turnover � number of employees � Fixed asset � Profitability: � gross margin on turnover � Productivity: � per capita turnover 18