Independent and conditionally independent counterfactuals Marcin - PowerPoint PPT Presentation

Independent and conditionally independent counterfactuals Marcin Wolski European Investment Bank M.Wolski@eib.org CRoNoS & MDA 2019 Limassol April 15, 2019 Views expressed in this study are those of the author only, and do not necessarily represent the position of the European Investment Bank. Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 1 / 24

Overview Introduction 1 Motivation Framework 2 Unconditional dependence Conditional setup Numerics (unconditional) 3 Monte Carlo setup (unconditional) Empirical application 4 Understanding the (Granger) dependence in the US grain market Conclusions 5 The main take-aways References Supplementary materials 6 Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 2 / 24

Counterfactuals The goal of the counterfactual analysis is the comparison between what actually happened to what would have happened under an alternative scenario. How to define alternative scenarios? Exogenous policy change (Rothe, 2010), treatment group (Chernozhukov et al., 2013), filter the dependence between variables (this paper). Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 3 / 24

Quick literature overview The vast majority of impact evaluation studies focus on parametric models treatment effect models (Heckman, 1978), propensity score matching (Rosenbaum and Rubin, 1983), matching estimators models (Abadie and Imbens, 2002), OLS, diff-in-diff estimators (Gertler et al., 2010). Non-parametric methods propensity score through a nonparametric regression model (Heckman, et al. (1997, 1998)), non-parametric/parametric method (Chernozhukov et al., 2013) under an assumption called conditional exogeneity counterfactual effects can be interpreted as causal effects. fully nonparametric approach total effects (Rothe, 2010), partial effects (Rothe, 2012). Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 4 / 24

This paper Theoretical contribution provide a fully non-parametric dependence filtering framework unconditional dependence, conditional dependence, consistent inference methods Gaussian and bootstrap confidence bounds, utilize smooth estimates (improved MSE performance), numerical verification. Empirical contribution build a link to hypothesis testing, understand the dependence structure in the US grain market. Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 5 / 24

Unconditional setup (I) General assumptions Y outcome variable (1d) with CDF given by F Y ( y ), X covariate (vector) with CDF/PDF given by F X ( x ) and f X ( x ), i.i.d sample { ( Y i , X i ) : i = 1 , ..., n } . Variable dependence F Y | X ( y | x ) � = F Y ( y ) for some x , y . The dependence filtering idea counterfactual outcome variable Y ′ with realizations y ′ , F Y ′ | X ( y | x ) = F Y ( y ) for all x , y , estimate realizations y ′ . Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 6 / 24

Unconditional setup (II) Filtering through data sharpening (Hall and Minote, 2002) assume Y ′ = φ ( Y , X = x ) ≡ φ ( Y ), so that y ′ = φ ( y , x ) ≡ φ ( y ) φ : R d X +1 → R and localy invertible. Then the plug-in estimator of the joint density becomes F Y ′ | X ( y , x ) = n − 1 � n i =1 ¯ K H Y ( y − φ ( Y i )) K H X ( x − X i ) ˆ = F Y ( y ) , (1) n − 1 � n i =1 K H X ( x − X i ) where H H Y and H H X are bandwidth matrices, ¯ K H Y is a cumulative and K H X is a scaled (multivariate) kernel function satisfying the standard regularity conditions (Wand and Jones, 1995). Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 7 / 24

Unconditional setup (III) Theorem Suppose that we have an i.i.d. sample { ( Y i , X i ) : i = 1 , ..., n } from a continuous distribution with well-defined and sufficiently smooth PDFs. Then, the counterfactual random variable Y ′ with realizations y ′ , satisfying the independence condition given by F Y ′ | X ( y | x ) = F Y ( y ) , follows F Y ( y ′ ) = F Y | X ( y | x ) , (2) where F Y | X is the conditional distribution function of Y given X = x , for any y and x in the support of ( Y , X ) . Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 8 / 24

Unconditional setup (estimation) The estimator of the independent counterfactuals y ′ = ˆ F − 1 Y (ˆ ˆ F Y | X ( y | x )) . (3) Theorem Suppose that Assumptions 1-4 hold (see Appendix). Then √ n y ′ − y ′ � d → N (0 , σ 2 ) , � ˆ − (4) conditional on the data, where σ 2 is given by � K ( u ) 2 d u σ 2 = F Y | X ( y | x )(1 − F Y | X ( y | x )) F Y | X ( y | x )(1 − F Y | X ( y | x )) + . F − 1 f X ( x )Π d X F − 1 � � � � f Y Y ( F Y | X ( y | x )) j =1 h jXY f Y Y ( F Y | X ( y | x )) (5) Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 9 / 24

Unconditional setup (consistency) y ′ achieved under uniform convergence of estimators Consistency of ˆ satisfied for 1-dimensional X , higher dimensions require lower estimate bias (higher order kernels) Assumption (Bandwidths of conditional CDF) As n → ∞ , n 1 / 2 h Y / (log n ) 1 / 2 + n 1 / 2 h r Y → 0 , (i) n 1 / 2 h X / log n + n 1 / 2 h r X → 0 , (ii) where r is the kernel order. Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 10 / 24

Unconditional setup (example) Consider a stylized mean-dependent process X ∼ N (0 , 1) , (6) � 1 − a 2 ε t , y t = ax t + a ∈ (0 , 1) . Independent counterfactuals are equal to the error term y ′ t ≡ φ ( y t , x t ) = F − 1 Y ( F Y | X ( y t | x t )) � y t − ax t √ (7) � �� = y t − ax t 2 erf − 1 √ √ = erf 1 − a 2 = ε t . 2 − 2 a 2 More generally, under error exogeneity for nonseparable model y t = m ( x t , ε t ) , (8) y ′ t = F − 1 Y ( F ε ( ε t )) . Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 11 / 24

Conditional setup(I) General assumptions Y outcome variable with CDF given by F Y ( y ), Q variable(s) with CDF/PDF given by F Q ( q ) and f q ( q ), X covariate (vector) with CDF/PDF given by F X ( x ) and f X ( x ), i.i.d sample { ( Y i , Q i , X i ) : i = 1 , ..., n } . Variable dependence (CDF version of Diks and Panchenko, 2006) F Y | Q , X ( y | q , x ) � = F Y | Q ( y | q ) for some y , q , x . The filtering idea counterfactual outcome variable Y ′′ with realizations y ′′ , F Y ′′ | Q , X ( y | q , x ) = F Y | Q ( y | q ) for all y , q , x , estimate realizations y ′′ . Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 12 / 24

Conditional setup (II) Theorem Suppose that we have an i.i.d. sample { ( Y i , Q i , X i ) : i = 1 , ..., n } from a continuous distribution with well-defined and sufficiently smooth PDFs. Then, the counterfactual random variable Y ′′ , satisfying the conditional independence condition given by F Y ′′ | Q , X ( y | q , x ) = F Y | Q ( y | q ) , follows asymptotically F Y | Q ( y ′′ | q ) = F Y | Q , X ( y | q , x ) . (9) Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 13 / 24

Monte Carlo setup Process specification (Diks and Wolski (2016)) X i ∼ N (0 , 1) , (10) 0 , c + aX 2 � � Y i ∼ N , i with c > 0 and 1 > a > 0. Filtering Mean Squared Error (MSE) is given by n � 2 � � ˆ Y (ˆ y ′ ) = n − 1 F − 1 F − i Y | X ( y | x )) − F − 1 MSE (ˆ Y ( F Y | X ( y | x )) . i =1 Technicalities compare step-wise and smooth kernel estimators (normal-scale, process-driven, LS-CV bandwidths) 1000 replications. Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 14 / 24

MSE performance Table: Median MSE estimates of independent counterfactuals. Bandwidth selector n=50 n=100 n=200 n=500 n=1000 no smoothing 0.584 0.406 0.274 0.169 0.107 smoothing 0.292 0.232 0.178 0.116 0.080 Notes: Medians taken over 1000 Monte Carlo results for the ARCH process. Band- width selectors are chosen as: ‘no smoothing’ for step-wise estimators and ‘smooth- ing’ for normal-scale bandwidth selector. Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 15 / 24

Granger causality in the US grain market Null hypothesis { X t } is not Granger causing { Y t } . Diks & Wolski (2016) framework Y t +1 | ( X t , Y t ) ∼ Y t +1 | Y t , test statistic (implication of the null) q ≡ E [ f X , Y , Z ( X , Y , Z ) f Y ( Y ) − f X , Y ( X , Y ) f Y , Z ( Y , Z )] = 0 . Conditionally independent counterfactuals framework F Y ′′ t +1 | Y t , X t ( y t +1 | y t , x t ) = F Y t +1 | Y t ( y t +1 | y t ) , test statistic y ′′ z ≡ ˆ t +1 − y t +1 . σ ˆ Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 16 / 24

Granger causality in the US grain market Table: US grain market results (Diks & Wolski, 2016). Variables Linear Granger Causality Nonlinear Granger Causality (N) Raw data VAR residuals Raw data VAR residuals X Y X → Y Y → X X → Y Y → X X → Y Y → X X → Y Y → X Corn Wheat *** *** *** *** Corn Beans * Beans Wheat Notes: causality results for the pairwise relations of the log returns on the US grain market. (*), (**), (***) denote statistical significance at 10%, 5% and 1%. Period: 09/01/2010–03/06/2013. Nonlinear tests are performed on standardized data, transformed to (N)ormal marginals. The number of lags is l X = l Y = 1 from the Bayesian Information Criterion. Marcin Wolski (EIB) Independent counterfactuals April 15, 2019 17 / 24