Notes Slide Set 12 Model Specification and Identification Pietro Coretto pcoretto@unisa.it Econometrics Master in Economics and Finance (MEF) Università degli Studi di Napoli “Federico II” Version: Saturday 28 th December, 2019 (h16:07) P. Coretto • MEF Model Specification and Identification 1 / 33 Summary Notes Specification of the model Consequences of model misspecification in predictive and explanatory modeling Identification of stochastic models Endogeneity Instrumental variables P. Coretto • MEF Model Specification and Identification 2 / 33
Notes Part I Specification P. Coretto • MEF Model Specification and Identification 3 / 33 Specification Notes Specification: is the art of finding: (i) the set of regressors that explain/predict the Y appropriately ; (ii) and appropriate settings for the error term Variable selection: the choice of the set of regressors. It is the central issue of model specification although choosing a model means more Predictive paradigm: specification is easier, because it is possible to assess the main goal (prediction error). However, even in this case we can get easily in the overfitting trap. Explanatory/Causal paradigm: specification is much harder! It relies on a combination of theoretical insights and empirical experience. Nobody really knows when the model is well specified! In general we try to exclude causes of bad specification P. Coretto • MEF Model Specification and Identification 4 / 33
Variables Selection Notes Consider two competing models Model 1: y = Xβ + ε Model 2: y = X 0 θ 0 + Xθ + ν Questions: 1 Predictive paradigm: which model better predicts Y 2 Explanatory/Causal paradigm: which model explains accurately the relationships between Y and X s? Which model is “ true ”? P. Coretto • MEF Model Specification and Identification 5 / 33 Overifitting trap Notes Consider the OLS fits of the previous models y = Xb + e (12.1) and y = X 0 t 0 + Xt + v (12.2) From which Xb + e = X 0 t 0 + Xt + v (12.3) Since these are all OLS fits, then e ′ X = v ′ X 0 = v ′ X = 0 . P. Coretto • MEF Model Specification and Identification 6 / 33
Notes Multiply both sides of (12.3) by e ′ and v ′ , and obtain e ′ e = e ′ X 0 t 0 + e ′ v v ′ e = v ′ v which implies e ′ X 0 t 0 = e ′ e − v ′ v (12.4) Now we show that going from the first OLS fit to the second, the sum of squared residuals cannot increase, that is: v ′ v ≤ e ′ e P. Coretto • MEF Model Specification and Identification 7 / 33 Notes v ′ v = ( y − X 0 t 0 − Xt ) ′ ( y − X 0 t 0 − Xt ) = ( Xb + e − X 0 t 0 − Xt ) ′ ( Xb + e − X 0 t 0 − Xt ) = ( X ( b − t ) − X 0 t 0 + e ) ′ ( X ( b − t ) − X 0 t 0 + e ) set a = X ( b − t ) − X 0 t 0 = a ′ a + 2 e ′ a + e ′ e = a ′ a − 2 e ′ X 0 t 0 + e ′ e = a ′ a − 2( e ′ e − v ′ v ) + e ′ e = a ′ a − e ′ e + 2 v ′ v Observe that v ′ v = e ′ e − a ′ a implies that v ′ v ≤ e ′ e Therefore, adding a new set of regressors will reduce the RSS and increases the R 2 P. Coretto • MEF Model Specification and Identification 8 / 33
Predictive paradigm Notes Main goal: ability to predict Y as good as possible. A low prediction error (RSS) is targeted Overfitting: whenever the model becomes richer/more flexible (adding regressors), it will fit the observed data not worse. Therefore, any measure of fit based on in-sample information will get into the overfitting trap Two approaches to solve this problem Penalized (in-sample) measures of fit: an in-sample statistics of fit that penalizes for the addition of regressors Out-of-sample prediction error: estimate the expected prediction error on samples not used to fit the model P. Coretto • MEF Model Specification and Identification 9 / 33 Penalized (in-sample) measures of fit Notes Adjusted R 2 RSS = 1 − n − 1 R 2 = 1 − n − K (1 − R 2 ) TSS n − K n − 1 � �� � > 1 You wan to maximize it. It is not based on distributional assumptions for the error ter Akaike’s information criterion � RSS � + 2 K AIC = log n n You wan to minimize it. Implicitly assumes that the error term is at least approximately normal Bayes information criterion � RSS � + K log( n ) BIC = log n n You wan to minimize it. Implicitly assumes that the error term is at least approximately normal P. Coretto • MEF Model Specification and Identification 10 / 33
Out-of-sample prediction error via cross-validation Notes Step 1: Fix H < n and divide the n units into two subsets: Train set: randomly(*) select n − H units Test set: select the remaining H units. Let y o 1 , y o 2 , . . . , y o H the Y -values in this set. Step 2: In-sample estimation apply the OLS to estimate b using observations in the Train set Step 3: Out-of-sample prediction y o y o y o predict the Y -values on the Test set using b . Let ˆ 1 , ˆ 2 , . . . , ˆ H the predicted values (*) Remark : the appropriate random split depends on the sampling design. In case of random sampling we may sample n − H observation uniformly without replacement. In case of time series data we take n − H consecutive observations (blocks) P. Coretto • MEF Model Specification and Identification 11 / 33 Notes Step 4: measure the expected prediction error The most popular prediction accuracy measure is the root mean square prediction error � � H � 1 � � ( y o y o h ) 2 RMSPE = h − ˆ H h =1 Another popular measure is the mean absolute prediction error H MAE = 1 � | y o y o h − ˆ h | H h =1 Since the splitting introduces additional random variations, steps 1–4 are repeated a number of times and the resulting RMSPE or MAE values are averaged across splittings. P. Coretto • MEF Model Specification and Identification 12 / 33
Explanatory/causal paradigm Notes Main goal: assess the structural relationships between X and Y , produce “ good estimates ” and valid inference about partial effects. Example: given a model log(wage) i = β 1 + β 2 SchoolYears i + . . . + ε i we want a a “ good estimate ” of β 2 , that is the expected variation of log(wage) produced by a unit change of SchoolYears ( ceteris paribus ) we want that Se( b 2 ) is accurate we want that hypothesis tests are optimal Relevant aspects: here we are interested in how a model misspecification impact the quality of estimates and the resulting inference P. Coretto • MEF Model Specification and Identification 13 / 33 Effects of misspecification on estimation/inference Notes Reconsider the two alternative models Model ( m = 1) : y = X 1 β 1 + ε Model ( m = 2) : y = X 1 β 1 + X 2 β 2 + u Two cases: inclusion of irrelevant variables: the truth = Model 1, but we estimate Model 2? omission of relevant variables: the truth = Model 2, but we estimate Model 1? Notations: b ( m ) is the OLS estimate of β h based on Model ( m ) . Let h b ( m ) be the overall OLS estimator under Model ( m ) . P. Coretto • MEF Model Specification and Identification 14 / 33
Inclusion of irrelevant variables Notes Assume Model 1 generates data (true model), but we estimate Model 2. We are failing at imposing the restriction β 2 = 0 Consequences the OLS is still unbiased and consistent for β 1 � � � � E[ b (2) b (2) | X 1 , X 2 ] = β 1 β 1 p 1 1 and − → E[ b (2) b (2) = 0 0 | X 1 , X 2 ] 2 2 s 2 is unbiased for σ 2 � b (2) � is consistent � AVar However, Var[ b (2) | X 1 , X 2 ] � Var[ b (1) | X 1 ] = ⇒ eventually some 1 1 efficiency is lost. The loss of efficiency increases with the correlation between X 1 and X 2 Test power is reduced because probability of false negatives increases (not rejecting H 0 when H 1 is true). E.g. the default test tempts to exclude relevant variables P. Coretto • MEF Model Specification and Identification 15 / 33 Omission of relevant variables Notes Assume Model 2 = truth, i.e. it generates data, but we estimate estimate Model 1. Assume β 2 � = 0 , that is X 2 correlates with y Observe that from true Model 2: y − X 1 β 1 = X 2 β 2 + u from wrong Model 1: ε = y − X 1 β 1 Therefore ε = X 2 β 2 + u X 2 is called “ unobserved heterogeneity ” : variations across individual units not accounted by the specified model, and that will go into the error term of the misspecified model Endogeneity : if the unobserved heterogeneity X 2 correlates with X 1 , then in the misspecified model the error correlates with the regressors violating orthogonality assumptions. Failure of orthogonality leads to non identifiable models (see later). Effects of endogeneity are devastating P. Coretto • MEF Model Specification and Identification 16 / 33
Recommend
More recommend
