BS2247 Introduction to Econometrics Lecture 6: The multiple regression model OLS Unbiasedness, OLS Variances, Including irrelevant variables and excluding relevant variables, the Gauss-Markov Theorem Dr. Kai Sun Aston Business School 1 / 18
Assumptions for Unbiasedness MLR for “Multiple Linear Regression” ◮ Assumption MLR.1: The population model is linear in parameters, i.e., y = β 0 + β 1 x 1 + β 2 x 2 + . . . + β k x k + u ◮ Assumption MLR.2: Use a random sample of size n , { ( x 1 i , . . . , x ki , y i ) : i = 1 , 2 , . . . , n } ◮ Assumption MLR.3: No perfect collinearity: no exact linear relationship among the independent variables. For example, we cannot estimate a model like this: y = β 0 + β 1 x 1 + β 2 x 2 + u , where x 1 = 2 x 2 . 2 / 18
Assumptions for Unbiasedness MLR for “Multiple Linear Regression” ◮ Assumption MLR.1: The population model is linear in parameters, i.e., y = β 0 + β 1 x 1 + β 2 x 2 + . . . + β k x k + u ◮ Assumption MLR.2: Use a random sample of size n , { ( x 1 i , . . . , x ki , y i ) : i = 1 , 2 , . . . , n } ◮ Assumption MLR.3: No perfect collinearity: no exact linear relationship among the independent variables. For example, we cannot estimate a model like this: y = β 0 + β 1 x 1 + β 2 x 2 + u , where x 1 = 2 x 2 . 2 / 18
Assumptions for Unbiasedness MLR for “Multiple Linear Regression” ◮ Assumption MLR.1: The population model is linear in parameters, i.e., y = β 0 + β 1 x 1 + β 2 x 2 + . . . + β k x k + u ◮ Assumption MLR.2: Use a random sample of size n , { ( x 1 i , . . . , x ki , y i ) : i = 1 , 2 , . . . , n } ◮ Assumption MLR.3: No perfect collinearity: no exact linear relationship among the independent variables. For example, we cannot estimate a model like this: y = β 0 + β 1 x 1 + β 2 x 2 + u , where x 1 = 2 x 2 . 2 / 18
Assumptions for Unbiasedness ◮ Assumption MLR.4: Zero conditional mean, i.e., E ( u i | x 1 i , . . . , x ki ) = E ( u i ) = 0: the error term is not correlated with any of the regressors (i.e., all the regressors are exogenous) MLR.1 - MLR.4 guarantee that: E (ˆ β j ) = β j , ∀ j = 0 , 1 , 2 , . . . , k . ˆ β j is unbiased for β j . Proof needs linear algebra - not required. 3 / 18
Assumptions for Unbiasedness ◮ Assumption MLR.4: Zero conditional mean, i.e., E ( u i | x 1 i , . . . , x ki ) = E ( u i ) = 0: the error term is not correlated with any of the regressors (i.e., all the regressors are exogenous) MLR.1 - MLR.4 guarantee that: E (ˆ β j ) = β j , ∀ j = 0 , 1 , 2 , . . . , k . ˆ β j is unbiased for β j . Proof needs linear algebra - not required. 3 / 18
Including irrelevant variables ◮ Suppose that the true model is y = ˆ β 0 + ˆ β 1 x 1 + ˆ β 2 x 2 + ˆ u ◮ However, we estimate the model y = ˜ β 0 + ˜ β 1 x 1 + ˜ β 2 x 2 + ˜ β 3 x 3 + ˜ u ◮ This means that we included an irrelevant variable, x 3 . ◮ But if the estimated model still satisfies MLR.1 - MLR.4, then E (˜ β 0 ) = β 0 , E (˜ β 1 ) = β 1 , E (˜ β 2 ) = β 2 , E (˜ β 3 ) = β 3 = 0. (although the variances of the parameters in the estimated model would be larger than those in the true model) 4 / 18
Including irrelevant variables ◮ Suppose that the true model is y = ˆ β 0 + ˆ β 1 x 1 + ˆ β 2 x 2 + ˆ u ◮ However, we estimate the model y = ˜ β 0 + ˜ β 1 x 1 + ˜ β 2 x 2 + ˜ β 3 x 3 + ˜ u ◮ This means that we included an irrelevant variable, x 3 . ◮ But if the estimated model still satisfies MLR.1 - MLR.4, then E (˜ β 0 ) = β 0 , E (˜ β 1 ) = β 1 , E (˜ β 2 ) = β 2 , E (˜ β 3 ) = β 3 = 0. (although the variances of the parameters in the estimated model would be larger than those in the true model) 4 / 18
Excluding relevant variables: Omitted Variable Bias ◮ Suppose that the true model is y = ˆ β 0 + ˆ β 1 x 1 + ˆ β 2 x 2 + ˆ u ◮ However, we estimate the model y = ˜ β 0 + ˜ β 1 x 1 + ˜ u ◮ This means that we omitted a possibly relevant variable, x 2 . ◮ So the estimated model is actually a simple regression model, � i ( x 1 i − ¯ x 1 ) y i and ˜ β 1 = x 1 ) 2 � i ( x 1 i − ¯ ◮ We want to link ˜ β 1 (in the estimated model) with ˆ β 1 (in the true model) 5 / 18
Excluding relevant variables: Omitted Variable Bias ◮ Suppose that the true model is y = ˆ β 0 + ˆ β 1 x 1 + ˆ β 2 x 2 + ˆ u ◮ However, we estimate the model y = ˜ β 0 + ˜ β 1 x 1 + ˜ u ◮ This means that we omitted a possibly relevant variable, x 2 . ◮ So the estimated model is actually a simple regression model, � i ( x 1 i − ¯ x 1 ) y i and ˜ β 1 = x 1 ) 2 � i ( x 1 i − ¯ ◮ We want to link ˜ β 1 (in the estimated model) with ˆ β 1 (in the true model) 5 / 18
Excluding relevant variables: Omitted Variable Bias So plug in the true model, y i = ˆ β 0 + ˆ β 1 x 1 i + ˆ u i , into ˜ β 2 x 2 i + ˆ β 1 : � x 1 )(ˆ β 0 + ˆ β 1 x 1 i + ˆ i ( x 1 i − ¯ β 2 x 2 i + ˆ u i ) ˜ = ˆ β 1 + ˆ β 2 ˜ � β 1 = δ 1 , x 1 ) 2 i ( x 1 i − ¯ where ˜ δ 1 is the coefficient from regressing x 2 on x 1 . 6 / 18
Excluding relevant variables: Omitted Variable Bias Take the conditional expectation for both sides of ˜ β 1 = ˆ β 1 + ˆ β 2 ˜ δ 1 , E (˜ β 1 | x 1 , x 2 ) = E (ˆ β 1 + ˆ β 2 ˜ δ 1 | x 1 , x 2 ) = E (ˆ β 1 | x 1 , x 2 ) + ˜ δ 1 E (ˆ β 2 | x 1 , x 2 ) = β 1 + ˜ δ 1 β 2 7 / 18
◮ Define the bias of ˜ β 1 as bias (˜ β 1 ) = E (˜ β 1 | x 1 , x 2 ) − β 1 = ˜ δ 1 β 2 ◮ So we can say that bias (˜ β 1 ) = 0 if and only if (1) ˜ δ 1 = 0: x 1 and x 2 are uncorrelated, or (2) β 2 = 0: x 2 is irrelevant ◮ The bias is positive if and only if (1) ˜ δ 1 > 0 and β 2 > 0, or (2) ˜ δ 1 < 0 and β 2 < 0 ◮ The bias is negative if and only if (1) ˜ δ 1 > 0 and β 2 < 0, or (2) ˜ δ 1 < 0 and β 2 > 0 8 / 18
◮ Define the bias of ˜ β 1 as bias (˜ β 1 ) = E (˜ β 1 | x 1 , x 2 ) − β 1 = ˜ δ 1 β 2 ◮ So we can say that bias (˜ β 1 ) = 0 if and only if (1) ˜ δ 1 = 0: x 1 and x 2 are uncorrelated, or (2) β 2 = 0: x 2 is irrelevant ◮ The bias is positive if and only if (1) ˜ δ 1 > 0 and β 2 > 0, or (2) ˜ δ 1 < 0 and β 2 < 0 ◮ The bias is negative if and only if (1) ˜ δ 1 > 0 and β 2 < 0, or (2) ˜ δ 1 < 0 and β 2 > 0 8 / 18
◮ Define the bias of ˜ β 1 as bias (˜ β 1 ) = E (˜ β 1 | x 1 , x 2 ) − β 1 = ˜ δ 1 β 2 ◮ So we can say that bias (˜ β 1 ) = 0 if and only if (1) ˜ δ 1 = 0: x 1 and x 2 are uncorrelated, or (2) β 2 = 0: x 2 is irrelevant ◮ The bias is positive if and only if (1) ˜ δ 1 > 0 and β 2 > 0, or (2) ˜ δ 1 < 0 and β 2 < 0 ◮ The bias is negative if and only if (1) ˜ δ 1 > 0 and β 2 < 0, or (2) ˜ δ 1 < 0 and β 2 > 0 8 / 18
◮ Define the bias of ˜ β 1 as bias (˜ β 1 ) = E (˜ β 1 | x 1 , x 2 ) − β 1 = ˜ δ 1 β 2 ◮ So we can say that bias (˜ β 1 ) = 0 if and only if (1) ˜ δ 1 = 0: x 1 and x 2 are uncorrelated, or (2) β 2 = 0: x 2 is irrelevant ◮ The bias is positive if and only if (1) ˜ δ 1 > 0 and β 2 > 0, or (2) ˜ δ 1 < 0 and β 2 < 0 ◮ The bias is negative if and only if (1) ˜ δ 1 > 0 and β 2 < 0, or (2) ˜ δ 1 < 0 and β 2 > 0 8 / 18
Variance of OLS ◮ Now we know that the sampling distribution of our estimate is centered around the true parameter (unbiasedness) ◮ Want to think about how spread out this distribution is ◮ Much easier to think about this variance under an additional assumption, so Assumption MLR.5: Homoskedasticity, i.e., Var ( u | x 1 , x 2 , . . . , x k ) = σ 2 , or Var ( y | x 1 , x 2 , . . . , x k ) = σ 2 The conditional variance is a constant (not a function of x ’s). 9 / 18
Variance of OLS ◮ Now we know that the sampling distribution of our estimate is centered around the true parameter (unbiasedness) ◮ Want to think about how spread out this distribution is ◮ Much easier to think about this variance under an additional assumption, so Assumption MLR.5: Homoskedasticity, i.e., Var ( u | x 1 , x 2 , . . . , x k ) = σ 2 , or Var ( y | x 1 , x 2 , . . . , x k ) = σ 2 The conditional variance is a constant (not a function of x ’s). 9 / 18
Variance of OLS ◮ Now we know that the sampling distribution of our estimate is centered around the true parameter (unbiasedness) ◮ Want to think about how spread out this distribution is ◮ Much easier to think about this variance under an additional assumption, so Assumption MLR.5: Homoskedasticity, i.e., Var ( u | x 1 , x 2 , . . . , x k ) = σ 2 , or Var ( y | x 1 , x 2 , . . . , x k ) = σ 2 The conditional variance is a constant (not a function of x ’s). 9 / 18
Variance of OLS ◮ The MLR.1-MLR.4 assumptions for unbiasedness, plus the homoskedasticity assumption, MLR.5, are known as the Gauss-Markov assumptions. ◮ Given the Gauss-Markov assumptions, σ 2 Var (ˆ β j ) = j ) , ∀ j = 1 , . . . , k SST j (1 − R 2 where SST j = � x j ) 2 : total sum of squares for x j i ( x ji − ¯ j : R 2 from regressing x j on all other regressors, including R 2 intercept 10 / 18
Variance of OLS ◮ The MLR.1-MLR.4 assumptions for unbiasedness, plus the homoskedasticity assumption, MLR.5, are known as the Gauss-Markov assumptions. ◮ Given the Gauss-Markov assumptions, σ 2 Var (ˆ β j ) = j ) , ∀ j = 1 , . . . , k SST j (1 − R 2 where SST j = � x j ) 2 : total sum of squares for x j i ( x ji − ¯ j : R 2 from regressing x j on all other regressors, including R 2 intercept 10 / 18
Recommend
More recommend
