Introduction There will be a number of assignments, which collectively will account for 30% of the final mark. These assignments will make extensive use of the computer. The final examination will be worth 70%. It will be open-book (but with limited time), because I do not know how to administer conventional examinations remotely. James G. MacKinnon Economics 850 September, 2020 5 / 26
Introduction There will be a number of assignments, which collectively will account for 30% of the final mark. These assignments will make extensive use of the computer. The final examination will be worth 70%. It will be open-book (but with limited time), because I do not know how to administer conventional examinations remotely. Just what the final examination will look like and how it will be administered is not yet known. James G. MacKinnon Economics 850 September, 2020 5 / 26
Introduction There will be a number of assignments, which collectively will account for 30% of the final mark. These assignments will make extensive use of the computer. The final examination will be worth 70%. It will be open-book (but with limited time), because I do not know how to administer conventional examinations remotely. Just what the final examination will look like and how it will be administered is not yet known. In the past, the assignments have been worth 20%, the midterm 20%, and the final exam 60%. James G. MacKinnon Economics 850 September, 2020 5 / 26
Introduction There will be a number of assignments, which collectively will account for 30% of the final mark. These assignments will make extensive use of the computer. The final examination will be worth 70%. It will be open-book (but with limited time), because I do not know how to administer conventional examinations remotely. Just what the final examination will look like and how it will be administered is not yet known. In the past, the assignments have been worth 20%, the midterm 20%, and the final exam 60%. The T.A. is Mehtab Hanzroh. James G. MacKinnon Economics 850 September, 2020 5 / 26
Introduction There will be a number of assignments, which collectively will account for 30% of the final mark. These assignments will make extensive use of the computer. The final examination will be worth 70%. It will be open-book (but with limited time), because I do not know how to administer conventional examinations remotely. Just what the final examination will look like and how it will be administered is not yet known. In the past, the assignments have been worth 20%, the midterm 20%, and the final exam 60%. The T.A. is Mehtab Hanzroh. Tutorials? Virtual office hours? James G. MacKinnon Economics 850 September, 2020 5 / 26
Introduction There will be a number of assignments, which collectively will account for 30% of the final mark. These assignments will make extensive use of the computer. The final examination will be worth 70%. It will be open-book (but with limited time), because I do not know how to administer conventional examinations remotely. Just what the final examination will look like and how it will be administered is not yet known. In the past, the assignments have been worth 20%, the midterm 20%, and the final exam 60%. The T.A. is Mehtab Hanzroh. Tutorials? Virtual office hours? http://qed.econ.queensu.ca/pub/faculty/mackinnon/econ850/ James G. MacKinnon Economics 850 September, 2020 5 / 26
Some Properties of PDFs and CDFs Some Properties of PDFs and CDFs Random variables may be discrete (binary, counts) or continuous. A continuous random variable x can take on real values. The realized value of x is often denoted X . James G. MacKinnon Economics 850 September, 2020 6 / 26
Some Properties of PDFs and CDFs Some Properties of PDFs and CDFs Random variables may be discrete (binary, counts) or continuous. A continuous random variable x can take on real values. The realized value of x is often denoted X . The distribution of x is described by a cumulative distribution function, or CDF: F ( x ) = Pr ( X ≤ x ) . James G. MacKinnon Economics 850 September, 2020 6 / 26
Some Properties of PDFs and CDFs Some Properties of PDFs and CDFs Random variables may be discrete (binary, counts) or continuous. A continuous random variable x can take on real values. The realized value of x is often denoted X . The distribution of x is described by a cumulative distribution function, or CDF: F ( x ) = Pr ( X ≤ x ) . 0 ≤ F ( x ) ≤ 1. James G. MacKinnon Economics 850 September, 2020 6 / 26
Some Properties of PDFs and CDFs Some Properties of PDFs and CDFs Random variables may be discrete (binary, counts) or continuous. A continuous random variable x can take on real values. The realized value of x is often denoted X . The distribution of x is described by a cumulative distribution function, or CDF: F ( x ) = Pr ( X ≤ x ) . 0 ≤ F ( x ) ≤ 1. F ( x ) tends to 0 as x → − ∞ . F ( x ) tends to 1 as x → + ∞ . James G. MacKinnon Economics 850 September, 2020 6 / 26
Some Properties of PDFs and CDFs Some Properties of PDFs and CDFs Random variables may be discrete (binary, counts) or continuous. A continuous random variable x can take on real values. The realized value of x is often denoted X . The distribution of x is described by a cumulative distribution function, or CDF: F ( x ) = Pr ( X ≤ x ) . 0 ≤ F ( x ) ≤ 1. F ( x ) tends to 0 as x → − ∞ . F ( x ) tends to 1 as x → + ∞ . F ( x ) must be a weakly increasing function of x . James G. MacKinnon Economics 850 September, 2020 6 / 26
Some Properties of PDFs and CDFs The probability that x = X is always zero. James G. MacKinnon Economics 850 September, 2020 7 / 26
Some Properties of PDFs and CDFs The probability that x = X is always zero. If a < b , then Pr ( X ≤ b ) = Pr ( X ≤ a ) + Pr ( a < X ≤ b ) . James G. MacKinnon Economics 850 September, 2020 7 / 26
Some Properties of PDFs and CDFs The probability that x = X is always zero. If a < b , then Pr ( X ≤ b ) = Pr ( X ≤ a ) + Pr ( a < X ≤ b ) . Therefore, Pr ( a ≤ X ≤ b ) = F ( b ) − F ( a ) . James G. MacKinnon Economics 850 September, 2020 7 / 26
Some Properties of PDFs and CDFs The probability that x = X is always zero. If a < b , then Pr ( X ≤ b ) = Pr ( X ≤ a ) + Pr ( a < X ≤ b ) . Therefore, Pr ( a ≤ X ≤ b ) = F ( b ) − F ( a ) . If b = a , then we get F ( a ) − F ( a ) = 0. James G. MacKinnon Economics 850 September, 2020 7 / 26
Some Properties of PDFs and CDFs The probability that x = X is always zero. If a < b , then Pr ( X ≤ b ) = Pr ( X ≤ a ) + Pr ( a < X ≤ b ) . Therefore, Pr ( a ≤ X ≤ b ) = F ( b ) − F ( a ) . If b = a , then we get F ( a ) − F ( a ) = 0. The probability density function, or PDF, is just the derivative of the CDF: f ( x ) ≡ F ′ ( x ) . Evidently, � ∞ � ∞ − ∞ F ′ ( x ) d x = F ( ∞ ) − F ( − ∞ ) = 1. − ∞ f ( x ) d x = (1) James G. MacKinnon Economics 850 September, 2020 7 / 26
Some Properties of PDFs and CDFs The probability that x = X is always zero. If a < b , then Pr ( X ≤ b ) = Pr ( X ≤ a ) + Pr ( a < X ≤ b ) . Therefore, Pr ( a ≤ X ≤ b ) = F ( b ) − F ( a ) . If b = a , then we get F ( a ) − F ( a ) = 0. The probability density function, or PDF, is just the derivative of the CDF: f ( x ) ≡ F ′ ( x ) . Evidently, � ∞ � ∞ − ∞ F ′ ( x ) d x = F ( ∞ ) − F ( − ∞ ) = 1. − ∞ f ( x ) d x = (1) More generally, � b a f ( x ) d x = Pr ( a ≤ X ≤ b ) = F ( b ) − F ( a ) . (2) James G. MacKinnon Economics 850 September, 2020 7 / 26
Some Properties of PDFs and CDFs It is evident that f ( x ) ≥ 0, because F ( x ) is non-decreasing. James G. MacKinnon Economics 850 September, 2020 8 / 26
Some Properties of PDFs and CDFs It is evident that f ( x ) ≥ 0, because F ( x ) is non-decreasing. f ( x ) is not bounded above by unity, because the value of a PDF at a point x is not a probability. James G. MacKinnon Economics 850 September, 2020 8 / 26
Some Properties of PDFs and CDFs It is evident that f ( x ) ≥ 0, because F ( x ) is non-decreasing. f ( x ) is not bounded above by unity, because the value of a PDF at a point x is not a probability. The PDF of the standard normal distribution is � − 1 φ ( x ) = ( 2 π ) − 1/2 exp 2 x 2 � . (3) James G. MacKinnon Economics 850 September, 2020 8 / 26
Some Properties of PDFs and CDFs It is evident that f ( x ) ≥ 0, because F ( x ) is non-decreasing. f ( x ) is not bounded above by unity, because the value of a PDF at a point x is not a probability. The PDF of the standard normal distribution is � − 1 φ ( x ) = ( 2 π ) − 1/2 exp 2 x 2 � . (3) The CDF of the standard normal distribution is � x Φ ( x ) = − ∞ φ ( y ) d y . (4) This has no closed-form solution. James G. MacKinnon Economics 850 September, 2020 8 / 26
Some Properties of PDFs and CDFs It is evident that f ( x ) ≥ 0, because F ( x ) is non-decreasing. f ( x ) is not bounded above by unity, because the value of a PDF at a point x is not a probability. The PDF of the standard normal distribution is � − 1 φ ( x ) = ( 2 π ) − 1/2 exp 2 x 2 � . (3) The CDF of the standard normal distribution is � x Φ ( x ) = − ∞ φ ( y ) d y . (4) This has no closed-form solution. The maximum of the PDF φ ( x ) is at x = 0, where the slope of the CDF Φ ( x ) is steepest. James G. MacKinnon Economics 850 September, 2020 8 / 26
Some Properties of PDFs and CDFs Standard Normal CDF: Φ( x ) 1.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . − 3 − 2 − 1 0 1 2 3 Standard Normal PDF: φ ( x ) 0.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . − 3 − 2 − 1 0 1 2 3 James G. MacKinnon Economics 850 September, 2020 9 / 26
Some Properties of PDFs and CDFs For a continuous random variable, � ∞ µ ≡ E ( x ) ≡ − ∞ x f ( x ) d x . (5) Since x can range from − ∞ to ∞ , this integral may well diverge. James G. MacKinnon Economics 850 September, 2020 10 / 26
Some Properties of PDFs and CDFs For a continuous random variable, � ∞ µ ≡ E ( x ) ≡ − ∞ x f ( x ) d x . (5) Since x can range from − ∞ to ∞ , this integral may well diverge. The k th uncentered moment of x is � ∞ − ∞ x k f ( x ) d x . m k ( x ) ≡ (6) James G. MacKinnon Economics 850 September, 2020 10 / 26
Some Properties of PDFs and CDFs For a continuous random variable, � ∞ µ ≡ E ( x ) ≡ − ∞ x f ( x ) d x . (5) Since x can range from − ∞ to ∞ , this integral may well diverge. The k th uncentered moment of x is � ∞ − ∞ x k f ( x ) d x . m k ( x ) ≡ (6) The k th central moment of the distribution of x is � ∞ µ k ≡ E ( x − µ ) k = − ∞ ( x − µ ) k f ( x ) d x . (7) James G. MacKinnon Economics 850 September, 2020 10 / 26
Some Properties of PDFs and CDFs The second central moment is the variance , Var ( x ) = σ 2 . The square root of the variance, σ , is called the standard deviation . James G. MacKinnon Economics 850 September, 2020 11 / 26
Some Properties of PDFs and CDFs The second central moment is the variance , Var ( x ) = σ 2 . The square root of the variance, σ , is called the standard deviation . Estimates of standard deviations of parameter estimates are called standard errors . James G. MacKinnon Economics 850 September, 2020 11 / 26
Some Properties of PDFs and CDFs The second central moment is the variance , Var ( x ) = σ 2 . The square root of the variance, σ , is called the standard deviation . Estimates of standard deviations of parameter estimates are called standard errors . If ¯ x is the sample mean of x i , i = 1, . . . , N , then the sample standard deviation is 1/2 � � N 1 ∑ x ) 2 s.d. ( x ) = ( x i − ¯ . (8) N − 1 i = 1 James G. MacKinnon Economics 850 September, 2020 11 / 26
Some Properties of PDFs and CDFs The second central moment is the variance , Var ( x ) = σ 2 . The square root of the variance, σ , is called the standard deviation . Estimates of standard deviations of parameter estimates are called standard errors . If ¯ x is the sample mean of x i , i = 1, . . . , N , then the sample standard deviation is 1/2 � � N 1 ∑ x ) 2 s.d. ( x ) = ( x i − ¯ . (8) N − 1 i = 1 Under the assumption that the x i are uncorrelated, 1 s.e. ( ¯ x ) = √ s.d. ( x ) . (9) N James G. MacKinnon Economics 850 September, 2020 11 / 26
Joint Distributions Joint Distributions A continuous, bivariate random variable ( x 1 , x 2 ) has the distribution function � ( X 1 ≤ x 1 ) ∩ ( X 2 ≤ x 2 ) � F ( x 1 , x 2 ) = Pr . (10) Thus the joint CDF F ( x 1 , x 2 ) is the joint probability that both X 1 ≤ x 1 and X 2 ≤ x 2 . James G. MacKinnon Economics 850 September, 2020 12 / 26
Joint Distributions Joint Distributions A continuous, bivariate random variable ( x 1 , x 2 ) has the distribution function � ( X 1 ≤ x 1 ) ∩ ( X 2 ≤ x 2 ) � F ( x 1 , x 2 ) = Pr . (10) Thus the joint CDF F ( x 1 , x 2 ) is the joint probability that both X 1 ≤ x 1 and X 2 ≤ x 2 . The joint density function is f ( x 1 , x 2 ) = ∂ 2 F ( x 1 , x 2 ) . (11) ∂ x 1 ∂ x 2 James G. MacKinnon Economics 850 September, 2020 12 / 26
Joint Distributions Joint Distributions A continuous, bivariate random variable ( x 1 , x 2 ) has the distribution function � ( X 1 ≤ x 1 ) ∩ ( X 2 ≤ x 2 ) � F ( x 1 , x 2 ) = Pr . (10) Thus the joint CDF F ( x 1 , x 2 ) is the joint probability that both X 1 ≤ x 1 and X 2 ≤ x 2 . The joint density function is f ( x 1 , x 2 ) = ∂ 2 F ( x 1 , x 2 ) . (11) ∂ x 1 ∂ x 2 Like all densities, this joint PDF integrates to one: � ∞ � ∞ − ∞ f ( x 1 , x 2 ) d x 1 d x 2 = 1. (12) − ∞ James G. MacKinnon Economics 850 September, 2020 12 / 26
Joint Distributions The joint CDF is related to the joint PDF by � x 2 � x 1 F ( x 1 , x 2 ) = − ∞ f ( y 1 , y 2 ) d y 1 d y 2 . (13) − ∞ James G. MacKinnon Economics 850 September, 2020 13 / 26
Joint Distributions The joint CDF is related to the joint PDF by � x 2 � x 1 F ( x 1 , x 2 ) = − ∞ f ( y 1 , y 2 ) d y 1 d y 2 . (13) − ∞ X 1 and X 2 are said to be independent if F ( x 1 , x 2 ) is the product of the marginal CDFs of x 1 and x 2 : F ( x 1 , x 2 ) = F ( x 1 , ∞ ) F ( ∞ , x 2 ) = F ( x 1 ) F ( x 2 ) . (14) James G. MacKinnon Economics 850 September, 2020 13 / 26
Joint Distributions The joint CDF is related to the joint PDF by � x 2 � x 1 F ( x 1 , x 2 ) = − ∞ f ( y 1 , y 2 ) d y 1 d y 2 . (13) − ∞ X 1 and X 2 are said to be independent if F ( x 1 , x 2 ) is the product of the marginal CDFs of x 1 and x 2 : F ( x 1 , x 2 ) = F ( x 1 , ∞ ) F ( ∞ , x 2 ) = F ( x 1 ) F ( x 2 ) . (14) The marginal density of x 1 is � ∞ − ∞ f ( x 1 , x 2 ) d x 2 = ∂ F ( x 1 , ∞ ) f ( x 1 ) = . (15) ∂ x 1 Thus f ( x 1 ) is obtained by integrating x 2 out of the joint density. James G. MacKinnon Economics 850 September, 2020 13 / 26
Joint Distributions If x 1 and x 2 are independent, so that (14) holds, then f ( x 1 , x 2 ) = f ( x 1 ) f ( x 2 ) . (16) James G. MacKinnon Economics 850 September, 2020 14 / 26
Joint Distributions If x 1 and x 2 are independent, so that (14) holds, then f ( x 1 , x 2 ) = f ( x 1 ) f ( x 2 ) . (16) Suppose that A and B are any two events. Then Pr ( A | B ) and is defined implicitly by the equation Pr ( A ∩ B ) = Pr ( B ) Pr ( A | B ) . (17) Evidently, Pr ( B ) � = 0, since we cannot condition on B when Pr ( B ) = 0. James G. MacKinnon Economics 850 September, 2020 14 / 26
Joint Distributions If x 1 and x 2 are independent, so that (14) holds, then f ( x 1 , x 2 ) = f ( x 1 ) f ( x 2 ) . (16) Suppose that A and B are any two events. Then Pr ( A | B ) and is defined implicitly by the equation Pr ( A ∩ B ) = Pr ( B ) Pr ( A | B ) . (17) Evidently, Pr ( B ) � = 0, since we cannot condition on B when Pr ( B ) = 0. Equation (17) underlies all of Bayesian statistics. Pr ( A ∩ B ) = Pr ( A ) Pr ( B | A ) (18) is just (17) with A and B interchanged. James G. MacKinnon Economics 850 September, 2020 14 / 26
Joint Distributions Equations (17) and (18) imply that Pr ( A | B ) = Pr ( B | A ) Pr ( A ) . (19) Pr ( B ) James G. MacKinnon Economics 850 September, 2020 15 / 26
Joint Distributions Equations (17) and (18) imply that Pr ( A | B ) = Pr ( B | A ) Pr ( A ) . (19) Pr ( B ) For Bayesian estimation, the sample y plays the role of B , and the parameter vector θ plays the role of A . Thus we have f ( θ | y ) = f ( y | θ ) f ( θ ) , (20) f ( y ) where the f ( · ) denote densities. This is one version of Bayes’ Rule . James G. MacKinnon Economics 850 September, 2020 15 / 26
Joint Distributions Equations (17) and (18) imply that Pr ( A | B ) = Pr ( B | A ) Pr ( A ) . (19) Pr ( B ) For Bayesian estimation, the sample y plays the role of B , and the parameter vector θ plays the role of A . Thus we have f ( θ | y ) = f ( y | θ ) f ( θ ) , (20) f ( y ) where the f ( · ) denote densities. This is one version of Bayes’ Rule . In words, the posterior density is equal to the likelihood times the prior density , divided by the unconditional density of y . If we ignore the denominator, then posterior ∝ prior × likelihood . James G. MacKinnon Economics 850 September, 2020 15 / 26
Joint Distributions The conditional density , or conditional PDF , of x 1 for a given value x 2 is f ( x 1 | x 2 ) = f ( x 1 , x 2 ) f ( x 2 ) . (21) James G. MacKinnon Economics 850 September, 2020 16 / 26
Joint Distributions The conditional density , or conditional PDF , of x 1 for a given value x 2 is f ( x 1 | x 2 ) = f ( x 1 , x 2 ) f ( x 2 ) . (21) If we let y denote x 1 and x denote x 2 , then the conditional expectation of y given x is E ( y | x ) = h ( x ) , (22) where h ( x ) could be any sort of function. It is a (deterministic) function that gives us E ( y ) for every possible value of x . James G. MacKinnon Economics 850 September, 2020 16 / 26
Joint Distributions The conditional density , or conditional PDF , of x 1 for a given value x 2 is f ( x 1 | x 2 ) = f ( x 1 , x 2 ) f ( x 2 ) . (21) If we let y denote x 1 and x denote x 2 , then the conditional expectation of y given x is E ( y | x ) = h ( x ) , (22) where h ( x ) could be any sort of function. It is a (deterministic) function that gives us E ( y ) for every possible value of x . A very simple example is the regression function E ( y ) = β 1 + β 2 x . (23) James G. MacKinnon Economics 850 September, 2020 16 / 26
Joint Distributions The Law of Iterated Expectations is very useful. It tells us that � = E ( x 1 ) . � E ( x 1 | x 2 ) E (24) In words, the unconditional expectation of x 1 is equal to the expectation (using f ( x 2 ) ) of the conditional expectation. James G. MacKinnon Economics 850 September, 2020 17 / 26
Joint Distributions The Law of Iterated Expectations is very useful. It tells us that � = E ( x 1 ) . � E ( x 1 | x 2 ) E (24) In words, the unconditional expectation of x 1 is equal to the expectation (using f ( x 2 ) ) of the conditional expectation. Any deterministic function of a conditioning variable x 2 is its own conditional expectation. Thus E ( x 2 2 | x 2 ) = x 2 E ( x 2 | x 2 ) = x 2 and 2 . (25) James G. MacKinnon Economics 850 September, 2020 17 / 26
Joint Distributions The Law of Iterated Expectations is very useful. It tells us that � = E ( x 1 ) . � E ( x 1 | x 2 ) E (24) In words, the unconditional expectation of x 1 is equal to the expectation (using f ( x 2 ) ) of the conditional expectation. Any deterministic function of a conditioning variable x 2 is its own conditional expectation. Thus E ( x 2 2 | x 2 ) = x 2 E ( x 2 | x 2 ) = x 2 and 2 . (25) Similarly, � = h ( x 2 ) E ( x 1 | x 2 ) � x 1 h ( x 2 ) | x 2 E (26) for any deterministic function h ( · ) . James G. MacKinnon Economics 850 September, 2020 17 / 26
Joint Distributions An important special case arises when E ( x 1 | x 2 ) = 0. In that case, for any function h ( · ) , E ( x 1 h ( x 2 )) = 0, because � = E � � � x 1 h ( x 2 ) E ( x 1 h ( x 2 ) | x 2 ) E � � = E h ( x 2 ) E ( x 1 | x 2 ) (27) � = 0. = E � ( h ( x 2 ) 0 James G. MacKinnon Economics 850 September, 2020 18 / 26
Joint Distributions An important special case arises when E ( x 1 | x 2 ) = 0. In that case, for any function h ( · ) , E ( x 1 h ( x 2 )) = 0, because � = E � � � x 1 h ( x 2 ) E ( x 1 h ( x 2 ) | x 2 ) E � � = E h ( x 2 ) E ( x 1 | x 2 ) (27) � = 0. = E � ( h ( x 2 ) 0 The first two equalities follow from (24) and (26). Since E ( x 1 | x 2 ) = 0, the third equality then follows immediately. James G. MacKinnon Economics 850 September, 2020 18 / 26
The Specification of Regression Models The Specification of Regression Models Because E ( u i | x i ) = 0, E ( y i | x i ) = β 1 + β 2 x i + E ( u i | x i ) = β 1 + β 2 x i . (28) James G. MacKinnon Economics 850 September, 2020 19 / 26
The Specification of Regression Models The Specification of Regression Models Because E ( u i | x i ) = 0, E ( y i | x i ) = β 1 + β 2 x i + E ( u i | x i ) = β 1 + β 2 x i . (28) Suppose that we estimate the model (28) when in fact y i = β 1 + β 2 x i + β 3 x 2 i + v i . (29) James G. MacKinnon Economics 850 September, 2020 19 / 26
The Specification of Regression Models The Specification of Regression Models Because E ( u i | x i ) = 0, E ( y i | x i ) = β 1 + β 2 x i + E ( u i | x i ) = β 1 + β 2 x i . (28) Suppose that we estimate the model (28) when in fact y i = β 1 + β 2 x i + β 3 x 2 i + v i . (29) Then � = β 3 x 2 β 3 x 2 � E ( u i | x i ) = E i + v i | x i i , (30) which must be nonzero unless x i = 0. James G. MacKinnon Economics 850 September, 2020 19 / 26
The Specification of Regression Models The Specification of Regression Models Because E ( u i | x i ) = 0, E ( y i | x i ) = β 1 + β 2 x i + E ( u i | x i ) = β 1 + β 2 x i . (28) Suppose that we estimate the model (28) when in fact y i = β 1 + β 2 x i + β 3 x 2 i + v i . (29) Then � = β 3 x 2 β 3 x 2 � E ( u i | x i ) = E i + v i | x i i , (30) which must be nonzero unless x i = 0. Should the sample be described as t = 1, . . . , T or i = 1, . . . , N ? James G. MacKinnon Economics 850 September, 2020 19 / 26
The Specification of Regression Models The Specification of Regression Models Because E ( u i | x i ) = 0, E ( y i | x i ) = β 1 + β 2 x i + E ( u i | x i ) = β 1 + β 2 x i . (28) Suppose that we estimate the model (28) when in fact y i = β 1 + β 2 x i + β 3 x 2 i + v i . (29) Then � = β 3 x 2 β 3 x 2 � E ( u i | x i ) = E i + v i | x i i , (30) which must be nonzero unless x i = 0. Should the sample be described as t = 1, . . . , T or i = 1, . . . , N ? ETM mostly uses t = 1, . . . , n , but my slides will use i = 1, . . . , N except for time series. James G. MacKinnon Economics 850 September, 2020 19 / 26
The Specification of Regression Models Information set: The set of potential explanatory variables, denoted Ω i . It is what we condition on. Instead of (28), E ( y i | Ω i ) = β 1 + β 2 x i . (31) James G. MacKinnon Economics 850 September, 2020 20 / 26
The Specification of Regression Models Information set: The set of potential explanatory variables, denoted Ω i . It is what we condition on. Instead of (28), E ( y i | Ω i ) = β 1 + β 2 x i . (31) Exogenous and endogenous variables. James G. MacKinnon Economics 850 September, 2020 20 / 26
The Specification of Regression Models Information set: The set of potential explanatory variables, denoted Ω i . It is what we condition on. Instead of (28), E ( y i | Ω i ) = β 1 + β 2 x i . (31) Exogenous and endogenous variables. Disturbances rather than error terms . James G. MacKinnon Economics 850 September, 2020 20 / 26
The Specification of Regression Models Information set: The set of potential explanatory variables, denoted Ω i . It is what we condition on. Instead of (28), E ( y i | Ω i ) = β 1 + β 2 x i . (31) Exogenous and endogenous variables. Disturbances rather than error terms . These are often assumed to be independent and identically distributed , or IID . James G. MacKinnon Economics 850 September, 2020 20 / 26
The Specification of Regression Models Information set: The set of potential explanatory variables, denoted Ω i . It is what we condition on. Instead of (28), E ( y i | Ω i ) = β 1 + β 2 x i . (31) Exogenous and endogenous variables. Disturbances rather than error terms . These are often assumed to be independent and identically distributed , or IID . Serial correlation can arise when observations are ordered by time. Then E ( u t | u s ) � = 0, perhaps only when | t − s | is small. James G. MacKinnon Economics 850 September, 2020 20 / 26
The Specification of Regression Models Information set: The set of potential explanatory variables, denoted Ω i . It is what we condition on. Instead of (28), E ( y i | Ω i ) = β 1 + β 2 x i . (31) Exogenous and endogenous variables. Disturbances rather than error terms . These are often assumed to be independent and identically distributed , or IID . Serial correlation can arise when observations are ordered by time. Then E ( u t | u s ) � = 0, perhaps only when | t − s | is small. Heteroskedasticity means that Var ( u i ) is not constant. It may depend on X i , or it may depend on lagged values of Var ( u i ) . James G. MacKinnon Economics 850 September, 2020 20 / 26
The Specification of Regression Models Information set: The set of potential explanatory variables, denoted Ω i . It is what we condition on. Instead of (28), E ( y i | Ω i ) = β 1 + β 2 x i . (31) Exogenous and endogenous variables. Disturbances rather than error terms . These are often assumed to be independent and identically distributed , or IID . Serial correlation can arise when observations are ordered by time. Then E ( u t | u s ) � = 0, perhaps only when | t − s | is small. Heteroskedasticity means that Var ( u i ) is not constant. It may depend on X i , or it may depend on lagged values of Var ( u i ) . Clustering implies that Cov ( u gi u gj ) � = 0. Here the sample is divided into clusters indexed by g . James G. MacKinnon Economics 850 September, 2020 20 / 26
The Specification of Regression Models Equation (31), by itself, is not a complete specification . If a model is completely specified, we can simulate it. For the regression model (28): James G. MacKinnon Economics 850 September, 2020 21 / 26
The Specification of Regression Models Equation (31), by itself, is not a complete specification . If a model is completely specified, we can simulate it. For the regression model (28): Fix the sample size, N . James G. MacKinnon Economics 850 September, 2020 21 / 26
The Specification of Regression Models Equation (31), by itself, is not a complete specification . If a model is completely specified, we can simulate it. For the regression model (28): Fix the sample size, N . Choose β 1 and β 2 , the parameters of the deterministic specification . James G. MacKinnon Economics 850 September, 2020 21 / 26
The Specification of Regression Models Equation (31), by itself, is not a complete specification . If a model is completely specified, we can simulate it. For the regression model (28): Fix the sample size, N . Choose β 1 and β 2 , the parameters of the deterministic specification . Obtain the N values x i , i = 1, . . . , N , of the explanatory variable. James G. MacKinnon Economics 850 September, 2020 21 / 26
The Specification of Regression Models Equation (31), by itself, is not a complete specification . If a model is completely specified, we can simulate it. For the regression model (28): Fix the sample size, N . Choose β 1 and β 2 , the parameters of the deterministic specification . Obtain the N values x i , i = 1, . . . , N , of the explanatory variable. Evaluate β 1 + β 2 x i for i = 1, . . . , N . James G. MacKinnon Economics 850 September, 2020 21 / 26
The Specification of Regression Models Equation (31), by itself, is not a complete specification . If a model is completely specified, we can simulate it. For the regression model (28): Fix the sample size, N . Choose β 1 and β 2 , the parameters of the deterministic specification . Obtain the N values x i , i = 1, . . . , N , of the explanatory variable. Evaluate β 1 + β 2 x i for i = 1, . . . , N . Choose the distribution of the disturbances, if necessary specifying parameters such as mean and variance. James G. MacKinnon Economics 850 September, 2020 21 / 26
The Specification of Regression Models Equation (31), by itself, is not a complete specification . If a model is completely specified, we can simulate it. For the regression model (28): Fix the sample size, N . Choose β 1 and β 2 , the parameters of the deterministic specification . Obtain the N values x i , i = 1, . . . , N , of the explanatory variable. Evaluate β 1 + β 2 x i for i = 1, . . . , N . Choose the distribution of the disturbances, if necessary specifying parameters such as mean and variance. Use a random-number generator , or RNG , to generate values of u i . James G. MacKinnon Economics 850 September, 2020 21 / 26
The Specification of Regression Models Equation (31), by itself, is not a complete specification . If a model is completely specified, we can simulate it. For the regression model (28): Fix the sample size, N . Choose β 1 and β 2 , the parameters of the deterministic specification . Obtain the N values x i , i = 1, . . . , N , of the explanatory variable. Evaluate β 1 + β 2 x i for i = 1, . . . , N . Choose the distribution of the disturbances, if necessary specifying parameters such as mean and variance. Use a random-number generator , or RNG , to generate values of u i . Form the simulated values y i by adding the disturbances to the values of the regression function. James G. MacKinnon Economics 850 September, 2020 21 / 26
The Specification of Regression Models Equation (31), by itself, is not a complete specification . If a model is completely specified, we can simulate it. For the regression model (28): Fix the sample size, N . Choose β 1 and β 2 , the parameters of the deterministic specification . Obtain the N values x i , i = 1, . . . , N , of the explanatory variable. Evaluate β 1 + β 2 x i for i = 1, . . . , N . Choose the distribution of the disturbances, if necessary specifying parameters such as mean and variance. Use a random-number generator , or RNG , to generate values of u i . Form the simulated values y i by adding the disturbances to the values of the regression function. For a dynamic model like y t = β 1 + β 2 y t − 1 + u t , the data need to be generated recursively. James G. MacKinnon Economics 850 September, 2020 21 / 26
The Specification of Regression Models Alternative models for the mean of y i conditional on x i : y i = β 1 + β 2 x i + β 3 x 2 i + u i (32) y i = γ 1 + γ 2 log x i + u i (33) 1 y i = δ 1 + δ 2 + u i . (34) x i James G. MacKinnon Economics 850 September, 2020 22 / 26
The Specification of Regression Models Alternative models for the mean of y i conditional on x i : y i = β 1 + β 2 x i + β 3 x 2 i + u i (32) y i = γ 1 + γ 2 log x i + u i (33) 1 y i = δ 1 + δ 2 + u i . (34) x i These are all linear models. A nonlinear (but rarely sensible) model is y i = e β 1 x β 2 i 2 x β 3 i 3 + u i . (35) James G. MacKinnon Economics 850 September, 2020 22 / 26
The Specification of Regression Models Alternative models for the mean of y i conditional on x i : y i = β 1 + β 2 x i + β 3 x 2 i + u i (32) y i = γ 1 + γ 2 log x i + u i (33) 1 y i = δ 1 + δ 2 + u i . (34) x i These are all linear models. A nonlinear (but rarely sensible) model is y i = e β 1 x β 2 i 2 x β 3 i 3 + u i . (35) A better model is y i = e β 1 x β 2 i 2 x β 3 i 3 e v i . (36) James G. MacKinnon Economics 850 September, 2020 22 / 26
The Specification of Regression Models Alternative models for the mean of y i conditional on x i : y i = β 1 + β 2 x i + β 3 x 2 i + u i (32) y i = γ 1 + γ 2 log x i + u i (33) 1 y i = δ 1 + δ 2 + u i . (34) x i These are all linear models. A nonlinear (but rarely sensible) model is y i = e β 1 x β 2 i 2 x β 3 i 3 + u i . (35) A better model is y i = e β 1 x β 2 i 2 x β 3 i 3 e v i . (36) If we take logarithms of both sides, we get log y i = β 1 + β 2 log x i 2 + β 3 log x i 3 + v i , (37) which is a loglinear regression model . James G. MacKinnon Economics 850 September, 2020 22 / 26
Method-of-Moments Estimation Method-of-Moments Estimation The method of moments , or MM , replaces population quantities by sample analogs. James G. MacKinnon Economics 850 September, 2020 23 / 26
Recommend
More recommend
