Lecture 8 AR, MA, and ARMA Models 9/27/2018 1 AR models AR(p) - PowerPoint PPT Presentation

Lecture 8 AR, MA, and ARMA Models 9/27/2018 1

AR models

AR(p) models We can generalize from an AR(1) to an AR(p) model by simply adding additional autoregressive terms to the model. 𝐵𝑆(𝑞) ∶ 𝑞 ∑ 𝑗=1 What are the properities of 𝐵𝑆(𝑞) , 1. Expected value? 2. Autocovariance / autocorrelation? 3. Stationarity conditions? 2 𝑧 𝑢 = 𝜀 + 𝜚 1 𝑧 𝑢−1 + 𝜚 2 𝑧 𝑢−2 + ⋯ + 𝜚 𝑞 𝑧 𝑢−𝑞 + 𝑥 𝑢 = 𝜀 + 𝑥 𝑢 + 𝜚 𝑗 𝑧 𝑢−𝑗

AR(p) models We can generalize from an AR(1) to an AR(p) model by simply adding additional autoregressive terms to the model. 𝐵𝑆(𝑞) ∶ 𝑞 ∑ 𝑗=1 What are the properities of 𝐵𝑆(𝑞) , 1. Expected value? 2. Autocovariance / autocorrelation? 3. Stationarity conditions? 2 𝑧 𝑢 = 𝜀 + 𝜚 1 𝑧 𝑢−1 + 𝜚 2 𝑧 𝑢−2 + ⋯ + 𝜚 𝑞 𝑧 𝑢−𝑞 + 𝑥 𝑢 = 𝜀 + 𝑥 𝑢 + 𝜚 𝑗 𝑧 𝑢−𝑗

𝑀 2 𝑧 𝑢 = 𝑀 (𝑀 𝑧 𝑢 ) 𝑀 𝑙 𝑧 𝑢 = 𝑧 𝑢−𝑙 Lag operator The lag operator is convenience notation for writing out AR (and other) time series models. We define the lag operator 𝑀 as follows, this can be generalized where, = 𝑀 𝑧 𝑢−1 = 𝑧 𝑢−2 therefore, 3 𝑀 𝑧 𝑢 = 𝑧 𝑢−1

Lag operator The lag operator is convenience notation for writing out AR (and other) time series models. We define the lag operator 𝑀 as follows, this can be generalized where, = 𝑀 𝑧 𝑢−1 = 𝑧 𝑢−2 therefore, 3 𝑀 𝑧 𝑢 = 𝑧 𝑢−1 𝑀 2 𝑧 𝑢 = 𝑀 (𝑀 𝑧 𝑢 ) 𝑀 𝑙 𝑧 𝑢 = 𝑧 𝑢−𝑙

𝑧 𝑢 − 𝜚 1 𝑀 𝑧 𝑢 − 𝜚 2 𝑀 2 𝑧 𝑢 − … − 𝜚 𝑞 𝑀 𝑞 𝑧 𝑢 = 𝜀 + 𝑥 𝑢 (1 − 𝜚 1 𝑀 − 𝜚 2 𝑀 2 − … − 𝜚 𝑞 𝑀 𝑞 ) 𝑧 𝑢 = 𝜀 + 𝑥 𝑢 𝜚 𝑞 (𝑀) = (1 − 𝜚 1 𝑀 − 𝜚 2 𝑀 2 − ⋯ − 𝜚 𝑞 𝑀 𝑞 ) Lag polynomial Lets rewrite the 𝐵𝑆(𝑞) model using the lag operator, This polynomial of lags is called the characteristic polynomial of the AR process. 4 𝑧 𝑢 = 𝜀 + 𝜚 1 𝑧 𝑢−1 + 𝜚 2 𝑧 𝑢−2 + … + 𝜚 𝑞 𝑧 𝑢−𝑞 + 𝑥 𝑢 𝑧 𝑢 = 𝜀 + 𝜚 1 𝑀 𝑧 𝑢 + 𝜚 2 𝑀 2 𝑧 𝑢 + … + 𝜚 𝑞 𝑀 𝑞 𝑧 𝑢 + 𝑥 𝑢

𝜚 𝑞 (𝑀) = (1 − 𝜚 1 𝑀 − 𝜚 2 𝑀 2 − ⋯ − 𝜚 𝑞 𝑀 𝑞 ) Lag polynomial Lets rewrite the 𝐵𝑆(𝑞) model using the lag operator, This polynomial of lags is called the characteristic polynomial of the AR process. 4 𝑧 𝑢 = 𝜀 + 𝜚 1 𝑧 𝑢−1 + 𝜚 2 𝑧 𝑢−2 + … + 𝜚 𝑞 𝑧 𝑢−𝑞 + 𝑥 𝑢 𝑧 𝑢 = 𝜀 + 𝜚 1 𝑀 𝑧 𝑢 + 𝜚 2 𝑀 2 𝑧 𝑢 + … + 𝜚 𝑞 𝑀 𝑞 𝑧 𝑢 + 𝑥 𝑢 𝑧 𝑢 − 𝜚 1 𝑀 𝑧 𝑢 − 𝜚 2 𝑀 2 𝑧 𝑢 − … − 𝜚 𝑞 𝑀 𝑞 𝑧 𝑢 = 𝜀 + 𝑥 𝑢 (1 − 𝜚 1 𝑀 − 𝜚 2 𝑀 2 − … − 𝜚 𝑞 𝑀 𝑞 ) 𝑧 𝑢 = 𝜀 + 𝑥 𝑢

Lag polynomial Lets rewrite the 𝐵𝑆(𝑞) model using the lag operator, This polynomial of lags is called the characteristic polynomial of the AR process. 4 𝑧 𝑢 = 𝜀 + 𝜚 1 𝑧 𝑢−1 + 𝜚 2 𝑧 𝑢−2 + … + 𝜚 𝑞 𝑧 𝑢−𝑞 + 𝑥 𝑢 𝑧 𝑢 = 𝜀 + 𝜚 1 𝑀 𝑧 𝑢 + 𝜚 2 𝑀 2 𝑧 𝑢 + … + 𝜚 𝑞 𝑀 𝑞 𝑧 𝑢 + 𝑥 𝑢 𝑧 𝑢 − 𝜚 1 𝑀 𝑧 𝑢 − 𝜚 2 𝑀 2 𝑧 𝑢 − … − 𝜚 𝑞 𝑀 𝑞 𝑧 𝑢 = 𝜀 + 𝑥 𝑢 (1 − 𝜚 1 𝑀 − 𝜚 2 𝑀 2 − … − 𝜚 𝑞 𝑀 𝑞 ) 𝑧 𝑢 = 𝜀 + 𝑥 𝑢 𝜚 𝑞 (𝑀) = (1 − 𝜚 1 𝑀 − 𝜚 2 𝑀 2 − ⋯ − 𝜚 𝑞 𝑀 𝑞 )

(𝜇 𝑞 − 𝜚 1 𝜇 𝑞−1 − 𝜚 2 𝜇 𝑞−2 − ⋯ − 𝜚 𝑞−1 𝜇 − 𝜚 𝑞 ) Stationarity of 𝐵𝑆(𝑞) processes Claim : An 𝐵𝑆(𝑞) process is stationary if the roots of the characteristic polynomial lay outside the complex unit circle If we define 𝜇 = 1/𝑀 then we can rewrite the characteristic polynomial as then as a corollary of our claim the 𝐵𝑆(𝑞) process is stationary if the roots of this new polynomial are inside the complex unit circle (i.e. |𝜇| < 1 ). 5

Stationarity of 𝐵𝑆(𝑞) processes Claim : An 𝐵𝑆(𝑞) process is stationary if the roots of the characteristic polynomial lay outside the complex unit circle If we define 𝜇 = 1/𝑀 then we can rewrite the characteristic polynomial as then as a corollary of our claim the 𝐵𝑆(𝑞) process is stationary if the roots of this new polynomial are inside the complex unit circle (i.e. |𝜇| < 1 ). 5 (𝜇 𝑞 − 𝜚 1 𝜇 𝑞−1 − 𝜚 2 𝜇 𝑞−2 − ⋯ − 𝜚 𝑞−1 𝜇 − 𝜚 𝑞 )

Example AR(1) 6

Example AR(2) 7

AR(2) Stationarity Conditions From Shumway&Stofer4thed. 8

Proof Sketch ⎢ ⎦ ⎥ ⎥ ⎥ ⎤ 𝑧 𝑢−𝑞 ⋮ 𝑧 𝑢−3 𝑧 𝑢−2 𝑧 𝑢−1 ⎣ ⎢ ⎢ ⎡ ⎡ ⎦ ⎥ ⎥ ⎥ ⎤ 0 1 ⋯ 0 0 0 ⋮ + ⎢ We can rewrite the 𝐵𝑆(𝑞) model into an 𝐵𝑆(1) form using matrix ⎢ ⎦ ⎥ ⎥ ⎥ ⎤ 𝑧 𝑢−𝑞+1 ⋮ 𝑧 𝑢−2 𝑧 𝑢−1 𝑞 ⎣ ⎢ ⎢ ⎡ ⎢ = ⎦ ⎥ ⎥ ⎥ ⎤ 0 ⋮ 0 0 𝑥 𝑢 ⎣ ⎢ ⋮ ⋯ ⋮ ⎥ ⋮ 0 0 𝜀 ⎣ ⎢ ⎢ ⎢ ⎡ = ⎦ ⎥ ⎥ ⎤ ⎤ 𝑧 𝑢−𝑞+1 ⋮ 𝑧 𝑢−2 𝑧 𝑢−1 𝑧 𝑢 ⎣ ⎢ ⎢ ⎢ ⎡ where notation ⋮ 0 ⎥ 1 ⋮ 0 0 ⋯ 0 1 0 0 0 ⋯ 0 ⎥ 0 𝜚 𝑞 𝜚 𝑞−1 ⎥ ⎦ + ⎡ ⎢ ⎢ ⎢ ⎣ 𝜚 1 𝜚 2 𝜚 3 ⋯ 9 𝑧 𝑢 = 𝜀 + 𝜚 1 𝑧 𝑢−1 + 𝜚 2 𝑧 𝑢−2 + ⋯ + 𝜚 𝑞 𝑧 𝑢−𝑞 + 𝑥 𝑢 𝝄 𝑢 = 𝜺 + 𝐆 𝝄 𝑢−1 + 𝐱 𝑢 𝜀 + 𝑥 𝑢 + ∑ 𝑗=1 𝜚 𝑗 𝑧 𝑢−𝑗

Proof sketch (cont.) ∑ and therefore we need lim 𝑗=0 ∑ 𝑢 So just like the original 𝐵𝑆(1) we can expand out the autoregressive 𝑗=0 𝑢 = ( equation 10 𝝄 𝑢 = 𝜺 + 𝐱 𝑢 + 𝐆 𝝄 𝑢−1 = 𝜺 + 𝐱 𝑢 + 𝐆 (𝜺 + 𝐱 𝑢−1 ) + 𝐆 2 (𝜺 + 𝐱 𝑢−2 ) + ⋯ + 𝐆 𝑢−1 (𝜺 + 𝐱 1 ) + 𝐆 𝑢 (𝜺 + 𝐱 0 ) 𝐺 𝑗 )𝜺 + 𝐺 𝑗 𝑥 𝑢−𝑗 𝑢→∞ 𝐺 𝑢 → 0 .

𝐺 𝑗 )𝜺 + 𝐺 𝑗 𝑥 𝑢−𝑗 𝝄 𝑢 = ( 𝐑𝚳 𝑗 𝐑 −1 𝑥 𝑢−𝑗 Proof sketch (cont.) 𝑗=0 ∑ 𝑢 𝐑𝚳 𝑗 𝐑 −1 )𝜺 + 𝑗=0 ∑ 𝑢 = ( ∑ 𝑗=0 𝑢 𝑗=0 ∑ 𝑢 Using this property we can rewrite our equation from the previous slide as A useful property of the eigen decomposition is that corresponding eigenvalues. columns of 𝐑 are the eigenvectors of 𝐆 and 𝚳 is a diagonal matrix of the 11 We can find the eigen decomposition such that 𝐆 = 𝐑𝚳𝐑 −1 where the 𝐆 𝑗 = 𝐑𝚳 𝑗 𝐑 −1

Proof sketch (cont.) 𝑢 𝑗=0 ∑ 𝑢 𝐑𝚳 𝑗 𝐑 −1 )𝜺 + 𝑗=0 ∑ 𝑢 = ( 𝑗=0 ∑ 𝑗=0 ∑ 𝑢 Using this property we can rewrite our equation from the previous slide as A useful property of the eigen decomposition is that corresponding eigenvalues. columns of 𝐑 are the eigenvectors of 𝐆 and 𝚳 is a diagonal matrix of the 11 We can find the eigen decomposition such that 𝐆 = 𝐑𝚳𝐑 −1 where the 𝐆 𝑗 = 𝐑𝚳 𝑗 𝐑 −1 𝐺 𝑗 )𝜺 + 𝐺 𝑗 𝑥 𝑢−𝑗 𝝄 𝑢 = ( 𝐑𝚳 𝑗 𝐑 −1 𝑥 𝑢−𝑗

Proof sketch (cont.) ⎦ ⋯ 𝜇 𝑗 𝑞 ⎤ ⎥ ⎥ Therefore, 0 lim when lim which requires that |𝜇 𝑗 | < 1 for all 𝑗 0 ⋮ ⋱ 0 ⎡ ⎢ ⎢ ⎣ 𝜇 𝑗 1 ⋯ 0 0 𝜇 𝑗 2 ⋯ 0 ⋮ ⋮ 12 𝚳 𝑗 = 𝑢→∞ 𝐺 𝑢 → 0 𝑢→∞ Λ 𝑢 → 0

which if we multiply by 1/𝜇 𝑞 where 𝑀 = 1/𝜇 gives 1 − 𝜚 1 𝑀 − 𝜚 2 𝑀 2 − ⋯ − 𝜚 𝑞 1 𝑀 𝑞−1 − 𝜚 𝑞 𝑀 𝑞 = 0 Proof sketch (cont.) Eigenvalues are defined such that for 𝝁 , det (𝐆 − 𝝁 𝐉) = 0 based on our definition of 𝐆 our eigenvalues will therefore be the roots of 13 𝜇 𝑞 − 𝜚 1 𝜇 𝑞−1 − 𝜚 2 𝜇 𝑞−2 − ⋯ − 𝜚 𝑞 1 𝜇 1 − 𝜚 𝑞 = 0

Proof sketch (cont.) Eigenvalues are defined such that for 𝝁 , det (𝐆 − 𝝁 𝐉) = 0 based on our definition of 𝐆 our eigenvalues will therefore be the roots of 13 𝜇 𝑞 − 𝜚 1 𝜇 𝑞−1 − 𝜚 2 𝜇 𝑞−2 − ⋯ − 𝜚 𝑞 1 𝜇 1 − 𝜚 𝑞 = 0 which if we multiply by 1/𝜇 𝑞 where 𝑀 = 1/𝜇 gives 1 − 𝜚 1 𝑀 − 𝜚 2 𝑀 2 − ⋯ − 𝜚 𝑞 1 𝑀 𝑞−1 − 𝜚 𝑞 𝑀 𝑞 = 0

Properties of 𝐵𝑆(2) 𝑊 𝑏𝑠(𝑥 𝑢 ) = 𝜏 2 𝑥 14 For a stationary 𝐵𝑆(2) process where 𝑥 𝑢 has 𝐹(𝑥 𝑢 ) = 0 and

Properties of 𝐵𝑆(𝑞) 𝑊 𝑏𝑠(𝑥 𝑢 ) = 𝜏 2 𝑥 𝐹(𝑍 𝑢 ) = 𝜀 𝑊 𝑏𝑠(𝑧 𝑢 ) = 𝛿(0) = 𝜚 1 𝛿(1) + 𝜚 2 𝛿(2) + … + 𝜚 𝑞 𝛿(𝑞) + 𝜏 2 𝑥 𝛿(ℎ) = 𝜚 1 𝛿(ℎ − 1) + 𝜚 2 𝛿(ℎ − 2) + … + 𝜚 𝑞 𝛿(ℎ − 𝑞) 15 For a stationary 𝐵𝑆(𝑞) process where 𝑥 𝑢 has 𝐹(𝑥 𝑢 ) = 0 and 1 − 𝜚 1 − 𝜚 2 − ⋯ − 𝜚 𝑞 𝜍(ℎ) = 𝜚 1 𝜍(ℎ − 1) + 𝜚 2 𝜍(ℎ − 2) + … + 𝜚 𝑞 𝜍(ℎ − 𝑞)

Moving Average (MA) Processes