Lecture 9 ARIMA Models Colin Rundel 02/15/2017 1 2 MA ( ) 3 - PowerPoint PPT Presentation

Lecture 9 ARIMA Models Colin Rundel 02/15/2017 1

2 MA ( ∞ )

3 From last time, 0 w Properties: MA ( q ) MA ( q ) : y t = δ + w t + θ 1 w t − 1 + θ 2 w t − 2 + · · · + θ q w t − q E ( y t ) = δ Var ( y t ) = ( 1 + θ 2 1 + θ 2 + · · · + θ 2 q ) σ 2 { θ h + θ 1 θ 1 + h + θ 2 θ 2 + h + · · · + θ q − h θ q if | h | ≤ q Cov ( y t , y t + h ) = if | h | > q and is stationary for any values of θ i

4 Sometimes, a slightly strong condition called absolute summability, MA ( ∞ ) If we let q → ∞ then process will still be stationary if the moving average coefficients ( θ ’s) are square summable, ∞ ∑ i < ∞ θ 2 i = 1 since this is necessary for Var ( y t ) < ∞ . i = 1 | θ i | < ∞ , is necessary (e.g. for some CLT related asymptotic results) . ∑ ∞

Invertibility process then it is said that the MA process is invertible. 5 If a MA ( q ) process, y t = δ + θ q ( L ) w t , can be rewritten as a purely AR MA ( 1 ) w/ δ = 0 example:

p L y t w t can be q L are Invertibility vs Stationarity Conversely, an AR p process is stationary if rewritten as an exclusively MA process (of possibly infinite order), i.e. y t L w t . So using our results w.r.t. L it follows that if all of the roots of outside the complex unit circle then the moving average is invertible. 6 A MA ( q ) process is invertible if y t = δ + θ q ( L ) w t can be rewritten as an exclusively AR process (of possibly infinite order), i.e. ϕ ( L ) y t = α + w t .

q L are Invertibility vs Stationarity rewritten as an exclusively MA process (of possibly infinite order), i.e. So using our results w.r.t. L it follows that if all of the roots of outside the complex unit circle then the moving average is invertible. 6 A MA ( q ) process is invertible if y t = δ + θ q ( L ) w t can be rewritten as an exclusively AR process (of possibly infinite order), i.e. ϕ ( L ) y t = α + w t . Conversely, an AR ( p ) process is stationary if ϕ p ( L ) y t = δ + w t can be y t = δ + θ ( L ) w t .

Invertibility vs Stationarity rewritten as an exclusively MA process (of possibly infinite order), i.e. outside the complex unit circle then the moving average is invertible. 6 A MA ( q ) process is invertible if y t = δ + θ q ( L ) w t can be rewritten as an exclusively AR process (of possibly infinite order), i.e. ϕ ( L ) y t = α + w t . Conversely, an AR ( p ) process is stationary if ϕ p ( L ) y t = δ + w t can be y t = δ + θ ( L ) w t . So using our results w.r.t. ϕ ( L ) it follows that if all of the roots of θ q ( L ) are

Differencing 7

Difference operator We will need to define one more notational tool for indicating differencing just like the lag operator we will indicate repeated applications of this operator using exponents 8 ∆ y t = y t − y t − 1 ∆ 2 y t = ∆(∆ y t ) = (∆ y t ) − (∆ y t − 1 ) = ( y t − y t − 1 ) − ( y t − 1 − y t − 2 ) = y t − 2 y t − 1 + y t − 2 ∆ can also be expressed in terms of the lag operator L , ∆ d = ( 1 − L ) d

Differencing and Stocastic Trend Using the two component time series model stationary component. We have already shown that differencing can address deterministic trend follows a random walk. 9 y t = µ t + x t where µ t is a non-stationary trend component and x t is a mean zero (e.g. µ t = β 0 + β 1 t ). In fact, if µ t is any k -th order polynomial of t then ∆ k y t is stationary. Differencing can also address stochastic trend such as in the case where µ t

Stochastic trend - Example 1 10 Let y t = µ t + w t where w t is white noise and µ t = µ t − 1 + v t with v t stationary as well. Is ∆ y t stationary?

Stochastic trend - Example 2 is it stationary? 11 Let y t = µ t + w t where w t is white noise and µ t = µ t − 1 + v t but now v t = v t − 1 + e t with e t being stationary. Is ∆ y t stationary? What about ∆ 2 y t ,

ARIMA 12

ARIMA Models Autoregressive integrated moving average are just an extension of an ARMA model to include differencing of degree d to y t , which is most often used to address trend in the data. Box-Jenkins approach: 1. Transform data if necessary to stabilize variance 2. Choose order ( p , d , and q ) of ARIMA model 3. Estimate model parameters ( s and s) 4. Diagnostics 13 ARIMA ( p , d , q ) : ϕ p ( L ) ∆ d y t = δ + θ q ( L ) w t

ARIMA Models Autoregressive integrated moving average are just an extension of an ARMA model to include differencing of degree d to y t , which is most often used to address trend in the data. Box-Jenkins approach: 1. Transform data if necessary to stabilize variance 2. Choose order ( p , d , and q ) of ARIMA model 4. Diagnostics 13 ARIMA ( p , d , q ) : ϕ p ( L ) ∆ d y t = δ + θ q ( L ) w t 3. Estimate model parameters ( ϕ s and θ s)

Using forecast - random walk with drift ## BIC=1395.31 AICc=1386.91 ## AIC=1386.88 log likelihood=-691.44 ## sigma^2 estimated as 0.9323: ## 0.0431 ## s.e. 0.0641 drift Some of R’s base timeseries handling is a bit wonky, the forecast package ## ## Coefficients: ## ## ARIMA(0,1,0) with drift ## Series: rwd Arima (rwd, order = c (0,1,0), include.constant = TRUE) library (forecast) rwd = arima.sim (n=500, model= list (order= c (0,1,0)), mean=0.1) offers some useful alternatives and additional functionality. 14

EDA 15 40 3 2 30 1 diff(rwd) rwd 20 0 −1 10 −2 −3 0 0 100 200 300 400 500 0 100 200 300 400 500 0.10 0.8 ACF ACF 0.00 0.4 −0.10 0.0 0 5 10 15 20 25 0 5 10 15 20 25

Over differencing 16 0.5 4 2 0.3 diff(rwd, 2) ACF 0 0.1 −2 −0.1 −4 0 100 200 300 400 500 0 5 10 15 20 25 0.6 4 2 0.4 diff(rwd, 3) ACF 0 0.2 −2 0.0 −4 0 100 200 300 400 500 0 5 10 15 20 25

AR or MA? 17 0 60 −5 −10 40 ts1 −15 ts2 20 −20 −25 0 −30 0 50 100 150 200 250 0 50 100 150 200 250

EDA 18 ts2 ts1 0 20 40 60 −30 −20 −10 0 0 0 50 50 100 100 150 150 200 200 250 250 ACF ACF −0.2 0.2 0.6 1.0 −0.2 0.2 0.6 1.0 5 5 10 10 15 15 20 20 Partial ACF Partial ACF −0.2 0.2 0.6 1.0 −0.2 0.2 0.6 1.0 5 5 10 10 15 15 20 20

ts1 - Finding d 19 Partial ACF ACF diff(ts1) −0.2 0.0 0.2 0.4 −0.2 0.0 0.2 0.4 −3 −1 1 3 0 50 5 5 100 10 10 d=1 150 15 15 200 20 20 250 Partial ACF ACF diff(ts1, 2) −0.4 0.0 0.4 −0.2 0.2 0.6 −6 −2 0 2 4 0 50 5 5 100 10 10 d=2 150 15 15 200 20 20 250 Partial ACF ACF diff(ts1, 3) −0.4 0.0 0.4 0.8 −0.2 0.2 0.6 −5 0 5 0 50 5 5 100 10 10 d=3 150 15 15 200 20 20 250

ts2 - Finding d 20 Partial ACF ACF diff(ts2) −0.2 0.2 0.6 −0.2 0.2 0.6 −3 −1 1 2 3 4 0 50 5 5 100 10 10 d=1 150 15 15 200 20 20 250 Partial ACF ACF diff(ts2, 2) −0.2 0.2 0.6 −0.2 0.2 0.6 −6 −2 2 4 6 0 50 5 5 100 10 10 d=2 150 15 15 200 20 20 250 Partial ACF ACF diff(ts2, 3) −0.2 0.2 0.6 −0.2 0.2 0.6 −5 0 5 0 50 5 5 100 10 10 d=3 150 15 15 200 20 20 250

ts1 - Models 748.65 747.55 758.12 1 1 0 747.61 754.65 1 1 1 759.21 1 0 1 1 764.98 772.02 0 1 0 800.43 803.95 0 2 p 1 d q AIC BIC 0 1 2 729.43 740.00 1 2 758.38 731.23 745.31 2 1 2 731.57 749.18 2 1 1 744.29 21

ts2 - Models 754.66 686.38 696.95 1 1 0 719.16 726.20 0 1 2 765.22 1 0 1 1 804.44 811.48 0 1 0 890.32 893.85 1 1 p 1 d q AIC BIC 2 1 0 683.12 693.68 1 2 702.67 683.25 697.34 2 1 1 683.83 697.92 2 1 2 685.06 22

ts1 - Model Choice ## s.e. BIC=740 AICc=729.53 ## AIC=729.43 log likelihood=-361.72 ## sigma^2 estimated as 1.064: ## 0.0622 0.0547 0.4319 Arima (ts1, order = c (0,1,2)) 0.4138 ## ma2 ma1 ## ## Coefficients: ## ## ARIMA(0,1,2) ## Series: ts1 23

ts2 - Model Choice ## s.e. BIC=693.68 AICc=683.22 ## AIC=683.12 log likelihood=-338.56 ## sigma^2 estimated as 0.8822: ## 0.0587 0.0587 0.3770 Arima (ts2, order = c (2,1,0)) 0.4392 ## ar2 ar1 ## ## Coefficients: ## ## ARIMA(2,1,0) ## Series: ts2 24

Residuals 25 ts1 Residuals 0.2 0.2 3 Partial ACF ts1_resid 1 ACF 0.0 0.0 −1 −0.2 −0.2 −3 0 50 100 150 200 250 5 10 15 20 5 10 15 20 ts2 Residuals 0.2 0.2 3 Partial ACF 2 ts2_resid 1 ACF 0.0 0.0 0 −2 −0.2 −0.2 0 50 100 150 200 250 5 10 15 20 5 10 15 20

Electrical Equipment Sales 26

Data 27 elec_sales 110 100 90 80 2000 2005 2010 0.6 0.6 PACF ACF 0.2 0.2 −0.2 −0.2 0 5 10 15 20 25 30 35 0 5 10 15 20 25 30 35 Lag Lag

1st order differencing 28 diff(elec_sales, 1) 10 5 0 −5 −10 2000 2005 2010 0.3 0.3 0.1 0.1 PACF ACF −0.1 −0.1 −0.3 −0.3 0 5 10 15 20 25 30 35 0 5 10 15 20 25 30 35 Lag Lag

2nd order differencing 29 diff(elec_sales, 2) 10 5 0 −5 −10 2000 2005 2010 0.4 0.4 0.2 0.2 PACF ACF 0.0 0.0 −0.2 −0.2 0 5 10 15 20 25 30 35 0 5 10 15 20 25 30 35 Lag Lag