Time Series Analysis Henrik Madsen hm@imm.dtu.dk Informatics and - PowerPoint PPT Presentation

H. Madsen, Time Series Analysis, Chapmann Hall Time Series Analysis Henrik Madsen hm@imm.dtu.dk Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby Henrik Madsen 1

H. Madsen, Time Series Analysis, Chapmann Hall Outline of the lecture Regression based methods, 2nd part: Regression and exponential smoothing (Sec. 3.4) Time series with seasonal variations (Sec. 3.5) Henrik Madsen 2

H. Madsen, Time Series Analysis, Chapmann Hall Regression without explanatory variables During Lecture 2 we saw that assuming known independent variables x we can forecast the dependent variable Y To be able to do so we estimated θ in Y t = f ( x t , t ; θ ) + ε t If we do not have access to x we may use: Y t = f ( t ; θ ) + ε t During this lecture we shall consider models of this (last) form and we shall consider how � θ can be updated as more information becomes available Only models linear in θ will be considered Henrik Madsen 3

H. Madsen, Time Series Analysis, Chapmann Hall Model: Constant mean Y t = µ + ε t , ε t i.i.d. with mean zero and constant variance σ 2 (white noise). In vector form ( t = 1 , . . . , N ): Y = 1 µ + ε N � µ = ( 1 T 1 ) − 1 1 T Y = N − 1 Estimate: ˆ Y t = ¯ y · t =1 N � Prediction (the conditional mean): � µ = 1 Y N + ℓ | N = � Y t N t =1 Variance of the prediction error: � � Y N + ℓ − � = σ 2 (1 + 1 V Y N + ℓ | N N ) Henrik Madsen 4

H. Madsen, Time Series Analysis, Chapmann Hall Updating the estimate N � µ N = 1 Based on Y 1 , Y 2 , . . . , Y N we have ˆ Y t N t =1 When we get one more observation Y N +1 the best estimate is N +1 � 1 µ N +1 = ˆ Y t N +1 t =1 Recursive update: N +1 � 1 1 N µ N +1 = ˆ Y t = N + 1 Y N +1 + N + 1 ˆ µ N N + 1 t =1 Henrik Madsen 5

H. Madsen, Time Series Analysis, Chapmann Hall Model: Local constant mean In the constant mean model the variance of the forecast error decrease towards σ 2 as 1 /N Therefore, if N is sufficiently high (say 100) there is not much gained by increasing the number of observations If there is indications that the true (underlying) mean is actually changing slowly it can even be advantageous to “forget” old observations. One way of doing this is to base the estimate on a rolling window containing e.g. the 100 most recent observations An alternative is exponential smoothening Henrik Madsen 6

H. Madsen, Time Series Analysis, Chapmann Hall Exponential smoothening N − 1 � λ j Y N − j = c [ Y N + λY N − 1 + · · · + λ N − 1 Y 1 ] µ N = c ˆ j =0 c Weight 0 0 5 10 15 20 25 30 Observation number The constant c is chosen so that the weights sum to one, which implies that c = (1 − λ ) / (1 − λ N ) . For large N : µ N or � Y N + ℓ +1 | N +1 = (1 − λ ) Y N +1 + λ � µ N +1 = (1 − λ ) Y N +1 + λ ˆ ˆ Y N + ℓ | N Henrik Madsen 7

H. Madsen, Time Series Analysis, Chapmann Hall Choice of smoothing constant α = 1 − λ The smoothing constant α = 1 − λ determines how much the latest observation influence the prediction Given a data set t = 1 , . . . , N we can try different values before implementing the method on-line N � ( Y t − � Y t | t − 1 ( α )) 2 S ( α ) = t =1 If the data set is large we eliminate the influence of the initial estimate by dropping the first part of the errors when evaluating S ( α ) Henrik Madsen 8

H. Madsen, Time Series Analysis, Chapmann Hall Example – wind speed 76 m a.g.l. at Risø Measurements of wind speed every 10th minute Task: Forecast up to approximately 3 hours ahead using exponential smoothing 25 10min. avg. of wind speed (m/s) 20 15 10 5 Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep 2002 2003 Henrik Madsen 9

H. Madsen, Time Series Analysis, Chapmann Hall S ( α ) for horizons 10 and 70 minutes 10 minutes 70 minutes 50000 100000 40000 95000 SSE SSE 90000 30000 85000 20000 80000 0.2 0.4 0.6 0.8 0.2 0.4 0.6 0.8 Weight on most recent observation Weight on most recent observation 10 minutes (1-step): Use α = 0 . 95 or higher 70 minutes (7-step): Use α ≈ 0 . 7 Henrik Madsen 10

H. Madsen, Time Series Analysis, Chapmann Hall S ( α ) for horizons 130 and 190 minutes 130 minutes 190 minutes 145000 186000 184000 140000 182000 SSE SSE 180000 135000 178000 176000 130000 174000 0.2 0.4 0.6 0.8 0.2 0.4 0.6 0.8 Weight on most recent observation Weight on most recent observation 130 minutes (13-step): Use α ≈ 0 . 6 190 minutes (19-step): Use α ≈ 0 . 5 Henrik Madsen 11

H. Madsen, Time Series Analysis, Chapmann Hall Example of forecasts with optimal α Measurements 10 minute forecast 190 minute forecast 14 12 10 m/s 8 6 4 12:00 14:00 16:00 18:00 20:00 22:00 0:00 2:00 4:00 6:00 8:00 10:00 12:00 Jan 11 2003 Jan 12 2003 Henrik Madsen 12

H. Madsen, Time Series Analysis, Chapmann Hall Trend models Linear regression model Functions of time are taken as the independent variables Henrik Madsen 13

H. Madsen, Time Series Analysis, Chapmann Hall Linear trend Observations for t = 1 , . . . , N Naive formulation of the model: Y t = φ 0 + φ 1 t + ε t If we want to forecast Y N + j given information up to N we use Y N + j | N = ˆ φ 0 + ˆ � φ 1 ( N + j ) However, for on-line applications N + j can be arbitrary large The problem arise because φ 0 and φ 1 is defined w.r.t. the origin 0 Defining the parameters w.r.t. the origin n we obtain the model: Y t = θ 0 + θ 1 ( t − N ) + ε t Y N + j | N = ˆ θ 0 + ˆ Using this formulation we get: � θ 1 j Henrik Madsen 14

H. Madsen, Time Series Analysis, Chapmann Hall Linear trend in a general setting The general trend model: Y N + j = f T ( j ) θ + ε N + j � 1 � The linear trend model is obtained when: f ( j ) = j It follows that for N + 1 + j : �� 1 � � 1 � � T �� T 1 0 Y N +1+ j = θ + ε N +1+ j = θ + ε N +1+ j j + 1 1 1 j The 2 × 2 matrix L defines the transition from f ( j ) to f ( j + 1) Henrik Madsen 15

H. Madsen, Time Series Analysis, Chapmann Hall Trend models in general Model: Y N + j = f T ( j ) θ + ε N + j Requirement: f ( j + 1) = Lf ( j ) Initial value: f (0) In Section 3.4 some trend models which fulfill the requirement above are listed. Constant mean: Y N + j = θ 0 + ε N + j Linear trend: Y N + j = θ 0 + θ 1 j + ε N + j j 2 Quadratic trend: Y N + j = θ 0 + θ 1 j + θ 2 2 + ε n + j k ’th order polynomial trend: j k j 2 Y n + j = θ 0 + θ 1 j + θ 2 2 + · · · + θ k k ! + ε N + j Harmonic model with the period p : Y N + j = θ 0 + θ 1 sin 2 π p j + θ 2 cos 2 π p j + ε N + j Henrik Madsen 16

H. Madsen, Time Series Analysis, Chapmann Hall Estimation Model equations written for all observations Y 1 , . . . , Y N Y = x N θ + ε       f T ( − N + 1) Y 1 ε 1 f T ( − N + 2)       Y 2 ε 2        =  θ +       . . . . . .     . . . f T (0) Y N ε N OLS-estimates: � θ N = ( x T N x N ) − 1 x T N Y or N − 1 N − 1 � � f ( − j ) f T ( − j ) θ N = F − 1 � F N = h N = f ( − j ) Y N − j N h N j =0 j =0 Henrik Madsen 17

H. Madsen, Time Series Analysis, Chapmann Hall ℓ -step prediction Prediction: Y N + ℓ | N = f T ( ℓ ) � � θ N Variance of the prediction error: Y N + ℓ | N ] = σ 2 � � 1 + f T ( ℓ ) F − 1 V [ Y N + ℓ − � N f ( ℓ ) 100(1 − α )% prediction interval: � � Y N + ℓ | N ± t α/ 2 ( N − p ) V [ e N ( ℓ )] = � 1 + f T ( ℓ ) F − 1 � Y N + ℓ | N ± t α/ 2 ( N − p ) � σ N f ( ℓ ) σ 2 = ε T ε / ( N − p ) ( p is the number of estimated where ˆ parameters) Henrik Madsen 18

H. Madsen, Time Series Analysis, Chapmann Hall Updating the estimates when Y N +1 is available Task: Going from estimates based on t = 1 , . . . , N , i.e. � θ N to estimates based on t = 1 , . . . , N, N + 1 , i.e. � θ N +1 without redoing everything. . . Solution: F − 1 � = θ N +1 N +1 h N +1 F N + f ( − N ) f T ( − N ) = F N +1 L − 1 h N + f (0) Y N +1 = h N +1 Henrik Madsen 19

H. Madsen, Time Series Analysis, Chapmann Hall Local trend models We forget old observations in an exponential manner: � θ N = arg min θ S ( θ ; N ) where for 0 < λ < 1 N − 1 � λ j [ Y N − j − f T ( − j ) θ ] 2 S ( θ ; N ) = j =0 0.8 Weight 0.4 0.0 0 5 10 15 20 25 30 j (age of observation) Henrik Madsen 20

H. Madsen, Time Series Analysis, Chapmann Hall WLS formulation The criterion: N − 1 � λ j [ Y N − j − f T ( − j ) θ ] 2 S ( θ ; N ) = j =0 can be written as:   T     Y 1 − f T ( N − 1) θ Y 1 − f T ( N − 1) θ λ N − 1 0 0 · · · Y 2 − f T ( N − 2) θ Y 2 − f T ( N − 2) θ       λ N − 2 0 0 · · ·             . . . . . ... . . . . .       . . . . . Y N − f T (0) θ Y N − f T (0) θ 0 0 0 1 which is a WLS criterion with Σ = diag [1 /λ N − 1 , . . . , 1 /λ, 1] Henrik Madsen 21

H. Madsen, Time Series Analysis, Chapmann Hall WLS solution θ N = ( x T N Σ − 1 x N ) − 1 x T N Σ − 1 Y � or � F − 1 = θ N N h N N − 1 � λ j f ( − j ) f T ( − j ) = F N j =0 N − 1 � λ j f ( − j ) Y N − j = h N j =0 Henrik Madsen 22