Resampling for nonstationary stochastic models Jacek Le skow Anna - PowerPoint PPT Presentation

Resampling for nonstationary stochastic models Jacek Le´ skow Anna Dudek Łukasz Lenart Rafał Synowiecki

Plan of the presentation * Nonstationary stochastic models - PC and APC models, time domain - PC and APC frequency domain - periodic counting process models * Limit results * Why resampling? * Selected results * Future directions of research

Time domain approach { X t : t ∈ Z } - APC, when µ X ( t ) = E ( X t ) and the autocovariance function B X ( t , τ ) = cov ( X t , X t + τ ) are almost periodic function at t for every τ ∈ Z . Put B X ( t , τ ) = � a ( λ, τ ) e i λτ λ ∈ Λ Time domain approach ( µ X ( t ) ≡ 0): n − τ � 1 X ( t + τ ) X ( t ) e − i λ t ˆ a n ( λ, τ ) = n − τ t = 1

Frequency domain approach Harmonizable time series { X ( t ) : t ∈ Z } 2 π � e i ξ t Z ( d ξ ) . X ( t ) = 0 Spectral bimeasure is defined as R (( a , b ] × ( c , d ]) = E [( Z ( b ) − Z ( a ))( Z ( d ) − Z ( c ))] , with a support � { ( ξ 1 , ξ 2 ) ∈ ( 0 , 2 π ] 2 : ξ 2 = ξ 1 ± λ } . S = λ ∈ Λ

Spectral density estimator n n � � 1 ˆ K n ( s − t ) X t X s e − i ν t e i ω s . G n ( ν, ω ) = (1) 2 π n t = 1 s = 1 Support lines 2 Π 0 APC case 2 Π

Simulation example Time series Subsampling test for � Γ � Ν , Ω �� 2 7.5 5 2.5 0 -2.5 -5 -7.5 0 100 200 300 400 500 600 700 X t = ( 2 + sin ( 2 π t / 4 )) Y t − 1 + Y t , where Y t are i.i.d. from N ( 0 , 1 ) .

Nonstationary counting process X - counting process on [ 0 , T ] . Intensity of X is of the form λ ( t ) = λ 0 ( t ) Y ( t ) , t ∈ [ 0 , T ] λ 0 ( t ) – nonnegative deterministic periodic function Y ( t ) – nonnegative stochastic process

Nonstationary counting process Sieve estimator of λ 0 ( t ) The histogram maximum likelihood estimator of the periodic λ 0 ( t ) function is of the form: � n k = 1 X k ( B s n ) � λ n ( s ) = � n Y k ( u ) du 1 D n ( s ) , s ∈ [ 0 , P ] , � n k = 1 B s where s ∈ B s n is the interval of the length P / b that contains s and � n � D n = { Y k ( u ) du > 0 } . B s n k = 1

Real data example Incoming packets number in one hour non-overlapping bins - border between the network of University of Waikato and the internet provider.

Limit results

Asymptotic normality, PC time domain domain We have √ d n (ˆ a n ( λ, τ ) − a ( λ, τ )) − − − → N 2 ( 0 , Σ) , (2) where � σ 11 � σ 12 Σ = , σ 21 σ 22 T ∞ � � σ 11 = 1 B Z τ ( s , k ) cos ( λ s ) cos ( λ k ) , T s = 1 k = −∞ T ∞ � � σ 22 = 1 B Z τ ( s , k ) sin ( λ s ) sin ( λ k ) , T s = 1 k = −∞ T ∞ � � σ 12 = σ 21 = 1 B Z τ ( s , k ) cos ( λ s ) sin ( λ k ) , T s = 1 k = −∞ and Z ( t , τ ) = X ( t ) X ( t + τ ) − B X ( t , τ ) , B Z τ ( s , k ) = Cov ( Z ( s , τ ) , Z ( s + k , τ )) .

Asymptotic covariance, APC case Lemma (Lenart 2008) If (i) there exists δ > 0 such that sup t ∈ Z � X t � 6 + 3 δ ≤ ∆ < ∞ , � ∞ δ k 2 α ( k ) 2 + δ ≤ K < ∞ , (ii) k = 1 (iii) K n ( s − t ) = I {| s − t | ≤ w n } + additional regularity assumptions then we have a convergence � � n G n ( ν 1 , ω 1 ) , ˆ ˆ lim cov G n ( ν 2 , ω 2 ) = P ( ν 1 , ν 2 ) P ( ω 1 , ω 2 ) w n n →∞ + P ( ν 1 , 2 π − ω 2 ) P ( ν 2 , 2 π − ω 1 ) , for any ( ν 1 , ω 2 ) , ( ν 2 , ω 2 ) ∈ ( 0 , 2 π ] 2 .

Asymptotic normality, APC case Theorem (Lenart 2008) If (i) there exists δ > 0 such that sup t ∈ Z � X t � 6 + 3 δ ≤ ∆ < ∞ , (ii) w n = O ( n κ ) for some κ ∈ ( 0 , δ/ ( 4 + 4 δ ) , ∞ � δ 2 ( r + 1 )+ δ < ∞ , where r is the integer such that h 2 r α ( h ) (iii) h = 1 � � κ ( 1 + δ ) 1 + δ, 1 − κ r > max 4 κ , , δ − 2 κ ( 1 + δ ) then � n � � ˆ G n ( ν, ω ) − P ( ν, ω ) − → N ( 0 , Σ( ν, ω )) , w n where matrix Σ( ν, ω ) can be obtained by previous Theorem.

Asymptotic normality, counting process case Theorem (Dudek, 2008) If (i) The Y process is periodically correlated with period P and its mean function E ( Y ( s )) is bounded away from zero. (ii) Process Y has the third moment bounded. (iii) Process Y is α –mixing with α ( k ) = o ( k − 3 ) . � √ n � (iv) Each period P is divided into b parts, where b = O . (v) The periodic λ 0 function (with the period length equal to P ) and EY ( t ) fulfill the Lipschitz condition on [ 0 , P ] then � � � � � n λ 0 ( s ) � λ n ( s ) − λ 0 ( s ) ⇒ N 0 , . b E ( Y ( s ))

Why resampling? * APC time series case: too complicated asymptotic covariance matrix * periodic counting process case: slow convergence, need for simultaneous confidence bands

Subsampling for Fourier coefficient of autocovariance function Consistency holds for the estimator ˆ θ n = | ˆ a n ( λ, τ ) | . Let � √ � n ( | ˆ a n ( λ, τ ) | − | a ( λ, τ ) | ) ≤ x J n ( x , P ) = Prob P . By CLT for ˆ a n ( λ, τ ) and the delta method we have d J n ( P ) − − − → J ( P ) . We define correspondingly subsampling distribution in the form n − b + 1 √ � 1 b ( | ˆ a n , b , t ( λ, τ ) | − | ˆ L n , b ( P ) = 1 { a n ( λ, τ ) | ) ≤ x } n − b + 1 t = 1

Subsampling for Fourier coefficient of autocovariance function Theorem (Lenart, Le´ skow, Synowiecki, 2008) Let { X ( t ) : t ∈ Z } be APC time series. Assume that (i) b → ∞ but b / n → 0, (ii) sup t E | X ( t ) | 4 + 4 δ < ∞ , ∞ � δ ( k + 1 ) 2 α ( k ) 4 + δ < ∞ , (iii) k = 0 (iv) the function � � V ( t , τ 1 , τ 2 , τ 3 ) = E X ( t ) X ( t + τ 1 ) X ( t + τ 2 ) X ( t + τ 3 ) is almost periodic. Then subsampling is consistent, which means that P sup | J n ( x , P ) − L n , b ( x ) | − − − → 0 . x

Application of subsampling procedure for PC time series Testing problem: H 0 : B ( · , τ ) is periodic with period T 0 , H 1 : B ( · , τ ) is periodic with period T 1 . Test statistics (Lenart, Leskow, Synowiecki, 2008):   � √   . | ˆ U n ( τ ) = n a n ( λ, τ ) | λ ∈ Λ T 1 \ Λ T 0

Application of subsampling procedure for PC time series Under H 0 : d U n ( τ ) − − − → J . Under H 1 : U n ( τ ) − → ∞ . Large values of U n ( τ ) suggest that hypothesis H 1 is true. The rejection area is of the form [ c 1 − α , ∞ ) . In order to find c 1 − α subsampling may be applied.

Application of subsampling procedure for PC time series 0.5 1 0.4 0.8 0.3 0.6 0.2 0.4 0.1 0.2 50 100 150 200 250 300 350 50 100 150 200 250 300 350 (a) Probability of rejection H 0 pro- (b) Probability of rejection H 0 pro- vided that H 0 is true. vided that H 1 is true. Figure: Monte Carlo approximations of test errors.

Consistency of MBB for (almost) periodic time series Theorem (Synowiecki, 2007) Let { X t : t ∈ Z } be APC and α -mixing, let ( X ∗ 1 , . . . , X ∗ n ) be MBB sample, b → ∞ ale b / n → 0. Assume that (i) Λ = { λ : [ 0 , 2 π ) : M t ( EX t e − i λ t ) � = 0 } is finite, (ii) autocovariance is uniformly summable � � 4 � s + b − 1 1 ( X t − EX t ) < K (iii) sup s = 1 ,..., n − b + 1 E √ t = s b � � (iv) CLT holds, i.e. √ n d → N ( 0 , σ 2 ) X n − M t ( EX t ) − − − Then MBB procedure is consistent, which means that √ ∗ P Var ∗ ( → σ 2 − − − n X n ) and � �� � √ � � � − P ∗ � √ � � � � ∗ ∗ p n − E ∗ X sup � P n X n − µ ≤ x n X ≤ x → 0 . � n x ∈ R

Consistency of subsampling - APC case, spectral coherence Theorem (Lenart, 2008) Under regularity conditions the subsampling confidence intervals for coherence are consistent �� � γ n ( ν, ω ) | − | γ ( ν, ω ) | ) ≤ c γ n / w n ( | ˆ n , b ( 1 − α ) − → 1 − α, P where b = b ( n ) → ∞ , and b / n → 0, c γ n , b ( 1 − α ) = inf { x : L γ n , b ( x ) ≥ 1 − α } . n − b + 1 � � 1 L γ n , b ( x )= 1 { b / w b ( | ˆ γ n , b , t ( ν, ω ) | − | ˆ γ n ( ν, ω ) | ) ≤ x } . n − b + 1 t = 1

Consistency of bootstrap - counting process case Theorem (Dudek, 2008) � �� � � n � � P ∗ b ( � λ ∗ n ( s ) − � λ n ( s )) ≤ u sup � u ∈ R �� �� � n � b ( � − P λ n ( s ) − λ 0 ( s )) ≤ u � = o P ( 1 ) , � where � n k = 1 X ∗ k ( B s n ) λ ∗ � n ( s ) = � n Y k ( u ) du 1 D n ( s ) . � n B s k = 1

Real data example - counting process case Estimator of the intensity of the number of packets being received by one host together with 90 % confidence region: 500 400 300 200 5 10 15 20

Future directions of research * Subsampling - optimal selection of block size * Resampling in GACS signals * nonstationary random fields

References Chan V., Lahiri S., Meeker W. (2004) Block bootstrap estimation of the distribution of cumulative outdoor degradation Technometrics Dudek A., Go´ cwin M., Le´ skow J., (2008) Simultaneous confidence bands for periodic hazard function, submitted Lenart Ł., Le´ skow J., Synowiecki R. (2007) Subsampling in testing autocovariance for periodically correlated time series Journal of Time Series Analysis, to appear Lenart Ł., (2008) Asymptotic properties of periodogram for almost periodically correlated time series Probability and Mathematical Statistics, to appear