Estimating Models Based on Markov Jump Processes Given Fragmented - PowerPoint PPT Presentation

Estimating Models Based on Markov Jump Processes Given Fragmented Observation Series Markus Hahn Johann Radon Institute for Computational and Applied Mathematics (RICAM), Linz Austrian Academy of Sciences Joint work with S. Fr¨ uhwirth-Schnatter (JKU Linz) and J. Sass (TU Kaiserslautern) Linz, December 2, 2008 Work funded by FWF project P17947

Introduction Problem ◮ Estimation from set of observed series ◮ Independent series ◮ Data with breaks ◮ Each single series is (based) Markov process ◮ Same generator for all series ◮ How to estimate common generator? ◮ How to cope with short observation series? 2 / 37

Introduction Outline Introduction Markov Jump Processes Merged Markov Jump Processes Inference for Merged Markov Jump Processes Generalization: Markov Switching Models Conclusion 3 / 37

Markov Jump Processes Finite state Markov jump process (MJP) ◮ Y = ( Y t ) t ∈ [0 , T ) is a continuous time Markov process ◮ Finite state space { 1 , . . . , d } ◮ Y is time homogeneous ◮ Jumps of Y are governed by rate matrix Q ∈ R d × d ◮ Exponential rate of leaving state k : � λ k = − Q kk = Q kl < ∞ l � = k i.e. average waiting time for leaving k is 1 /λ k ◮ Conditional transition probability: P( Y t = l | Y t − = k , Y t � = Y t − ) = Q kl /λ k 4 / 37

Markov Jump Processes Inference about rate matrix ◮ O k occupation time of state k ◮ N kl number of jumps from k to l ◮ Maximum likelihood estimation: ˆ Q kl = N kl / O k ◮ Observing a path ( Y t ) t ∈ [0 , T ) , O k and N kl are sufficient for estimating Q kl ◮ Unbiased? E( ˆ Q kl | Q ) = ∞ E( ˆ Q kl | Q , O k > 0) = ∞ ◮ ˆ Q is consistent, i.e. T →∞ P( | ˆ lim Q kl − Q kl | > ǫ ) = 0 5 / 37

Markov Jump Processes Some Bayesian inference ◮ Using uninformative prior Q kl ∼ Ga(1 , 0): Q kl | Y ∼ Ga( N kl + 1 , O k ) ◮ Mean: ( N kl + 1) / O k ◮ Variance: ( N kl + 1) / O 2 k , hence decreasing with 1 / T ◮ Mode: N kl / O k 6 / 37

Markov Jump Processes Remarks on inference ◮ ˆ Q is well-known and looks nice ◮ In fact, estimation of Q is not so easy ◮ We need some observations for each transition k to l ◮ Basically, T needs to be as large as possible 7 / 37

Merged Markov Jump Processes Introduction Markov Jump Processes Merged Markov Jump Processes Inference for Merged Markov Jump Processes Generalization: Markov Switching Models Conclusion 8 / 37

Merged Markov Jump Processes Observing a number of series of MJPs ◮ M series of MJPs are observed ◮ All Y ( m ) characterized by same rate matrix Q ◮ Series may be independent or come from data with breaks 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0.05 0 0.05 0 0.05 0 0.05 0 0.05 Figure: Processes Y (1) , . . . , Y ( M ) 9 / 37

Merged Markov Jump Processes Observing merged MJPs ◮ Merged process of M single MJPs is observed ◮ Single MJPs characterized by same rate matrix Q 3 2 1 0 0.05 0.1 0.15 0.2 0.25 Figure: Merged process ¯ Y 10 / 37

Merged Markov Jump Processes Merged MJPs ◮ Given: Observation process ¯ Y = ( ¯ Y t ) t ∈ [0 , ¯ T ) ◮ ¯ Y is concatenation of MJPs Y (1) , . . . , Y ( M ) ◮ In detail, Y ( m ) = ( Y ( m ) ) t ∈ [0 , T ) , ¯ T = M T , and t Y t = Y (1) ¯ if 0 ≤ t < T , t Y t = Y (2) ¯ if T ≤ t < 2 T , t − T . . . Y t = Y ( m ) ¯ if ( m − 1) T ≤ t < mT t − mT ◮ All Y ( m ) characterized by same rate matrix Q ◮ NB: ¯ Y itself is not Markov! ◮ Assumption of equal length is for notational convenience only 11 / 37

Merged Markov Jump Processes Merged MJPs – Example � − 50 30 20 � M = 5, d = 3, T = 0 . 05, ¯ T = 0 . 25, Q = 20 − 40 20 30 40 − 70 3 2 1 0 0.05 0.1 0.15 0.2 0.25 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0.05 0 0.05 0 0.05 0 0.05 0 0.05 Figure: Merged process ¯ Y and single processes Y (1) , . . . , Y ( M ) 12 / 37

Inference for Merged Markov Jump Processes Introduction Markov Jump Processes Merged Markov Jump Processes Inference for Merged Markov Jump Processes Generalization: Markov Switching Models Conclusion 13 / 37

Inference for Merged Markov Jump Processes Splitting merged process ◮ Split ¯ Y into Y (1) , . . . , Y ( M ) 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0.05 0 0.05 0 0.05 0 0.05 0 0.05 ◮ Pooling: M Q kl = 1 ˆ � Q ( m ) ˆ kl M m =1 ◮ Problem: MLE does not exist if some O ( m ) = 0; k short occupation times lead to unstable results 14 / 37

Inference for Merged Markov Jump Processes Estimating directly from merged process ◮ Consider ¯ Y 3 2 1 0 0.05 0.1 0.15 0.2 0.25 ◮ Number of jumps and occupation time: M − 1 M ¯ � N ( m ) � + N + where N + N kl = kl , kl = � , I � kl Y ( m ) T − = k , Y ( m +1) = l 0 m =1 m =1 M ¯ � O ( m ) O k = k m =1 15 / 37

Inference for Merged Markov Jump Processes Estimating directly from merged process – first try ◮ First attempt: ˜ Q kl = ¯ N kl / ¯ O k ◮ Problem: Bias caused from “artificial” jumps N + kl / ¯ O k ◮ If N + kl is observed explicitly, things are easy ◮ We assume N + kl cannot be observed ◮ Location of splitting points unknown ◮ Process not directly observed 16 / 37

Inference for Merged Markov Jump Processes Estimating directly from merged process – bias ◮ Assume Y ( m ) ∼ π , where π stationary distribution 0 ◮ Extra jumps N + do not affect distribution of occupation times and stationary distribution π ( ˜ ◮ Hence, π = ¯ π and ˜ π = ˜ Q ) is “unbiased” estimate for π ◮ Joining two independent stationary processes generates a jump from k to l with probability π k π l : N + � � P kl | π = Bin( M − 1 , π k π l ) 17 / 37

Inference for Merged Markov Jump Processes Estimating directly from merged process – bias (2) N + ◮ P � � kl | π = Bin( M − 1 , π k π l ) is justified if ◮ Y ( m ) are independent series ◮ Data with breaks: Length of break τ such that ρ τ ( Y ) is close to zero, where P d k =1 π k ( X kk ( t ) − π k ) ρ t ( Y ) = P d k =1 π k (1 − π k ) and X kl ( t ) = P( Y t = l | Y 0 = k ) = exp( Q t ) kl I.e. X kk ( τ ) should be close to π k 18 / 37

Inference for Merged Markov Jump Processes Estimating directly from merged process – bias (3) kl and ¯ ◮ Note: If π is given, N + O k are independent N + ◮ Recall: P � � kl | π = Bin( M − 1 , π k π l ) ◮ As T 1 P − − − − → , ¯ π k O k T →∞ we have N + ≈ ( M − 1) π k π l M − 1 kl = π l ¯ π k T O k T ◮ As ˜ π is unbiased estimate for π , these quantities can be estimated knowing ˜ Q 19 / 37

Inference for Merged Markov Jump Processes Estimating directly from merged process – correction ◮ 2-step construction for corrected estimate: Q kl = ¯ ˜ N kl / ¯ 1) O k Q kl = ˜ ¯ π l / ¯ 2) Q kl − ( M − 1) ˜ T ◮ “Merging” 20 / 37

Inference for Merged Markov Jump Processes Comparison: Splitting vs. Merging ◮ Variance for Splitting for Ga(1 , 0) prior: M M N ( m ) � � 1 + 1 � � Q ( m ) � M − 1 � Y (1) , . . . , Y ( M ) kl Var = � kl � 2 M 2 O ( m ) � m =1 m =1 k ◮ Variance for Merging for Ga(1 , 0) prior: m =1 N ( m ) � M + 1 Q kl | ¯ � � kl Var = Y � 2 � � M m =1 O ( m ) k ◮ For rather short single observation times T we expect Merging to give more reliable results 21 / 37

Inference for Merged Markov Jump Processes Numerical example � − 100 60 40 � ◮ M = 100, d = 3, T = 0 . 25, ¯ T = 25, Q = 40 − 70 30 40 60 − 100 ◮ On average, about 22 jumps in each single process Y ( m ) ◮ Simulate merged data and apply Splitting and Merging ◮ Repeat 100 000 times and consider sampling distributions of ˆ Q kl and ¯ Q kl � − 2 . 8 1 . 8 1 . 0 � 0 . 25) and ¯ Q = ˜ ◮ π = (0 . 29 0 . 46 Q − 1 . 1 − 2 . 1 1 . 0 1 . 1 1 . 8 − 2 . 9 22 / 37

Inference for Merged Markov Jump Processes Numerical example – results 0.15 0.2 0.1 0.1 0.1 0.05 0.05 0 0 0 −140 −120 −100 −80 50 70 90 30 40 50 60 0.2 0.2 0.2 0.1 0.1 0.1 0 0 0 30 40 50 −90 −80 −70 −60 20 30 40 0.2 0.15 0.1 0.1 0.1 0.05 0.05 0 0 0 30 40 50 60 50 70 90 −140 −120 −100 −80 Green circles: true Q kl , blue: Merging ¯ Q kl , red: Splitting ˆ Q kl 23 / 37

Inference for Merged Markov Jump Processes Numerical example – remarks ◮ ¯ Q kl outperforms ˆ Q kl wrt. both location and dispersion ◮ ˆ Q kl is skewed towards over-estimation: If some O ( m ) is very small, ˆ Q ( m ) heavily over-estimates Q kl and k kl also ˆ Q kl is too high ◮ This is the more severe, the smaller T is 24 / 37

Inference for Merged Markov Jump Processes Numerical example (2) � − 800 450 350 � ◮ M = 62, d = 3, T = 1 / M , ¯ T = 1, Q = 100 − 200 100 300 400 − 700 ◮ On average, about 6 jumps in each single process Y ( m ) ◮ Probability that one state is visited never or only for a very short time is high! ◮ Simulate merged data and apply Splitting and Merging ◮ Repeat 100 000 times and consider sampling distributions of ˆ Q kl and ¯ Q kl � − 51 . 8 41 . 4 10 . 4 � 0 . 17) and ¯ Q = ˜ ◮ π = (0 . 15 0 . 68 Q − 9 . 1 − 19 . 6 10 . 5 9 . 1 41 . 4 − 50 . 5 25 / 37