Calculating bounds on expected return and first passage times in - PowerPoint PPT Presentation

Calculating bounds on expected return and first passage times in finite-state imprecise birth-death chains Stavros Lopatatzidis, Jasper De Bock and Gert de Cooman

Birth-death chains Birth-death chain special type of Markov chain X : = { 0 ,..., L } Finite state space , with L ∈ N X k : n Random variable and a sequence of variables , with and X n k , n ∈ N k ≤ n A sequence can be infinite as well X k : ∞ x 1: n : = x 1 ,..., x n Sequence of state values in X n E n + 1 ( ·| x 1: n ) = E n + 1 ( ·| x n ) , ∀ x 1: n ∈ X n Markov condition p ( X n + 1 | x n ) E n + 1 ( ·| x n ) where is the expectation operator with p.m.f 0 1 r 0 p 0 0 · · · · · · 0 q 1 r 1 p 1 0 · · · 0 B C . . B C ... ... ... ... for time-homogeneous . . P = B C p . . B C B C 0 · · · 0 q L − 1 r L − 1 p L − 1 @ A 0 · · · · · · 0 q L r L

Imprecise birth-death chains Consider a matrix with p.m.f. not precisely known P For every , the p.m.f. of the row belong to a credal set i ∈ X M i i φ i and consists of elements of the form  if j = i − 1 q i   if j = L − 1 if j = 0  q L r 0    if j = i  r i    φ L ( j ) = φ 0 ( j ) = if j = L φ i ( j ) = i ∈ X \{ 0 , L } if j = 1 r L p 0 if j = i + 1 p i    0 otherwise 0 otherwise      0 otherwise i ∈ X \{ 0 , L } q i , r i , p i Positivity assumption: and for all strictly positive r 0 , p 0 , r L , q L

Imprecise Markov condition f Lower and upper expectations of real-valued function on X ⇢ � ∑ E ( f | i ) : = min E φ i ( f ) = min φ i ( j ) f ( j ) φ i ∈ M i φ i ∈ M i j ∈ X ⇢ � ∑ E ( f | i ) : = max E φ i ( f ) = max φ i ( j ) f ( j ) φ i ∈ M i φ i ∈ M i j ∈ X x 1: n ∈ X n and for all , the imprecise Markov condition is E n + 1 ( ·| x 1: n ) = E n + 1 ( ·| x n ) : = E ( ·| x n )

Global uncertainty models Based on the notion of submartingales, we derive global uncertainty models These models satisfy a version of the Law of Iterated expectation X N For every and every real-valued function on n ∈ N g (time-homogeneity) E n +1: 1 ( g ( X n +1: 1 ) | i ) = E n +2: 1 ( g ( X n +2: 1 ) | i ) . f 0 i 0 2 X f 0 ( i 0 ) : = E n + 2: ∞ ( g ( i 0 , X n + 2: ∞ ) | i 0 ) By defining on by for all , then X E n + 1: ∞ ( g ( X n + 1: ∞ ) | i ) = E n + 1 ( f 0 | i ) = E ( f 0 | i )

First passage time The first passage time from to with is i , j ∈ X j i ( X n + 1 = j 1 τ i ! j ( i , X n + 1: ∞ ) : = 1 + τ X n + 1 ! j ( X n + 1 , X n + 2: ∞ ) X n + 1 6 = j = 1 + I j c ( X n + 1 ) τ X n + 1 ! j ( X n + 1 , X n + 2: ∞ ) where is the indicator function of j c : = X \{ j } I j c For , we have the return time i = j τ i → j , n : = E n + 1: ∞ ( τ i → j ( i , X n + 1: ∞ ) | i ) Due to time-homogeneity and τ i → j , n : = E n + 1: ∞ ( τ i → j ( i , X n + 1: ∞ ) | i ) will be denoted by and τ i → j τ i → j Due to positivity assumption and are real-valued and strictly positive τ i → j τ i → j τ i → j = 1 + E ( I j c τ • → j | i ) τ i → j = 1 + E ( I j c τ • → j | i ) and have the form and

Lower expected upward first passage time The first passage time from to with and i , j ∈ X j i i < j τ 0 → 1 = 1 p 0 i ∈ X \{ 0 , L } For all , we have that { q i τ i − 1 → i − p i τ i → i + 1 } = − 1 min φ i ∈ M i M i For all satisfying the positivity assumption, with , i ∈ X \{ 0 , L } and a real constant, then is strictly decreasing in { qc − p µ } min c µ φ i ∈ M i

Lower expected upward first passage time { q i τ i − 1 → i − p i τ i → i + 1 } = − 1 min φ i ∈ M i We can calculate recursively τ i → i + 1 Using a bisection method, as long as we have calculated … τ i − 1 → i Moreover, For all , s.t , we have that i ∈ X \{ 0 , L } i + 1 < j τ i → j = τ i → i + 1 + τ i + 1 → j j − 1 ∑ For all , such that , we have that i ∈ X τ i → j = i < j τ k → k + 1 k = i

Lower expected downward first passage time The first passage time from to with and i , j ∈ X j i i > j Similarly to the upward case… τ L → L − 1 = 1 q L For all , we have that { − q i τ i → i − 1 + p i τ i + 1 → i } = − 1 i ∈ X \{ 0 , L } min φ i ∈ M i i − 1 ∑ i ∈ X For all , such that , we have that i > j τ i → j = τ k + 1 → k k = j

Lower expected return time The first passage time from to with and i , j ∈ X j i i = j Combining the results from expected upward with these of downward first passage times { p 0 τ 1 → 0 } = 1 + p 0 τ 1 → 0 τ 0 → 0 = 1 + min φ 0 ∈ M 0 { q L τ L − 1 → L } = 1 + q L τ L − 1 → L τ L → L = 1 + min φ L ∈ M L and for all i ∈ X \{ 0 , L } { q i τ i − 1 → i + p i τ i + 1 → i } τ i → i = 1 + min φ i ∈ M i

Linear vacuous mixtures The set is a subset of the simplex M i Σ X For any , is the subset of containing p.m.f. Σ X i φ i i ∈ X Σ X ε i ∈ [ 0 , 1 ) Given precise and for any φ ∗ 0 , φ ∗ L , φ ∗ i ∈ X i � ( 1 � ε 0 ) φ ⇤ 0 + ε 0 φ 0 0 : φ 0 M 0 = 0 2 Σ X 0 � ( 1 � ε L ) φ ⇤ L + ε L φ 0 L : φ 0 M L = L 2 Σ X L and for all i ∈ X \{ 0 , L } � ( 1 � ε i ) φ ⇤ i + ε i φ 0 i : φ 0 M i = i 2 Σ X i

Linear vacuous mixtures We can also define q i : = ( 1 − ε i ) q ∗ q i : = ( 1 − ε i ) q ∗ i + ε i i ∈ X \{ 0 } and for all i p i : = ( 1 − ε i ) p ∗ i ∈ X \{ L } and for all p i : = ( 1 − ε i ) p ∗ i + ε i i Expected lower upward, downward first passage and return times ∏ i ∏ k − 1 ` = k + 1 q ` ` = i p ` τ i → i + 1 = ∑ i τ i → i − 1 = ∑ L k = 0 ∏ i k = i ∏ k m = k p m m = i q m τ i → i = 1 + q i τ i − 1 → i + p i τ i + 1 → i

Linear vacuous mixtures Q π ∗ X : = { 0 ,..., 4 } Consider state space , and ε i = ε = 0 . 4 q   0 . 55 0 . 45 0 0 0 then, for all 0 . 3 0 . 5 0 . 2 0 0   P ∗ =   0 0 . 3 0 . 5 0 . 2 0 i ∈ X \{ 0 , L } π ∗     0 0 0 . 3 0 . 5 0 . 2   0 0 0 0 . 6 0 . 4 p r we calculate lower and upper expected return times i τ i → i τ i → i 0 1.584 91.41 1 1.526 24.956 2 1.678 17.845 3 1.656 79.71 4 2.037 503.724

General example X : = { 0 ,..., 4 } Consider state space p 0 ∈ [ 0 . 15 , 0 . 4 ] q L ∈ [ 0 . 2 , 0 . 6 ] M 0 M L is determined by and by ( q i , r i , p i ) i ∈ X \{ 0 , L } M i For all , is characterised by triplets of the form q (0 . 65 , 0 . 15 , 0 . 2) , (0 . 6 , 0 . 25 , 0 . 15) , (0 . 5 , 0 . 4 , 0 . 1) , (0 . 43 , 0 . 45 , 0 . 12) , (0 . 33 , 0 . 5 , 0 . 17) , (0 . 27 , 0 . 43 , 0 . 3) , ⇒ (0 . 25 , 0 . 35 , 0 . 4) , (0 . 3 , 0 . 25 , 0 . 45) , (0 . 4 , 0 . 17 , 0 . 43) , (0 . 55 , 0 . 1 , 0 . 35) p r lower and upper expected upward and downward first passage times 2.5 1.666 τ 0 → 1 τ 4 → 3 3.889 2.051 τ 1 → 2 τ 3 → 2 4.814 2.169 τ 2 → 3 τ 2 → 1 5.432 2.206 τ 3 → 4 τ 1 → 0 6.666 5 τ 0 → 1 τ 4 → 3 43.333 12 τ 1 → 2 τ 3 → 2 226.666 23.2 τ 2 → 3 τ 2 → 1 1143.333 41.12 τ 3 → 4 τ 1 → 0

Conclusions and future work Simple methods for computing lower and upper expected first passage and return times Applying similar methods to other type of chains, e.g. Bonus-Malus systems Applying similar methods to continuous time systems