Imprecise Markov chains by Jasper De Bock & Thomas Krak - PowerPoint PPT Presentation

A tutorial on Imprecise Markov chains by Jasper De Bock & Thomas Krak SMPS/BELIEF 2018 September 17 now :-)

by Jasper De Bock & Thomas Krak

A tutorial on Imprecise Markov chains by Jasper De Bock & Thomas Krak ? SMPS/BELIEF 2018 September 17 now :-)

(Walley 1991) (Augustin et al. 2014) A tutorial on Imprecise Markov chains by Jasper De Bock & Thomas Krak ? SMPS/BELIEF 2018 September 17 now :-)

( E ( f ) = sup E ( f ) = − E ( − f ) P ∈ P P ∈ P E ( f ) = inf P ∈ P E ( f ) A tutorial on Imprecise Markov chains by Jasper De Bock & Thomas Krak ? SMPS/BELIEF 2018 September 17 now :-)

A tutorial on Imprecise Markov chains by Jasper De Bock & Thomas Krak SMPS/BELIEF 2018 September 17 now :-)

A tutorial on Imprecise Markov chains by Jasper De Bock & Thomas Krak SMPS/BELIEF 2018 September 17 now :-)

-time stochastic process X 0 X t t 0

R ≥ 0 n o continuous -time stochastic process discrete N X 0 X t t 0

R ≥ 0 n o continuous -time stochastic process discrete N P ( X 0 = x 0 ) P ( X t n +1 = y | X t 1 = x t 1 , ..., X t n − 1 = x t n − 1 , X t n = x ) X 0 X t 1 X t n − 1 X t n X t n +1 t n t n +1 0 t 1 t n − 1

R ≥ 0 n o continuous -time Markov chain discrete N P ( X 0 = x 0 ) P ( X t n +1 = y | X t 1 = x t 1 , ..., X t n − 1 = x t n − 1 , X t n = x ) = P ( X t n +1 = y | X t 1 = X t n = x ) X 0 X t 1 X t n − 1 X t n X t n +1 t n t n +1 0 t 1 t n − 1

R ≥ 0 homogeneous n o � continuous -time Markov chain discrete N P ( X 0 = x 0 ) P ( X t n +1 = y | X t 1 = x t 1 , ..., X t n − 1 = x t n − 1 , X t n = x ) = P ( X t n +1 = y | X t 1 = X t n = x ) only the time difference ∆ = t n +1 − t n matters! = T ∆ ( x, y ) X 0 X t 1 X t n − 1 X t n X t n +1 t n t n +1 0 t 1 t n − 1

R ≥ 0 homogeneous n o � continuous -time Markov chain discrete N P ( X 0 = x 0 ) P ( X t n +1 = y | X t 1 = x t 1 , ..., X t n − 1 = x t n − 1 , X t n = x ) = P ( X t n +1 = y | X t 1 = X t n = x ) only the time difference ∆ = t n +1 − t n matters! = T ∆ ( x, y ) X 0 X t 1 X t n − 1 X t n X t n +1 t n t n +1 0 t 1 t n − 1

R ≥ 0 homogeneous n o � continuous -time Markov chain discrete N that’s just a probability P ( X 0 = x 0 ) π 0 ( x 0 ) mass function initial distribution = T ∆ ( x t n , x t n +1 ) X 0 X t 1 X t n − 1 X t n X t n +1 t n t n +1 0 t 1 t n − 1

R ≥ 0 homogeneous n o � continuous -time Markov chain discrete N that’s just a probability P y T ( x, y ) = 1 P ( X 0 = x 0 ) π 0 ( x 0 ) mass function ( ∀ y ) T ( x, y ) ≥ 0 initial distribution transition matrix = T ∆ = T ∆ ( x t n , x t n +1 ) T := T 1 with X 0 X t 1 X t n − 1 X t n X t n +1 t n t n +1 0 t 1 t n − 1

  0 . 6 0 . 3 0 . 1 T = 0 . 2 0 . 3 0 . 5   0 . 4 0 . 1 0 . 5 P y T ( x, y ) = 1 ( ∀ y ) T ( x, y ) ≥ 0 transition matrix = T ∆ = T ∆ ( x t n , x t n +1 ) T := T 1 with X 0 X t 1 X t n − 1 X t n X t n +1 t n t n +1 0 t 1 t n − 1

R ≥ 0 homogeneous n o � continuous -time Markov chain discrete N that’s just a probability P ( X 0 = x 0 ) π 0 ( x 0 ) mass function initial distribution X 0 X t 1 X t n − 1 X t n X t n +1 t n t n +1 0 t 1 t n − 1

R ≥ 0 homogeneous n o � continuous -time Markov chain discrete N P y Q ( x, y ) = 0 that’s just a probability P ( X 0 = x 0 ) π 0 ( x 0 ) mass function ( 8 y 6 = x ) Q ( x, y ) � 0 initial distribution transition rate matrix ⌘ n T ∆ − I I + t ⇣ T ∆ = e Q ∆ := lim nQ Q := lim with ∆ n →∞ ∆ → 0 X 0 X t 1 X t n − 1 X t n X t n +1 t n t n +1 0 t 1 t n − 1

epression onfusion ickering morous P y Q ( x, y ) = 0 ( 8 y 6 = x ) Q ( x, y ) � 0 transition rate matrix 2 T ∆ − I Q := lim 2 ∆ ∆ → 0 1 3 4 3 1 2

R ≥ 0 homogeneous n o � continuous -time Markov chain discrete N ( transition rate Q matrix initial or π 0 ( x 0 ) distribution transition matrix T X 0 X t t 0

E ( f ( X t ) | X 0 = x ) = [ T t f ]( x ) (P y e Qt ( x, y ) f ( y ) X = T t ( x, y ) f ( y ) = P y T t ( x, y ) f ( y ) y = x = 0 f ( X t ) = I ( X t ) = X 0 X t t 0

E ( f ( X t ) | X 0 = x ) = [ T t f ]( x ) (P y e Qt ( x, y ) f ( y ) X = T t ( x, y ) f ( y ) = P y T t ( x, y ) f ( y ) y P ( X t = y | X 0 = x ) = E ( I y ( X t ) | X 0 = x ) = [ T t I y ]( x ) y = x = 0 ( 1 if X t = y f ( X t ) = I y ( X t ) = 0 otherwise X 0 X t t 0

E ( f ( X t ) | X 0 = x ) = [ T t f ]( x ) (P X E ∞ ( f ):= t → + ∞ E ( f ( X t ) | X 0 = x ) lim P ( X t = y | X 0 = x ) = E ( I y ( X t ) | X 0 = x ) = [ T t I y ]( x ) π ∞ ( y ):= t → + ∞ P ( X t = y | X 0 = x ) lim X 0 X t t 0

Reliability engineering (failure probabilities, …) Queuing theory (waiting in line …) - optimising supermarket waiting times - dimensioning of call centers - airport security lines - router queues on the internet Chemical reactions (time-evolution …) Pagerank …

So how about imprecision?

So how about imprecision? What if we don’t know or M Q T exactly?

Sets of transition (rate) matrices Don’t know T (or Q ) exactly But confident that T ∈ T for some set T of transition matrices (or that Q ∈ Q for some set Q of rate matrices) Induces imprecise Markov chain ; set of processes compatible with T . Jasper De Bock, Thomas Krak Imprecise Markov Chains

Sets of transition (rate) matrices Don’t know T (or Q ) exactly But confident that T 2 T for some set T of transition matrices (or that Q 2 Q for some set Q of rate matrices) Induces imprecise Markov chain ; set of processes compatible with T . Di ff erent versions: P HM T : all homogeneous Markov chains with T 2 T Jasper De Bock, Thomas Krak Imprecise Markov Chains

Sets of transition (rate) matrices Don’t know T (or Q ) exactly But confident that T ∈ T for some set T of transition matrices (or that Q ∈ Q for some set Q of rate matrices) Induces imprecise Markov chain ; set of processes compatible with T . Di ff erent versions: P HM T : all homogeneous Markov chains with T ∈ T T : all ( non -homogeneous) Markov chains with T ( t ) ∈ T P M Jasper De Bock, Thomas Krak Imprecise Markov Chains

Sets of transition (rate) matrices Don’t know T (or Q ) exactly But confident that T ∈ T for some set T of transition matrices (or that Q ∈ Q for some set Q of rate matrices) Induces imprecise Markov chain ; set of processes compatible with T . Di ff erent versions: P HM T : all homogeneous Markov chains with T ∈ T T : all ( non -homogeneous) Markov chains with T ( t ) ∈ T P M P T : all ( non -Markov) processes with T ( t , x u ) ∈ T Jasper De Bock, Thomas Krak Imprecise Markov Chains

Sets of transition (rate) matrices Don’t know T (or Q ) exactly But confident that T ∈ T for some set T of transition matrices (or that Q ∈ Q for some set Q of rate matrices) Induces imprecise Markov chain ; set of processes compatible with T . Di ff erent versions: P HM T : all homogeneous Markov chains with T ∈ T T : all ( non -homogeneous) Markov chains with T ( t ) ∈ T P M P T : all ( non -Markov) processes with T ( t , x u ) ∈ T Clearly P HM ⊆ P M T ⊆ P T T Jasper De Bock, Thomas Krak Imprecise Markov Chains

Lower expectations and lower probabilities Given an imprecise Markov chain P ∗ T , we are interested in E ∗ ⇥ ⇤ ⇥ ⇤ f ( X t ) | X 0 = x = inf f ( X t ) | X 0 = x E P T P ∈ P ∗ T ∗ ⇥ ⇤ (And E f ( X t ) | X 0 = x by conjugacy) T Lower- (and upper) probabilities a special case: P ∗ = E ∗ � � � � ⇥ ⇤ X t = y | X 0 = x = inf P X t = y | X 0 = x I y ( X t ) | X 0 = x T T P ∈ P ∗ T Because di ff erent types are nested, ⇥ ⇤ ≤ E M ⇥ ⇤ ≤ E HM ⇥ ⇤ f ( X t ) | X 0 = x f ( X t ) | X 0 = x f ( X t ) | X 0 = x E T T T Jasper De Bock, Thomas Krak Imprecise Markov Chains

Computing lower expectations, first try Recall that for a homogeneous Markov chain P with transition matrix T , ⇥ ⇤ ⇥ ⇤ E P f ( X 1 ) | X 0 = x = Tf ( x ) . Now consider P HM T . Then, E HM ⇥ ⇤ ⇥ ⇤ f ( X 1 ) | X 0 = x := inf E P f ( X 1 ) | X 0 = x T P ∈ P HM T ⇥ ⇤ = inf Tf ( x ) T ∈ T Linear optimisation problem with constraints given by T Relatively straightforward if T is “nice” Essentially solving a linear programming problem Jasper De Bock, Thomas Krak Imprecise Markov Chains

Computing lower expectations, first try Recall that for a homogeneous Markov chain P with transition matrix T , ⇥ ⇤ ⇥ T t f ⇤ f ( X t ) | X 0 = x = ( x ) . E P Now consider P HM T . Then, E HM ⇥ ⇤ ⇥ ⇤ f ( X t n ) | X 0 = x := inf f ( X t ) | X 0 = x E P T P ∈ P HM T T t f ⇥ ⇤ = inf ( x ) T ∈ T Non -linear optimisation problem with constraints given by T Not straightforward even if T is “nice” Jasper De Bock, Thomas Krak Imprecise Markov Chains