Markov chains and Markov decision processes in Isabelle/HOL - PowerPoint PPT Presentation

n n N n N n N U LEAST n n f n n c c lfp 1 tl 0 lfp otherwise 11 if lfp 1 tl gfp tl if tl lfp tl otherwise tl Queries on the trace space ∃ n . φ ω n Eventually φ : ♢ φ ω = Always φ : □ φ ω = ψ U φ ω ψ Until φ : = f φ ω = First hit φ : c φ ω Counting φ : =

N n N n N U LEAST n n f n n c c 1 lfp otherwise tl if 0 otherwise tl lfp lfp 1 tl if tl tl lfp tl gfp 11 Queries on the trace space ∃ n . φ ω n Eventually φ : ♢ φ ω = ∀ n . φ ω n Always φ : □ φ ω = ψ U φ ω ψ Until φ : = f φ ω = First hit φ : c φ ω Counting φ : =

U LEAST n n f n n c c if tl 1 lfp otherwise 0 if tl tl 1 otherwise lfp 11 tl lfp gfp lfp tl tl Queries on the trace space ∃ n . φ ω n Eventually φ : ♢ φ ω = ∀ n . φ ω n Always φ : □ φ ω = ψ U φ ω ∃ N . ( ∀ n < N . ψ ω n ) ∧ φ ω N ψ Until φ : = f φ ω = First hit φ : c φ ω Counting φ : =

U f n n c c 11 tl if 0 tl lfp otherwise tl lfp 1 gfp 1 tl if tl lfp tl otherwise lfp Queries on the trace space ∃ n . φ ω n Eventually φ : ♢ φ ω = ∀ n . φ ω n Always φ : □ φ ω = ψ U φ ω ∃ N . ( ∀ n < N . ψ ω n ) ∧ φ ω N ψ Until φ : = f φ ω LEAST n . φ ω n = First hit φ : c φ ω Counting φ : =

U f c c 11 lfp tl tl lfp if 0 otherwise gfp tl lfp 1 tl if tl lfp tl otherwise 1 Queries on the trace space ∃ n . φ ω n Eventually φ : ♢ φ ω = ∀ n . φ ω n Always φ : □ φ ω = ψ U φ ω ∃ N . ( ∀ n < N . ψ ω n ) ∧ φ ω N ψ Until φ : = f φ ω LEAST n . φ ω n = First hit φ : ∑ c φ ω Counting φ : = n ⌊ φ ω n ⌋

U f c c 11 tl tl lfp tl if 0 otherwise gfp lfp lfp 1 tl if lfp tl otherwise 1 Queries on the trace space ∃ n . φ ω n Eventually φ : ♢ φ ω = = φ ω ∨ ♢ φ ( tl ω ) ∀ n . φ ω n Always φ : □ φ ω = ψ U φ ω ∃ N . ( ∀ n < N . ψ ω n ) ∧ φ ω N ψ Until φ : = f φ ω LEAST n . φ ω n = First hit φ : ∑ c φ ω Counting φ : = n ⌊ φ ω n ⌋

U f c c gfp lfp lfp 1 tl if 0 11 otherwise tl 1 tl if lfp tl otherwise lfp Queries on the trace space ∃ n . φ ω n Eventually φ : ♢ φ ω = = φ ω ∨ ♢ φ ( tl ω ) ∀ n . φ ω n Always φ : □ φ ω = = φ ω ∧ □ φ ( tl ω ) ψ U φ ω ∃ N . ( ∀ n < N . ψ ω n ) ∧ φ ω N ψ Until φ : = f φ ω LEAST n . φ ω n = First hit φ : ∑ c φ ω Counting φ : = n ⌊ φ ω n ⌋

f c c 11 otherwise 1 tl if 0 gfp 1 lfp lfp tl if lfp tl otherwise lfp Queries on the trace space ∃ n . φ ω n Eventually φ : ♢ φ ω = = φ ω ∨ ♢ φ ( tl ω ) ∀ n . φ ω n Always φ : □ φ ω = = φ ω ∧ □ φ ( tl ω ) ψ U φ ω ∃ N . ( ∀ n < N . ψ ω n ) ∧ φ ω N ψ Until φ : = ( ) ψ U φ ( tl ω ) = ψ ω ∧ ∨ φ ω f φ ω LEAST n . φ ω n = First hit φ : ∑ c φ ω Counting φ : = n ⌊ φ ω n ⌋

c c 11 lfp otherwise tl lfp if tl 1 lfp otherwise gfp 0 lfp Queries on the trace space ∃ n . φ ω n Eventually φ : ♢ φ ω = = φ ω ∨ ♢ φ ( tl ω ) ∀ n . φ ω n Always φ : □ φ ω = = φ ω ∧ □ φ ( tl ω ) ψ U φ ω ∃ N . ( ∀ n < N . ψ ω n ) ∧ φ ω N ψ Until φ : = ( ) ψ U φ ( tl ω ) = ψ ω ∧ ∨ φ ω f φ ω LEAST n . φ ω n = First hit φ : { 1 + f φ ( tl ω ) if ¬ φ ω = ∑ c φ ω Counting φ : = n ⌊ φ ω n ⌋

11 gfp otherwise lfp otherwise lfp 0 lfp lfp Queries on the trace space ∃ n . φ ω n Eventually φ : ♢ φ ω = = φ ω ∨ ♢ φ ( tl ω ) ∀ n . φ ω n Always φ : □ φ ω = = φ ω ∧ □ φ ( tl ω ) ψ U φ ω ∃ N . ( ∀ n < N . ψ ω n ) ∧ φ ω N ψ Until φ : = ( ) ψ U φ ( tl ω ) = ψ ω ∧ ∨ φ ω f φ ω LEAST n . φ ω n = First hit φ : { 1 + f φ ( tl ω ) if ¬ φ ω = ∑ c φ ω Counting φ : = n ⌊ φ ω n ⌋ { 1 + c φ ( tl ω ) if φ ω = c φ ( tl ω )

C X monotone C f i C i i f C i g g f f g g f f g g f f g g f f g g lfp f f f f f f g g g f f f f d for f Borel-measurable g g g g g lfp g Rolling rule: g lfp f g lfp g f Iteration rule: lfp f f lfp f Nesting rule: lfp x lfp f x x f x x f g f g Transfer rule: lfp f lfp g 12 continuous lfp Interception: Least/Greatest Fixed Points Monotone functions f , g Least fixed point: lfp f = f ( lfp f ) ( ∀ x . f x ⩽ x = ⇒ lfp f ⩽ x ) ∞ � �� � lfp f = f ◦ f ◦ f ◦ f ◦ f ◦ f ◦ f ◦ f ◦ f ◦ · · · ◦ ⊥

C X monotone C f i C i i f C i g f g f g f g f g lfp f f f f f f g g g f f f f d for f Borel-measurable g g g g g lfp g Iteration rule: lfp f f lfp f Nesting rule: lfp x lfp f x x f x x f g f g Transfer rule: lfp f f f f f f f f f f lfp f lfp g 12 continuous lfp Interception: Least/Greatest Fixed Points Monotone functions f , g Least fixed point: lfp f = f ( lfp f ) ( ∀ x . f x ⩽ x = ⇒ lfp f ⩽ x ) Rolling rule: g ( lfp ( f ◦ g )) = lfp ( g ◦ f ) ���� ���� ���� ���� g ◦ f ◦ g ◦ f ◦ g ◦ f ◦ g ◦ f ◦ g ◦ . . .

C X monotone C f i C i i f C i lfp f f f f f f g g g f f f f d for f Borel-measurable g g g g g lfp g Iteration rule: lfp f f lfp f Nesting rule: lfp x lfp f x x f x x f g f g g f g f g f g f g Transfer rule: lfp f f f f f f f f f f lfp f lfp g 12 continuous lfp Interception: Least/Greatest Fixed Points Monotone functions f , g Least fixed point: lfp f = f ( lfp f ) ( ∀ x . f x ⩽ x = ⇒ lfp f ⩽ x ) Rolling rule: g ( lfp ( f ◦ g )) = lfp ( g ◦ f ) ���� ���� ���� ���� g ◦ f ◦ g ◦ f ◦ g ◦ f ◦ g ◦ f ◦ g ◦ . . .

C X monotone C f i C i i f C i lfp f f f f f f g g g f f f f d for f Borel-measurable g g g g g lfp g Nesting rule: lfp x lfp f x x f x x f g f g g g f f g g f f g g f f g g f f g g Transfer rule: lfp f f f f f f f f f f lfp f lfp g 12 continuous lfp Interception: Least/Greatest Fixed Points Monotone functions f , g Least fixed point: lfp f = f ( lfp f ) ( ∀ x . f x ⩽ x = ⇒ lfp f ⩽ x ) Rolling rule: g ( lfp ( f ◦ g )) = lfp ( g ◦ f ) Iteration rule: lfp ( f ◦ f ) = lfp f

lfp f f f f f f g g g f f f f d for f Borel-measurable g g g g g lfp g C X monotone C f i C i i f C i f g f g g g f f g g f f g g f f g g f f g g Transfer rule: lfp f f f f f f f f f f lfp f lfp g continuous 12 Interception: Least/Greatest Fixed Points Monotone functions f , g Least fixed point: lfp f = f ( lfp f ) ( ∀ x . f x ⩽ x = ⇒ lfp f ⩽ x ) Rolling rule: g ( lfp ( f ◦ g )) = lfp ( g ◦ f ) Iteration rule: lfp ( f ◦ f ) = lfp f Nesting rule: lfp ( λ x . lfp ( f x )) = lfp ( λ x . f x x )

g g f f g g f f g g f f g g f f g g lfp f f f f f f f f f f f f d for f Borel-measurable 12 Interception: Least/Greatest Fixed Points ∀ C ∈ N → X . monotone C = ⇒ f ( ⊔ i C i ) = ⊔ i f C i Monotone functions f , g Least fixed point: lfp f = f ( lfp f ) ( ∀ x . f x ⩽ x = ⇒ lfp f ⩽ x ) Rolling rule: g ( lfp ( f ◦ g )) = lfp ( g ◦ f ) Iteration rule: lfp ( f ◦ f ) = lfp f Nesting rule: lfp ( λ x . lfp ( f x )) = lfp ( λ x . f x x ) ⊔ − continuous α , f , g α ◦ f = g ◦ α α ⊥ = ⊥ Transfer rule: α ( lfp f ) = lfp g α ( lfp f ) α ◦ f ◦ f ◦ f ◦ f ◦ f ◦ · · · ◦ ⊥ = g ◦ g ◦ g ◦ α ◦ f ◦ f ◦ · · · ◦ ⊥ = g ◦ g ◦ g ◦ g ◦ g ◦ · · · ◦ α ⊥ = lfp g =

C X monotone C f i C i i f C i g g f f g g f f g g f f g g f f g g lfp f f f f f f f f f f lfp f f f f f f g g g f f f f d for f Borel-measurable g g g g g lfp g 12 Interception: Least/Greatest Fixed Points Monotone functions f , g Least fixed point: lfp f = f ( lfp f ) ( ∀ x . f x ⩽ x = ⇒ lfp f ⩽ x ) Rolling rule: g ( lfp ( f ◦ g )) = lfp ( g ◦ f ) Iteration rule: lfp ( f ◦ f ) = lfp f Nesting rule: lfp ( λ x . lfp ( f x )) = lfp ( λ x . f x x ) ⊔ − continuous α , f , g α ◦ f = g ◦ α α ⊥ = ⊥ Transfer rule: α ( lfp f ) = lfp g

C X monotone C f i C i i f C i g g f f g g f f g g f f g g f f g g lfp f f f f f f f f f f lfp f f f f f f g g g f f g g g g g lfp g 12 Interception: Least/Greatest Fixed Points Monotone functions f , g Least fixed point: lfp f = f ( lfp f ) ( ∀ x . f x ⩽ x = ⇒ lfp f ⩽ x ) Rolling rule: g ( lfp ( f ◦ g )) = lfp ( g ◦ f ) Iteration rule: lfp ( f ◦ f ) = lfp f Nesting rule: lfp ( λ x . lfp ( f x )) = lfp ( λ x . f x x ) ⊔ − continuous α , f , g α ◦ f = g ◦ α α ⊥ = ⊥ Transfer rule: α ( lfp f ) = lfp g ∫ α f = f d M for f Borel-measurable

f s f s f d lfp g s t g t dK s s s t 13 For finite state space: lfp is a system of linear equations! 1 otherwise Prove computation rule by transfer rule: 0 if 1 lfp def Equation for queries under integration Example (First hitting time φ on states) Define f : f φ = lfp ( λ f ( s · ω ) . ⌊ ¬ φ s ⌋ · ( 1 + f ω ))

f d lfp g s t g t dK s s s t 13 For finite state space: lfp is a system of linear equations! 1 Prove computation rule by transfer rule: otherwise 0 lfp def Equation for queries under integration Example (First hitting time φ on states) Define f : f φ = lfp ( λ f ( s · ω ) . ⌊ ¬ φ s ⌋ · ( 1 + f ω )) { 1 + f φ ω if ¬ φ s f φ ( s · ω ) =

13 lfp For finite state space: lfp is a system of linear equations! Prove computation rule by transfer rule: 0 otherwise def Equation for queries under integration Example (First hitting time φ on states) Define f : f φ = lfp ( λ f ( s · ω ) . ⌊ ¬ φ s ⌋ · ( 1 + f ω )) { 1 + f φ ω if ¬ φ s f φ ( s · ω ) = ∫ ∫ ( ) f φ ω d T s = lfp λ g s . ⌊ ¬ φ t ⌋ · ( 1 + g t ) dK s s t ω

13 lfp For finite state space: lfp is a system of linear equations! Prove computation rule by transfer rule: 0 otherwise def Equation for queries under integration Example (First hitting time φ on states) Define f : f φ = lfp ( λ f ( s · ω ) . ⌊ ¬ φ s ⌋ · ( 1 + f ω )) { 1 + f φ ω if ¬ φ s f φ ( s · ω ) = ∫ ∫ ( ) f φ ω d T s = lfp λ g s . ⌊ ¬ φ t ⌋ · ( 1 + g t ) dK s s t ω

Lemma (Finite hitting time) f t d if Pr s t 1 and finite state space s ! 65 Proof size is reduced to 14 Proofs employing fixed point reasoning Lemma (Fairness) ( ) if t ′ ∈ K t □♢ t = ⇒ □♢ ( t ∧ ⃝ t ′ ) = 1 Pr s Proof. Show that gfp ( λ g s . ( ¬ t ) U ( t · ¬ t ′ · g )) has probability 0.

14 Proof size is reduced to ! 65 Proofs employing fixed point reasoning Lemma (Fairness) ( ) if t ′ ∈ K t □♢ t = ⇒ □♢ ( t ∧ ⃝ t ′ ) = 1 Pr s Proof. Show that gfp ( λ g s . ( ¬ t ) U ( t · ¬ t ′ · g )) has probability 0. Lemma (Finite hitting time) ∫ f t ω d T s < ∞ if Pr s ( ♢ t ) = 1 and finite state space ω

14 Proofs employing fixed point reasoning Lemma (Fairness) ( ) if t ′ ∈ K t □♢ t = ⇒ □♢ ( t ∧ ⃝ t ′ ) = 1 Pr s Proof. Show that gfp ( λ g s . ( ¬ t ) U ( t · ¬ t ′ · g )) has probability 0. Lemma (Finite hitting time) ∫ f t ω d T s < ∞ if Pr s ( ♢ t ) = 1 and finite state space ω Proof size is reduced to ≈ 65 % !

15 • When essential and aperiodic: 2 1 lim 1 Stationary Distribution N is a stationary distribution iff ( N > = K ) = N > Or: K × N = N — K as transition matrix • When support set of N is essential (bottom SCC): ∫ f s ω d T s = N s − 1 ω n → ∞ Pr s ( ω n = t ) = N t ( ) • Stationary distribution for b): N = geometric

16 DTMC ( M :: α measure ) ( X :: nat ⇒ α ⇒ σ ) = σ U ) prob-space M ( ∀ n . X n ∈ M → ∧ ∧ ( ∃ S . countable S ∧ ∀ n . Pr ( X n ∈ S ) = 1 ) ∧ — The stochastic process X is memoryless : ∀ n s t . ( Pr ( ∀ n ′ ⩽ n . X n ′ = t n ′ ) ̸ = 0 − → Pr ( X ( n + 1 ) = s | ∀ n ′ ⩽ n . X n ′ = t n ′ ) = Pr ( X ( n + 1 ) = s | X n = t n ) ) ∧ — The stochastic process X is time-homogeneous : ∀ n n ′ s t . ( Pr ( X n = t ) ̸ = 0 ∧ Pr ( X n ′ = t ) ̸ = 0 − → Pr ( X ( n + 1 ) = s | X n = t ) = Pr ( X ( n ′ + 1 ) = s | X n ′ = t ) )

Markov decision processes

K K s sc sc h s K s Probabilistic & non-deterministic transitions • Kernels (coalgebras) of MDPs: pmf set • Traditional definition of schedulers: list pmf 18 Markov decision process

sc sc h s K s Probabilistic & non-deterministic transitions • Kernels (coalgebras) of MDPs: • Traditional definition of schedulers: list pmf 18 Markov decision process K :: σ ⇒ σ pmf set , K s ̸ = ∅

Probabilistic & non-deterministic transitions • Kernels (coalgebras) of MDPs: • Traditional definition of schedulers: 18 Markov decision process K :: σ ⇒ σ pmf set , K s ̸ = ∅ sc :: σ list ⇒ σ pmf , sc ( h · s ) ∈ K s

19 1 1 4 1 4 1 2 Configurations β s 0 s 1 α s 2 ( s 0 , α ) ( s 0 , β ) ( s 0 , α ) ( s 1 , α ) ( s 0 , α ) ( s 1 , α ) ( s 2 , α ) ( s 2 , α ) Attention: the configuration includes the entire tree!

20 Configurations on MDPs codatatype σ cfg = Cfg ( state : σ ) ( act : σ pmf ) ( cont : σ ⇒ σ cfg ) where state ( cont c s ) = s • Induces a Markov chain : K MC :: σ cfg ⇒ σ cfg pmf K MCc = map ( cont c ) ( act c ) • Trace space: T c = map measure ( map stream state ) T MCc • Valid Configuration: act is always compatible with K

Lemma (Iteration Rule) s f s f t dD t D K s 21 min min Definition (Minimal Expectation) ∫ � [ f ] f d T c E min = s c ∈ valid s

21 Definition (Minimal Expectation) ∫ � [ f ] f d T c E min = s c ∈ valid s Lemma (Iteration Rule) ∫ f � f ( t · ω ) dD [ ] [ ] E min = E min s s t D ∈ K s

22 1 2 1 2 1 1 4 1 4 3 1 8 4 2 Application: Reachability Problem Example p = 5 s 1 n = 1 α α p = 1 p = 1 s 0 s 2 n = 0 n = 1 γ β p = 0 p = 0 s 4 s 3 α n = 0 n = 0 p is Pr max ( S 1 U S 2 ) , n is Pr min ( S 1 U S 2 ) s s

S 1 U S 2 s Pr min S 1 U S 2 and Pr max S 1 U S 2 Currently: v v Goal: certify solutions to reachability problems in MDPs Formalize MDPs and reachability problems Pr min lfp Implement and verify certification algorithm Requires proof: optimal memoryless scheduler Import results by executing algorithm in Isabelle/HOL 23 Application: Reachability problems on MDPs

Pr min S 1 U S 2 and Pr max S 1 U S 2 Currently: v v Goal: certify solutions to reachability problems in MDPs Pr min Implement and verify certification algorithm Requires proof: optimal memoryless scheduler Import results by executing algorithm in Isabelle/HOL 23 Application: Reachability problems on MDPs ⇒ Formalize MDPs and reachability problems = ( S 1 U S 2 ) = lfp ( · · · ) s

Goal: certify solutions to reachability problems in MDPs Pr min Requires proof: optimal memoryless scheduler Import results by executing algorithm in Isabelle/HOL 23 Application: Reachability problems on MDPs ⇒ Formalize MDPs and reachability problems = ( S 1 U S 2 ) = lfp ( · · · ) s ⇒ Implement and verify certification algorithm = Currently: v ⩽ Pr min ( S 1 U S 2 ) and Pr max ( S 1 U S 2 ) ⩽ v

Goal: certify solutions to reachability problems in MDPs Pr min Import results by executing algorithm in Isabelle/HOL 23 Application: Reachability problems on MDPs ⇒ Formalize MDPs and reachability problems = ( S 1 U S 2 ) = lfp ( · · · ) s ⇒ Implement and verify certification algorithm = Currently: v ⩽ Pr min ( S 1 U S 2 ) and Pr max ( S 1 U S 2 ) ⩽ v ⇒ Requires proof: ∃ optimal memoryless scheduler =

Goal: certify solutions to reachability problems in MDPs Pr min 23 Application: Reachability problems on MDPs ⇒ Formalize MDPs and reachability problems = ( S 1 U S 2 ) = lfp ( · · · ) s ⇒ Implement and verify certification algorithm = Currently: v ⩽ Pr min ( S 1 U S 2 ) and Pr max ( S 1 U S 2 ) ⩽ v ⇒ Requires proof: ∃ optimal memoryless scheduler = ⇒ Import results by executing algorithm in Isabelle/HOL =

24 Abort Par pgcl pgcl Present the pGCL semantics similar to [Gretz, Katoen, McIver (2014)]: Seq pgcl pgcl Skip pgcl Application: pGCL semantics := | | Assign ( σ ⇒ σ ) | | | If ( σ ⇒ bool ) pgcl pgcl | Prob [ 0 , 1 ] pgcl pgcl While ( σ ⇒ bool ) pgcl pgcl |

25 Weakest pre-expectation transformer ( ) ( ) wp :: pgcl ⇒ σ ⇒ R ∞ ⇒ σ ⇒ R ∞ ⩾ 0 ⩾ 0 wp Skip f f = wp Abort f ⊥ = wp ( Assign u ) f f ◦ u = wp ( Seq c 1 c 2 ) f wp c 1 ( wp c 2 f ) = wp ( Par c 1 c 2 ) f wp c 1 f ⊓ wp c 2 f = wp ( If b c 1 c 2 ) f λ s . if b s then wp c 1 f s else wp c 2 f s = wp ( Prob p c 1 c 2 ) f λ s . p · wp c 1 f s + ( 1 − p ) · wp c 2 f s = wp ( While b c ) f lfp ( λ g s . if b s then wp c g s else f s ) =

26 else else Operational semantics as MDP K :: ( pgcl × σ ) ⇒ ( pgcl × σ ) pmf set K ( Skip , s ) ≪ Skip , s ≫ = K ( Abort , s ) ≪ Abort , s ≫ = K ( Assign u , s ) ≪ Skip , u s ≫ = K ( Seq c 1 c 2 , s ) = { } ] [ ( Seq c ′ 1 c 2 , s ′ ) if c ′ 1 ̸ = Skip K ( c 1 , s ) λ ( c ′ 1 , s ′ ) . ( c 2 , s ′ ) K ( Par c 1 c 2 , s ) ≪ c 1 , s ≫ ∪ ≪ c 2 , s ≫ = K ( If b c 1 c 2 , s ) if b s then K ( c 1 , s ) else K ( c 2 , s ) = K ( Prob p c 1 c 2 , s ) {{ ( c 1 , s ) �→ p , ( c 2 , s ) �→ ( 1 − p )) }} = { ≪ Seq c ( While g c ) , s ≫ if g s K ( While g c , s ) = ≪ Skip , s ≫

Theorem (Operational semantics equals denotational semantics) c s r f wp c f s 27 else lfp min Equate wp and K Definition (Result of a Trace)  r f ω if c ̸ = Skip    r f (( c , s ) · ω ) = f s   

27 lfp else Equate wp and K Definition (Result of a Trace)  r f ω if c ̸ = Skip    r f (( c , s ) · ω ) = f s    Theorem (Operational semantics equals denotational semantics) ( c , s ) ( r f ) = wp c f s E min

28 else else Proof { }   ∫ g ( c , s ) if c ̸ = Skip ( c , s ) ( r f ) = lfp  λ g s . � d µ  ( c , s ) E min f s ( c , s ) µ ∈ K ( c , s ) Case c = Seq c 1 c 2 : ( Seq c 1 c 2 , s ) ( r f ) = E min r λ s ′ . E min ( c 2 , s ′ ) ( r f ) ( ( )) E min ( c 1 , s ) Case c = While b c ′ : E min ( While g c ′ , s ) ( r f ) = lfp w s   g ( d , t ) if d ̸ = Skip ∫     w g s = � g ( c ′ , t ) if b t d µ   ( d , t )  f t  µ ∈ K s

Probabilistic Hierarchy

Ana Sokolva – Coalgebraic Analysis of Probabilistic Systems (2005): 30 Zoo of Probabilistic System Types H., Traytel & Lochbihler [ITP 2015]

How to … …model system types? …compare systems of same type? …compare different system types? Coalgebras Bisimulation Embedding respecting bisimulation …formalize it in Isabelle/HOL? codatatype + Probability Mass Func. 31 Hierarchy of Probabilistic Systems Types

How to … …model system types? …compare systems of same type? …compare different system types? Coalgebras Bisimulation Embedding respecting bisimulation …formalize it in Isabelle/HOL? codatatype + Probability Mass Func. 31 Hierarchy of Probabilistic Systems Types

How to … …model system types? …compare systems of same type? …compare different system types? Coalgebras Bisimulation Embedding respecting bisimulation …formalize it in Isabelle/HOL? codatatype + Probability Mass Func. 31 Hierarchy of Probabilistic Systems Types

How to … …model system types? …compare systems of same type? …compare different system types? Coalgebras Bisimulation Embedding respecting bisimulation …formalize it in Isabelle/HOL? codatatype + Probability Mass Func. 31 Hierarchy of Probabilistic Systems Types

How to … …model system types? …compare systems of same type? …compare different system types? Coalgebras Bisimulation Embedding respecting bisimulation …formalize it in Isabelle/HOL? codatatype + Probability Mass Func. 31 Hierarchy of Probabilistic Systems Types

How to … …model system types? …compare systems of same type? …compare different system types? Coalgebras Bisimulation Embedding respecting bisimulation …formalize it in Isabelle/HOL? codatatype + Probability Mass Func. 31 Hierarchy of Probabilistic Systems Types

How to … …model system types? …compare systems of same type? …compare different system types? Coalgebras Bisimulation Embedding respecting bisimulation …formalize it in Isabelle/HOL? codatatype + Probability Mass Func. 31 Hierarchy of Probabilistic Systems Types

How to … …model system types? …compare systems of same type? …compare different system types? Coalgebras Bisimulation Embedding respecting bisimulation …formalize it in Isabelle/HOL? codatatype + Probability Mass Func. 31 Hierarchy of Probabilistic Systems Types

How to … …model system types? …compare systems of same type? …compare different system types? Coalgebras Bisimulation Embedding respecting bisimulation …formalize it in Isabelle/HOL? codatatype + Probability Mass Func. 31 Hierarchy of Probabilistic Systems Types

How to … …model system types? …compare systems of same type? …compare different system types? Coalgebras Bisimulation Embedding respecting bisimulation …formalize it in Isabelle/HOL? codatatype + Probability Mass Func. 31 Hierarchy of Probabilistic Systems Types

How to model all F -coalgebras as type? F F F Example (Labeled Markov Chains where F codatatype C pmf): codatatype mc MC mc pmf 32 Coalgebras as Codatatypes in Isabelle/HOL Idea: Analyse transition systems modulo bisimulation ! Equality : ⇐ ⇒ Bisimulation

F F F Example (Labeled Markov Chains where F codatatype C pmf): codatatype mc MC mc pmf 32 Coalgebras as Codatatypes in Isabelle/HOL Idea: Analyse transition systems modulo bisimulation ! Equality : ⇐ ⇒ Bisimulation How to model all F -coalgebras as type?

Example (Labeled Markov Chains where F pmf): codatatype mc MC mc pmf 32 Coalgebras as Codatatypes in Isabelle/HOL Idea: Analyse transition systems modulo bisimulation ! Equality : ⇐ ⇒ Bisimulation How to model all F -coalgebras as type? codatatype τ F = C ( τ F F )

32 Coalgebras as Codatatypes in Isabelle/HOL Idea: Analyse transition systems modulo bisimulation ! Equality : ⇐ ⇒ Bisimulation How to model all F -coalgebras as type? codatatype τ F = C ( τ F F ) Example (Labeled Markov Chains where F = α × □ pmf): codatatype α mc = MC ( α × α mc pmf )

33 Simple Segala system Generative system Det. automaton Stratified system 1 Alternating system Labeled MDP Segala system Non-det. automaton Labeled MC MC Bundle system Markov chain Pnueli-Zuck system Most general system Reactive system System Types Name Functor Codatatype σ pmf α × σ pmf α LMC α × σ pmf set κ α LMDP κ α ⇒ σ option α DLTS ( α × σ ) set κ α LTS κ α ⇒ σ pmf option α React ( α × σ ) pmf option α Gen σ pmf + ( α × σ ) option α Str σ pmf + ( α × σ ) set κ α Alt κ ( α × σ pmf ) set κ α SSeg κ α Seg κ ( α × σ ) pmf set κ α Bun κ ( α × σ ) set κ pmf ( α × σ ) set κ 1 pmf set κ 2 α PZ κ 1 , κ 2 ( α × σ + σ ) set κ 1 pmf set κ 2 α MG κ 1 , κ 2

34 MC Hierarchy α option PZ κ 1 , κ 2 α MG κ 1 , κ 2 κ ⩽ κ 1 κ ⩽ κ 2 α option Bun κ α option Seg κ α option SSeg κ α PZ κ 1 , κ 2 κ ⩽ κ 1 κ ⩽ κ 2 α Seg κ α Bun κ α SSeg κ α Alt κ α Gen α LMDP κ α Str α set ⩽ κ α LTS κ α LMC α React α set ⩽ κ α DLTS

35 Ana Sokolva – Coalgebraic Analysis of Probabilistic Systems (2005): Hierarchy of Probabilistic System Types

Conclusion

• Formal verification of probabilistic algorithms • Formal reasoning about classified Markov chains in HOL • Probabilistic guarded commands mechanized in HOL • Proofs of randomized algorithms in Coq • Verifying probabilistic correctness in Isabelle with pGCL 37 Related Work Formalizing probabilistic trace spaces: Hurd [thesis 2002] Liu, Hasan, Aravantinos, and Tahar [ITP 2013] Formalizing probabilistic transition systems: Hurd, McIver, and Morgan [Theor. Comput. Sci. 2005] Audebaud and Paulin-Mohring [MPC 2006] Cock [SSV 2012]

Found two flaws • Future Work: • Coalgebraic & Fixed point approach simplified out theory (also smaller proofs) • Very usable for our applications • Probabilistic model checking • pGCL semantics equivalence • Small examples on fixed models • Formalized hierarchy of probabilistic systems types • Probability theory also used for: • Density Compiler [Eberl, H., Nipkow (ESOP 2015)] • Central Limit Theorem [Avigad, H., Serafin (2014)] Average Runtime Analysis, Probabilistic Programming 38 Conclusion