Principles of Probabilistic Programming Lectures at EWSCS 2020 - PowerPoint PPT Presentation

Principles of Probabilistic Programming Lectures at EWSCS 2020 Winter School Joost-Pieter Katoen EWSCS 2020, Palmse, Estonia Joost-Pieter Katoen Principles of Probabilistic Programming 1/222

What is probabilistic programming? Probabilistic programs What? Programs with random assignments, conditioning and usual control-flow constructs Why? Z Random assignments: to describe randomised algorithms Z Conditioning: to describe stochastic decision making Joost-Pieter Katoen Principles of Probabilistic Programming 10/222

What is probabilistic programming? Applications Joost-Pieter Katoen Principles of Probabilistic Programming 11/222

What is probabilistic programming? Planning in AI: robot navigation Uncertainty: noisy sensors and actuators, unknown environment 5 5 Evans et al. , Modeling Agents with Probabilistic Programs, 2019 Joost-Pieter Katoen Principles of Probabilistic Programming 14/222

What is probabilistic programming? Security: The RSA-OAEP protocol Correctness proof took more than 20 years Joost-Pieter Katoen Principles of Probabilistic Programming 16/222

What is probabilistic programming? Printer troubleshooting in Windows 95 How likely is it that your print is garbled given that the ps-file is not and the page orientation is portrait? [Ramanna et al. , Emerging Paradigms in Machine Learning, 2013] Joost-Pieter Katoen Principles of Probabilistic Programming 19/222

What is probabilistic programming? Languages Tp Pyro Ub O Joost-Pieter Katoen Principles of Probabilistic Programming 21/222

What is probabilistic programming? Issue 1: Program correctness Z Classical programs: Z A program is correct with respect to a (formal) specification “for input array A, the output array B is sorted and contains all elements contained in A” Z Defines a deterministic input-output relation Z Partial correctness: if an output is produced, it is correct Z Total correctness: in addition, the program terminates Z Probabilistic programs: Z They do not always generate the same output Z They generate a probability distribution over possible outputs Joost-Pieter Katoen Principles of Probabilistic Programming 23/222

What is probabilistic programming? Issue 2: Termination a 112 ± ' Z Classical programs: a Z They terminate (on a given/all inputs), or they do not Z If they terminate, they take finitely many steps to do so * Z Showing program termination is undecidable (halting problem) Z Probabilistic programs: Z They terminate (or not) with a certain likelihood Z They may have diverging runs whose likelihood is zero x Z They may take infinitely many steps (on average) to terminate x even if they terminate with probability one! Z Showing “probability-one” termination is “more” undecidable Z and showing they do in finite time on average, even more! Joost-Pieter Katoen Principles of Probabilistic Programming 24/222

What is probabilistic programming? Issue 3: The program’s runtime Z Classical programs: Z They have a deterministic, fixed run-time for a given input Z Runtimes of terminating programs in sequence are compositional: if P and Q terminate in n and k steps, then P;Q halts in n + k steps Z Analysis techniques: recurrence equations, tree analysis, etc. Z Probabilistic programs: Z Every runtime has a probability; their runtime is a distribution Z Runtimes of “probability-one” terminating programs may not sum up if P and Q terminate in n and k steps on average, then P;Q may need infinitely many steps on average Z Analysis techniques: involve reasoning about expected values etc. Joost-Pieter Katoen Principles of Probabilistic Programming 25/222

What is probabilistic programming? This EWSCS 2020 tutorial Z The probabilistic guarded command language pGCL Z examples, syntax, semantics (Markov chains), conditioning, recursion Z Proving correctness of probabilistic programs Z weakest pre-conditions, loop invariants, post-conditions, conditioning Z Almost-sure termination Z positive a.s.-termination, (a bit of) hardness, stochastic ranking functions Z Analysing runtimes of probabilistic programs Z examples, finite versus infinite expected runtime, wp-reasoning Z Verifying and runtime analysis of Bayesian networks Joost-Pieter Katoen Principles of Probabilistic Programming 26/222

What is probabilistic programming? Overview What is probabilistic programming? 1 Probabilistic Guarded Command Language 2 Weakest preconditions 3 Conditioning 4 Expected runtime analysis 5 Analysing Bayesian networks 6 Http Recursion 7 Loop invariant synthesis 8 Epilogue 9 Joost-Pieter Katoen Principles of Probabilistic Programming 27/222

Probabilistic Guarded Command Language Overview What is probabilistic programming? 1 Probabilistic Guarded Command Language 2 Weakest preconditions 3 Conditioning 4 Expected runtime analysis 5 Analysing Bayesian networks 6 Recursion 7 Mawgan Loop invariant synthesis 8 Epilogue 9 Joost-Pieter Katoen Principles of Probabilistic Programming 28/222

Probabilistic Guarded Command Language Dijkstra’s guarded command language Z skip empty statement Z diverge divergence Z x := E assignment Z prog1 ; prog2 sequential composition Z if (G) prog1 else prog2 choice Z prog1 [] prog2 non-deterministic choice Z while (G) prog iteration Joost-Pieter Katoen Principles of Probabilistic Programming 29/222

Probabilistic Guarded Command Language Probabilistic GCL I Kozen McIver Morgan p - P I p - Xt 7 Z skip empty statement Z diverge divergence Z x := E assignment P Z observe (G) ✓ conditioning Z prog1 ; prog2 sequential composition Z if (G) prog1 else prog2 choice Z prog1 [p] prog2 probabilistic choice - n p - Z while (G) prog iteration Joost-Pieter Katoen Principles of Probabilistic Programming 30/222

Probabilistic Guarded Command Language Let’s start simple x := 0 [0.5] x := 1; y := -1 [0.5] y := 0 This program admits four runs and yields the outcome: Pr [ x = 0, y = 0 ] = Pr [ x = 0, y = � 1 ] = Pr [ x = 1, y = 0 ] = Pr [ x = 1, y = � 1 ] = 1 / 4 Joost-Pieter Katoen Principles of Probabilistic Programming 31/222

Probabilistic Guarded Command Language A loopy program For 0 < p < 1 an arbitrary probability: bool c := true ; int i := 0; while (c) { i++; (c := false [p] c := true ) } The loopy program models a geometric distribution with parameter p . Pr [ i = N ] = ( 1 � p ) N � 1 � p for N > 0 false ↳ time - C ie one times - tune N : r c - Joost-Pieter Katoen Principles of Probabilistic Programming 32/222

Probabilistic Guarded Command Language On termination bool c := true ; int i := 0; while (c) { i++; (c := false [p] c := true ) } - This program does not always terminate. It almost surely terminates. Joost-Pieter Katoen Principles of Probabilistic Programming 33/222

Probabilistic Guarded Command Language Conditioning Joost-Pieter Katoen Principles of Probabilistic Programming 34/222

Probabilistic Guarded Command Language Let’s start simple x := 0 [0.5] x := 1; y := -1 [0.5] y := 0; observe (x+y = 0) This program blocks two runs as they violate x+y = 0 . Outcome: Pr [ x = 0, y = 0 ] = Pr [ x = 1, y = � 1 ] = 1 / 2 ⇒ t ' . Joost-Pieter Katoen Principles of Probabilistic Programming 35/222

Probabilistic Guarded Command Language Let’s start simple x := 0 [0.5] x := 1; y := -1 [0.5] y := 0; observe (x+y = 0) This program blocks two runs as they violate x+y = 0 . Outcome: Pr [ x = 0, y = 0 ] = Pr [ x = 1, y = � 1 ] = 1 / 2 Observations thus normalize the probability of the “feasible” program runs Joost-Pieter Katoen Principles of Probabilistic Programming 35/222

Probabilistic Guarded Command Language A loopy program For 0 < p < 1 an arbitrary probability: bool c := true ; C p ) { int i : = 0; while (c) { geom i++; (c := false [p] c := true ) } observe (odd(i)) ⇒ " Joost-Pieter Katoen Principles of Probabilistic Programming 36/222

Probabilistic Guarded Command Language A loopy program For 0 < p < 1 an arbitrary probability: bool c := true ; int i : = 0; while (c) { i++; (c := false [p] c := true ) } observe (odd(i)) 1 The feasible program runs have a probability 8 N ≥ 0 ( 1 � p ) 2 N � p = 2 � p This program models the distribution: Pr [ i = 2 N + 1 ] = ( 1 � p ) 2 N � p � ( 2 � p ) for N ≥ 0 Pr [ i = 2 N ] = 0 Joost-Pieter Katoen Principles of Probabilistic Programming 36/222

Probabilistic Guarded Command Language Why formal semantics matters discrete ~ Z Unambiguous meaning to (almost) all probabilistic programs in Z Operational interpretation to weakest pre-expectations - Z Basis for proving correctness Z of programs Z of program transformations Z of program equivalence Z of static analysis Z of compilers Z . . . . . . Joost-Pieter Katoen Principles of Probabilistic Programming 37/222

Probabilistic Guarded Command Language Andrei Andrejewitsch Markow Joost-Pieter Katoen Principles of Probabilistic Programming 38/222

s OF Probabilistic Guarded Command Language ÷ Markov chains " O O 7 2 3 4 A Markov chain (MC) is a triple ( Σ , σ I , P ) with: Z Σ being a countable set of states Z σ I ∈ Σ the initial state, and Z P ⇥ Σ � Dist ( Σ ) the transition probability function where Dist ( Σ ) is a discrete probability measure on Σ . rhet . ) P ( 2 with gli )= =p I , )= } its )=o star refs ,3 Joost-Pieter Katoen Principles of Probabilistic Programming 39/222