Conditioning in Probabilistic Programming MFPS XXXI 2015 Friedrich Gretz Nils Jansen Benjamin Kaminski Joost-Pieter Katoen Annabelle McIver Federico Olmedo 23.6.2015 Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 1
Motivation Syntax of pGCL Motivation Syntax of pGCL Programs [McIver & Morgan ’06] P − → x := E | if ( G ) { P } else { P } | { P } [ p ] { P } | { P } � { P } | while ( G ) { P } Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 2
Motivation Syntax of pGCL Motivation Syntax of pGCL Programs [McIver & Morgan ’06] P − → x := E | if ( G ) { P } else { P } | { P } [ p ] { P } | { P } � { P } | while ( G ) { P } Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 2
Motivation Syntax of pGCL Motivation Syntax of pGCL Programs [McIver & Morgan ’06] P − → x := E | if ( G ) { P } else { P } | { P } [ p ] { P } | { P } � { P } | while ( G ) { P } Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 2
Motivation Syntax of pGCL Motivation Syntax of pGCL Programs [McIver & Morgan ’06] P − → x := E | if ( G ) { P } else { P } | { P } [ p ] { P } | { P } � { P } | while ( G ) { P } Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 2
Motivation Syntax of pGCL Motivation Syntax of pGCL Programs [McIver & Morgan ’06] P − → x := E | if ( G ) { P } else { P } | { P } [ p ] { P } | { P } � { P } | while ( G ) { P } Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 2
Motivation Syntax of pGCL Motivation Syntax of pGCL Programs [McIver & Morgan ’06] P − → x := E | if ( G ) { P } else { P } | { P } [ p ] { P } | { P } � { P } | while ( G ) { P } Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 2
Motivation Syntax of pGCL Motivation Syntax of pGCL Programs [McIver & Morgan ’06] P − → x := E | if ( G ) { P } else { P } | { P } [ p ] { P } | { P } � { P } | while ( G ) { P } Syntax of cpGCL Programs P − → “see pGCL” | observe G Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 2
Motivation Syntax of pGCL Motivation Syntax of pGCL Programs [McIver & Morgan ’06] P − → x := E | if ( G ) { P } else { P } | { P } [ p ] { P } | { P } � { P } | while ( G ) { P } Syntax of cpGCL Programs P − → “see pGCL” | observe G Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 2
Motivation Syntax of pGCL Motivation Syntax of pGCL Programs [McIver & Morgan ’06] P − → x := E | if ( G ) { P } else { P } | { P } [ p ] { P } | { P } � { P } | while ( G ) { P } Syntax of cpGCL Programs P − → “see pGCL” | observe G Given a probabilistic program P and a random variable f : What is the conditional expected value of f after termination of P given that no observation is violated while executing P ? Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 2
Motivation Expectations Expectations Unbounded and Bounded Expectations S = { σ | σ : V ars → R } denotes the set of program states. Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 3
Motivation Expectations Expectations Unbounded and Bounded Expectations S = { σ | σ : V ars → R } denotes the set of program states. The set of expectations E and the set of bounded expectations E ≤ 1 are � f : S → R ∞ � � � E = f E ≤ 1 = { g | g : S → [0 , 1] } . ≥ 0 Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 3
Motivation Expectations Expectations Unbounded and Bounded Expectations S = { σ | σ : V ars → R } denotes the set of program states. The set of expectations E and the set of bounded expectations E ≤ 1 are � f : S → R ∞ � � � E = f E ≤ 1 = { g | g : S → [0 , 1] } . ≥ 0 random variable = expectation Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 3
Motivation Expectations Expectations Unbounded and Bounded Expectations S = { σ | σ : V ars → R } denotes the set of program states. The set of expectations E and the set of bounded expectations E ≤ 1 are � f : S → R ∞ � � � E = f E ≤ 1 = { g | g : S → [0 , 1] } . ≥ 0 random variable = expectation � = expected value Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 3
Operational Semantics for cpGCL Operational Semantics for cpGCL Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 4
Operational Semantics for cpGCL Operational Reward Markov Decision Process (RMDP) Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 5
Operational Semantics for cpGCL Operational Reward Markov Decision Process (RMDP) Given: Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 5
Operational Semantics for cpGCL Operational Reward Markov Decision Process (RMDP) Given: P ∈ cpGCL, Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 5
Operational Semantics for cpGCL Operational Reward Markov Decision Process (RMDP) Given: P ∈ cpGCL, f ∈ E . Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 5
Operational Semantics for cpGCL Operational Reward Markov Decision Process (RMDP) Given: P ∈ cpGCL, f ∈ E . Construct an operational RMDP ` a la Gretz et al. and define conditional expected rewards under a minimizing scheduler. Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 5
Operational Semantics for cpGCL Operational Reward Markov Decision Process (RMDP) Given: P ∈ cpGCL, f ∈ E . Construct an operational RMDP ` a la Gretz et al. and define conditional expected rewards under a minimizing scheduler. Schematically such RMDPs look as follows: � � � ↓ � init � ↓ ↓ ↓ � sink � ↓ ↓ diverge Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 5
Operational Semantics for cpGCL Operational Reward Markov Decision Process (RMDP) Given: P ∈ cpGCL, f ∈ E . Construct an operational RMDP ` a la Gretz et al. and define conditional expected rewards under a minimizing scheduler. Schematically such RMDPs look as follows: � � � ↓ � init � ↓ ↓ ↓ � sink � ↓ ↓ diverge Only terminal states can contribute positive non–zero reward! Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 5
Denotational Semantics Denotational Semantics for pGCL Denotational Semantics Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 6
Denotational Semantics Denotational Semantics for pGCL Denotational Semantics for (unconditioned) pGCL Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 6
Denotational Semantics Denotational Semantics for pGCL Programs as Unconditional Expectation Transformers Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 7
Denotational Semantics Denotational Semantics for pGCL Programs as Unconditional Expectation Transformers Think of a pGCL program P as a state–to–distribution transformer σ �→ � P � ( σ ) . Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 7
Denotational Semantics Denotational Semantics for pGCL Programs as Unconditional Expectation Transformers Think of a pGCL program P as a state–to–distribution transformer σ �→ � P � ( σ ) . An expectation wp [ P ]( f ) is called the weakest pre–expectation of P with respect to post–expectation f Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 7
Denotational Semantics Denotational Semantics for pGCL Programs as Unconditional Expectation Transformers Think of a pGCL program P as a state–to–distribution transformer σ �→ � P � ( σ ) . An expectation wp [ P ]( f ) is called the weakest pre–expectation of P with respect to post–expectation f , if � � E � P � ( σ ) ( f ) = E δ σ wp [ P ]( f ) Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 7
Denotational Semantics Denotational Semantics for pGCL Programs as Unconditional Expectation Transformers Think of a pGCL program P as a state–to–distribution transformer σ �→ � P � ( σ ) . An expectation wp [ P ]( f ) is called the weakest pre–expectation of P with respect to post–expectation f , if � � E � P � ( σ ) ( f ) = E δ σ wp [ P ]( f ) = wp [ P ]( f )( σ ) . Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 7
Denotational Semantics Denotational Semantics for pGCL Programs as Unconditional Expectation Transformers Think of a pGCL program P as a state–to–distribution transformer σ �→ � P � ( σ ) . An expectation wp [ P ]( f ) is called the weakest pre–expectation of P with respect to post–expectation f , if � � E � P � ( σ ) ( f ) = E δ σ wp [ P ]( f ) = wp [ P ]( f )( σ ) . For g ∈ E ≤ 1 , an expectation wlp [ P ]( g ) is called the weakest liberal pre–expectation of P with respect to post–expectation g Benjamin Kaminski Conditioning in Probabilistic Programming 23.6.2015 7
Recommend
More recommend
