Operational semantics for disintegration Chung-chieh Shan (Indiana - PowerPoint PPT Presentation

Operational semantics for disintegration Chung-chieh Shan (Indiana University) Norman Ramsey (Tufts University) Mathematical Foundations of Programming Semantics 2016-05-25 1

What is semantics for? 1. Language for composing generative stories do { x � normal 0 10 ; (monadic bind, unit) 2. Guarantee that terms denote distributions (product measure tricky ) 2

What is semantics for? 1. Language for composing generative stories do { x � normal 0 10 ; normal x 1 } (monadic bind, unit) 2. Guarantee that terms denote distributions (product measure tricky ) 2

What is semantics for? 1. Language for composing generative stories do { x � normal 0 10 ; return e x } (monadic bind, unit) 2. Guarantee that terms denote distributions (product measure tricky ) 2

What is semantics for? 1. Language for composing generative stories do { x � normal 0 10 ; return e x } (monadic bind, unit) 2. Guarantee that terms denote distributions do { x � m ; do { y � n ; return ( x , y ) }} (product measure tricky ) 2

What is semantics for? 1. Language for composing generative stories do { x � normal 0 10 ; return e x } (monadic bind, unit) 2. Guarantee that terms denote distributions do { x � m ; = do { x � m ; do { y � n ; y � n ; return ( x , y ) }} return ( x , y ) } (product measure tricky ) 2

What is semantics for? 3. Equational reasoning by semantics-preserving rewriting! Fubini do { x � m ; = do { y � n ; y � n ; x � m ; return ( x , y ) } return ( x , y ) } Conjugacy √ do { x � normal 0 1 ; = do { factor ( 1 / 3 ); √ factor e − x 2 ; x � normal 0 3 ; return x } return x } 3

What is semantics for? 3. Equational reasoning by semantics-preserving rewriting! Fubini do { x � m ; = do { y � n ; y � n ; x � m ; return ( x , y ) } return ( x , y ) } Conjugacy √ do { x � normal 0 1 ; = do { factor ( 1 / 3 ); √ factor e − x 2 ; x � normal 0 3 ; return x } return x } 3

What is semantics for? 3. Equational reasoning by semantics-preserving rewriting! Fubini do { x � m ; = do { y � n ; y � n ; x � m ; return ( x , y ) } return ( x , y ) } Conjugacy √ do { x � normal 0 1 ; = do { factor ( 1 / 3 ); √ factor e − x 2 ; x � normal 0 3 ; return x } return x } (convenient factor expresses non-(sub)probability measures) 3

Two program transformations specified semantically 1. Simplification m ′ = m 2. Disintegration m = do { t � m 1 ; x � m 2 ; return ( t , x ) } (typically t appears free in m 2 ) 4

Two program transformations specified semantically 1. Simplification m ′ = m 2. Disintegration m = do { t � m 1 ; x � m 2 ; return ( t , x ) } (typically t appears free in m 2 ) 4

Two program transformations specified semantically 1. Simplification m ′ = m 2. Disintegration m = do { t � m 1 ; x � m 2 ; return ( t , x ) } (typically t appears free in m 2 ) 4

Two program transformations specified semantically 1. Simplification m ′ = m 2. Disintegration m = do { t � m 1 ; = do { t ⋆ ∼ m 1 ; x � m 2 ; m 2 } return ( t , x ) } (typically t appears free in m 2 ) 4

What is disintegration for? = do { t � m 1 ; = do { t ⋆ ∼ m 1 ; m x � m 2 ; m 2 } return ( t , x ) } 5

What is disintegration for? = do { t � m 1 ; = do { t ⋆ ∼ m 1 ; m x � m 2 ; m 2 } return ( t , x ) } Disintegration defines conditional distributions do { x � normal 0 10 ; -- latent cause t t � normal x 1 ; -- observed effect return ( t , x ) } x 5

What is disintegration for? = do { t � m 1 ; = do { t ⋆ ∼ m 1 ; m x � m 2 ; m 2 } return ( t , x ) } Disintegration defines conditional distributions do { x � normal 0 10 ; -- latent cause t t � normal x 1 ; -- observed effect return ( t , x ) } x √ = do { t � normal 0 101 ; √ x � normal ( t × 100 / 101 ) ( 10 / 101 ); return ( t , x ) } 5

What is disintegration for? = do { t � m 1 ; = do { t ⋆ ∼ m 1 ; m x � m 2 ; m 2 } return ( t , x ) } Disintegration defines conditional distributions do { x � normal 0 10 ; -- latent cause t t � normal x 1 ; -- observed effect return ( t , x ) } x √ = do { t ⋆ 101 ; ∼ normal 0 √ normal ( t × 100 / 101 ) ( 10 / 101 ) } 5

What is disintegration for? = do { t � m 1 ; = do { t ⋆ ∼ m 1 ; m x � m 2 ; m 2 } return ( t , x ) } Disintegration allows an uncountable space of observations 1 do { x � uniform 0 1 ; y � uniform 0 1 ; ; t � return ( t , ( x , y )) } 0 1 6

What is disintegration for? = do { t � m 1 ; = do { t ⋆ ∼ m 1 ; m x � m 2 ; m 2 } return ( t , x ) } Disintegration allows an uncountable space of observations 1 do { x � uniform 0 1 ; y � uniform 0 1 ; t � return ( y − 2 × x ); return ( t , ( x , y )) } 0 1 6

What is disintegration for? = do { t � m 1 ; = do { t ⋆ ∼ m 1 ; m x � m 2 ; m 2 } return ( t , x ) } Disintegration allows an uncountable space of observations 1 do { x � uniform 0 1 ; y � uniform 0 1 ; t � return ( y / x ); return ( t , ( x , y )) } 0 1 6

From denotation to operation Denotation: equivalence relation Operation: directed graph So denotation justifies soundness of operation 7

From denotation to operation Denotation: equivalence relation Operation: directed graph So denotation justifies soundness of operation 7

From denotation to operation Denotation: equivalence relation Operation: directed graph So denotation justifies soundness of operation This setup is typical for execution, but what about disintegration? Rest of this talk: a disintegrator that justifies its work, step by step A contextual operational semantics, implemented in PLT Redex 7

Syntax � � do { x � m ; m } � � do { x ⋆ Term e , m ::= do { factor e ; m } ∼ m ; m } � return e � uniform e e � normal e e � lebesgue � � � � � ( e , e ) � · · · � � � fst e � � snd e � � number � � e − e � � e / e � � x � (not shown: conditionals) 8

Syntax Stratified syntax during disintegration � � do { x � m ; s } � � do { delimit ; t } State s ::= do { factor e ; s } � do { x � m ; t } � do { x ⋆ � � t ::= do { factor e ; t } ∼ m ; m } Thread � do { x � m ; m } � Term e , m ::= do { factor e ; m } � return e � uniform e e � normal e e � lebesgue � � � � � ( e , e ) � · · · � � � fst e � � snd e � � number � � e − e � � e / e � � x � (not shown: conditionals) 8

Example disintegration do { delimit ; r � do { x � uniform 0 1 ; y � uniform 0 1 ; return ( y / x , ( x , y )) } ∼ return ( fst r ); t ⋆ return ( snd r ) } Stage 1: Isolate (evaluate) Stage 2: Invert (unevaluate) 9

Example disintegration do { delimit ; r � do { x � uniform 0 1 ; y � uniform 0 1 ; return ( y / x , ( x , y )) } ; ∼ return ( fst r ); t ⋆ return ( snd r ) } 9

Example disintegration (step 1) bind do { delimit ; − − → do { delimit ; r � do { x � uniform 0 1 ; x � uniform 0 1 ; y � uniform 0 1 ; r � do { y � uniform 0 1 ; return ( y / x , ( x , y )) } ; return ( y / x , ( x , y )) } ; ∼ return ( fst r ); ∼ return ( fst r ); t ⋆ t ⋆ return ( snd r ) } return ( snd r ) } 9

Example disintegration (step 1) bind do { delimit ; − − → do { delimit ; r � do { x � uniform 0 1 ; x � uniform 0 1 ; y � uniform 0 1 ; r � do { y � uniform 0 1 ; return ( y / x , ( x , y )) } ; return ( y / x , ( x , y )) } ; ∼ return ( fst r ); ∼ return ( fst r ); t ⋆ t ⋆ return ( snd r ) } return ( snd r ) } bind do { delimit ; − − → do { delimit ; H ; H ; r � do { x � m 1 ; x � m 1 ; m 2 } ; r � m 2 ; P [ r ] } P [ r ] } H is a heap of bindings P [ r ] is a program that demands r 9

Example disintegration (step 1) do { delimit ; x � uniform 0 1 ; r � do { y � uniform 0 1 ; return ( y / x , ( x , y )) } ; ∼ return ( fst r ); t ⋆ return ( snd r ) } 9

Example disintegration (step 2) do { delimit ; x � uniform 0 1 ; r � do { y � uniform 0 1 ; return ( y / x , ( x , y )) } ; ∼ return ( fst r ); t ⋆ return ( snd r ) } 10

Example disintegration (step 2) bind do { delimit ; − − → do { delimit ; x � uniform 0 1 ; x � uniform 0 1 ; r � do { y � uniform 0 1 ; y � uniform 0 1 ; return ( y / x , ( x , y )) } ; r � return ( y / x , ( x , y )); ∼ return ( fst r ); ∼ return ( fst r ); t ⋆ t ⋆ return ( snd r ) } return ( snd r ) } bind do { delimit ; − − → do { delimit ; H ; H ; r � do { x � m 1 ; x � m 1 ; m 2 } ; r � m 2 ; P [ r ] } P [ r ] } 10

Example disintegration (step 3) return do { delimit ; − − − → do { delimit ; x � uniform 0 1 ; x � uniform 0 1 ; y � uniform 0 1 ; y � uniform 0 1 ; r � return ( y / x , ( x , y )); r � return ( y / x , ( x , y )); ∼ return ( fst r ); ∼ return ( fst ( y / x , ( x , y ))); t ⋆ t ⋆ return ( snd r ) } return ( snd r ) } return do { delimit ; − − − → do { delimit ; H ; H ; r � return v ; r � return v ; P [ r ] } P [ v ] } v is head normal form for lazy evaluation 11