Symbolic Bayesian inference by lazy partial evaluation Chung-chieh - PowerPoint PPT Presentation

Symbolic Bayesian inference by lazy partial evaluation Chung-chieh Shan (Indiana University) Norman Ramsey (Tufts University) November 2015 This research was supported by DARPA grants FA8750-14-2-0007 and FA8750-14-C-0002, NSF grant CNS-0723054, Lilly Endowment, Inc. (through its support for the Indiana University Pervasive Technology Institute), and the Indiana METACyt Initiative. The Indiana METACyt Initiative at IU is also supported in part by Lilly Endowment, Inc. 1

Program transformations galore Expectation Total Computer algebra Normalization Disintegration Simplification Conditioning Density Exact inference Gibbs sampling MH sampling 2

Disintegration for medical diagnosis Diseases A and B are equally prevalent. Disease A causes one of symptoms 1 , 2 , 3 with equal probability. Disease B causes one of symptoms 1 , 2 with equal probability. do { disease � � A �→ 1 / 2 , B �→ 1 / 2 � : M Disease ; 3

Disintegration for medical diagnosis Diseases A and B are equally prevalent. Disease A causes one of symptoms 1 , 2 , 3 with equal probability. Disease B causes one of symptoms 1 , 2 with equal probability. do { disease � � A �→ 1 / 2 , B �→ 1 / 2 � ; symptom � case disease of A → � 1 �→ 1 / 3 , 2 �→ 1 / 3 , 3 �→ 1 / 3 � B → � 1 �→ 1 / 2 , 2 �→ 1 / 2 � ; return ( symptom , disease ) } : M ( Symptom × Disease ) 3

Disintegration for medical diagnosis Diseases A and B are equally prevalent. Disease A causes one of symptoms 1 , 2 , 3 with equal probability. Disease B causes one of symptoms 1 , 2 with equal probability. do { disease � � A �→ 1 / 2 , B �→ 1 / 2 � ; symptom � case disease of A → � 1 �→ 1 / 3 , 2 �→ 1 / 3 , 3 �→ 1 / 3 � B → � 1 �→ 1 / 2 , 2 �→ 1 / 2 � ; return ( symptom , disease ) } : M ( Symptom × Disease ) = � ( A , 1 ) �→ 1 / 6 , ( A , 2 ) �→ 1 / 6 , ( A , 3 ) �→ 1 / 6 , ( B , 1 ) �→ 1 / 4 , ( B , 2 ) �→ 1 / 4 � A 1 / 6 1 / 6 1 / 6 1 / 4 1 / 4 0 B 1 2 3 3

Disintegration for medical diagnosis Diseases A and B are equally prevalent. Disease A causes one of symptoms 1 , 2 , 3 with equal probability. Disease B causes one of symptoms 1 , 2 with equal probability. do { disease � � A �→ 1 / 2 , B �→ 1 / 2 � ; symptom � case disease of A → � 1 �→ 1 / 3 , 2 �→ 1 / 3 , 3 �→ 1 / 3 � B → � 1 �→ 1 / 2 , 2 �→ 1 / 2 � ; return ( symptom , disease ) } : M ( Symptom × Disease ) = � ( A , 1 ) �→ 1 / 6 , ( A , 2 ) �→ 1 / 6 , ( A , 3 ) �→ 1 / 6 , ( B , 1 ) �→ 1 / 4 , ( B , 2 ) �→ 1 / 4 � ⇒ λ symptom . case symptom of 1 → � A �→ 1 / 6 , B �→ 1 / 4 � 2 → � A �→ 1 / 6 , B �→ 1 / 4 � A 1 / 6 1 / 6 1 / 6 3 → � A �→ 1 / 6 � 1 / 4 1 / 4 0 B : Symptom → M Disease 1 2 3 3

Disintegration on a zero-probability observation Diseases A and B are equally prevalent. Disease A causes a symptom chosen uniformly from [ 0 , 3 ] ⊂ R . Disease B causes a symptom chosen uniformly from [ 0 , 2 ] ⊂ R . do { disease � � A �→ 1 / 2 , B �→ 1 / 2 � : M Disease ; 4

Disintegration on a zero-probability observation Diseases A and B are equally prevalent. Disease A causes a symptom chosen uniformly from [ 0 , 3 ] ⊂ R . Disease B causes a symptom chosen uniformly from [ 0 , 2 ] ⊂ R . do { disease � � A �→ 1 / 2 , B �→ 1 / 2 � ; symptom � case disease of A → uniform 0 3 B → uniform 0 2 ; return ( symptom , disease ) } : M ( Symptom × Disease ) 4

Disintegration on a zero-probability observation Diseases A and B are equally prevalent. Disease A causes a symptom chosen uniformly from [ 0 , 3 ] ⊂ R . Disease B causes a symptom chosen uniformly from [ 0 , 2 ] ⊂ R . do { disease � � A �→ 1 / 2 , B �→ 1 / 2 � ; symptom � case disease of A → uniform 0 3 B → uniform 0 2 ; return ( symptom , disease ) } : M ( Symptom × Disease ) A B 0 1 2 3 4

Disintegration on a zero-probability observation Diseases A and B are equally prevalent. Disease A causes a symptom chosen uniformly from [ 0 , 3 ] ⊂ R . Disease B causes a symptom chosen uniformly from [ 0 , 2 ] ⊂ R . do { disease � � A �→ 1 / 2 , B �→ 1 / 2 � ; symptom � case disease of A → uniform 0 3 B → uniform 0 2 ; return ( symptom , disease ) } : M ( Symptom × Disease ) ⇒ λ symptom . if symptom ≤ 2 then � A �→ 1 / 6 , B �→ 1 / 4 � A else � A �→ 1 / 6 � : Symptom → M Disease B 0 1 2 3 4

Disintegration on a zero-probability observation Choose disease uniformly from [ 1 , 3 ] ⊂ R . Choose symptom uniformly from [ 0 , disease ] ⊂ R . do { disease � uniform 1 3 : M Disease ; 5

Disintegration on a zero-probability observation Choose disease uniformly from [ 1 , 3 ] ⊂ R . Choose symptom uniformly from [ 0 , disease ] ⊂ R . do { disease � uniform 1 3 ; symptom � uniform 0 disease ; return ( symptom , disease ) } : M ( Symptom × Disease ) 5

Disintegration on a zero-probability observation Choose disease uniformly from [ 1 , 3 ] ⊂ R . Choose symptom uniformly from [ 0 , disease ] ⊂ R . do { disease � uniform 1 3 ; symptom � uniform 0 disease ; return ( symptom , disease ) } : M ( Symptom × Disease ) 3 2 1 1 0 2 3 5

Disintegration on a zero-probability observation Choose disease uniformly from [ 1 , 3 ] ⊂ R . Choose symptom uniformly from [ 0 , disease ] ⊂ R . do { disease � uniform 1 3 ; symptom � uniform 0 disease ; return ( symptom , disease ) } : M ( Symptom × Disease ) ⇒ λ symptom . do { disease � uniform 1 3 ; if 0 ≤ symptom ≤ disease then � disease �→ 1 / disease � else �� } : Symptom → M Disease 3 2 1 1 0 2 3 5

Measure semantics � M α � = ( α → R ) → R 6

Measure semantics � M α � = ( α → R ) → R λ c . c ( A ) + c ( B ) �� A �→ 1 / 2 , B �→ 1 / 2 �� = 2 2 � return ( symptom , disease ) � = λ c . c ( symptom , disease ) = �� ( symptom , disease ) �→ 1 �� � 3 c ( x ) � uniform 1 3 � = λ c . d x 2 1 � ∞ � lebesgue � = λ c . c ( x ) d x −∞ � do { x � m ; k } � = λ c . � m � ( λ x . � k � c ) 6

Measure semantics � M α � = ( α → R ) → R λ c . c ( A ) + c ( B ) �� A �→ 1 / 2 , B �→ 1 / 2 �� = 2 2 � return ( symptom , disease ) � = λ c . c ( symptom , disease ) = �� ( symptom , disease ) �→ 1 �� � 3 c ( x ) � uniform 1 3 � = λ c . d x 2 1 � ∞ � lebesgue � = λ c . c ( x ) d x −∞ � do { x � m ; k } � = λ c . � m � ( λ x . � k � c ) � � do { d � uniform 1 3 ; � 3 � d c ( s , d ) � � s � uniform 0 d ; = λ c . d s d d 2 · d 1 0 return ( s , d ) } 6

Disintegration specification � m � = � do { s � lebesgue ; d � k ; return ( s , d ) } � 7

Disintegration specification � m � = � do { s � lebesgue ; d � k ; return ( s , d ) } � m = do { d � uniform 1 3 ; k = do { d � uniform 1 3 ; s � uniform 0 d ; if 0 ≤ s ≤ d return ( s , d ) } then � d �→ 1 / d � else �� } 7

Disintegration specification � m � = � do { s � lebesgue ; d � k ; return ( s , d ) } � m = do { d � uniform 1 3 ; k = do { d � uniform 1 3 ; s � uniform 0 d ; if 0 ≤ s ≤ d return ( s , d ) } then � d �→ 1 / d � else �� } if 0 ≤ s ≤ d then c ( d ) � 3 else 0 d � k � = λ c . d d 2 1 � do { s � lebesgue ; d � k ; return ( s , d ) } � if 0 ≤ s ≤ d then c ( s , d ) � 3 � ∞ else 0 d = λ c . d d d s 2 1 −∞ � 3 � d c ( s , d ) = λ c . d s d d = � m � 2 · d 1 0 7

Useful but unspecified and thus unautomated before Figure 12 . Gibbs sampler. Borel paradox 8

Radio Yerevan Question: Is it correct that Grigori Grigorievich Grigoriev won a luxury car at the All-Union Championship in Moscow? Answer: In principle, yes. But first of all it was not Grigori Grigorievich Grigoriev, but Vassili Vassilievich Vassiliev. Second, it was not at the All-Union Championship in Moscow, but at a Collective Farm Sports Festival in Smolensk. Third, it was not a car, but a bicycle. And fourth he didn’t win it, but rather it was stolen from him. 9

Automatic disintegrator Question: Is it correct that our disintegrator is a lazy evaluator? Answer: In principle, yes. evaluate : ⌈ α ⌉ → H → ( α × H ) 10

Automatic disintegrator Question: Is it correct that our disintegrator is a lazy evaluator? Answer: In principle, yes. But first of all it is not an evaluator, but a partial evaluator. evaluate : ⌈ α ⌉ → H → ( ⌊ α ⌋ × H ) 10

Automatic disintegrator Question: Is it correct that our disintegrator is a lazy evaluator? Answer: In principle, yes. But first of all it is not an evaluator, but a partial evaluator. Second, it not only evaluates terms, but also performs random choices. evaluate : ⌈ α ⌉ → H → ( ⌊ α ⌋ → H → ⌊ M γ ⌋ ) → ⌊ M γ ⌋ perform : ⌈ M α ⌉ → H → ( ⌊ α ⌋ → H → ⌊ M γ ⌋ ) → ⌊ M γ ⌋ 10