Approximate Knowledge Compilation by Online Collapsed Importance - PowerPoint PPT Presentation

Approximate Knowledge Compilation by Online Collapsed Importance Sampling Tal Friedman and Guy Van den Broeck

Motivation Factor Graphs: 1

Motivation Factor Graphs: Great! But asking queries is hard 2

Motivation Factor Graphs: 3

Motivation: Arithmetic Circuit • Exact inference: Use Knowledge Compilation (e.g. BDD, SPN) • Tractable form: easy queries + operations • Take advantage of further independence properties, logical structure 4

But they don’t scale ! 5

Knowledge Compilation Sampling Exact Scalable Independence Anytime Properties Logical Structure This work 6

Collapsed Sampling (Rao-Blackwell) Sampling on some variables, exact inference conditioned on sample 7

Collapsed Sampling (Rao-Blackwell) Sampling on some variables, exact inference conditioned on sample 8

Collapsed Sampling (Rao-Blackwell) Sampling on some variables, exact inference conditioned on sample Sample A,B 9

Collapsed Sampling (Rao-Blackwell) Sampling on some variables, exact inference conditioned on sample Observe sampled values 10

Collapsed Sampling (Rao-Blackwell) Sampling on some variables, exact inference conditioned on sample Compute exactly P(C|A,B) 11

What to Sample? • Is it even possible to pick a correct set a priori? • Consider a network of potential smokers, with friendships sampled Sample 1 Sample 2 12

Online Collapsed Sampling Choose on-the-fly which variable to sample next, based on result of sampling previous variables Theorem : Still unbiased 13

How? 1. What/when do we sample? 14

How? 1. What/when do we sample? 2. How do we sample? 15

How do we Sample? • Importance Sampling • Need a proposal for any variable conditioned on any other variables 16

How? 1. What/when do we sample? 2. How do we sample? 3. How do we do exact inference? 17

Exact Inference How do we do exact inference conditioned on different variables? 18

Exact Inference • How do we do exact inference conditioned on different variables? 19

Collapsed Compilation Big Circuit? Exact Sampling Inference Small Circuit? Result: A circuit for factor graph with some sampled variables 20

Collapsed Compilation Big Circuit? 1. What/when do we sample? Exact Sampling 2. How do we sample? Inference 3. How do we do exact inference? Small Circuit? 21

Collapsed Compilation Big Circuit? 1. What/when do we sample? Exact Sampling 2. How do we sample? Inference 3. How do we do exact inference? Small Circuit? 22

What/when do we sample? When : Circuit too big What : Heuristic on current circuit 23

Collapsed Compilation Big Circuit? 1. What/when do we sample? Exact Sampling 2. How do we sample? Inference 3. How do we do exact inference? Small Circuit? 24

Motivation: Arithmetic Circuit • Exact inference: Use Knowledge Compilation (e.g. BDD, SPN) • Tractable form: easy queries + operations • Take advantage of further independence properties, logical structure 25

How do we sample? Compute the marginal of the variable in the current circuit! 26

Collapsed Compilation Big Circuit? 1. What/when do we sample? Exact Sampling 2. How do we sample? Inference 3. How do we do exact inference? Small Circuit? 27

Conditional Exact Inference Result is a circuit: any joint can be computed efficiently & exactly 28

Online Collapsed Importance Sampling 0.6 x 0.9 x Approximate any query! 1.1 x 29

Experiments • Approximate marginal in factor graph • Algorithmically limit exact inference 30

Experiments 31

Knowledge Compilation Collapsed Compilation Sampling Scalable Exact Scalable Anytime Independence Anytime Properties Independence Properties Logical Structure Logical Structure 32

Thanks! Poster: Room 210 #5 Code: github.com/UCLA-StarAI/Collapsed-Compilation 33