Reducing the Cost of Probabilistic Knowledge Compilation Giso H. - PowerPoint PPT Presentation

Reducing the Cost of Probabilistic Knowledge Compilation Giso H. Dal, Steffen Michels and Peter J.F. Lucas Radboud University, Nijmegen (The Netherlands)

Outline The Problem 1 Exact Inference by Weighted Model Counting 2 Bayesian Networks Encoding Bayesian Networks Compilation and Inference The Framework 3 Partition and Compile Assembly and Inference Reducing Representation Size 4 An Upperbound Empirical Evaluation 5

Outline The Problem The Problem 1 Inf. by WMC Exact Inference by Weighted Model Counting Framework 2 Reducing Size Bayesian Networks Empirical Encoding Bayesian Networks Evaluation Compilation and Inference The Framework 3 Partition and Compile Assembly and Inference Reducing Representation Size 4 An Upperbound Empirical Evaluation 5

The Problem The Problem Inf. by WMC Framework Reducing Size Exact inference is nice, Empirical Evaluation but... knowledge compilation is computationally intensive.

Outline The Problem The Problem 1 Inf. by WMC Bayesian Networks Exact Inference by Weighted Model Counting 2 Encoding Compilation Bayesian Networks Framework Encoding Bayesian Networks Reducing Size Compilation and Inference Empirical Evaluation The Framework 3 Partition and Compile Assembly and Inference Reducing Representation Size 4 An Upperbound Empirical Evaluation 5

Running Example The Problem Inf. by WMC Bayesian Networks P ( a = 1) P ( a = 2) P ( a = 3) Encoding a b P ( X ) Compilation a b 0.8 0.1 0.1 Framework 1 1 0.4 1 2 0.4 Reducing Size P ( b =1 | a ) P ( b =2 | a ) a 2 1 0.05 Empirical P ( X ) = P ( b | a ) P ( a ) 1 0.5 0.5 Evaluation 2 2 0.05 2 0.5 0.5 3 1 0 3 0 1 3 2 0.1 Joint distribution Bayesian network CPTs

Encoding The Problem Inf. by WMC Bayesian Networks Let a BN be defined over variables x . We encode it as Boolean function f by Encoding Compilation adding for x ∈ X : Framework Reducing Size n n � � ( x 1 ∨ · · · ∨ x n ) ∧ ( x i ∨ x j ) Empirical . Evaluation i =1 j = i +1 � �� � � �� � at-most-once at-least-once Unique symbolic weights ω j identify distinct probabilities local to x ’s CPT. We introduce into f clauses that imply each weight ω j .

Encoding The Problem P ( a = 1) P ( a = 2) P ( a = 3) Inf. by WMC 0.8 0.1 0.1 Bayesian Networks Encoding Compilation a P ( b =1 | a ) P ( b =2 | a ) Framework 1 0.5 0.5 Reducing Size 2 0.5 0.5 3 0 1 Empirical Evaluation

Encoding The Problem P ( a = 1) P ( a = 2) P ( a = 3) P ( a = 1) P ( a = 2) P ( a = 3) Inf. by WMC ω 1 ω 2 ω 2 0.8 0.1 0.1 Bayesian Networks Encoding Compilation a P ( b =1 | a ) P ( b =2 | a ) a P ( b =1 | a ) P ( b =2 | a ) Framework 1 0.5 0.5 1 ω 3 ω 3 Reducing Size 2 0.5 0.5 2 ω 3 ω 3 3 0 1 3 Empirical ω 4 ω 5 Evaluation

Encoding The Problem P ( a = 1) P ( a = 2) P ( a = 3) P ( a = 1) P ( a = 2) P ( a = 3) Inf. by WMC ω 1 ω 2 ω 2 0.8 0.1 0.1 Bayesian Networks Encoding Compilation a P ( b =1 | a ) P ( b =2 | a ) a P ( b =1 | a ) P ( b =2 | a ) Framework 1 0.5 0.5 1 ω 3 ω 3 Reducing Size 2 0.5 0.5 2 ω 3 ω 3 3 0 1 3 Empirical ω 4 ω 5 Evaluation Variables: ( a 1 ∨ a 2 ∨ a 3 ) ∧ ( a 1 ∨ a 2 ) ∧ ( a 1 ∨ a 3 ) ∧ ( a 2 ∨ a 3 ) ∧ ( b 1 ∨ b 2 ) ∧ ( b 1 ∨ b 2 ) Probabilities: ( a 1 ∨ ω 1 ) ∧ ( a 2 ∨ ω 2 ) ∧ ( a 3 ∨ ω 2 ) ∧ ( a 1 ∨ b 1 ∨ ω 3 ) ∧ ( a 1 ∨ b 2 ∨ ω 3 ) ∧ ( a 2 ∨ b 1 ∨ ω 3 ) ∧ ( a 2 ∨ b 2 ∨ ω 3 ) ∧ ( a 3 ∨ b 1 ∨ ω 4 ) ∧ ( a 3 ∨ b 2 ∨ ω 5 ) .

Inference by Weighted Model Counting The Problem Inf. by WMC Bayesian Networks a 1 Encoding Compilation a 3 Framework ω 1 Reducing Size ω 2 b 1 b 1 Empirical Evaluation b 2 ω 3 ω 4 ω 5 1 0 WPBDD

Inference by Weighted Model Counting The Problem Inf. by WMC Bayesian Networks a 1 ∨ Encoding Compilation ∧ ∧ a 3 Framework ω 1 a 3 ∨ ∧ ∧ Reducing Size ω 2 b 1 b 1 Empirical a 1 a 2 ω 1 ∧ ω 2 ∨ Evaluation ∨ ω 3 ∧ ∧ b 2 ω 3 ω 4 ω 4 ω 5 ω 5 b 1 b 2 b 1 b 2 1 0 Logical circuit WPBDD

Inference by Weighted Model Counting The Problem Inf. by WMC Bayesian Networks a 1 ∨ + Encoding Compilation ∗ ∗ ∧ ∧ a 3 Framework ω 1 a 3 + ∗ ∗ ∨ ∧ ∧ Reducing Size 1 ω 2 b 1 b 1 Empirical a 1 a 2 ω 1 ∧ ω 2 ∨ ∗ + 1 1 0 . 1 0 . 8 Evaluation ∨ ω 3 ∧ ∧ + ∗ ∗ b 2 0 . 5 ω 3 ω 4 ω 4 ω 5 ω 5 b 1 b 2 b 1 b 2 1 0 1 0 0 1 1 0 Logical circuit Instantiated arithmetic circuit for WPBDD P ( b = 1)

Outline The Problem The Problem 1 Inf. by WMC Exact Inference by Weighted Model Counting Framework 2 Partition and Bayesian Networks Compile Assembly and Encoding Bayesian Networks Inference Reducing Size Compilation and Inference Empirical Evaluation The Framework 3 Partition and Compile Assembly and Inference Reducing Representation Size 4 An Upperbound Empirical Evaluation 5

The Framework The Problem Inf. by WMC Framework Partition and Compile Assembly and Inference 1 Partition Reducing Size 2 Compilation Empirical Evaluation 3 Assembly 4 Inference

Partition and Compile cut The Problem a Inf. by WMC b a 1 Framework Partition 1 Partition 2 Partition and Compile a 3 Assembly and Partitioning Inference a 3 Reducing Size b 1 b 1 Empirical a 1 P ( a = 1) P ( a = 2) P ( a = 3) ω 2 Evaluation ω 1 b 2 ω 3 ω 1 ω 2 ω 2 1 0 1 0 Partition 1. P ( b =1 | a ) P ( b =2 | a ) a Compilation with Partition 2. 1 ω 3 ω 3 order: Compilation with order: 2 ω 3 ω 3 a 3 ≤ a 2 ≤ a 1 a 1 ≤ a 2 ≤ a 3 ≤ b 1 ≤ b 2 3 ω 4 ω 5 Capture symbolic structure in CPTs Compilation of a partitioned BN,

Partition and Compile: What to compile? The Problem Partition 1: Inf. by WMC Framework ( a 1 ∨ a 2 ∨ a 3 ) ∧ ( a 1 ∨ a 2 ) ∧ ( a 1 ∨ a 3 ) ∧ ( a 2 ∨ a 3 ) ∧ Partition and Compile Assembly and ( a 1 ∨ ω 1 ) ∧ ( a 2 ∨ ω 2 ) ∧ ( a 3 ∨ ω 2 ) Inference Reducing Size Empirical Evaluation Partition 2: ( a 1 ∨ a 2 ∨ a 3 ) ∧ ( a 1 ∨ a 2 ) ∧ ( a 1 ∨ a 3 ) ∧ ( a 2 ∨ a 3 ) ∧ ( b 1 ∨ b 2 ) ∧ ( b 1 ∨ b 2 ) ∧ ( a 1 ∨ b 1 ∨ ω 3 ) ∧ ( a 1 ∨ b 2 ∨ ω 3 ) ∧ ( a 2 ∨ b 1 ∨ ω 3 ) ∧ ( a 2 ∨ b 2 ∨ ω 3 ) ∧ ( a 3 ∨ b 1 ∨ ω 4 ) ∧ ( a 3 ∨ b 2 ∨ ω 5 ) .

Assemby and Inference The Problem Inf. by WMC a 3 Partition 1 a 3 Framework a 1 Partition 1 a 1 Partition and ω 2 ω 1 Compile Tier 1 ω 1 0 ω 2 Assembly and 0 Tier 1 1 Inference 1 3 } a { Reducing Size a 1 Partition 2 1 , 2 b 1 Empirical a 3 Partition 2 b 3 Evaluation b 1 b 1 0 Partition 2 b 1 1 Conditioned on a 3 b 2 ω 3 Tier 2 0 ω 3 1 0 Conditioned on a 1 , a 2 Tier 2 1 Dynamic conditioning Static conditioning

Outline The Problem The Problem 1 Inf. by WMC Exact Inference by Weighted Model Counting Framework 2 Reducing Size Bayesian Networks An Upperbound Encoding Bayesian Networks Empirical Evaluation Compilation and Inference The Framework 3 Partition and Compile Assembly and Inference Reducing Representation Size 4 An Upperbound Empirical Evaluation 5

An upperbound Let a Bayesian network be defined over n variables X . The Problem 1 We formulate it as a set of constraints C x ∈ C , where C x represents the Inf. by WMC dependencies of variable x ∈ X : Framework C x = { x } ∪ Parents ( x ) . Reducing Size An Upperbound 2 Compile using ordering O = { o 1 , . . . , o n } imposed on X by π , where o i = π ( x ) , Empirical Evaluation o j ≤ o k for j < k . 3 The upperbound is based on spanning variables S l ∈ S associated with each evalutation depth, or level l : � S l = { o 1 , . . . , o l − 1 } ∩ C x { C x ∈ C : { o 1 ,...,o l − 1 } ⊂ C x } 4 A generalized upperbound is computed by: n � � D ( S l ) , where D ( Y ) = | x | . l =1 x ∈ Y

An Upperbound a 1 a D ( S 1 ∪ { a } ) The Problem = 3 nodes = 1 node level 1 P ( a =1) P ( a =2) P ( a =3) D ( S 1 ) 1 2 3 a 2 Inf. by WMC ω 1 ω 2 ω 3 a 3 ω 1 ω 2 ω 3 ω 1 Framework ω 2 ω 3 Reducing Size b 1 b 1 b 1 D ( S 2 ∪ { b } ) An Upperbound a P ( b =1 | a ) P ( b =2 | a ) b b b = 3 nodes = 6 nodes D ( S 2 ) level 2 Empirical 1 ω 4 ω 5 1 2 1 2 1 2 Evaluation ω 8 2 ω 6 ω 7 ω 6 ω 6 ω 7 ω 4 b 2 b 2 b 2 ω 5 ω 8 3 ω 8 ω 9 ω 5 ω 4 ω 9 ω 7 ω 9 1 1 0

An Upperbound a 1 a D ( S 1 ∪ { a } ) The Problem = 3 nodes = 1 node level 1 P ( a =1) P ( a =2) P ( a =3) D ( S 1 ) 1 2 3 a 2 Inf. by WMC ω 1 ω 2 ω 3 a 3 ω 1 ω 2 ω 3 ω 1 Framework ω 2 ω 3 Reducing Size b 1 b 1 b 1 D ( S 2 ∪ { b } ) An Upperbound a P ( b =1 | a ) P ( b =2 | a ) b b b = 3 nodes = 6 nodes D ( S 2 ) level 2 Empirical 1 ω 4 ω 5 1 2 1 2 1 2 Evaluation ω 8 2 ω 6 ω 7 ω 6 ω 6 ω 7 ω 4 b 2 b 2 b 2 ω 5 ω 8 3 ω 8 ω 9 ω 5 ω 4 ω 9 ω 7 ω 9 1 1 0 C a = { a } , C b = { a, b } .