lifted probabilistic inference by first order knowledge
play

Lifted Probabilistic Inference by First-Order Knowledge Compilation - PowerPoint PPT Presentation

May 09, 2023 •138 likes •947 views

Lifted Probabilistic Inference by First-Order Knowledge Compilation Guy Van den Broeck Nima Taghipour Wannes Meert Jesse Davis Luc De Raedt Lifted Inference in Probabilistic Logical Models - Tutorial - IJCAI11 18/07/11 Outline Overview

  1. Lifted Probabilistic Inference by First-Order Knowledge Compilation Guy Van den Broeck Nima Taghipour Wannes Meert Jesse Davis Luc De Raedt Lifted Inference in Probabilistic Logical Models - Tutorial - IJCAI11 18/07/11

  2. Outline ● Overview Approach ● First-Order d-DNNF Circuits ● First-Order Knowledge Compilation ● Experiments ● Conclusions 18/07/11 First-Order Knowledge Compilation 2

  3. Outline ● Overview Approach ● First-Order d-DNNF Circuits ● First-Order Knowledge Compilation ● Experiments ● Conclusions 18/07/11 First-Order Knowledge Compilation 3

  4. Context Variable Belief Knowledge Elimination Propagation Compilation Ground [Zhang94] [Pearl82] [Darwiche03] Lifted [Poole03] [Singla08] Our approach 18/07/11 First-Order Knowledge Compilation 4

  5. Advantages of Knowledge Compilation ● Compile once, then run polytime inference for multiple queries and evidence ● Efficient data structures ● Principled logical approach ● Exploits context-specific independences ● State of the art for exact inference in ● Bayesian networks ● Statistical relational learning ● Used in many domains, not just probabilistic reasoning 18/07/11 First-Order Knowledge Compilation 5

  6. Question? ● Can we lift knowledge compilation to a first-order setting? ● First step taken: first-order d-DNNFs for ● weighted first-order model counting ● lifted probabilistic inference ● Many open questions remaining! 18/07/11 First-Order Knowledge Compilation 6

  7. What is Lifted Inference? 18/07/11 First-Order Knowledge Compilation 7

  8. What is Lifted Inference? ● Variables X,Y range over domain People ● Represents propositional model for given domain (50 people) ● Propositional inference in factor graph is expensive ● However: symmetries 18/07/11 First-Order Knowledge Compilation 8

  9. What is Lifted Inference? ● We compile to a circuit independent of |People| ● Inference linear in |People| → Lifted Inference 18/07/11 First-Order Knowledge Compilation 9

  10. Knowledge Compilation Bayesian Network Factor Graph MLN ... 18/07/11 First-Order Knowledge Compilation 10

  11. Knowledge Compilation Bayesian Network 1 Factor Graph Weighted CNF MLN ... ● Step : Convert model to weighted CNF 1 18/07/11 First-Order Knowledge Compilation 11

  12. Knowledge Compilation Bayesian Network 1 2 Factor Graph Weighted d-DNNF CNF Circuit MLN ... ● Step : Convert model to weighted CNF 1 ● Step : Convert CNF to d-DNNF circuit 2 18/07/11 First-Order Knowledge Compilation 12

  13. Knowledge Compilation Bayesian Network 1 2 3 Factor Graph Weighted d-DNNF Weighted CNF Circuit Model Counting MLN ... ● Step : Convert model to weighted CNF 1 ● Step : Convert CNF to d-DNNF circuit 2 ● Step : Perform weighted model counting 3 18/07/11 First-Order Knowledge Compilation 13

  14. Our Approach: First-Order Knowledge Compilation: Parfactor 1 2 3 Graph Weighted FO d-DNNF Weighted FO MLN FO CNF Circuit Model Counting ... ● Step : Convert model to weighted FO CNF 1 ● Step : Convert CNF to FO d-DNNF circuit 2 ● Step : Perform weighted FO model counting 3 18/07/11 First-Order Knowledge Compilation 14

  15. Step 1: Converting to Weighted FO CNF MLN 18/07/11 First-Order Knowledge Compilation 15

  16. Step 1: Converting to Weighted FO CNF MLN Weighted FO Theory 18/07/11 First-Order Knowledge Compilation 16

  17. Step 1: Converting to Weighted FO CNF MLN Weighted FO Theory Weighted FO CNF 18/07/11 First-Order Knowledge Compilation 17

  18. Step 3: Weighted FO Model Counting ● Weight function on ground atoms 18/07/11 First-Order Knowledge Compilation 18

  19. Step 3: Weighted FO Model Counting ● Weight function on ground atoms ● Weight of a model (possible world) 18/07/11 First-Order Knowledge Compilation 19

  20. Step 3: Weighted FO Model Counting ● Weight function on ground atoms ● Weight of a model (possible world) ● Weight of all models is 18/07/11 First-Order Knowledge Compilation 20

  21. Step 3: Weighted FO Model Counting ● Weight function on ground atoms ● Weight of a model (possible world) ● Weight of all models is Weight of models where Alice smokes is 18/07/11 First-Order Knowledge Compilation 21

  22. Step 3: Weighted FO Model Counting ● Weight function on ground atoms ● Weight of a model (possible world) ● Weight of all models is Weight of models where Alice smokes is ● 18/07/11 First-Order Knowledge Compilation 22

  23. Outline ● Overview Approach ● First-Order d-DNNF Circuits ● First-Order Knowledge Compilation ● Experiments ● Conclusions 18/07/11 First-Order Knowledge Compilation 23

  24. Propositional d-DNNF Circuits [Darwiche and Marquis, 2002] Logical theory: 18/07/11 First-Order Knowledge Compilation 24

  25. Propositional d-DNNF Circuits [Darwiche and Marquis, 2002] Logical theory: Literal (leaf) 18/07/11 First-Order Knowledge Compilation 25

  26. Propositional d-DNNF Circuits [Darwiche and Marquis, 2002] Logical theory: Logical operators (inner node) Literal (leaf) 18/07/11 First-Order Knowledge Compilation 26

  27. Propositional d-DNNF Circuits [Darwiche and Marquis, 2002] Logical theory: Logical operators (inner node) Deterministic disjunction Literal (leaf) 18/07/11 First-Order Knowledge Compilation 27

  28. Propositional d-DNNF Circuits [Darwiche and Marquis, 2002] Logical theory: Logical operators (inner node) Deterministic disjunction Decomposable conjunction Literal (leaf) 18/07/11 First-Order Knowledge Compilation 28

  29. First-Order d-DNNF Circuits Logical Theory: 18/07/11 First-Order Knowledge Compilation 29

  30. First-Order d-DNNF Circuits Logical Theory: First-Order Literal (leaf) 18/07/11 First-Order Knowledge Compilation 30

  31. First-Order d-DNNF Circuits Logical Theory: First-Order Literal (leaf) ● Deterministic disjunction ● Decomposable conjunction ● 3 additional first-order operators (inner nodes) 18/07/11 First-Order Knowledge Compilation 31

  32. Outline ● Overview Approach ● First-Order d-DNNF Circuits ● First-Order Knowledge Compilation ● Experiments ● Conclusions 18/07/11 First-Order Knowledge Compilation 32

  33. Step 2: Our Compilation Algorithm ● Recursively apply Weighted FO d-DNNF FO CNF Circuit ● Unit Propagation ● Independence ● Inclusion-Exclusion (Shannon Decomposition) ● Shattering ● Independent Partial Grounding ● Atom Counting ● (Grounding) 18/07/11 First-Order Knowledge Compilation 33

  34. Step 2: Our Compilation Algorithm ● Recursively apply Weighted FO d-DNNF FO CNF Circuit ● Unit Propagation ● Independence ● Inclusion-Exclusion (Shannon Decomposition) ● Shattering ● Independent Partial Grounding ● Atom Counting ● (Grounding) Generate first-order operators in inner nodes 18/07/11 First-Order Knowledge Compilation 34

  35. Step 2: Our Compilation Algorithm ● Recursively apply Weighted FO d-DNNF FO CNF Circuit ● Unit Propagation ● Independence ● Inclusion-Exclusion (Shannon Decomposition) ● Shattering ● Independent Partial Grounding ● Atom Counting ● (Grounding) 18/07/11 First-Order Knowledge Compilation 35

  36. Unit Propagation 18/07/11 First-Order Knowledge Compilation 36

  37. Unit Propagation Unit clause 18/07/11 First-Order Knowledge Compilation 37

  38. Unit Propagation Unit clause 18/07/11 First-Order Knowledge Compilation 38

  39. Unit Propagation Clauses split w.r.t. unit clause atom 'residuals' → independent 18/07/11 First-Order Knowledge Compilation 39

  40. Unit Propagation Resolvent of unit and 2 nd clause 18/07/11 First-Order Knowledge Compilation 40

  41. Step 2: Our Compilation Algorithm ● Recursively apply Weighted FO d-DNNF FO CNF Circuit ● Unit Propagation ● Independence ● Inclusion-Exclusion (Shannon Decomposition) ● Shattering ● Independent Partial Grounding ● Atom Counting ● (Grounding) 18/07/11 First-Order Knowledge Compilation 41

  42. Atom Counting 18/07/11 First-Order Knowledge Compilation 42

  43. Atom Counting Atom with one logical variable X ∈ {luc,jesse} 18/07/11 First-Order Knowledge Compilation 43

  44. Atom Counting Atom with 1 logical variable 18/07/11 First-Order Knowledge Compilation 44

  45. Atom Counting All partial interpretations for fun(X) - deterministic - 2 |People| 18/07/11 First-Order Knowledge Compilation 45

  46. Atom Counting Same weighted model count 18/07/11 First-Order Knowledge Compilation 46

  47. Atom Counting 2 |People| → |People|+1 18/07/11 First-Order Knowledge Compilation 47

  48. Atom Counting Isomorphic circuits 18/07/11 First-Order Knowledge Compilation 48

  49. Atom Counting |People|+1 → 1 18/07/11 First-Order Knowledge Compilation 49

  50. Atom Counting 18/07/11 First-Order Knowledge Compilation 50

  51. Outline ● Overview Approach ● First-Order d-DNNF Circuits ● First-Order Knowledge Compilation ● Experiments ● Conclusions 18/07/11 First-Order Knowledge Compilation 51

  52. Evaluated Models ● Sick Death [de Salvo Braz 2005] ● WebKB [Lowd 2007] ● Competing Workshops [Milch 2008] ● Workshop Attributes [Milch 2008] ● Friends Smoker [Singla 2008] ● Friends Smoker Drinker 18/07/11 First-Order Knowledge Compilation 52

Recommend

Generating and Sampling Orbits for Lifted Probabilistic Inference Steven Holtzen , Todd Millstein,

Generating and Sampling Orbits for Lifted Probabilistic Inference Steven Holtzen , Todd Millstein,

Generating and Sampling Orbits for Lifted Probabilistic Inference Steven Holtzen , Todd Millstein, Guy Van den Broeck Computer Science Department, University of California, Los Angeles {sholtzen, todd, guyvdb}@cs.ucla.edu UAI 2019 1 Motivation:

563 views • 28 slides
Outline First-order logic Syntax and semantics Inference First-order Logic

Outline First-order logic Syntax and semantics Inference First-order Logic

Outline First-order logic Syntax and semantics Inference First-order Logic Propositionalization with ground inference Lifted resolution CS 486/686 Sept 25, 2008 University of Waterloo 1 2 CS486/686 Lecture Slides (c)

81 views • 6 slides
Lifted Inference in Statistical Relational Models Guy Van den Broeck BUDA Invited Tutorial June

Lifted Inference in Statistical Relational Models Guy Van den Broeck BUDA Invited Tutorial June

Lifted Inference in Statistical Relational Models Guy Van den Broeck BUDA Invited Tutorial June 22 nd 2014 Overview 1. What are statistical relational models? 2. What is lifted inference? 3. How does lifted inference work? 4. Theoretical

1.5k views • 118 slides
Asymmetric Graphical Models Guy Van den Broeck and Mathias Niepert Jan 28, 2015, AAAI Take-Away

Asymmetric Graphical Models Guy Van den Broeck and Mathias Niepert Jan 28, 2015, AAAI Take-Away

Lifted Probabilistic Inference for Asymmetric Graphical Models Guy Van den Broeck and Mathias Niepert Jan 28, 2015, AAAI Take-Away Message Two problems: 1. Lifted inference gives exponential speedups in symmetric graphical models. But what

807 views • 16 slides
15-780 Graduate Artificial Intelligence: Probabilistic inference J. Zico Kolter (this

15-780 Graduate Artificial Intelligence: Probabilistic inference J. Zico Kolter (this

15-780 Graduate Artificial Intelligence: Probabilistic inference J. Zico Kolter (this lecture) and Nihar Shah Carnegie Mellon University Spring 2020 1 Outline Probabilistic graphical models Probabilistic inference Exact inference

665 views • 39 slides
Inference in First-Order Logic C H A P T E R 9 H A S S A N K H O S R A V I S P R I N G 2 0 1 1

Inference in First-Order Logic C H A P T E R 9 H A S S A N K H O S R A V I S P R I N G 2 0 1 1

Inference in First-Order Logic C H A P T E R 9 H A S S A N K H O S R A V I S P R I N G 2 0 1 1 Outline Reducing first-order inference to propositional inference Unification Generalized Modus Ponens Forward chaining Backward

856 views • 58 slides
Successes and Challenges Guy Van den Broeck IJCAI Early Career Spotlight Jul 14, 2016 Overview

Successes and Challenges Guy Van den Broeck IJCAI Early Career Spotlight Jul 14, 2016 Overview

First-Order Probabilistic Reasoning: Successes and Challenges Guy Van den Broeck IJCAI Early Career Spotlight Jul 14, 2016 Overview 1. Why first-order probabilistic models ? 2. Why first-order probabilistic reasoning ? 3. How does lifted

1.4k views • 129 slides
First-Order Knowledge Compilation for Probabilistic Reasoning Guy Van den Broeck based on joint

First-Order Knowledge Compilation for Probabilistic Reasoning Guy Van den Broeck based on joint

First-Order Knowledge Compilation for Probabilistic Reasoning Guy Van den Broeck based on joint work with Adnan Darwiche, Dan Suciu, and many others MOTIVATION 1 A Simple Reasoning Problem ... ? Probability that Card1 is Hearts? [Van den

1.54k views • 106 slides
Inference in first-order logic Chapter 9 1 Outline Reducing first-order inference to

Inference in first-order logic Chapter 9 1 Outline Reducing first-order inference to

Inference in first-order logic Chapter 9 1 Outline Reducing first-order inference to propositional inference Unification Generalized Modus Ponens Forward and backward chaining Logic programming Resolution 2 Reasoning in

987 views • 40 slides
Lifted Message Passing Rorschach Test K. Kersting 2 Lifted Message Passing GRAPHBOT@ ROS 2010

Lifted Message Passing Rorschach Test K. Kersting 2 Lifted Message Passing GRAPHBOT@ ROS 2010

GRAPHBOT 2010. IROS WORKSHOP ON PROBABILISTIC GRAPHICAL Sriraam Babak Fabian Scott Youssef El Natarajan Ahmadi Hadiji Sanner Massaoudi MODELS IN ROBOTICS Taipei, Taiwan, October 22. 2010 A E Kristian Kersting Lifted Message

431 views • 28 slides
Automorphism Groups of Graphical Models and Lifted Variational Inference Hung Hai Bui 1 Tuyen N.

Automorphism Groups of Graphical Models and Lifted Variational Inference Hung Hai Bui 1 Tuyen N.

Automorphism Groups of Graphical Models and Lifted Variational Inference Hung Hai Bui 1 Tuyen N. Huynh 2 Sebastian Riedel 3 1 Laboratory for Natural Language Understanding, Nuance Communications 2 AI Center, SRI International 3 Department of

501 views • 26 slides
Inference in first-order logic Chapter 9 Chapter 9 1 Outline Reducing first-order inference

Inference in first-order logic Chapter 9 Chapter 9 1 Outline Reducing first-order inference

Inference in first-order logic Chapter 9 Chapter 9 1 Outline Reducing first-order inference to propositional inference Unification Generalized Modus Ponens Forward and backward chaining Resolution Chapter 9 2 Universal

789 views • 42 slides
First-Order Knowledge Compilation Guy Van den Broeck Dagstuhl Sept 18, 2017 Overview 1.

First-Order Knowledge Compilation Guy Van den Broeck Dagstuhl Sept 18, 2017 Overview 1.

First-Order Knowledge Compilation Guy Van den Broeck Dagstuhl Sept 18, 2017 Overview 1. Propositional Refresher 2. Primer: A First-Order Tractable Language 3. Probabilistic Databases 4. Symmetric First-Order Model Counting 5. Lots of

2.09k views • 181 slides
@ Leuven H. Blockeel, J. Davis, L. De Raedt, D. Fierens, W. Meert, N. Taghipour, G. Van den

@ Leuven H. Blockeel, J. Davis, L. De Raedt, D. Fierens, W. Meert, N. Taghipour, G. Van den

Recent advances in lifted inference @ Leuven H. Blockeel, J. Davis, L. De Raedt, D. Fierens, W. Meert, N. Taghipour, G. Van den Broeck SML, April 19, 2012 Outline Introduction to lifted inference Four contributions Arbitrary

873 views • 49 slides
Inference in first-order logic Chapter 9 Chapter 9 1 Outline Reducing first-order inference

Inference in first-order logic Chapter 9 Chapter 9 1 Outline Reducing first-order inference

Inference in first-order logic Chapter 9 Chapter 9 1 Outline Reducing first-order inference to propositional inference Unification Generalized Modus Ponens Forward and backward chaining Logic programming Resolution

1.2k views • 46 slides
Inference in first-order logic Chapter 9 Chapter 9 1 Outline Reducing first-order inference

Inference in first-order logic Chapter 9 Chapter 9 1 Outline Reducing first-order inference

Inference in first-order logic Chapter 9 Chapter 9 1 Outline Reducing first-order inference to propositional inference Unification Generalized Modus Ponens Forward and backward chaining Logic programming Resolution

793 views • 48 slides
On Computational and Probabilistic Inference Rajat Mani Thomas Objectives: Revisiting Bayesian

On Computational and Probabilistic Inference Rajat Mani Thomas Objectives: Revisiting Bayesian

On Computational and Probabilistic Inference Rajat Mani Thomas Objectives: Revisiting Bayesian inference . A look at Likelihood and the prior Probabilistic programming: Automating Bayesian (like) Inference Probabilistic Toolkit:

833 views • 59 slides
Inference in first-order logic Russell and Norvig Chapter 9 Outline n Reducing first-order

Inference in first-order logic Russell and Norvig Chapter 9 Outline n Reducing first-order

Inference in first-order logic Russell and Norvig Chapter 9 Outline n Reducing first-order inference to propositional inference n Generalized Modus Ponens n Forward chaining n Backward chaining n Resolution 2 FOL to PL n First order inference

853 views • 64 slides
Advanced inference in probabilistic programs Brooks Paige Inference thus far Likelihood

Advanced inference in probabilistic programs Brooks Paige Inference thus far Likelihood

Advanced inference in probabilistic programs Brooks Paige Inference thus far Likelihood weighting / importance sampling MCMC (single-dimension, coded by hand) Lightweight Metropolis-Hastings (update one random choice at a time,

995 views • 86 slides
CSC421 Intro to Artificial Intelligence UNIT 15: Inference in First-Order Logic + Knowledge

CSC421 Intro to Artificial Intelligence UNIT 15: Inference in First-Order Logic + Knowledge

CSC421 Intro to Artificial Intelligence UNIT 15: Inference in First-Order Logic + Knowledge Engineering Outline Logic Programming Resolution Ontological Engineering Categories & Objects Physical Composition

340 views • 18 slides
Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh

Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh

Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives MAP inference and its complexity exact & approximate MAP inference max-product and max-sum

957 views • 74 slides
Probabilistic Partial Evaluation: Exploiting rule structure in probabilistic inference David

Probabilistic Partial Evaluation: Exploiting rule structure in probabilistic inference David

To appear, Proc. 15th International Joint Conference on AI (IJCAI-97), Nagoya, Japan, August, 1997 Probabilistic Partial Evaluation: Exploiting rule structure in probabilistic inference David Poole Department of Computer Science University

220 views • 8 slides
Applying Search Based Probabilistic Inference Algorithms to Probabilistic Conformant Planning:

Applying Search Based Probabilistic Inference Algorithms to Probabilistic Conformant Planning:

Applying Search Based Probabilistic Inference Algorithms to Probabilistic Conformant Planning: Preliminary Results Junkyu Lee * , Radu Marinescu ** and Rina Dechter * * University of California, Irvine ** IBM Research, Ireland ISAIM 2016

592 views • 35 slides
Probabilistic Partial Evaluation: Exploiting rule structure in probabilistic inference David

Probabilistic Partial Evaluation: Exploiting rule structure in probabilistic inference David

Probabilistic Partial Evaluation: Exploiting rule structure in probabilistic inference David Poole University of British Columbia 1 Overview Belief Networks Variable Elimination Algorithm Parent Contexts & Structured

259 views • 21 slides

More recommend