probabilistic inference in bn2t models by weighted model
play

Probabilistic Inference in BN2T Models by Weighted Model Counting - PowerPoint PPT Presentation

Sep 11, 2022 •314 likes •741 views

Probabilistic Inference in BN2T Models by Weighted Model Counting Jirka Vomlel Institute of Information Theory and Automation Academy of Sciences of the Czech Republic http://www.utia.cz/vomlel Aalborg, Denmark, November, 21, 2013 A medical

  1. Probabilistic Inference in BN2T Models by Weighted Model Counting Jirka Vomlel Institute of Information Theory and Automation Academy of Sciences of the Czech Republic http://www.utia.cz/vomlel Aalborg, Denmark, November, 21, 2013

  2. A medical example: Carcinoid Heart Disease (CHD), van Gerven (2003) • Eleven CHD risk factors X 1 , . . . , X 11 measured at patient admission to the clinic: diarrhea, hepatic metastases, etc.

  3. A medical example: Carcinoid Heart Disease (CHD), van Gerven (2003) • Eleven CHD risk factors X 1 , . . . , X 11 measured at patient admission to the clinic: diarrhea, hepatic metastases, etc. • Dependent variable Y has two values: either 0 if CHD does not develop of 1 if it does.

  4. A medical example: Carcinoid Heart Disease (CHD), van Gerven (2003) • Eleven CHD risk factors X 1 , . . . , X 11 measured at patient admission to the clinic: diarrhea, hepatic metastases, etc. • Dependent variable Y has two values: either 0 if CHD does not develop of 1 if it does. • The conditional probability P ( Y | X 1 , . . . , X 11 ) is modeled by a noisy threshold model with ℓ = 6 .

  5. A medical example: Carcinoid Heart Disease (CHD), van Gerven (2003) • Eleven CHD risk factors X 1 , . . . , X 11 measured at patient admission to the clinic: diarrhea, hepatic metastases, etc. • Dependent variable Y has two values: either 0 if CHD does not develop of 1 if it does. • The conditional probability P ( Y | X 1 , . . . , X 11 ) is modeled by a noisy threshold model with ℓ = 6 . • The threshold model (without noise) implies that the CHD develop if at least 6 risk factors are positive.

  6. A medical example: Carcinoid Heart Disease (CHD), van Gerven (2003) • Eleven CHD risk factors X 1 , . . . , X 11 measured at patient admission to the clinic: diarrhea, hepatic metastases, etc. • Dependent variable Y has two values: either 0 if CHD does not develop of 1 if it does. • The conditional probability P ( Y | X 1 , . . . , X 11 ) is modeled by a noisy threshold model with ℓ = 6 . • The threshold model (without noise) implies that the CHD develop if at least 6 risk factors are positive. • The noise on inputs allows a non-zero probability of no CHD even if at least 6 risk factors are positive.

  7. BN2T - Bayesian Network with 2 Layers Consisting of Noisy Threshold Models X 1 X 2 X 3 X 4 Y 1 Y 2 Y j takes value 1 iff at least ℓ out of k parents X i take value 1 .

  8. BN2T - Bayesian Network with 2 Layers Consisting of Noisy Threshold Models X 1 X 2 X 3 X 4 Y 1 Y 2 Y j takes value 1 iff at least ℓ out of k parents X i take value 1 . Assume ℓ = 2 .

  9. BN2T - Bayesian Network with 2 Layers Consisting of Noisy Threshold Models For deterministic threshold the value of p = P ( Y 1 = 1 | X 1 , X 2 , X 3 ) is X 1 X 2 X 3 X 4 X 1 X 2 X 3 p 0 0 0 0 0 0 1 0 0 1 0 0 Y 1 Y 2 0 1 1 1 Y j takes value 1 iff at 1 0 0 0 least ℓ out of k parents 1 0 1 1 1 1 0 1 X i take value 1 . 1 1 1 1 Assume ℓ = 2 .

  10. BN2T - Bayesian Network with 2 Layers Consisting of Noisy Threshold Models For noisy threshold the value of p ′ = P ( Y 1 = 1 | X 1 , X 2 , X 3 ) is X 1 X 2 X 3 X 4 p ′ X 1 X 2 X 3 p 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 Y 1 Y 2 0 1 1 1 (1 − p 2 )(1 − p 3 ) Y j takes value 1 iff at 1 0 0 0 0 least ℓ out of k parents 1 0 1 1 (1 − p 1 )(1 − p 3 ) 1 1 0 1 (1 − p 1 )(1 − p 2 ) X i take value 1 . 1 1 1 1 (1 − p 1 )(1 − p 2 )(1 − p 3 ) Assume ℓ = 2 . + p 1 (1 − p 2 )(1 − p 3 ) +(1 − p 1 ) p 2 (1 − p 3 ) +(1 − p 1 )(1 − p 2 ) p 3 where p 1 , p 2 , p 3 are inhibitory probabilities.

  11. BN2T - Bayesian Network with 2 Layers Consisting of Noisy Threshold Models The joint probability of the Bayesian network X 1 X 2 X 3 X 4 P ( X 1 , . . . , X n , Y 1 , . . . , Y m ) n m � � = P ( X i ) P ( Y j | pa ( Y j )) . Y 1 Y 2 i =1 j =1 Y j takes value 1 iff at least ℓ out of k parents X i take value 1 . Assume ℓ = 2 .

  12. Noisy-threshold with explicit deterministic and noisy parts X 1 X 2 X k . . . X ′ X ′ X ′ 1 2 . . . k Y

  13. Noisy-threshold with explicit deterministic and noisy parts P X ′ i | X i � 1 � p i = X 1 X 2 X k . . . 0 1 − p i X ′ X ′ X ′ 1 2 . . . k Y

  14. Noisy-threshold with explicit deterministic and noisy parts P X ′ i | X i � 1 � p i = X 1 X 2 X k . . . 0 1 − p i P Y | X ′ 1 , X ′ 2 , X ′ 3 � � � � X ′ X ′ X ′   1 1 1 0 1 2 . . . k 1 0 0 0     = .   � � � � 0 0 0 1     Y 0 1 1 1 where we visualize the CPT as a tensor using nested matrices.

  15. Tensor of the threshold (for ℓ = 1 ) as 4D cube and its decomposition 1 1 0 0 1 1 0 0 1 1 0 0 0 1 X ′ 3 1 0 Y X ′ 2 X ′ 1

  16. Tensor of the threshold (for ℓ = 1 ) as 4D cube and its decomposition 1 1 0 1 1 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 + = 1 1 1 1 0 0 0 0 0 0 0 0 −1 0 1 1 1 0 X ′ 3 1 0 0 0 1 0 Y X ′ 2 X ′ 1

  17. Tensor of the threshold (for ℓ = 1 ) as 4D cube and its decomposition 1 1 0 1 1 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 + = 1 1 1 1 0 0 0 0 0 0 0 0 −1 0 1 1 1 0 X ′ 3 1 0 0 0 1 0 Y X ′ 2 X ′ 1 1 −1 1 0 1 0 + = 0 1 1 1 1 1 1 1 1 0

  18. CP tensor decomposition of the threshold CPT (Vomlel, Tichavsk´ y, 2012) X ′ X ′ X ′ 1 2 k . . . Y ′

  19. CP tensor decomposition of the threshold CPT (Vomlel, Tichavsk´ y, 2012) P Y =1 | X ′ 1 , X ′ 2 , X ′ 3 3 � � = ψ X ′ i , Y ′ , X ′ X ′ X ′ 1 2 k . . . i =1 Y ′ Y ′

  20. CP tensor decomposition of the threshold CPT (Vomlel, Tichavsk´ y, 2012) P Y =1 | X ′ 1 , X ′ 2 , X ′ 3 3 � � = ψ X ′ i , Y ′ , X ′ X ′ X ′ 1 2 k . . . i =1 Y ′ ψ X ′ i , Y ′   � � � 1 1 1 3 3 − 3 Y ′ 6 3 2 =   � �   1 1 2 3 − 3 0 6 3 for i = 1 , 2 , 3 .

  21. CP tensor decomposition of the threshold CPT (Vomlel, Tichavsk´ y, 2012) P Y =1 | X ′ 1 , X ′ 2 , X ′ 3 3 � � = ψ X ′ i , Y ′ , X ′ X ′ X ′ 1 2 k . . . i =1 Y ′ ψ X ′ i , Y ′   � � � 1 1 1 3 3 − 3 Y ′ 6 3 2 =   � �   1 1 2 3 − 3 0 6 3 for i = 1 , 2 , 3 . Instead of an array with 2 k entries we get k arrays with 2 k entries!

  22. Probabilistic inference by weighted model counting (WMC) • The basic idea of WMC is to encode a Bayesian network using a conjunctive normal form (CNF),

  23. Probabilistic inference by weighted model counting (WMC) • The basic idea of WMC is to encode a Bayesian network using a conjunctive normal form (CNF), • associate weights to literals according to the CPTs of the Bayesian network, and

  24. Probabilistic inference by weighted model counting (WMC) • The basic idea of WMC is to encode a Bayesian network using a conjunctive normal form (CNF), • associate weights to literals according to the CPTs of the Bayesian network, and • compute the probability of evidence as the sum of weights of all logical models consistent with that evidence.

  25. Probabilistic inference by weighted model counting (WMC) • The basic idea of WMC is to encode a Bayesian network using a conjunctive normal form (CNF), • associate weights to literals according to the CPTs of the Bayesian network, and • compute the probability of evidence as the sum of weights of all logical models consistent with that evidence. • The weight of a logical model is the product of weights of all literals.

  26. Probabilistic inference by weighted model counting (WMC) • The basic idea of WMC is to encode a Bayesian network using a conjunctive normal form (CNF), • associate weights to literals according to the CPTs of the Bayesian network, and • compute the probability of evidence as the sum of weights of all logical models consistent with that evidence. • The weight of a logical model is the product of weights of all literals. • Efficient WMC solvers exploiting several advanced techniques such as clause learning, component caching can be used – e.g. Cachet.

  27. Probabilistic inference by weighted model counting (WMC) • The basic idea of WMC is to encode a Bayesian network using a conjunctive normal form (CNF), • associate weights to literals according to the CPTs of the Bayesian network, and • compute the probability of evidence as the sum of weights of all logical models consistent with that evidence. • The weight of a logical model is the product of weights of all literals. • Efficient WMC solvers exploiting several advanced techniques such as clause learning, component caching can be used – e.g. Cachet. • If the Bayesian network exhibit a lot of determinism this is much more efficient than standard techniques.

  28. Encoding transformed BN2T as a CNF using Chavira and Darwiche (2008) encoding Clauses for indicator ( λ ) and parameter ( θ ) logical variables: ⊕ x ∈ X i λ x ”states of X i are mutualy exclusive” X i

  29. Encoding transformed BN2T as a CNF using Chavira and Darwiche (2008) encoding Clauses for indicator ( λ ) and parameter ( θ ) logical variables: ⊕ x ∈ X i λ x ”states of X i are mutualy exclusive” X i ”states of Y ′ j λ y j are mutualy exclusive” ⊕ y ∈ Y ′ Y ′ j

Recommend

Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh

Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh

Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives MAP inference and its complexity exact & approximate MAP inference max-product and max-sum

957 views • 74 slides
15-780 Graduate Artificial Intelligence: Probabilistic inference J. Zico Kolter (this

15-780 Graduate Artificial Intelligence: Probabilistic inference J. Zico Kolter (this

15-780 Graduate Artificial Intelligence: Probabilistic inference J. Zico Kolter (this lecture) and Nihar Shah Carnegie Mellon University Spring 2020 1 Outline Probabilistic graphical models Probabilistic inference Exact inference

665 views • 39 slides
Probabilistic Graphical Models Guest Lecture by Narges Razavian Machine Learning Class April 14

Probabilistic Graphical Models Guest Lecture by Narges Razavian Machine Learning Class April 14

Probabilistic Graphical Models Guest Lecture by Narges Razavian Machine Learning Class April 14 2017 Today What is probabilistic graphical model and why it is useful? Bayesian Networks Basic Inference Generative Models Fancy Inference

785 views • 51 slides
Probabilistic Graphical Models Inference & Learning in DL Zhiting Hu Lecture 19, March 29,

Probabilistic Graphical Models Inference & Learning in DL Zhiting Hu Lecture 19, March 29,

Probabilistic Graphical Models Inference & Learning in DL Zhiting Hu Lecture 19, March 29, 2017 Reading: 1 Deep Generative Models l Explicit probabilistic models Provide an explicit parametric specification of the distribution of l

514 views • 50 slides
Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak

Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak

Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak Ravanbakhsh Fall 2019 Learning objective Learning objective an intuition for inference in graphical models why is it difficult? exact inference by

1.49k views • 62 slides
Graphical models Why All probabilistic inference and learning amount at repeated applications

Graphical models Why All probabilistic inference and learning amount at repeated applications

Graphical models Why All probabilistic inference and learning amount at repeated applications of the sum and product rules Probabilistic graphical models are graphical representations of the qualitative aspects of probability distribu-

551 views • 23 slides
Approximate inference: Sampling methods Probabilistic Graphical Models Sharif University of

Approximate inference: Sampling methods Probabilistic Graphical Models Sharif University of

Approximate inference: Sampling methods Probabilistic Graphical Models Sharif University of Technology Spring 2018 Soleymani Approximate inference Approximate inference techniques Deterministic approximation Variational algorithms

1.96k views • 97 slides
Approximate Inference: Mean Field Methods Probabilistic Graphical Models (10- Probabilistic

Approximate Inference: Mean Field Methods Probabilistic Graphical Models (10- Probabilistic

School of Computer Science Approximate Inference: Mean Field Methods Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 17, Nov 12, 2007 Receptor A Receptor A X 1 X 1 X 1 Receptor B Receptor B X 2

672 views • 18 slides
Probabilistic Graphical Models Lecture 15 Inference as Optimization CS/CNS/EE 155 Andreas

Probabilistic Graphical Models Lecture 15 Inference as Optimization CS/CNS/EE 155 Andreas

Probabilistic Graphical Models Lecture 15 Inference as Optimization CS/CNS/EE 155 Andreas Krause Announcements Homework 3 due next Monday (Nov 23) Project poster session on Friday December 4 (tentative) Final writeup (8 pages NIPS

805 views • 36 slides
Lifted Probabilistic Inference by First-Order Knowledge Compilation Guy Van den Broeck Nima

Lifted Probabilistic Inference by First-Order Knowledge Compilation Guy Van den Broeck Nima

Lifted Probabilistic Inference by First-Order Knowledge Compilation Guy Van den Broeck Nima Taghipour Wannes Meert Jesse Davis Luc De Raedt Lifted Inference in Probabilistic Logical Models - Tutorial - IJCAI11 18/07/11 Outline Overview

947 views • 80 slides
Variational inference Probabilistic Graphical Models Sharif University of Technology Soleymani

Variational inference Probabilistic Graphical Models Sharif University of Technology Soleymani

Variational inference Probabilistic Graphical Models Sharif University of Technology Soleymani Spring 2018 Some slides are adapted from Xing s slides Exact methods for inference Variable elimination Message Passing: shared terms

1.2k views • 84 slides
Probabilistic Graphical Models Probabilistic Graphical Models Markov Chain Monte Carlo Inference

Probabilistic Graphical Models Probabilistic Graphical Models Markov Chain Monte Carlo Inference

Probabilistic Graphical Models Probabilistic Graphical Models Markov Chain Monte Carlo Inference Fall 2019 Siamak Ravanbakhsh Learning objectives Learning objectives Markov chains the idea behind Markov Chain Monte Carlo (MCMC) two

1.03k views • 71 slides
Computationally efficient probabilistic inference with noisy threshold models based on a CP

Computationally efficient probabilistic inference with noisy threshold models based on a CP

Computationally efficient probabilistic inference with noisy threshold models based on a CP tensor decomposition Jirka Vomlel and Petr Tichavsk y Institute of Information Theory and Automation ( UTIA) Academy of Sciences of the Czech

1.06k views • 53 slides
Monte Carlo methods Draw random samples from the desired distribution Yield a stochastic

Monte Carlo methods Draw random samples from the desired distribution Yield a stochastic

School of Computer Science Approximate Inference: Monte Carlo Inference Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 18, Nov 19, 2007 Receptor A Receptor A X 1 X 1 X 1 Receptor B Receptor B X

503 views • 24 slides
Tractable Representations Inference Probabilistic Learning Models Applications Guy Van den

Tractable Representations Inference Probabilistic Learning Models Applications Guy Van den

Tractable Representations Inference Probabilistic Learning Models Applications Guy Van den Broeck University of California, Los Angeles based on joint AAAI-2020 and UAI-2019 tutorials with Antonio Vergari YooJung Choi University of

2.07k views • 177 slides
From probabilistic inference to Bayesian unfolding (passing through a toy model) Giulio

From probabilistic inference to Bayesian unfolding (passing through a toy model) Giulio

From probabilistic inference to Bayesian unfolding (passing through a toy model) Giulio DAgostini University and INFN Section of Roma1 Helmholtz School Advanced Topics in Statistics G ottingen, 17-20 October 2010 G.

1.57k views • 127 slides
Probabilistic Graphical Models Probabilistic Graphical Models Exponential family &

Probabilistic Graphical Models Probabilistic Graphical Models Exponential family &

Probabilistic Graphical Models Probabilistic Graphical Models Exponential family & Variational Inference I Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives entropy exponential family distribution duality in

1.38k views • 85 slides
Variable Elimination Probabilistic Graphical Models Sharif University of Technology Spring 2018

Variable Elimination Probabilistic Graphical Models Sharif University of Technology Spring 2018

Exact Inference: Variable Elimination Probabilistic Graphical Models Sharif University of Technology Spring 2018 Soleymani Probabilistic Inference and Learning We now have compact representations of probability distributions

697 views • 31 slides
III.3 Probabilistic Retrieval Models 1. Probabilistic Ranking Principle 2. Binary Independence

III.3 Probabilistic Retrieval Models 1. Probabilistic Ranking Principle 2. Binary Independence

III.3 Probabilistic Retrieval Models 1. Probabilistic Ranking Principle 2. Binary Independence Model 3. Okapi BM25 4. Tree Dependence Model 5. Bayesian Networks for IR ! ! Based on MRS Chapter 11 IR&DM 13/14 ! 48

628 views • 30 slides
Scalable Inference and Learning for High-Level Probabilistic Models Guy Van den Broeck KU Leuven

Scalable Inference and Learning for High-Level Probabilistic Models Guy Van den Broeck KU Leuven

Scalable Inference and Learning for High-Level Probabilistic Models Guy Van den Broeck KU Leuven Outline Motivation Why high-level representations? Why high-level reasoning? Intuition: Inference rules Liftability theory:

1.96k views • 152 slides
Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model Atlm

Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model Atlm

Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model Atlm Gne Baydin , Lukas Heinrich, Wahid Bhimji, Lei Shao, Saeid Naderiparizi, Andreas Munk, Jialin Liu, Bradley Gram-Hansen, Gilles Louppe, Lawrence

991 views • 67 slides
Chapter 4: Advanced IR Models 4.1 Probabilistic IR 4.2 Statistical Language Models (LMs) 4.3

Chapter 4: Advanced IR Models 4.1 Probabilistic IR 4.2 Statistical Language Models (LMs) 4.3

Chapter 4: Advanced IR Models 4.1 Probabilistic IR 4.2 Statistical Language Models (LMs) 4.3 Latent-Concept Models 4.3.1 Foundations from Linear Algebra 4.3.2 Latent Semantic Indexing (LSI) 4.3.3 Probabilistic Aspect Model (pLSI) 4-1 IRDM

330 views • 32 slides
Reducing the Cost of Probabilistic Knowledge Compilation Giso H. Dal, Steffen Michels and Peter

Reducing the Cost of Probabilistic Knowledge Compilation Giso H. Dal, Steffen Michels and Peter

Reducing the Cost of Probabilistic Knowledge Compilation Giso H. Dal, Steffen Michels and Peter J.F. Lucas Radboud University, Nijmegen (The Netherlands) Outline The Problem 1 Exact Inference by Weighted Model Counting 2 Bayesian Networks

440 views • 30 slides
Part 3: Probabilistic Inference in Graphical Models Sebastian Nowozin and Christoph H. Lampert

Part 3: Probabilistic Inference in Graphical Models Sebastian Nowozin and Christoph H. Lampert

Belief Propagation Variational Inference Sampling Break Part 3: Probabilistic Inference in Graphical Models Sebastian Nowozin and Christoph H. Lampert Colorado Springs, 25th June 2011 Sebastian Nowozin and Christoph H. Lampert Part 3:

951 views • 69 slides

More recommend