belief network inference
play

Belief network inference Four main approaches to determine posterior - PowerPoint PPT Presentation

Mar 29, 2023 •40 likes •330 views

Belief network inference Four main approaches to determine posterior distributions in belief networks: Variable Elimination: exploit the structure of the network to eliminate (sum out) the non-observed, non-query variables one at a time.

  1. Belief network inference Four main approaches to determine posterior distributions in belief networks: Variable Elimination: exploit the structure of the network to eliminate (sum out) the non-observed, non-query variables one at a time. Search-based approaches: enumerate some of the possible worlds, and estimate posterior probabilities from the worlds generated. Stochastic simulation: random cases are generated according to the probability distributions. Variational methods: find the closest tractable distribution to the (posterior) distribution we are interested in. � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 1

  2. Factors A factor is a representation of a function from a tuple of random variables into a number. We will write factor f on variables X 1 , . . . , X j as f ( X 1 , . . . , X j ). We can assign some or all of the variables of a factor: f ( X 1 = v 1 , X 2 , . . . , X j ), where v 1 ∈ dom ( X 1 ), is a factor on X 2 , . . . , X j . f ( X 1 = v 1 , X 2 = v 2 , . . . , X j = v j ) is a number that is the value of f when each X i has value v i . The former is also written as f ( X 1 , X 2 , . . . , X j ) X 1 = v 1 , etc. � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 2

  3. Example factors val Y Z X Y Z val t t 0.1 t t t 0.1 r ( X = t , Y , Z ): t f t t f 0.9 f t t f t 0.2 f f r ( X , Y , Z ): t f f 0.8 f t t 0.4 f t f 0.6 f f t 0.3 f f f 0.7 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 3

  4. Example factors val Y Z X Y Z val t t 0.1 t t t 0.1 r ( X = t , Y , Z ): t f 0.9 t t f 0.9 f t 0.2 t f t 0.2 f f 0.8 r ( X , Y , Z ): t f f 0.8 f t t 0.4 f t f 0.6 r ( X = t , Y , Z = f ): f f t 0.3 f f f 0.7 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 4

  5. Example factors val Y Z X Y Z val t t 0.1 t t t 0.1 r ( X = t , Y , Z ): t f t t f 0.9 f t t f t 0.2 f f r ( X , Y , Z ): t f f 0.8 f t t 0.4 Y val f t f 0.6 r ( X = t , Y , Z = f ): t f f t 0.3 f f f f 0.7 r ( X = t , Y = f , Z = f ) = � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 5

  6. Example factors val Y Z X Y Z val t t 0.1 t t t 0.1 r ( X = t , Y , Z ): t f t t f 0.9 f t t f t 0.2 f f r ( X , Y , Z ): t f f 0.8 f t t 0.4 Y val f t f 0.6 r ( X = t , Y , Z = f ): t 0.9 f f t 0.3 f 0.8 f f f 0.7 r ( X = t , Y = f , Z = f ) = 0 . 8 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 6

  7. Multiplying factors The product of factor f 1 ( X , Y ) and f 2 ( Y , Z ), where Y are the variables in common, is the factor ( f 1 × f 2 )( X , Y , Z ) defined by: ( f 1 × f 2 )( X , Y , Z ) = f 1 ( X , Y ) f 2 ( Y , Z ) . � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 7

  8. Multiplying factors example A B val t t 0.1 val A B C f 1 : t f 0.9 t t t 0.03 f t 0.2 t t f f f 0.8 t f t f 1 × f 2 : t f f B C val f t t t t 0.3 f t f f 2 : t f 0.7 f f t f t 0.6 f f f f f 0.4 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 8

  9. Multiplying factors example A B val t t 0.1 val A B C f 1 : t f 0.9 t t t 0.03 f t 0.2 t t f 0.07 f f 0.8 t f t 0.54 f 1 × f 2 : t f f 0.36 B C val f t t 0.06 t t 0.3 f t f 0.14 f 2 : t f 0.7 f f t 0.48 f t 0.6 f f f 0.32 f f 0.4 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 9

  10. Summing out variables We can sum out a variable, say X 1 with domain { v 1 , . . . , v k } , from factor f ( X 1 , . . . , X j ), resulting in a factor on X 2 , . . . , X j defined by: � ( f )( X 2 , . . . , X j ) X 1 = f ( X 1 = v 1 , . . . , X j ) + · · · + f ( X 1 = v k , . . . , X j ) � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 10

  11. Summing out a variable example val A B C t t t 0.03 val t t f 0.07 A C t f t 0.54 t t 0.57 f 3 : t f f 0.36 � B f 3 : t f f t t 0.06 f t f t f 0.14 f f f f t 0.48 f f f 0.32 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 11

  12. Summing out a variable example val A B C t t t 0.03 val t t f 0.07 A C t f t 0.54 t t 0.57 f 3 : t f f 0.36 � B f 3 : t f 0.43 f t t 0.06 f t 0.54 f t f 0.14 f f 0.46 f f t 0.48 f f f 0.32 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 12

  13. Exercise Given factors: val A B A val t t 0.6 A val s: t 0.75 t: t f 0.4 o: t 0.3 f 0.25 f t 0.2 f 0.1 f f 0.8 What is? (a) s × t (b) � A s × t (c) � B s × t (d) s × t × o (e) � A s × t × o (f) � b s × t × o � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 13

  14. Evidence If we want to compute the posterior probability of Z given evidence Y 1 = v 1 ∧ . . . ∧ Y j = v j : P ( Z | Y 1 = v 1 , . . . , Y j = v j ) = � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 14

  15. Evidence If we want to compute the posterior probability of Z given evidence Y 1 = v 1 ∧ . . . ∧ Y j = v j : P ( Z | Y 1 = v 1 , . . . , Y j = v j ) P ( Z , Y 1 = v 1 , . . . , Y j = v j ) = P ( Y 1 = v 1 , . . . , Y j = v j ) = � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 15

  16. Evidence If we want to compute the posterior probability of Z given evidence Y 1 = v 1 ∧ . . . ∧ Y j = v j : P ( Z | Y 1 = v 1 , . . . , Y j = v j ) P ( Z , Y 1 = v 1 , . . . , Y j = v j ) = P ( Y 1 = v 1 , . . . , Y j = v j ) P ( Z , Y 1 = v 1 , . . . , Y j = v j ) = � Z P ( Z , Y 1 = v 1 , . . . , Y j = v j ) . So the computation reduces to the probability of P ( Z , Y 1 = v 1 , . . . , Y j = v j ). We normalize at the end. � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 16

  17. Probability of a conjunction Suppose the variables of the belief network are X 1 , . . . , X n . To compute P ( Z , Y 1 = v 1 , . . . , Y j = v j ), we sum out the other variables, Z 1 , . . . , Z k = { X 1 , . . . , X n } − { Z } − { Y 1 , . . . , Y j } . We order the Z i into an elimination ordering. P ( Z , Y 1 = v 1 , . . . , Y j = v j ) = � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 17

  18. Probability of a conjunction Suppose the variables of the belief network are X 1 , . . . , X n . To compute P ( Z , Y 1 = v 1 , . . . , Y j = v j ), we sum out the other variables, Z 1 , . . . , Z k = { X 1 , . . . , X n } − { Z } − { Y 1 , . . . , Y j } . We order the Z i into an elimination ordering. P ( Z , Y 1 = v 1 , . . . , Y j = v j ) � � = · · · P ( X 1 , . . . , X n ) Y 1 = v 1 ,..., Y j = v j . Z k Z 1 = � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 18

  19. Probability of a conjunction Suppose the variables of the belief network are X 1 , . . . , X n . To compute P ( Z , Y 1 = v 1 , . . . , Y j = v j ), we sum out the other variables, Z 1 , . . . , Z k = { X 1 , . . . , X n } − { Z } − { Y 1 , . . . , Y j } . We order the Z i into an elimination ordering. P ( Z , Y 1 = v 1 , . . . , Y j = v j ) � � = · · · P ( X 1 , . . . , X n ) Y 1 = v 1 ,..., Y j = v j . Z k Z 1 n � � � = · · · P ( X i | parents ( X i )) Y 1 = v 1 ,..., Y j = v j . Z k Z 1 i =1 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 19

  20. Computing sums of products Computation in belief networks reduces to computing the sums of products. How can we compute ab + ac efficiently? � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 20

  21. Computing sums of products Computation in belief networks reduces to computing the sums of products. How can we compute ab + ac efficiently? Distribute out the a giving a ( b + c ) � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 21

  22. Computing sums of products Computation in belief networks reduces to computing the sums of products. How can we compute ab + ac efficiently? Distribute out the a giving a ( b + c ) � n How can we compute � i =1 P ( X i | parents ( X i )) Z 1 efficiently? � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 22

  23. Computing sums of products Computation in belief networks reduces to computing the sums of products. How can we compute ab + ac efficiently? Distribute out the a giving a ( b + c ) � n How can we compute � i =1 P ( X i | parents ( X i )) Z 1 efficiently? Distribute out those factors that don’t involve Z 1 . � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 23

  24. Variable elimination algorithm To compute P ( Z | Y 1 = v 1 ∧ . . . ∧ Y j = v j ): Construct a factor for each conditional probability. Set the observed variables to their observed values. Sum out each of the other variables (the { Z 1 , . . . , Z k } ) according to some elimination ordering. Multiply the remaining factors. Normalize by dividing the resulting factor f ( Z ) by � Z f ( Z ). � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 24

Recommend

Inference in Belief Networks CMPUT 366: Intelligent Systems P&M 8.4 Lecture Outline

Inference in Belief Networks CMPUT 366: Intelligent Systems P&M 8.4 Lecture Outline

Inference in Belief Networks CMPUT 366: Intelligent Systems P&M 8.4 Lecture Outline 1. Recap 2. Factors 3. Variable Elimination 4. Efficiency Recap: Belief Networks Definition: A belief network (or Bayesian network )

1.27k views • 23 slides
Bayesian Belief Network 14.4 Inference Decision Theoretic Agents Introduction to Probability

Bayesian Belief Network 14.4 Inference Decision Theoretic Agents Introduction to Probability

RN, Chapter Bayesian Belief Network 14.4 Inference Decision Theoretic Agents Introduction to Probability [Ch13] Belief networks [Ch14] Introduction [Ch14.1-14.2] Bayesian Net Inference [Ch14.4] (Bucket Elimination) Dynamic

763 views • 33 slides
Variable Elimination 1 Inference Exact inference Enumeration Variable elimination

Variable Elimination 1 Inference Exact inference Enumeration Variable elimination

Variable Elimination 1 Inference Exact inference Enumeration Variable elimination Junction trees and belief propagation Approximate inference Loopy belief propagation Sampling methods: likelihood weighting, Markov

447 views • 12 slides
Overview Independence Belief Networks Conditional Independence Belief networks Chris

Overview Independence Belief Networks Conditional Independence Belief networks Chris

Overview Independence Belief Networks Conditional Independence Belief networks Chris Williams Constructing belief networks Inference in belief networks School of Informatics, University of Edinburgh Learning in belief

498 views • 6 slides
Belief Networks Some Belief Network references E. Charniak Bayesian Networks without

Belief Networks Some Belief Network references E. Charniak Bayesian Networks without

Belief Networks Some Belief Network references E. Charniak Bayesian Networks without Tears, AI Magazine Winter 1991, pp 50-63 Chris Williams, School of Informatics University of Edinburgh D. Heckerman, A Tutorial on Learning

486 views • 6 slides
Belief Propagation Algorithm Interest Group presentation by Eli Chertkov Inference Statistical

Belief Propagation Algorithm Interest Group presentation by Eli Chertkov Inference Statistical

Belief Propagation Algorithm Interest Group presentation by Eli Chertkov Inference Statistical inference is the determination of an underlying probability distribution from observed data. Probabilistic Graphical Models 1 1 =

522 views • 17 slides
Belief Propagation for Spatial Network Embeddings Andrew Frank Alex Ihler Padhraic Smyth

Belief Propagation for Spatial Network Embeddings Andrew Frank Alex Ihler Padhraic Smyth

Belief Propagation for Spatial Network Embeddings Andrew Frank Alex Ihler Padhraic Smyth Department of Computer Science UC Irvine August 25, 2009 Outline 1 Graphical Models Markov Random Fields Inference 2 Self-Localization Problem

1.04k views • 79 slides
Travel Time Estimation using Approximate Belief States on a Hidden Markov Model Walid Krichene

Travel Time Estimation using Approximate Belief States on a Hidden Markov Model Walid Krichene

Travel Time Estimation using Approximate Belief States on a Hidden Markov Model Walid Krichene Overview Context Inference on a HMM Modeling framework and exact inference Approximate Inference: the Boyen-Koller algorithm Graph Partitioning

1.04k views • 82 slides
Approximate inference on planar graphs using Loop Calculus and Belief Propagation Vicen Gmez 1

Approximate inference on planar graphs using Loop Calculus and Belief Propagation Vicen Gmez 1

Approximate inference on planar graphs using Loop Calculus and Belief Propagation Vicen Gmez 1 Hilbert J.Kappen 1 M. Chertkov 2 1 Department of Biophysics Radboud University, Nijmegen, The Netherlands 2 Theoretical Division and Center of

711 views • 28 slides
A DYNAMIC BAYESIAN BELIEF NETWORK APPROACH FOR MODELING THE ATM NETWORK DELAYS Yi it Bekir

A DYNAMIC BAYESIAN BELIEF NETWORK APPROACH FOR MODELING THE ATM NETWORK DELAYS Yi it Bekir

A DYNAMIC BAYESIAN BELIEF NETWORK APPROACH FOR MODELING THE ATM NETWORK DELAYS Yi it Bekir Kaya Data Science Researcher at CAL, Aeronautics Graduate Student Prof. Gkhan nalhan Director of CAL Istanbul Technical University Controls

715 views • 67 slides
Learning a Belief Network If you know the structure have observed all of the variables

Learning a Belief Network If you know the structure have observed all of the variables

Learning a Belief Network If you know the structure have observed all of the variables have no missing data you can learn each conditional probability separately. D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture

479 views • 17 slides
Learning a Belief Network If you know the structure have observed all of the variables

Learning a Belief Network If you know the structure have observed all of the variables

Learning a Belief Network If you know the structure have observed all of the variables have no missing data you can learn each conditional probability separately. D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture

434 views • 40 slides
Generative vs. discriminative Generative Discriminative Belief network A is more More

Generative vs. discriminative Generative Discriminative Belief network A is more More

Generative vs. discriminative Generative Discriminative Belief network A is more More robust modular Dont need precise model Class-conditional densities are specification, so long as it is likely to be local, characteristic

1.09k views • 15 slides
Meta-Amoruized Variational Inference and Learning Kristy Choi CS236: December 4th, 2019

Meta-Amoruized Variational Inference and Learning Kristy Choi CS236: December 4th, 2019

Meta-Amoruized Variational Inference and Learning Kristy Choi CS236: December 4th, 2019 Probabilistic Inference Probabilistic inference is a paruicular way of viewing the world: + = Observations Updated (posterior) Prior belief Typically

452 views • 28 slides
Bayesian Belief Networks Decision Theoretic Agents Introduction to Probability [Ch13]

Bayesian Belief Networks Decision Theoretic Agents Introduction to Probability [Ch13]

RN, Chapter 14 Bayesian Belief Networks Decision Theoretic Agents Introduction to Probability [Ch13] Belief networks [Ch14] Introduction [Ch14.1-14.2] Bayesian Net Inference [Ch14.4] (Bucket Elimination) Dynamic Belief

634 views • 49 slides
GPU INFERENCE ENGINE Michael Andersch, 7 th April 2016 WHAT IS INFERENCE, ANYWAYS? Building a

GPU INFERENCE ENGINE Michael Andersch, 7 th April 2016 WHAT IS INFERENCE, ANYWAYS? Building a

April 4-7, 2016 | Silicon Valley NVIDIA GIE: HIGH-PERFORMANCE GPU INFERENCE ENGINE Michael Andersch, 7 th April 2016 WHAT IS INFERENCE, ANYWAYS? Building a deep neural network based application Step 1: Use data to train the neural network -

1.02k views • 17 slides
Adaptive Distributed Convolutional Neural Network Inference at the Network Edge with ADCNN 17

Adaptive Distributed Convolutional Neural Network Inference at the Network Edge with ADCNN 17

Adaptive Distributed Convolutional Neural Network Inference at the Network Edge with ADCNN 17 August 2020 ICPP 2020 Sai Qian Zhang , Jieyu Lin, Qi Zhang Executing DNN Inference Tasks for End Users Option 2: cloud only Option 1: edge only

579 views • 17 slides
Variational Inference for Tutorial Outline Structured NLP Models 1. Structured Models and Factor

Variational Inference for Tutorial Outline Structured NLP Models 1. Structured Models and Factor

Variational Inference for Tutorial Outline Structured NLP Models 1. Structured Models and Factor Graphs 2. Mean Field 3. Structured Mean Field 4. Belief Propagation 5. Structured Belief Propagation ACL, August 4, 2013 David Burkett and Dan

390 views • 36 slides
Exact inference (Ch. 14) Bayesian Network A Bayesian network (Bayes net) is: (1) a directed

Exact inference (Ch. 14) Bayesian Network A Bayesian network (Bayes net) is: (1) a directed

Exact inference (Ch. 14) Bayesian Network A Bayesian network (Bayes net) is: (1) a directed graph (2) acyclic Additionally, Bayesian networks are assumed to be defined by conditional probability tables (3) P(x | Parents(x) ) We have actually

673 views • 28 slides
Inference Suppose you are given a Bayesian network with the graph structure and the parameters

Inference Suppose you are given a Bayesian network with the graph structure and the parameters

Inference Suppose you are given a Bayesian network with the graph structure and the parameters CS 331: Artificial Intelligence all figured out Bayesian Networks (Inference) Now you would like to use it to do inference You need

417 views • 5 slides
Inference Suppose you are given a Bayesian network with the graph structure and the parameters

Inference Suppose you are given a Bayesian network with the graph structure and the parameters

CS 331: Artificial Intelligence Bayesian Networks (Inference) 1 Inference Suppose you are given a Bayesian network with the graph structure and the parameters all figured out Now you would like to use it to do inference You need

465 views • 13 slides
Introduction: Belief vs Degrees of Belief Hannes Leitgeb LMU Munich October 2014 My three

Introduction: Belief vs Degrees of Belief Hannes Leitgeb LMU Munich October 2014 My three

Introduction: Belief vs Degrees of Belief Hannes Leitgeb LMU Munich October 2014 My three lectures will be devoted to answering this question: How does rational (all-or-nothing) belief relate to degrees of belief? But first I will prepare the

1.03k views • 33 slides
Lecture 15: Basic graph concepts, Belief Network and HMM Dr. Chengjiang Long Computer Vision

Lecture 15: Basic graph concepts, Belief Network and HMM Dr. Chengjiang Long Computer Vision

Lecture 15: Basic graph concepts, Belief Network and HMM Dr. Chengjiang Long Computer Vision Researcher at Kitware Inc. Adjunct Professor at RPI. Email: longc3@rpi.edu About Final Projects No. Project name Authors 1 Neural Style Transfer

657 views • 54 slides
Variational Network Inference: Strong and Stable with Concrete Support Amir Dezfouli, Edwin V.

Variational Network Inference: Strong and Stable with Concrete Support Amir Dezfouli, Edwin V.

Variational Network Inference: Strong and Stable with Concrete Support Amir Dezfouli, Edwin V. Bonilla and Richard Nock Network Structure Discovery: A Flexible Approach N nodes, T observations: = { y i , t i } 1 Goal: Learn network

428 views • 7 slides

More recommend