probabilistic graphical models
play

Probabilistic Graphical Models Christian Borgelt Dept. of Knowledge - PowerPoint PPT Presentation

Sep 15, 2023 •111 likes •1.06k views

Probabilistic Graphical Models Christian Borgelt Dept. of Knowledge Processing and Language Engineering Otto-von-Guericke-University of Magdeburg Universit atsplatz 2, D-39106 Magdeburg, Germany E-mail: borgelt@iws.cs.uni-magdeburg.de WWW:

  1. Probabilistic Graphical Models Christian Borgelt Dept. of Knowledge Processing and Language Engineering Otto-von-Guericke-University of Magdeburg Universit¨ atsplatz 2, D-39106 Magdeburg, Germany E-mail: borgelt@iws.cs.uni-magdeburg.de WWW: http://fuzzy.cs.uni-magdeburg.de/~borgelt 1

  2. Contents ✎ Inference Networks / Graphical Models ✎ Relational Networks: Decomposition and Reasoning ✎ Probabilistic Networks: Decomposition and Reasoning ✎ Conditional Independence Graphs ✎ Graphs and Decompositions ✎ Evidence Propagation in Graphs ✎ Danish Jersey Cattle Blood Type Determination: An Example ✎ Learning Relational Networks from Data ✎ Learning Probabilistic Networks from Data ✎ Probabilistic Networks and Causality ✎ Fault Analysis at DaimlerChrysler: An Application 2

  3. Inference Networks / Graphical Models ✎ Decomposition: Under certain conditions a distribution ✍ (e.g. a probability distribution) on a multi-dimensional domain, which encodes prior or generic knowledge about this domain, can be decomposed into a set ❢ ✍ 1 ❀ ✿ ✿ ✿ ❀ ✍ s ❣ of (overlapping) distributions on lower-dimensional subspaces. ✎ Simplified Reasoning: If such a decomposition is possible, it is sufficient to know the distributions on the subspaces to draw all inferences in the domain under consideration that can be drawn using the original distribution ✍ . ✎ Since such a decomposition is usually represented as a network and since it is used to draw inferences, it can be called an inference network . The edges of the network indicate the paths along which evidence has to be propagated. ✎ Another popular name is graphical model , where “graphical” indicates that it is based on a graph in the sense of graph theory. 3

  4. ✔ ✕ ✖ ✗ ✘ ✙ ✚ ✛ ✜ ✢ A Simple Example Example World Relation ☛ ✍ color shape size ☛ ✍ small ✡ ✍ medium ✡ ✍ small ✡ ☞ medium ✡ ☞ medium ✟ ✌ large ✠ ✌ medium ✠ ☞ medium ✠ ☞ medium ✎ 10 simple geometrical objects, 3 attributes. large ✎ One object is chosen at random and examined. ✎ Inferences are drawn about the unobserved attributes. 4

  5. ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✥ ✪ ✥ ✥ ✥ ✥ ✥ ✪ ✏ ✩ The Reasoning Space ☛ ✡ ✟ ✠ ✟ ☞ ☞ ✌ ✍ large medium medium small ✎ The reasoning space consists of a finite set Ω of world states. ✎ The world states are described by a set of attributes ❆ ✐ , whose domains ❢ ❛ ( ✐ ) 1 ❀ ✿ ✿ ✿ ❀ ❛ ( ✐ ) ❦ i ❣ can be seen as sets of propositions or events. ✎ The events in a domain are mutually exclusive and exhaustive. ✎ The reasoning space is assumed to contain the true, but unknown state ✦ 0 . 5

  6. ✦ ✦ ★ ★ ★ ★ ✦ ★ ★ ✥ ✧ ✩ ✩ ★ The Relation in the Reasoning Space Relation Relation in the Reasoning Space ☛ ✡ ✟ ✠ ☛ ✍ color shape size ☞ ☛ ✍ small ✡ ✍ ✌ medium ✡ ✍ small ✍ ✡ ☞ medium ✡ ☞ medium ✟ ✌ large large ✠ ✌ medium medium small ✠ ☞ medium ✠ ☞ medium large Each cube represents one tuple. 6

  7. ✦ ✩ ❆ ✧❆ ❅ ❅ ✩ ✩ ✩ ★ ★ ✦ ✥ ✩ ✩ Reasoning ✎ Let it be known (e.g. from an observation) that the given object is green. This information considerably reduces the space of possible value combinations. ✎ From the prior knowledge it follows that the given object must be ✍ either a triangle or a square and ✍ either medium or large. ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ large large medium medium small small 7

  8. ❅ ✩ ❆ ❆ ❆ ❆ ❆ ✩ ✩ ✩ ✩ ✩ ✩ ✪ ❆ ❅ ❅ ❅ ❅ ❅ ✪❆ ✩ ✩ ✩ ✩ ✩ ❆ ✩ ✩ ✧ ❇ ❇ ❇ ❇ ✪❇ ✩ ✩ ✩ ✩ ✩ ✩ ✥ ✩ ★ ★ ✦ ★ ★ ★ ★ ✦ ✦ ★ ✩ ✩ Prior Knowledge and Its Projections ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ large large medium medium small small ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ large large medium medium small small 8

  9. ✦ ✄ ✥ ★ ✦ ✦ ✦ ✦ ★ ✧ ✩ ✩ ✂ ★ ✧ ✩ ✩ ✧ ✥ ★ ★ ✦ ★ ★ ★ ★ ✥ ★ ✧ ✧ ✥ ✥ ✥ ✩ ✩ ✩ ★ ★ ★ ✪ ✪ ✪ ✪ ✪ Cylindrical Extensions and Their Intersection ☛ ✡ ✟ ✠ ☞ Intersecting the cylindrical ex- ✌ tensions of the projection to the subspace formed by color ✍ and shape and of the projec- tion to the subspace formed by large shape and size yields the origi- medium nal three-dimensional relation. small ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ large large medium medium small small 9

  10. ❅ ❈ ❅ ❅ ❅ ❅ ❅ ❈ � ❈ ❈ � ❈ ❈ ✁ ❈ ❈❅ ❅ ❈ ❈ ✄ ❈ ❅ ❅ ❅ ❈ ❈ ❈ Reasoning with Projections The same result can be obtained using only the projections to the subspaces without reconstructing the original three-dimensional space: ☛ ✡ ✟ ✠ s m l size color project extend shape ☞ ☞ ✌ ✌ project extend ✍ ✍ ☛ ✡ ✟ ✠ s m l ✛ ✘ ✛ ✘ ✛ ✘ This justifies a network representation: color shape size ✚ ✙ ✚ ✙ ✚ ✙ 10

  11. ✩ ★ ✥ ✥ ✥ ✩ ✩ ✦ ✦ ✦ ★ ✥ ✄ ✂ ★ ✦ ✦ ★ ★ ★ ✥ ✥ ✦ ★ ✥ ✥ ✥ ✧ ✧ ✧ ✥ ✥ ★ ✦ ✦ ✦ ✦ ✦ ✦ ✦ ✦ ✦ ✩ ★ ★ ✧ ✦ ✦ ★ ★ ✦ ✦ ✧ ★ ✥ ✥ ✥ ✧ ✥ ✩ ✩ ★ ✦ ✩ ✩ ✥ ✧ Using other Projections ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ large large medium medium small small ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ large large medium medium small small 11

  12. ❅ ✩ ❆ ❆ ❆ ❆ ❆ ❆ ✪❆ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✪ ❅ ❅ ❅ ✩ ❅ ❅ ✩ ✩ ✩ ❆ ✩ ✩ ✧ ❇ ❇ ❇ ❇ ✪❇ ✩ ✩ ✩ ✩ ✩ ✩ ✥ ✩ ★ ★ ✦ ★ ★ ★ ★ ✦ ✦ ★ ✦ ✦ ✩ Is Decomposition Always Possible? ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ 2 large large medium medium 1 small small ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ large large medium medium small small 12

  13. Possibility-Based Formalization Definition: Let Ω be a (finite) sample space. A discrete possibility measure ❘ on Ω is a function ❘ : 2 Ω ✦ ❢ 0 ❀ 1 ❣ satisfying 1. ❘ ( ❀ ) = 0 and 2. ✽ ❊ 1 ❀ ❊ 2 ✒ Ω : ❘ ( ❊ 1 ❬ ❊ 2 ) = max ❢ ❘ ( ❊ 1 ) ❀ ❘ ( ❊ 2 ) ❣ . ✎ Similar to Kolmogorov’s axioms of probability theory. ✎ If an event ❊ can occur (if it is possible), then ❘ ( ❊ ) = 1 ❀ otherwise (if ❊ cannot occur/is impossible) then ❘ ( ❊ ) = 0 ✿ ✎ ❘ (Ω) = 1 is not required, because this would exclude the empty relation. ✎ From the axioms it follows ❘ ( ❊ 1 ❭ ❊ 2 ) ✔ min ❢ ❘ ( ❊ 1 ) ❀ ❘ ( ❊ 2 ) ❣ ✿ ✎ Attributes are introduced as random variables (as in probability theory). ✎ ❘ ( ❆ = ❛ ) is an abbreviation of ❘ ( ❢ ✦ ❥ ❆ ( ✦ ) = ❛ ❣ ) 13

  14. Possibility-Based Formalization (continued) Definition: Let ❳ = ❢ ❆ 1 ❀ ✿ ✿ ✿ ❀ ❆ ♥ ❣ be a set of attributes defined on a (finite) sample space Ω with respective domains dom( ❆ ✐ ), ✐ = 1 ❀ ✿ ✿ ✿ ❀ ♥ . A relation r ❳ over ❳ is the restriction of a discrete possibility measure ❘ on Ω to the set of all events that can be defined by stating values for all attributes in ❳ . That is, r ❳ = ❘ ❥ ❊ X , where � � � ❊ ✷ 2 Ω ❊ ❳ = � ✾ ❛ 1 ✷ dom( ❆ 1 ) : ✿ ✿ ✿ ✾ ❛ ♥ ✷ dom( ❆ ♥ ) : � � ❊ � = ❆ ❥ = ❛ ❥ ❆ j ✷ ❳ � � � ❊ ✷ 2 Ω = � ✾ ❛ 1 ✷ dom( ❆ 1 ) : ✿ ✿ ✿ ✾ ❛ ♥ ✷ dom( ❆ ♥ ) : � � �� � � ❊ = ✦ ✷ Ω ❆ ❥ ( ✦ ) = ❛ ❥ ✿ � ❆ j ✷ ❳ ✎ Corresponds to the notion of a probability distribution. ✎ Advantage of this formalization: No index transformation functions are needed for projections, there are just fewer terms in the conjunctions. 14

  15. Possibility-Based Formalization (continued) Definition: Let ❯ = ❢ ❆ 1 ❀ ✿ ✿ ✿ ❀ ❆ ♥ ❣ be a set of attributes and r ❯ a relation over ❯ . Furthermore, let ▼ = ❢ ▼ 1 ❀ ✿ ✿ ✿ ❀ ▼ ♠ ❣ ✒ 2 ❯ be a set of nonempty (but not neces- sarily disjoint) subsets of ❯ satisfying � ▼ = ❯✿ ▼ ✷▼ r ❯ is called decomposable w.r.t. ▼ iff ✽ ❛ 1 ✷ dom( ❆ 1 ) : ✿ ✿ ✿ ✽ ❛ ♥ ✷ dom( ❆ ♥ ) : � � � � �� � � r ❯ ❆ ✐ = ❛ ✐ = min r ▼ ❆ ✐ = ❛ ✐ ✿ ▼ ✷▼ ❆ i ✷ ❯ ❆ i ✷ ▼ If r ❯ is decomposable w.r.t. ▼ , the set of relations ❘ ▼ = ❢ r ▼ 1 ❀ ✿ ✿ ✿ ❀ r ▼ m ❣ = ❢ r ▼ ❥ ▼ ✷ ▼❣ is called the decomposition of r ❯ . 15

Recommend

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019 Siamak Ravanbakhsh Learning objectives Learning objectives Markov networks: how it represents a prob. dist. independence assumptions

941 views • 57 slides
Computer Science Let me be provocative Probabilistic graphical models is how we do probabilistic

Computer Science Let me be provocative Probabilistic graphical models is how we do probabilistic

Computer Science Let me be provocative Probabilistic graphical models is how we do probabilistic AI! Graphical models of variable-level (in)dependence are a broken abstraction. [VdB KRR15] Let me be provocative Probabilistic graphical models

782 views • 63 slides
Probabilistic Graphical Models Probabilistic Graphical Models Parameter learning in Bayesian

Probabilistic Graphical Models Probabilistic Graphical Models Parameter learning in Bayesian

Probabilistic Graphical Models Probabilistic Graphical Models Parameter learning in Bayesian networks Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives likelihood function and MLE role of the sufficient statistics MLE for

644 views • 60 slides
Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak

Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak

Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak Ravanbakhsh Fall 2019 Learning objective Learning objective an intuition for inference in graphical models why is it difficult? exact inference by

1.49k views • 62 slides
Probabilistic Graphical Models Probabilistic Graphical Models Exponential family &

Probabilistic Graphical Models Probabilistic Graphical Models Exponential family &

Probabilistic Graphical Models Probabilistic Graphical Models Exponential family & Variational Inference I Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives entropy exponential family distribution duality in

1.38k views • 85 slides
The Elimination Algorithm Probabilistic Graphical Models (10- Probabilistic Graphical Models

The Elimination Algorithm Probabilistic Graphical Models (10- Probabilistic Graphical Models

School of Computer Science The Elimination Algorithm Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 4, Sep 26, 2007 Receptor A Receptor A Receptor B Receptor B X 1 X 1 X 1 X 2 X 2 X 2 Eric Xing

457 views • 17 slides
Probabilistic Graphical Models CMSC 678 UMBC Probabilistic Graphical Models A graph G that

Probabilistic Graphical Models CMSC 678 UMBC Probabilistic Graphical Models A graph G that

Probabilistic Graphical Models CMSC 678 UMBC Probabilistic Graphical Models A graph G that represents a probability distribution over random variables 1 , , Probabilistic Graphical Models A graph G that represents a

1.74k views • 129 slides
Probabilistic Graphical Models Probabilistic Graphical Models parameter learning in undirected

Probabilistic Graphical Models Probabilistic Graphical Models parameter learning in undirected

Probabilistic Graphical Models Probabilistic Graphical Models parameter learning in undirected models Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives the form of likelihood for undirected models why is it difficult to

826 views • 49 slides
Probabilistic Graphical Models Probabilistic Graphical Models Gaussian Network Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Gaussian Network Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Gaussian Network Models Fall 2019 Siamak Ravanbakhsh Learning objectives Learning objectives multivariate Gaussian density: different parametrizations marginalization and

540 views • 39 slides
Probabilistic Graphical Models Probabilistic Graphical Models Relationship between the directed

Probabilistic Graphical Models Probabilistic Graphical Models Relationship between the directed

Probabilistic Graphical Models Probabilistic Graphical Models Relationship between the directed & undirected models Siamak Ravanbakhsh Fall 2019 Learning Objective Learning Objective understand the relationship between CIs in directed

694 views • 23 slides
CS 6782: Fall 2010 Probabilistic Graphical Models Guozhang Wang December 10, 2010 1

CS 6782: Fall 2010 Probabilistic Graphical Models Guozhang Wang December 10, 2010 1

CS 6782: Fall 2010 Probabilistic Graphical Models Guozhang Wang December 10, 2010 1 Introduction to Probabilistic Graphical Models In a probabilistic graphical model, each node represents a random variable, and the links express probabilistic

380 views • 21 slides
Probabilistic Graphical Models Probabilistic Graphical Models Loopy BP and Bethe Free Energy

Probabilistic Graphical Models Probabilistic Graphical Models Loopy BP and Bethe Free Energy

Probabilistic Graphical Models Probabilistic Graphical Models Loopy BP and Bethe Free Energy Fall 2019 Siamak Ravanbakhsh Learning objective Learning objective loopy belief propagation its variational derivation: Bethe approximation So

832 views • 58 slides
Probabilistic Graphical Models Probabilistic Graphical Models introduction to learning Siamak

Probabilistic Graphical Models Probabilistic Graphical Models introduction to learning Siamak

Probabilistic Graphical Models Probabilistic Graphical Models introduction to learning Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives different goals of learning a graphical model effect of goals on the learning setup

273 views • 23 slides
Directed Graphical Models: Bayesian Networks Probabilistic Graphical Models Sharif University of

Directed Graphical Models: Bayesian Networks Probabilistic Graphical Models Sharif University of

Directed Graphical Models: Bayesian Networks Probabilistic Graphical Models Sharif University of Technology Soleymani Spring 2018 Basics Multivariate distributions with large number of variables Independency assumptions are useful

503 views • 36 slides
Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh

Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh

Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives MAP inference and its complexity exact & approximate MAP inference max-product and max-sum

957 views • 74 slides
Graphical models Why All probabilistic inference and learning amount at repeated applications

Graphical models Why All probabilistic inference and learning amount at repeated applications

Graphical models Why All probabilistic inference and learning amount at repeated applications of the sum and product rules Probabilistic graphical models are graphical representations of the qualitative aspects of probability distribu-

551 views • 23 slides
Probabilistic Graphical Models Probabilistic Graphical Models Learning with partial observations

Probabilistic Graphical Models Probabilistic Graphical Models Learning with partial observations

Probabilistic Graphical Models Probabilistic Graphical Models Learning with partial observations Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives different types of missing data learning with missing data and hidden vars:

1.04k views • 80 slides
Sampling Techniques for Probabilistic and Deterministic Graphical models ICS 276, Fall 2014

Sampling Techniques for Probabilistic and Deterministic Graphical models ICS 276, Fall 2014

Sampling Techniques for Probabilistic and Deterministic Graphical models ICS 276, Fall 2014 Bozhena Bidyuk Rina Dechter Reading Darwiche chapter 15, related papers Overview 1. Probabilistic Reasoning/Graphical models 2. Importance

923 views • 63 slides
Sampling Techniques for Probabilistic and Deterministic Graphical models ICS 276, Spring 2017

Sampling Techniques for Probabilistic and Deterministic Graphical models ICS 276, Spring 2017

Sampling Techniques for Probabilistic and Deterministic Graphical models ICS 276, Spring 2017 Bozhena Bidyuk Rina Dechter Reading Darwiche chapter 15, related papers Overview 1. Probabilistic Reasoning/Graphical models 2. Importance

1.03k views • 54 slides
Sampling Techniques for Probabilistic and Deterministic Graphical models ICS 276, Spring 2017

Sampling Techniques for Probabilistic and Deterministic Graphical models ICS 276, Spring 2017

Sampling Techniques for Probabilistic and Deterministic Graphical models ICS 276, Spring 2017 Bozhena Bidyuk Rina Dechter Reading Darwiche chapter 15, related papers Overview 1. Probabilistic Reasoning/Graphical models 2. Importance

889 views • 63 slides
Two types of GMs Directed edges give causality relationships ( Bayesian Network or Directed

Two types of GMs Directed edges give causality relationships ( Bayesian Network or Directed

School of Computer Science Undirected Graphical Models Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 2, Sep 17, 2007 Receptor A Receptor A X 1 X 1 X 1 Receptor B Receptor B X 2 X 2 X 2 Eric

692 views • 20 slides
Two types of GMs Directed edges give causality relationships ( Bayesian Network or Directed

Two types of GMs Directed edges give causality relationships ( Bayesian Network or Directed

School of Computer Science Undirected Graphical Models Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 2, Sep 17, 2007 Receptor A Receptor A X 1 X 1 X 1 Receptor B Receptor B X 2 X 2 X 2 Eric

485 views • 21 slides
Graphical Models Graphical Models Bayesian Networks Siamak Ravanbakhsh Fall 2019 Previously on

Graphical Models Graphical Models Bayesian Networks Siamak Ravanbakhsh Fall 2019 Previously on

Graphical Models Graphical Models Bayesian Networks Siamak Ravanbakhsh Fall 2019 Previously on Previously on Probabilistic Graphical Models Probabilistic Graphical Models Probability distribution and density functions Random variable

1.39k views • 104 slides
Probabilistic Graphical Models Probabilistic Graphical Models Structure learning in Bayesian

Probabilistic Graphical Models Probabilistic Graphical Models Structure learning in Bayesian

Probabilistic Graphical Models Probabilistic Graphical Models Structure learning in Bayesian networks Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives why structure learning is hard? two approaches to structure learning

1.04k views • 74 slides

More recommend