bayes networks
play

Bayes Networks Robert Platt Northeastern University Some images, - PowerPoint PPT Presentation

Jan 27, 2023 •528 likes •1.35k views

Bayes Networks Robert Platt Northeastern University Some images, slides, or ideas are used from: 1. AIMA 2. Berkeley CS188 3. Chris Amato What is a Bayes Net? What is a Bayes Net? Suppose we're given this distribution: cavity P(T,C)

  1. Bayes Networks Robert Platt Northeastern University Some images, slides, or ideas are used from: 1. AIMA 2. Berkeley CS188 3. Chris Amato

  2. What is a Bayes Net?

  3. What is a Bayes Net? Suppose we're given this distribution: cavity P(T,C) P(T,!C) P(!T,C) P(!T,!C) true 0.16 0.018 0.018 0.002 false 0.048 0.19 0.11 0.448 Variables: Cavity Toothache (T) Catch (C)

  4. What is a Bayes Net? Suppose we're given this distribution: Can we summarize aspects of this probability distribution with a graph? cavity P(T,C) P(T,!C) P(!T,C) P(!T,!C) true 0.16 0.018 0.018 0.002 false 0.048 0.19 0.11 0.448 Variables: Cavity Toothache (T) Catch (C)

  5. What is a Bayes Net? cavity P(T,C) P(T,!C) P(!T,C) P(!T,!C) true 0.16 0.018 0.018 0.002 false 0.048 0.19 0.11 0.448 This diagram captures important information that is hard to extract from table by looking at it: Cavity toothache catch

  6. What is a Bayes Net? cavity P(T,C) P(T,!C) P(!T,C) P(!T,!C) true 0.16 0.018 0.018 0.002 false 0.048 0.19 0.11 0.448 This diagram captures important information that is hard to extract from table by looking at it: Cavity causes Cavity causes toothache Cavity catch toothache catch

  7. What is a Bayes Net? Something that looks like this: Bubbles: random variables Arrows: dependency relationships between variables

  8. What is a Bayes Net? Something that looks like this: Bubbles: random variables Arrows: dependency relationships between variables A Bayes net is a compact way of representing a probability distribution

  9. Bayes net example Diagram encodes the fact that toothache is conditionally independent of catch given Cavity cavity – therefore, all we need are the following distributions toothache catch cavity P(T|cav) cavity P(C|cav) P(cavity) = 0.2 true 0.9 true 0.9 false 0.3 false 0.2 Prob of toothache Prob of catch Prior probability given cavity given cavity of cavity

  10. Bayes net example Diagram encodes the fact that toothache is conditionally independent of catch given Cavity cavity – therefore, all we need are the following distributions This is called a “factored” representation toothache catch cavity P(T|cav) cavity P(C|cav) P(cavity) = 0.2 true 0.9 true 0.9 false 0.3 false 0.2 Prob of toothache Prob of catch Prior probability given cavity given cavity of cavity

  11. Bayes net example Cavity cavity P(T|cav) cavity P(C|cav) true 0.9 true 0.9 false 0.3 false 0.2 catch toothache P(cavity) = 0.2 How do we recover joint distribution from factored representation? cavity P(T,C) P(T,!C) P(!T,C) P(!T,!C) true 0.16 0.018 0.018 0.002 false 0.048 0.19 0.11 0.448

  12. Bayes net example Cavity cavity P(T|cav) cavity P(C|cav) true 0.9 true 0.9 false 0.3 false 0.2 catch toothache P(cavity) = 0.2 P(T,C,cavity) = P(T,C|cav)P(cav) What is this step? = P(T|cav)P(C|cav)P(cav) What is this step? cavity P(T,C) P(T,!C) P(!T,C) P(!T,!C) true 0.16 0.018 0.018 0.002 false 0.048 0.19 0.11 0.448

  13. Bayes net example Cavity cavity P(T|cav) cavity P(C|cav) true 0.9 true 0.9 false 0.3 false 0.2 catch toothache P(cavity) = 0.2 P(T,C,cavity) = P(T,C|cav)P(cav) = P(T|cav)P(C|cav)P(cav) cavity P(T,C) P(T,!C) P(!T,C) P(!T,!C) true 0.16 0.018 0.018 0.002 false 0.048 0.19 0.11 0.448 How calculate these?

  14. Bayes net example Cavity cavity P(T|cav) cavity P(C|cav) true 0.9 true 0.9 false 0.3 false 0.2 catch toothache P(cavity) = 0.2 P(T,C,cavity) = P(T,C|cav)P(cav) In general: = P(T|cav)P(C|cav)P(cav) cavity P(T,C) P(T,!C) P(!T,C) P(!T,!C) true 0.16 0.018 0.018 0.002 false 0.048 0.19 0.11 0.448 How calculate these?

  15. Another example

  16. Another example ?

  17. Another example

  18. Another example How much space did the BN representation save?

  19. A simple example Structure of Bayes network Parameters of Bayes network winter P(S|W) P(winter)=0.5 true 0.3 winter false 0.01 snow Joint distribution implied by bayes network winter !winter snow 0.15 0.005 !snow 0.35 0.495

  20. A simple example Structure of Bayes network Parameters of Bayes network snow P(W|S) P(snow)=0.155 true 0.968 snow false 0.414 winter Joint distribution implied by bayes network winter !winter snow 0.15 0.005 !snow 0.35 0.495

  21. A simple example Structure of Bayes network Parameters of Bayes network snow P(W|S) P(snow)=0.155 true 0.968 snow false 0.414 What does this say about causality winter and bayes net semantics? – what does bayes net topology encode? Joint distribution implied by bayes network winter !winter snow 0.15 0.005 !snow 0.35 0.495

  22. D-separation What does bayes network structure imply about conditional independence among variables? L Are D and T independent? Are D and T conditionally independent given R? R B Are D and T conditionally independent given L? D T D-separation is a method of answering these questions... T’

  23. D-separation Causal chain: X Y Z Z is conditionally independent of X given Y If Y is unknown, then Z is correlated with X For example: X = I was hungry Y = I put pizza in the oven Z = house caught fire Fire is conditionally independent of Hungry given Pizza... – Hungry and Fire are dependent if Pizza is unknown – Hungry and Fire are independent if Pizza is known

  24. D-separation Causal chain: X Y Z Z is conditionally independent of X given Y. Exercise: Prove it! For example: X = I was hungry Y = I put pizza in the oven Z = house caught fire Fire is conditionally independent of Hungry given Pizza... – Hungry and Fire are dependent if Pizza is unknown – Hungry and Fire are independent if Pizza is known

  25. D-separation Exercise: Prove it! Causal chain: Z is conditionally independent of X given Y. For example: X = I was hungry Y = I put pizza in the oven Z = house caught fire Fire is conditionally independent of Hungry given Pizza... – Hungry and Fire are dependent if Pizza is unknown – Hungry and Fire are independent if Pizza is known

  26. D-separation Y Common cause: X Z Z is conditionally independent of X given Y. If Y is unknown, then Z is correlated with X For example: X = john calls Y = alarm Z = mary calls

  27. D-separation Y Common cause: Exercise: Prove it! X Z Z is conditionally independent of X given Y. If Y is unknown, then Z is correlated with X For example: X = john calls Y = alarm Z = mary calls

  28. D-separation X Y Common effect: Z If Z is unknown, then X, Y are independent If Z is known, then X, Y are correlated For example: X = burglary Y = earthquake Z = alarm

  29. D-separation Given an arbitrary Bayes Net, you can find out whether two variables are independent just by looking at the graph.

  30. D-separation Given an arbitrary Bayes Net, you can find out whether two variables are independent just by looking at the graph. How?

  31. D-separation Given an arbitrary Bayes Net, you can find out whether two variables are independent just by looking at the graph. Are X, Y independent given A, B, C? 1. enumerate all paths between X and Y 2. figure out whether any of these paths are active 3. if no active path, then X and Y are independent

  32. D-separation What's an active path? Are X, Y independent given A, B, C? 1. enumerate all paths between X and Y 2. figure out whether any of these paths are active 3. if no active path, then X and Y are independent

  33. Active path Active triples Inactive triples Any path that has an inactive triple on it is inactive If a path has only active triples, then it is active

  34. Example

  35. Example

  36. Example

  37. D-separation What Bayes Nets do: – constrain probability distributions that can be represented – reduce the number of parameters Constrained by conditional independencies induced by structure – can figure out what these are by using d-separation Is there a Bayes Net can represent any distribution?

  38. Exact Inference P(winter)=0.5 winter winter P(S|W) true 0.3 snow Given this false 0.01 Bayes Network snow P(C|S) true 0.1 crash false 0.01 Calculate P(C) Calculate P(C|W)

  39. Exact Inference P(winter)=0.5 winter winter P(S|W) true 0.3 snow Given this false 0.01 Bayes Network snow P(C|S) true 0.1 crash false 0.01 Calculate P(C) Calculate P(C|W)

  40. Exact Inference P(winter)=0.5 winter winter P(S|W) true 0.3 snow Given this false 0.01 Bayes Network snow P(C|S) true 0.1 crash false 0.01 Calculate P(C) Calculate P(C|W)

  41. Inference by enumeration How exactly calculate this? Inference by enumeration: 1. calculate joint distribution 2. marginalize out variables we don't care about.

  42. Inference by enumeration How exactly calculate this? Inference by enumeration: 1. calculate joint distribution 2. marginalize out variables we don't care about. Joint distribution P(winter)=0.5 winter P(S|W) winter snow P(c,s,w) true 0.3 true true 0.015 false 0.1 false true 0.005 snow P(C|S) true false 0.0035 true 0.1 false false 0.0045 false 0.01

Recommend

Bayes Networks 2 Robert Platt Northeastern University All slides in this file are adapted from

Bayes Networks 2 Robert Platt Northeastern University All slides in this file are adapted from

Bayes Networks 2 Robert Platt Northeastern University All slides in this file are adapted from CS188 UC Berkeley Bayes Nets A Bayes net is an effjcient encoding of a probabilistic model of a domain Questions we can ask:

410 views • 30 slides
4 Bayesian Belief Networks (also called Bayes Nets) Interesting because: The Naive Bayes

4 Bayesian Belief Networks (also called Bayes Nets) Interesting because: The Naive Bayes

40. 4 Bayesian Belief Networks (also called Bayes Nets) Interesting because: The Naive Bayes assumption of conditional independence of attributes is too restrictive. (But its intractable without some such assumptions...) Bayesian

476 views • 15 slides
Bayes Networks 3 Robert Platt Northeastern University All slides in this file are adapted from

Bayes Networks 3 Robert Platt Northeastern University All slides in this file are adapted from

Bayes Networks 3 Robert Platt Northeastern University All slides in this file are adapted from CS188 UC Berkeley Bayes Nets Representation Conditional Independences Probabilistic Inference Enumeration (exact, exponential

464 views • 31 slides
1 Problem with Brute force Nave Bayes ( ) ( ) ( ) It cannot generalize to unseen

1 Problem with Brute force Nave Bayes ( ) ( ) ( ) It cannot generalize to unseen

Outline Bayes theorem Discrete Bayesian classifiers Maximum likelihood classification Brute force Bayesian learning Nave Bayes Lecture 5 Bayesian Belief Networks Bayes theorem Maximum likelihood ( ) ( ) ( ) P x

72 views • 4 slides
Reasoning with Bayes Bayes Networks Networks Reasoning with Course: CS40022 Course: CS40022

Reasoning with Bayes Bayes Networks Networks Reasoning with Course: CS40022 Course: CS40022

Reasoning with Bayes Bayes Networks Networks Reasoning with Course: CS40022 Course: CS40022 Instructor: Dr. Pallab Dasgupta Pallab Dasgupta Instructor: Dr. Department of Computer Science & Engineering Department of Computer Science

438 views • 27 slides
Bayesian Learning Bayes Theorem MAP, ML hypotheses MAP learners Minimum description

Bayesian Learning Bayes Theorem MAP, ML hypotheses MAP learners Minimum description

Bayesian Learning Bayes Theorem MAP, ML hypotheses MAP learners Minimum description length principle Bayes optimal classifier Naive Bayes learner Example: Learning over text data Bayesian belief networks

1.11k views • 51 slides
Bayes Nets (Ch. 14) Announcements Homework 1 posted Bayesian Network A Bayesian network (Bayes

Bayes Nets (Ch. 14) Announcements Homework 1 posted Bayesian Network A Bayesian network (Bayes

Bayes Nets (Ch. 14) Announcements Homework 1 posted Bayesian Network A Bayesian network (Bayes net) is: (1) a directed graph (2) acyclic Additionally, Bayesian networks are assumed to be defined by conditional probability tables (3) P(x |

463 views • 32 slides
Bayes Theorem Thomas Bayes (1701-1761) Simple form of Bayes Theorem, for

Bayes Theorem Thomas Bayes (1701-1761) Simple form of Bayes Theorem, for

Nave Bayes Classifica/on Debapriyo Majumdar Data Mining Fall 2014 Indian Statistical Institute Kolkata August 14, 2014 Bayes Theorem Thomas Bayes (1701-1761) Simple form of Bayes Theorem, for two

1.04k views • 20 slides
Intention Recognition with Evolution Prospection and Causal Bayes Networks Lu s Moniz

Intention Recognition with Evolution Prospection and Causal Bayes Networks Lu s Moniz

Intention Recognition with Evolution Prospection and Causal Bayes Networks Lu s Moniz Pereira and Han The Anh lmp@di.fct.unl.pt, h.anh@fct.unl.pt Centro de Intelig encia Artificial (CENTRIA) Departamento de Inform atica, Faculdade de

836 views • 31 slides
Intention Recognition via Causal Bayes Networks plus Plan Generation Lu s Moniz Pereira and

Intention Recognition via Causal Bayes Networks plus Plan Generation Lu s Moniz Pereira and

Intention Recognition via Causal Bayes Networks plus Plan Generation Lu s Moniz Pereira and Han The Anh lmp @ di.fct.unl.pt , h.anh @ fct.unl.pt Centro de Intelig encia Artificial (CENTRIA) Universidade Nova de Lisboa, 2829-516 Caparica,

160 views • 12 slides
A Na ve Bayes model based on overlapping groups for link prediction in online social networks

A Na ve Bayes model based on overlapping groups for link prediction in online social networks

A Na ve Bayes model based on overlapping groups for link prediction in online social networks Jorge Valverde-Rebaza and Alneu de Andrade Lopes Laboratory of Computational Intelligence (LABIC) University of S ao Paulo (USP) Brazil

651 views • 33 slides
1 Bayes Nets: Assumptions Independence in a BN Assumptions we are required to make to define

1 Bayes Nets: Assumptions Independence in a BN Assumptions we are required to make to define

1 2 Recap: Bayes Nets CS 473: Artificial Intelligence Bayes Nets: Independence A Bayes net is an efficient encoding of a probabilistic model of a domain Questions we can ask: Inference: given a fixed BN, what is P(X | e)?

197 views • 5 slides
Bayes Classifiers Nave Bayes Classification Patrick Mair Bayes Classifiers Weather data

Bayes Classifiers Nave Bayes Classification Patrick Mair Bayes Classifiers Weather data

Bayes Classifiers Nave Bayes Classification Patrick Mair Bayes Classifiers Weather data set Predictors: Outlook, Temperature, Humidity, Windy. Response: Play (yes/no) Starting point Cross classification: Predictor/Response

98 views • 6 slides
How Good is the Bayes Posterior in Deep Neural Networks Really? Florian Wenzel (Google Research

How Good is the Bayes Posterior in Deep Neural Networks Really? Florian Wenzel (Google Research

Code: github.com/google-research/google-research/tree/ master/cold_posterior_bnn How Good is the Bayes Posterior in Deep Neural Networks Really? Florian Wenzel (Google Research Berlin) Joint first authors: Kevin Roth, Bas Veeling, and: Jakub

385 views • 38 slides
Conditional independence, Nave Bayes and Bayesian Networks Jo Joint Probability Distribution

Conditional independence, Nave Bayes and Bayesian Networks Jo Joint Probability Distribution

Fundamentals of AI Introduction and the most basic concepts Conditional independence, Nave Bayes and Bayesian Networks Jo Joint Probability Distribution Banana -shaped probability distribution Probability of any combination of

456 views • 35 slides
Bayesian Networks Part 2 CS 760@UW-Madison Goals for the lecture you should understand the

Bayesian Networks Part 2 CS 760@UW-Madison Goals for the lecture you should understand the

Bayesian Networks Part 2 CS 760@UW-Madison Goals for the lecture you should understand the following concepts the parameter learning task for Bayes nets the structure learning task for Bayes nets maximum likelihood estimation

545 views • 37 slides
An Empirical-Bayes Score for Discrete Bayesian Networks Marco Scutari scutari@stats.ox.ac.uk

An Empirical-Bayes Score for Discrete Bayesian Networks Marco Scutari scutari@stats.ox.ac.uk

An Empirical-Bayes Score for Discrete Bayesian Networks Marco Scutari scutari@stats.ox.ac.uk Department of Statistics University of Oxford September 8, 2016 Bayesian Network Structure Learning Learning a BN B = ( G , ) from a data set D is

2.17k views • 19 slides
Nave Bayes in a Nutshell Bayes rule: Assuming conditional independence among X i s: So,

Nave Bayes in a Nutshell Bayes rule: Assuming conditional independence among X i s: So,

Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 25, 2011 Today: Readings: Nave Bayes Required: discrete-valued X i s Mitchell: Nave Bayes and Document

624 views • 15 slides
Statistical Reasoning Probability and Bayes' Theorem Certainty Factors and Rule- Based

Statistical Reasoning Probability and Bayes' Theorem Certainty Factors and Rule- Based

Statistical Reasoning Probability and Bayes' Theorem Certainty Factors and Rule- Based Systems Bayesian Networks Dempster-Shafer Theory Fuzzy Logic Chapter 8 1 Probability and Bayes' Theorem P(H i|E) = the probability that

281 views • 13 slides
Bayesian Networks 2 Recap of last lecture: Modeling causal relationships with Bayes nets Direct

Bayesian Networks 2 Recap of last lecture: Modeling causal relationships with Bayes nets Direct

Artificial Intelligence 15-381 Mar 29, 2007 Bayesian Networks 2 Recap of last lecture: Modeling causal relationships with Bayes nets Direct cause Indirect cause Common cause Common effect A A A A B B B B C C C P(B|A) P(B|A)

439 views • 22 slides
DATA MINING: NAVE BAYES 1 Nave Bayes Classifier Thomas Bayes 1702 - 1761 We will start off

DATA MINING: NAVE BAYES 1 Nave Bayes Classifier Thomas Bayes 1702 - 1761 We will start off

DATA MINING: NAVE BAYES 1 Nave Bayes Classifier Thomas Bayes 1702 - 1761 We will start off with some mathematical background. But first we start with some visual intuition. 2 Grasshoppers Katydids 10 9 8 7 Antenna Length 6 5 4

1.29k views • 61 slides
La th eorie PAC-Bayes en apprentissage supervis e Pr esentation au LRI de luniversit

La th eorie PAC-Bayes en apprentissage supervis e Pr esentation au LRI de luniversit

The mathematics of the PAC-Bayes Theory PAC-Bayes bounds and algorithms References La th eorie PAC-Bayes en apprentissage supervis e Pr esentation au LRI de luniversit e Paris XI Fran cois Laviolette, Laboratoire du GRAAL,

615 views • 50 slides
Learning Bayesian Networks: Learning Bayesian Networks: Na ve and non ve and non- -Na

Learning Bayesian Networks: Learning Bayesian Networks: Na ve and non ve and non- -Na

Learning Bayesian Networks: Learning Bayesian Networks: Na ve and non ve and non- -Na Na ve Bayes ve Bayes Na Hypothesis Space Hypothesis Space fixed size fixed size stochastic stochastic continuous

377 views • 18 slides
STAT 339 Naive Bayes Classification 8-10 March 2017 Colin Reimer Dawson Outline Naive Bayes

STAT 339 Naive Bayes Classification 8-10 March 2017 Colin Reimer Dawson Outline Naive Bayes

STAT 339 Naive Bayes Classification 8-10 March 2017 Colin Reimer Dawson Outline Naive Bayes Classification Naive Bayes Classifier Using MLE Parameter Estimates Bayesian Naive Bayes Continuous Features Mixed Feature Types Back to Supervised

530 views • 21 slides

More recommend