where do the probabilities come from
play

Where do the probabilities come from? Probabilities come from: - PowerPoint PPT Presentation

Jan 15, 2023 •328 likes •643 views

Where do the probabilities come from? Probabilities come from: Experts Data D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 1 / 6 Learning probabilities the simplest case Observe tosses of thumbtack:

  1. Where do the probabilities come from? Probabilities come from: ◮ Experts ◮ Data � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 1 / 6

  2. Learning probabilities — the simplest case Observe tosses of thumbtack: Tails Heads n 0 instances of Heads = false n 1 instances of Heads = true what should we use as P ( heads )? � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 2 / 6

  3. Learning probabilities — the simplest case Observe tosses of thumbtack: Tails Heads n 0 instances of Heads = false n 1 instances of Heads = true what should we use as P ( heads )? n 1 Empirical frequency: P ( heads ) = n 0 + n 1 � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 2 / 6

  4. Learning probabilities — the simplest case Observe tosses of thumbtack: Tails Heads n 0 instances of Heads = false n 1 instances of Heads = true what should we use as P ( heads )? n 1 Empirical frequency: P ( heads ) = n 0 + n 1 n 1 + 1 Laplace smoothing [1812]: P ( heads ) = n 0 + n 1 + 2 � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 2 / 6

  5. Learning probabilities — the simplest case Observe tosses of thumbtack: Tails Heads n 0 instances of Heads = false n 1 instances of Heads = true what should we use as P ( heads )? n 1 Empirical frequency: P ( heads ) = n 0 + n 1 n 1 + 1 Laplace smoothing [1812]: P ( heads ) = n 0 + n 1 + 2 n 1 + c 1 Informed priors: P ( heads ) = n 0 + n 1 + c 0 + c 1 for some informed pseudo counts c 0 , c 1 > 0. c 0 = 1, c 1 = 1, expressed ignorance (uniform prior) Pseudo-counts convey prior knowledge. Consider: “how much more would I believe α if I had seen one example with α true than if I has seen no examples with α true?” � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 2 / 6

  6. Learning probabilities — the simplest case Observe tosses of thumbtack: Tails Heads n 0 instances of Heads = false n 1 instances of Heads = true what should we use as P ( heads )? n 1 Empirical frequency: P ( heads ) = n 0 + n 1 n 1 + 1 Laplace smoothing [1812]: P ( heads ) = n 0 + n 1 + 2 n 1 + c 1 Informed priors: P ( heads ) = n 0 + n 1 + c 0 + c 1 for some informed pseudo counts c 0 , c 1 > 0. c 0 = 1, c 1 = 1, expressed ignorance (uniform prior) Pseudo-counts convey prior knowledge. Consider: “how much more would I believe α if I had seen one example with α true than if I has seen no examples with α true?” — empirical frequency overfits to the data. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 2 / 6

  7. Example of Overfitting We have a web site where people rate restaurants with 1 to 5 stars. We want to report the most liked restaurant(s) — the one predicted to have the best future ratings. How can we determine the most liked restaurant? � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 3 / 6

  8. Example of Overfitting We have a web site where people rate restaurants with 1 to 5 stars. We want to report the most liked restaurant(s) — the one predicted to have the best future ratings. How can we determine the most liked restaurant? Are the restaurants with the highest average rating the most liked restaurants? � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 3 / 6

  9. Example of Overfitting We have a web site where people rate restaurants with 1 to 5 stars. We want to report the most liked restaurant(s) — the one predicted to have the best future ratings. How can we determine the most liked restaurant? Are the restaurants with the highest average rating the most liked restaurants? Which restaurants have the highest average rating? � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 3 / 6

  10. Example of Overfitting We have a web site where people rate restaurants with 1 to 5 stars. We want to report the most liked restaurant(s) — the one predicted to have the best future ratings. How can we determine the most liked restaurant? Are the restaurants with the highest average rating the most liked restaurants? Which restaurants have the highest average rating? Which restaurants have a rating of 5? � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 3 / 6

  11. Example of Overfitting We have a web site where people rate restaurants with 1 to 5 stars. We want to report the most liked restaurant(s) — the one predicted to have the best future ratings. How can we determine the most liked restaurant? Are the restaurants with the highest average rating the most liked restaurants? Which restaurants have the highest average rating? Which restaurants have a rating of 5? ◮ Only restaurants with few ratings have an average rating of 5. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 3 / 6

  12. Example of Overfitting We have a web site where people rate restaurants with 1 to 5 stars. We want to report the most liked restaurant(s) — the one predicted to have the best future ratings. How can we determine the most liked restaurant? Are the restaurants with the highest average rating the most liked restaurants? Which restaurants have the highest average rating? Which restaurants have a rating of 5? ◮ Only restaurants with few ratings have an average rating of 5. Solution: add some “average” ratings for each restaurant! � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 3 / 6

  13. Bayesian Learning Probability of Heads … Toss 1 Toss 2 Toss 11 aispace: http://artint.info/code/aispace/beta.xml Probablity of Heads is a random variable representing the probability of heads. Range is { 0 . 0 , 0 . 1 , 0 . 2 , . . . , 0 . 9 , 1 . 0 } or interval [0 , 1]. P ( Toss # n = Heads | Probablity of Heads = v ) = � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 4 / 6

  14. Bayesian Learning Probability of Heads … Toss 1 Toss 2 Toss 11 aispace: http://artint.info/code/aispace/beta.xml Probablity of Heads is a random variable representing the probability of heads. Range is { 0 . 0 , 0 . 1 , 0 . 2 , . . . , 0 . 9 , 1 . 0 } or interval [0 , 1]. P ( Toss # n = Heads | Probablity of Heads = v ) = v Toss # i is independent of Toss # j (for i � = j ) given Probablity of Heads i.i.d. or independent and identically distributed. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 4 / 6

  15. Naive Bayes Classifier: User’s request for help H "able" "absent" "add" "zoom" . . . H is the help page the user is interested in. We observe the words in the query. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 5 / 6

  16. Naive Bayes Classifier: User’s request for help H "able" "absent" "add" "zoom" . . . H is the help page the user is interested in. We observe the words in the query. What probabilities are required? � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 5 / 6

  17. Naive Bayes Classifier: User’s request for help H "able" "absent" "add" "zoom" . . . H is the help page the user is interested in. We observe the words in the query. What probabilities are required? What counts are required? � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 5 / 6

  18. Naive Bayes Classifier: User’s request for help H "able" "absent" "add" "zoom" . . . H is the help page the user is interested in. We observe the words in the query. What probabilities are required? What counts are required? number of times each help page h i is the best one number of times word w j is used when h i is the help page. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 5 / 6

  19. Naive Bayes Classifier: User’s request for help H "able" "absent" "add" "zoom" . . . H is the help page the user is interested in. We observe the words in the query. What probabilities are required? What counts are required? number of times each help page h i is the best one number of times word w j is used when h i is the help page. When can the counts be updated? � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 5 / 6

  20. Naive Bayes Classifier: User’s request for help H "able" "absent" "add" "zoom" . . . H is the help page the user is interested in. We observe the words in the query. What probabilities are required? What counts are required? number of times each help page h i is the best one number of times word w j is used when h i is the help page. When can the counts be updated? When the correct page is found. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 5 / 6

  21. Naive Bayes Classifier: User’s request for help H "able" "absent" "add" "zoom" . . . H is the help page the user is interested in. We observe the words in the query. What probabilities are required? What counts are required? number of times each help page h i is the best one number of times word w j is used when h i is the help page. When can the counts be updated? When the correct page is found. What prior counts should be used? Can they be zero? � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.1 5 / 6

Recommend

Review: Probabilities DISCRETE PROBABILITIES Intro We have all been exposed to informal

Review: Probabilities DISCRETE PROBABILITIES Intro We have all been exposed to informal

Review: Probabilities DISCRETE PROBABILITIES Intro We have all been exposed to informal probabilities. However it is instructive to be more precise. The goal today is to refresh our ability to reason with probabilities in simple cases.

528 views • 8 slides
Should we think of quantum probabilities as Bayesian probabilities? Carlton M. Caves C. M.

Should we think of quantum probabilities as Bayesian probabilities? Carlton M. Caves C. M.

Should we think of quantum probabilities as Bayesian probabilities? Carlton M. Caves C. M. Caves, C. A. Fuchs, R. Schack, Subjective probability and quantum certainty, Studies in History and Philosophy of Modern Physics 38 , 255--274

234 views • 21 slides
1 A simplified definition Example: Rolling a dice Let be the sample space of an experiment. A

1 A simplified definition Example: Rolling a dice Let be the sample space of an experiment. A

Outline Probabilities Conditional probabilities Bayes theorem Distributions Discrete Continuous Advanced Herd Management Distribution functions Probabilities and distributions Sampling from distributions Estimation

545 views • 10 slides
Probabilities and Expectations A. Rupam Mahmood September 10, 2015 Probabilities

Probabilities and Expectations A. Rupam Mahmood September 10, 2015 Probabilities

Probabilities and Expectations A. Rupam Mahmood September 10, 2015 Probabilities Probability is a measure of uncertainty Being uncertain is much more than I dont know We can make informed guesses about uncertain events

286 views • 26 slides
Comonotone lower probabilities for bivariate Introduction and discrete structures Comonotonicity

Comonotone lower probabilities for bivariate Introduction and discrete structures Comonotonicity

Comonotone lower probabilities for bivariate Introduction and discrete structures Comonotonicity for lower probabilities Building Ignacio Montes and Sebastien Destercke comonotone lower probabilities Conclusions University of Oviedo,

422 views • 26 slides
Stochastic Simulation Idea: probabilities samples Get probabilities from samples: X count X

Stochastic Simulation Idea: probabilities samples Get probabilities from samples: X count X

Stochastic Simulation Idea: probabilities samples Get probabilities from samples: X count X probability x 1 n 1 x 1 n 1 / m . . . . . . . . . . . . x k n k x k n k / m total m If we could sample from a variables

476 views • 47 slides
Stochastic Simulation Idea: probabilities samples Get probabilities from samples: X count X

Stochastic Simulation Idea: probabilities samples Get probabilities from samples: X count X

Stochastic Simulation Idea: probabilities samples Get probabilities from samples: X count X probability x 1 n 1 n 1 / m x 1 . . . . . . . . . . . . x k n k x k n k / m total m If we could sample from a variables

714 views • 18 slides
Partially specified Probabilities: decisions and games May 2007 Ehud Lehrer The problem

Partially specified Probabilities: decisions and games May 2007 Ehud Lehrer The problem

Partially specified probabilities Partially specified Probabilities: decisions and games May 2007 Ehud Lehrer The problem Partially specified probabilities Nash equilibrium implicitly assumes that each player knows what other players

653 views • 38 slides
N-Gram Model Formulas Estimating Probabilities N-gram conditional probabilities can be

N-Gram Model Formulas Estimating Probabilities N-gram conditional probabilities can be

N-Gram Model Formulas Estimating Probabilities N-gram conditional probabilities can be estimated Word sequences from raw text based on the relative frequency of word sequences. Chain rule of probability Bigram: N-gram: Bigram

413 views • 4 slides
Logistic regression to predict probabilities SU P E R VISE D L E AR N IN G IN R : R E G R E

Logistic regression to predict probabilities SU P E R VISE D L E AR N IN G IN R : R E G R E

Logistic regression to predict probabilities SU P E R VISE D L E AR N IN G IN R : R E G R E SSION Nina Z u mel and John Mo u nt Win - Vector LLC Predicting Probabilities Predicting w hether an e v ent occ u rs (y es / no ): classi cation

1.01k views • 38 slides
STA 331 2.0 Stochastic Processes 4. Limiting Probabilities Dr Thiyanga S. Talagala August 25,

STA 331 2.0 Stochastic Processes 4. Limiting Probabilities Dr Thiyanga S. Talagala August 25,

STA 331 2.0 Stochastic Processes 4. Limiting Probabilities Dr Thiyanga S. Talagala August 25, 2020 Department of Statistics, University of Sri Jayewardenepura Markov chains: A Random Walk In-class 2 Limiting probabilities - Example Suppose

561 views • 15 slides
Foundations of Computing II Lecture 7: Conditional Probabilities Stefano Tessaro

Foundations of Computing II Lecture 7: Conditional Probabilities Stefano Tessaro

CSE 312 Foundations of Computing II Lecture 7: Conditional Probabilities Stefano Tessaro tessaro@cs.washington.edu 1 Reminder Gradescope enroll code: M8YYEZ Homework due tonight by 11:59pm. 2 Conditional Probabilities Often we want to know

706 views • 32 slides
Probabilities Sven Koenig, USC Russell and Norvig, 3 rd Edition, Chapter 13 These slides are new

Probabilities Sven Koenig, USC Russell and Norvig, 3 rd Edition, Chapter 13 These slides are new

12/18/2019 Probabilities Sven Koenig, USC Russell and Norvig, 3 rd Edition, Chapter 13 These slides are new and can contain mistakes and typos. Please report them to Sven (skoenig@usc.edu). 1 Probabilities Robots face lots of uncertainty.

319 views • 10 slides
Lecture 20/Chapter 17 Psychological Influences on Personal Probabilities Definitions of

Lecture 20/Chapter 17 Psychological Influences on Personal Probabilities Definitions of

Lecture 20/Chapter 17 Psychological Influences on Personal Probabilities Definitions of Various Phenomena Examples Calibrating Experts Personal Probabilities Psychological Influences on Risk Perception Certainty effect: people

452 views • 22 slides
Preventing Arbitrage from Collusion when Eliciting Probabilities Rupert Freeman David M. Pennock

Preventing Arbitrage from Collusion when Eliciting Probabilities Rupert Freeman David M. Pennock

Preventing Arbitrage from Collusion when Eliciting Probabilities Rupert Freeman David M. Pennock Dominik Peters Bo Waggoner Microsoft Research Rutgers Carnegie Mellon CU Boulder Poster #71 1 Eliciting Probabilities We want to know

351 views • 18 slides
Empirical Interpretation of Imprecise Probabilities Marco Cattaneo School of Mathematics and

Empirical Interpretation of Imprecise Probabilities Marco Cattaneo School of Mathematics and

Empirical Interpretation of Imprecise Probabilities Marco Cattaneo School of Mathematics and Physical Sciences University of Hull ISIPTA 17 ECSQARU 2017, Lugano, Switzerland 10 July 2017 introduction imprecise probabilities can have

311 views • 8 slides
Section 7.2 Assigning Probabilities Laplaces definition from the previous section, assumes

Section 7.2 Assigning Probabilities Laplaces definition from the previous section, assumes

Section 7.2 Assigning Probabilities Laplaces definition from the previous section, assumes that all outcomes are equally likely. Now we introduce a more general definition of probabilities that avoids this restriction. Let S be a sample

426 views • 18 slides
Natural Language Processing Spring 2017 Unit 1: Sequence Models Lecture 4a: Probabilities and

Natural Language Processing Spring 2017 Unit 1: Sequence Models Lecture 4a: Probabilities and

Natural Language Processing Spring 2017 Unit 1: Sequence Models Lecture 4a: Probabilities and Estimations Lecture 4b: Weighted Finite-State Machines required optional Liang Huang Probabilities experiment (e.g., toss a coin 3

753 views • 41 slides
Conditional Probabilities Anders Ringgaard Kristensen Department of Veterinary and Animal

Conditional Probabilities Anders Ringgaard Kristensen Department of Veterinary and Animal

Department of Veterinary and Animal Sciences Conditional Probabilities Anders Ringgaard Kristensen Department of Veterinary and Animal Sciences Outline Probabilities Conditional probabilities Bayes theorem Slide 2 Department of

400 views • 17 slides
Zeroes When working with n-gram models, zero probabilities can be real show-stoppers

Zeroes When working with n-gram models, zero probabilities can be real show-stoppers

Zeroes When working with n-gram models, zero probabilities can be real show-stoppers Smoothing and Backoff Examples: Zero probabilities are a problem p( w 1 w 2 w 3 ...w n ) p( w 1 ) p( w 2 | w 1 ) p( w 3 | w 2 )...p( w n |

364 views • 4 slides
(Wu 1995) Standard probabilistic context-free grammars: probabilities over rewrite rules

(Wu 1995) Standard probabilistic context-free grammars: probabilities over rewrite rules

(Wu 1995) Standard probabilistic context-free grammars: probabilities over rewrite rules define probabilities over trees, strings, in one language 6.864 (Fall 2007) Transduction grammars : Simultaneously generate strings in two languages

250 views • 5 slides
Integrable gap probabilities for the Generalized Bessel process Manuela Girotti SISSA,

Integrable gap probabilities for the Generalized Bessel process Manuela Girotti SISSA,

Integrable gap probabilities for the Generalized Bessel process Integrable gap probabilities for the Generalized Bessel process Manuela Girotti SISSA, Trieste, June 7th 2017 1 / 27 Integrable gap probabilities for the

877 views • 35 slides
Some Remarks on Sets of Lexicographic Probabilities and Sets of Desirable Gambles Fabio G. Cozman

Some Remarks on Sets of Lexicographic Probabilities and Sets of Desirable Gambles Fabio G. Cozman

Some Remarks on Sets of Lexicographic Probabilities and Sets of Desirable Gambles Fabio G. Cozman Universidade de S ao Paulo July 16, 2015 Overview Goal: to examine a few properties of sets of lexicographic probabilities and sets of

576 views • 18 slides
Summer School 2008, Disentis Gap Probabilities for Random Matrix Ensembles Felix Rubin July 21,

Summer School 2008, Disentis Gap Probabilities for Random Matrix Ensembles Felix Rubin July 21,

Introduction Gaussian Unitary Ensemble (GUE) Gap Probabilities and Distribution of Largest Eigenvalue Matrix Ensembles with Generalized Cauchy Weights Summer School 2008, Disentis Gap Probabilities for Random Matrix Ensembles Felix Rubin July

308 views • 18 slides

More recommend