Probabilistic Classifiers -- Generative Naive Bayes Announcements - PowerPoint PPT Presentation

Nov 17, 2023 •185 likes •384 views

Probabilistic Classifiers -- Generative Naive Bayes Announcements Math for Visual Computing Course Agenda: First classifier using our knowledge of probabilities Quick overview of Linear Algebra Least Square

Probabilistic Classifiers -- Generative Naive Bayes
Announcements ● Math for Visual Computing Course ● Agenda: ○ First classifier using our knowledge of probabilities ○ Quick overview of Linear Algebra ○ Least Square Regression
Definition Classifier outputs a probability distribution over the class labels: X → P(y|x)
Types of Classifiers ● Generative: learns to generate the examples ○ Learn P(x|y) and P(y) from the training data and then apply Bayes rule to find P(y|x)
Types of Classifiers ● Conditional Classifiers: Model conditional distribution P(y|x) directly. ○ Example: logistic regression classifier ○ Neural Networks.
Naive Bayes Classifier: A Generative Classifer Each attribute � is conditionally independent given the class label Formula:
Notion of conditional independence
Training a Naïve Bayesian Classifier ● Given training data: ● Estimating P(y): A multinomial distribution. ● Estimating � ○ If j-th attribute is categorical: P(xj|y) is estimated as the relative freq of samples having value di as j-th attribute in class y ○ If j-th attribute is continuous: P(xj|y) is estimated through a continuous density function: eg. Gaussian density function ● Computationally easy in both cases
Play-tennis example: estimating P(x i |C) outlook P(sunny|p) = 2/9 P(sunny|n) = 3/5 P(overcast|p) = 4/9 P(overcast|n) = 0 P(rain|p) = 3/9 P(rain|n) = 2/5 temperature P(hot|p) = 2/9 P(hot|n) = 2/5 P(mild|p) = 4/9 P(mild|n) = 2/5 P(cool|p) = 3/9 P(cool|n) = 1/5 humidity P(high|p) = 3/9 P(high|n) = 4/5 P(normal|p) = 6/9 P(normal|n) = 2/5 P(p) = 9/14 windy P(n) = 5/14 P(true|p) = 3/9 P(true|n) = 3/5 P(false|p) = 6/9 P(false|n) = 2/5
Naive Bayesian Classifier (II) ● Given a training set, we can compute the probabilities
Play-tennis example: classifying X ● An unseen sample X = <rain, hot, high, false> ● P(X|p)·P(p) = P(rain|p)·P(hot|p)·P(high|p)·P(false|p)·P(p) = 3/9·2/9·3/9·6/9·9/14 = 0.010582 ● ● P(X|n)·P(n) =P(rain|n)·P(hot|n)·P(high|n)·P(false|n)·P(n) = 2/5·2/5·4/5·2/5·5/14 = 0.018286 ● ● Sample X is classified in class n (don’t play)
Example: MNIST digit classification Jupyter Notebook: http://d2l.ai/chapter_crashcourse/naive-bayes.html
Naive Bayes for Spam Filtering.
Summary of Naïve Bayes Classifier ● Creates a distribution over class label given x P(y|x) by applying Bayes rule. ○ Requires estimating P(x|y) for each class y and P(y) ● Estimates P(x|y) by assuming that each attributes of x are conditionally independent given the class label ○ Very easy computationally. ● Many applications in spite of simplistic assumption. ● Can we do better?

Recommend

Logistic Regression, Generative and Discriminative Classifiers Recommended reading: Ng and

Logistic Regression, Generative and Discriminative Classifiers Recommended reading: Ng and Jordan paper On Discriminative vs. Generative classifiers: A comparison of logistic regression and nave Bayes, A. Ng and M. Jordan, NIPS

392 views • 28 slides

Data Warehousing and Machine Learning Probabilistic Classifiers Thomas D. Nielsen Aalborg

Data Warehousing and Machine Learning Probabilistic Classifiers Thomas D. Nielsen Aalborg University Department of Computer Science Spring 2008 DWML Spring 2008 1 / 34 Probabilistic Classifiers Conditional class probabilities Id. Savings

761 views • 59 slides

Modals, conditionals, and probabilistic generative models Topic 2: Indicative conditionals

Modals, conditionals, and probabilistic generative models Topic 2: Indicative conditionals Separating semantics from reasoning Dan Lassiter, Stanford Linguistics Universit de Paris VII, 2/12/19 Overall plan 1. probability, generative

997 views • 52 slides

Augmented Statistical Models: Exploiting Generative Models in Discriminative Classifiers Martin

Augmented Statistical Models: Exploiting Generative Models in Discriminative Classifiers Martin Layton & Mark Gales 9 December 2005 Cambridge University Engineering Department NIPS 2005 Augmented Statistical Models: Exploiting Generative

726 views • 25 slides

Probabilistic Models of Cognition: Generative models Table of Contents Chapter

Probabilistic Models of Cognition: Generative models Table of Contents Chapter Content Generative Model Example: Plinko Machine Working Model can be used captures some for simulation structure of the world in useful way

825 views • 78 slides

Adversarial BoltzmannMachines Belief Nets Networks Variational NADE

Unsupervised Learning Non-probabilistic Models Probabilistic Sparse Coding (Generative) Models Autoencoders Others (e.g. k-means) Tractable Models Non-Tractable Models Generative Fully observed Adversarial

815 views • 58 slides

Modals, conditionals, and probabilistic generative models Topic 1: intro to probability &

Modals, conditionals, and probabilistic generative models Topic 1: intro to probability & generative models; a bit on modality Dan Lassiter, Stanford Linguistics Universit de Paris VII, 25/11/19 4 lectures: The plan 1. probability,

694 views • 56 slides

A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine

A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine Learning Summer School Buenos Aires, Argentina June 2018 lips.cs.princeton.edu @ryan_p_adams Tutorial Outline What is generative modeling?

2.33k views • 156 slides

Generative Models and Nave Bayes Ke Chen Reading: [14.3, EA], [3.5, KPM], [1.5.4, CMB]

Generative Models and Nave Bayes Ke Chen Reading: [14.3, EA], [3.5, KPM], [1.5.4, CMB] COMP24111 Machine Learning Outline Background and Probability Basics Probabilistic Classification Principle Probabilistic discriminative

330 views • 18 slides

CSE 571 A simple algorithm for learning robust Probabilistic Robotics classifiers Freund

11/13/2007 Why boosting? CSE 571 A simple algorithm for learning robust Probabilistic Robotics classifiers Freund & Shapire, 1995 Friedman, Hastie, Tibshhirani, 1998 Boosting Provides efficient algorithm for sparse feature

739 views • 8 slides

Nonlinear Classifiers II 2 Nonlinear Classifiers: Introduction Classifiers Supervised

1 Nonlinear Classifiers II 2 Nonlinear Classifiers: Introduction Classifiers Supervised Classifiers XOR problem Linear Classifiers Perceptron Least Squares Methods Linear Support Vector Machine Nonlinear

355 views • 20 slides

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling Jiajun Wu* Chengkai Zhang* Tianfan Xue Bill Freeman Josh Tenenbaum NIPS 2016 (* indicates equal contribution) Outline Synthesizing 3D shapes

967 views • 65 slides

Outline CS 188: Artificial Intelligence Generative vs. Discriminative Binary Linear

Outline CS 188: Artificial Intelligence Generative vs. Discriminative Binary Linear Classifiers Perceptron Lecture 21: Perceptrons Multi-class Linear Classifiers Multi-class Perceptron Fixing the Perceptron:

182 views • 7 slides

Making Generative Classifiers Robust to Selection Bias Andrew Smith Charles Elkan November

Making Generative Classifiers Robust to Selection Bias Andrew Smith Charles Elkan November 30th, 2007 Outline What is selection bias? Types of selection bias. Overcoming learnable bias with weighting. Overcoming bias with

736 views • 59 slides

Classification with generative models 2 DSE 210 Classification with parametrized models

Classification with generative models 2 DSE 210 Classification with parametrized models Classifiers with a fixed number of parameters can represent a limited set of functions. Learning a model is about picking a good approximation. Typically the

427 views • 39 slides

Probabilistic Foundations of Statistical Network Analysis Chapter 4: Generative models Harry

Probabilistic Foundations of Statistical Network Analysis Chapter 4: Generative models Harry Crane Based on Chapter 4 of Probabilistic Foundations of Statistical Network Analysis Book website: http://www.harrycrane.com/networks.html Harry Crane

289 views • 13 slides

Introduction to Deep Models Part I: Classifiers and Generative Networks Nick Winovich Department

TensorFlow Workshop 2018 Introduction to Deep Models Part I: Classifiers and Generative Networks Nick Winovich Department of Mathematics Purdue University July 2018 SIAM@Purdue 2018 - Nick Winovich Introduction to Deep Models : Part I

554 views • 28 slides

A Probabilistic Model for Component Based Shape Synthesis Evangelos Kalogerakis, Siddhartha

A Probabilistic Model for Component Based Shape Synthesis Evangelos Kalogerakis, Siddhartha Chaudhuri, Daphne Koller, Vladlen Koltun Stanford University Goal: generative model of shape Goal: generative model of shape Challenge: understand shape

1.31k views • 55 slides

Logistic Regression Two Worlds: Probabilistic & Algorithmic We know two conceptual approaches

Logistic Regression Two Worlds: Probabilistic & Algorithmic We know two conceptual approaches to classification: Bayes es Classi ssifier er Probabilistic classifier with a classification class density data decision generative setup

526 views • 20 slides

Scalable Deep Generative Modeling for Sparse Graphs Hanjun Dai 1 , Azade Nazi 1 , Yujia Li 2 , Bo

Scalable Deep Generative Modeling for Sparse Graphs Hanjun Dai 1 , Azade Nazi 1 , Yujia Li 2 , Bo Dai 1 , Dale Schuurmans 1 1 Google Brain, 2 DeepMind Graph generative models Given a set of graphs {G 1 , G 2 , , G N }, fit a probabilistic model

516 views • 39 slides

Probabilistic classification CE-717: Machine Learning Sharif University of Technology M.

Probabilistic classification CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Topics Probabilistic approach Bayes decision theory Generative models Gaussian Bayes classifier Nave Bayes

845 views • 59 slides

Occasion-level Classifiers or Event-level Classifiers? -Evidence from Child Language Acquisition

Occasion-level Classifiers or Event-level Classifiers? -Evidence from Child Language Acquisition Yuan Xiaohe, Zhang Xiaoqian, Luo Yingyi, Hu Jianhua Institute of Linguistics Chinese Academy of Social Sciences Verbal classifiers in Mandarin

823 views • 24 slides

Machine Learning Nave Bayes classifiers Types of classifiers We can divide the large

10-701 Machine Learning Nave Bayes classifiers Types of classifiers We can divide the large variety of classification approaches into three major types 1. Instance based classifiers - Use observation directly (no models) - e.g. K

309 views • 28 slides

Bayesian Classifiers LM, session 2 CS6200: Information Retrieval Slides by: Jesse Anderton

Bayesian Classifiers LM, session 2 CS6200: Information Retrieval Slides by: Jesse Anderton Ranking with Probabilistic Models Imagine we have a function that gives us the probability that a document D is relevant to a query Q , P ( R =1| D, Q ) .

230 views • 9 slides