Linear classifiers : prediction eq u ations L IN E AR C L ASSIFIE R - PowerPoint PPT Presentation

Sep 02, 2022 •285 likes •513 views

Linear classifiers : prediction eq u ations L IN E AR C L ASSIFIE R S IN P YTH ON Michael ( Mike ) Gelbart Instr u ctor , The Uni v ersit y of British Col u mbia Dot Prod u cts x = np.arange(3) np.sum(x*y) x 14 array([0, 1, 2]) x@y y =

Linear classifiers : prediction eq u ations L IN E AR C L ASSIFIE R S IN P YTH ON Michael ( Mike ) Gelbart Instr u ctor , The Uni v ersit y of British Col u mbia
Dot Prod u cts x = np.arange(3) np.sum(x*y) x 14 array([0, 1, 2]) x@y y = np.arange(3,6) y 14 array([3, 4, 5]) x@y is called the dot prod u ct of x and y , and is x*y w ri � en x ⋅ y . array([0, 4, 10]) LINEAR CLASSIFIERS IN PYTHON
Linear classifier prediction raw model output = coefficients ⋅ features + intercept Linear classi � er prediction : comp u te ra w model o u tp u t , check the sign if positi v e , predict one class if negati v e , predict the other class This is the same for logistic regression and linear SVM fit is di � erent b u t predict is the same LINEAR CLASSIFIERS IN PYTHON
Ho w LogisticRegression makes predictions raw model output = coefficients ⋅ features + intercept lr = LogisticRegression() lr.fit(X,y) lr.predict(X)[10] 0 lr.predict(X)[20] 1 LINEAR CLASSIFIERS IN PYTHON
Ho w LogisticRegression makes predictions ( cont .) lr.coef_ @ X[10] + lr.intercept_ # raw model output array([-33.78572166]) lr.coef_ @ X[20] + lr.intercept_ # raw model output array([ 0.08050621]) LINEAR CLASSIFIERS IN PYTHON
The ra w model o u tp u t LINEAR CLASSIFIERS IN PYTHON
The ra w model o u tp u t LINEAR CLASSIFIERS IN PYTHON
The ra w model o u tp u t LINEAR CLASSIFIERS IN PYTHON
Let ' s practice ! L IN E AR C L ASSIFIE R S IN P YTH ON
What is a loss f u nction ? L IN E AR C L ASSIFIE R S IN P YTH ON Michael Gelbart Instr u ctor , The Uni v ersit y of British Col u mbia
Least sq u ares : the sq u ared loss scikit - learn ' s LinearRegression minimi z es a loss : n ∑ 2 (true i th target value − predicted i th target value) i =1 Minimi z ation is w ith respect to coe � cients or parameters of the model . Note that in scikit - learn model.score() isn ' t necessaril y the loss f u nction . LINEAR CLASSIFIERS IN PYTHON
Classification errors : the 0-1 loss Sq u ared loss not appropriate for classi � cation problems ( more on this later ). A nat u ral loss for classi � cation problem is the n u mber of errors . This is the 0-1 loss : it ' s 0 for a correct prediction and 1 for an incorrect prediction . B u t this loss is hard to minimi z e ! LINEAR CLASSIFIERS IN PYTHON
Minimi z ing a loss from scipy.optimize import minimize minimize(np.square, 0).x array([0.]) minimize(np.square, 2).x array([-1.88846401e-08]) LINEAR CLASSIFIERS IN PYTHON
Let ' s practice ! L IN E AR C L ASSIFIE R S IN P YTH ON
Loss f u nction diagrams L IN E AR C L ASSIFIE R S IN P YTH ON Michael ( Mike ) Gelbart Instr u ctor , The Uni v ersit y of British Col u mbia
The ra w model o u tp u t LINEAR CLASSIFIERS IN PYTHON
0-1 loss diagram LINEAR CLASSIFIERS IN PYTHON
Linear regression loss diagram LINEAR CLASSIFIERS IN PYTHON
Logistic loss diagram LINEAR CLASSIFIERS IN PYTHON
Hinge loss diagram LINEAR CLASSIFIERS IN PYTHON
Hinge loss diagram LINEAR CLASSIFIERS IN PYTHON
Let ' s practice ! L IN E AR C L ASSIFIE R S IN P YTH ON

Recommend

First look at structures CS 6355: Structured Prediction 1 So far Binary classifiers

First look at structures CS 6355: Structured Prediction 1 So far Binary classifiers Output: 0/1 Multiclass classifiers Output: one of a set of labels Linear classifiers for both Learning algorithms Winner-take-all

1.05k views • 75 slides

Directly and Efficiently Optimizing Prediction Error and AUC of Linear Classifiers Hiva Ghanbari

Directly and Efficiently Optimizing Prediction Error and AUC of Linear Classifiers Hiva Ghanbari Joint work with Prof. Katya Scheinberg Industrial and Systems Engineering Department US & Mexico Workshop on Optimization and its

586 views • 41 slides

Nonlinear Classifiers II 2 Nonlinear Classifiers: Introduction Classifiers Supervised

1 Nonlinear Classifiers II 2 Nonlinear Classifiers: Introduction Classifiers Supervised Classifiers XOR problem Linear Classifiers Perceptron Least Squares Methods Linear Support Vector Machine Nonlinear

355 views • 20 slides

Sketching Linear Classifiers Over Data Streams Kai Sheng Tai Vatsal Sharan, Peter Bailis,

Sketching Linear Classifiers Over Data Streams Kai Sheng Tai Vatsal Sharan, Peter Bailis, Gregory Valiant Stanford University High-dimensional linear classifiers on streams Ubiquitous: spam detection, ad click prediction, network traffic

604 views • 28 slides

Linear Classifiers and Regressors Borrowed with permission from Andrew Moore (CMU)

Linear Classifiers and Regressors Borrowed with permission from Andrew Moore (CMU) Single-Parameter Linear Regression Linear: Slide 2 Regression vs Classification Input Prediction of Classifier Attributes categorical output Input

1.08k views • 48 slides

Linear Classifiers and the Perceptron William Cohen February 4, 2008 1 Linear classifiers

Linear Classifiers and the Perceptron William Cohen February 4, 2008 1 Linear classifiers Lets assume that every instance is an n -dimensional vector of real numbers x R n , and there are only two possible classes, y = (+1) and y = (

269 views • 9 slides

CS440/ECE448 Lecture 22: Including Slides by Svetlana Lazebnik, 10/2016 Linear Classifiers

Mark Hasegawa-Johnson, 3/2020 CS440/ECE448 Lecture 22: Including Slides by Svetlana Lazebnik, 10/2016 Linear Classifiers License: CC-BY 4.0 Linear Classifiers Classifiers Perceptron Linear classifiers in general Logistic

581 views • 45 slides

Part-II Parametric Signal Modeling and Linear Prediction Theory 3. Linear Prediction Electrical

3 Linear Prediction Appendix: Detailed Derivations Part-II Parametric Signal Modeling and Linear Prediction Theory 3. Linear Prediction Electrical & Computer Engineering University of Maryland, College Park Acknowledgment: ENEE630 slides

392 views • 28 slides

Linear Classifiers and Perceptron E.D. And F. MAN TO BUY INTO

Text Classification Linear Classifiers and Perceptron E.D. And F. MAN TO BUY INTO HONG KONG FIRM CS678 Advanced Topics in Machine Learning The U.K. Based commodity

157 views • 3 slides

Linear classifiers CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall

Linear classifiers CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Topics Discriminant functions Linear classifiers Perceptron SVM will be covered in the later lectures Fisher Multi-class

616 views • 48 slides

Linear Classifiers: Expressiveness Machine Learning 1 Lecture outline Linear models:

Linear Classifiers: Expressiveness Machine Learning 1 Lecture outline Linear models: Introduction What functions do linear classifiers express? 2 Where are we? Linear models: Introduction What functions do linear classifiers

459 views • 27 slides

Linear classifiers CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall

Linear classifiers CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2019 Topics } Linear classifiers } Perceptron SVM will be covered in the later lectures } Fisher } Multi-class classification 2 Classification

881 views • 55 slides

Linear classifiers CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall

Linear classifiers CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2018 Topics } Linear classifiers } Perceptron SVM will be covered in the later lectures } Fisher } Multi-class classification 2 Classification

786 views • 50 slides

Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied

Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied linear regression ; the output/label space Y was R . 90 80 delay 70 60 50 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 duration 1 / 68 Linear

607 views • 38 slides

Linear, Binary SVM Classifiers COMPSCI 371D Machine Learning COMPSCI 371D Machine

Linear, Binary SVM Classifiers COMPSCI 371D Machine Learning COMPSCI 371D Machine Learning Linear, Binary SVM Classifiers 1 / 17 Outline 1 What Linear, Binary SVM Classifiers Do 2 Margin 3 Loss and Regularized Risk 4 Training an SVM is

510 views • 17 slides

Sequence Prediction Using Neural Network Classifiers Yanpeng Zhao ShanghaiTech University ICGI,

Sequence Prediction Using Neural Network Classifiers Yanpeng Zhao ShanghaiTech University ICGI, Oct 7 th , 2016, Delft, the Netherlands Sequence Prediction Whats the next symbol? 4 3 5 0 4 6 1 3 1 ? Classification Perspective -1

396 views • 15 slides

Predicting Structures: Conditional Models and Local Classifiers CS 6355: Structured Prediction 1

Predicting Structures: Conditional Models and Local Classifiers CS 6355: Structured Prediction 1 Outline Sequence models Hidden Markov models Inference with HMM Learning Conditional Models and Local Classifiers Global

1.1k views • 73 slides

Extending Binary Linear Classification One-Versus-All Classification (OVA) } In the presence of

Binary and Other Classification } We will generally discuss binary classifiers, which divide data into one of two classes } Linear classifiers as presented in last lecture are inherently binary, defining the classes based on two regions,

336 views • 7 slides

Week 1, video 3: Classifiers, Part 1 Prediction Develop a model which can infer a single

Week 1, video 3: Classifiers, Part 1 Prediction Develop a model which can infer a single aspect of the data (predicted variable) from some combination of other aspects of the data (predictor variables) Sometimes used to predict the future

883 views • 56 slides

L101: Feed Forward Neural Networks Linear classifiers e.g. binary logistic regression: And

L101: Feed Forward Neural Networks Linear classifiers e.g. binary logistic regression: And their limitations: http://www.ece.utep.edu/research/webfuzzy/docs/kk-thesis/kk-thesis-html/node19.html What if we could use multiple classifiers?

579 views • 19 slides

Linear Classifier Linear classifiers are single layer neural networks. x 2 4 x 2 = 2 x 1 Observe,

b b b Linear Classifier Linear classifiers are single layer neural networks. x 2 4 x 2 = 2 x 1 Observe, that x 2 = 2 x 1 can also be 3 expressed as 2 1 w 1 x 1 + w 2 x 2 = 0 x 2 = w 1 x 1 , 0 w 2 x 1 -1 where for instance -2 -3

768 views • 29 slides

STAT 113 Simple Linear Regression Colin Reimer Dawson Oberlin College Sept. 16, 2015 Outline

STAT 113 Simple Linear Regression Colin Reimer Dawson Oberlin College Sept. 16, 2015 Outline Prediction Whats a Good Prediction? Linear Prediction Equation Prediction Error Regression to the Mean Prediction Correlations give us a

359 views • 21 slides

Linear Classifiers CS 188: Artificial Intelligence Perceptrons and Logistic Regression Pieter

Linear Classifiers CS 188: Artificial Intelligence Perceptrons and Logistic Regression Pieter Abbeel & Dan Klein University of California, Berkeley Feature Vectors Some (Simplified) Biology Very loose inspiration: human neurons Hello,

86 views • 7 slides

CSC 311: Introduction to Machine Learning Lecture 3 - Linear Classifiers, Logistic Regression,

CSC 311: Introduction to Machine Learning Lecture 3 - Linear Classifiers, Logistic Regression, Multiclass Classification Roger Grosse Chris Maddison Juhan Bae Silviu Pitis University of Toronto, Fall 2020 Intro ML (UofT) CSC311-Lec3 1 /

699 views • 48 slides