csc 411 lecture 02 linear regression
play

CSC 411: Lecture 02: Linear Regression Class based on Raquel Urtasun - PowerPoint PPT Presentation

Apr 08, 2024 •175 likes •1.15k views

CSC 411: Lecture 02: Linear Regression Class based on Raquel Urtasun & Rich Zemels lectures Sanja Fidler University of Toronto Jan 13, 2016 (Most plots in this lecture are from Bishops book) Urtasun, Zemel, Fidler (UofT) CSC 411:

  1. CSC 411: Lecture 02: Linear Regression Class based on Raquel Urtasun & Rich Zemel’s lectures Sanja Fidler University of Toronto Jan 13, 2016 (Most plots in this lecture are from Bishop’s book) Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 1 / 22

  2. Problems for Today What should I watch this Friday? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 2 / 22

  3. Problems for Today What should I watch this Friday? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 2 / 22

  4. Problems for Today Goal : Predict movie rating automatically! Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 2 / 22

  5. Problems for Today Goal: How many followers will I get? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 2 / 22

  6. Problems for Today Goal: Predict the price of the house Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 2 / 22

  7. Regression What do all these problems have in common? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  8. Regression What do all these problems have in common? ◮ Continuous outputs, we’ll call these t (eg, a rating: a real number between 0-10, # of followers, house price) Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  9. Regression What do all these problems have in common? ◮ Continuous outputs, we’ll call these t (eg, a rating: a real number between 0-10, # of followers, house price) What do I need in order to predict these outputs? Predicting continuous outputs is called regression Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  10. Regression What do all these problems have in common? ◮ Continuous outputs, we’ll call these t (eg, a rating: a real number between 0-10, # of followers, house price) What do I need in order to predict these outputs? Predicting continuous outputs is called regression ◮ Features (inputs), we’ll call these x (or x if vectors) Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  11. Regression What do all these problems have in common? ◮ Continuous outputs, we’ll call these t (eg, a rating: a real number between 0-10, # of followers, house price) What do I need in order to predict these outputs? Predicting continuous outputs is called regression ◮ Features (inputs), we’ll call these x (or x if vectors) ◮ Training examples, many x ( i ) for which t ( i ) is known (eg, many movies for which we know the rating) Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  12. Regression What do all these problems have in common? ◮ Continuous outputs, we’ll call these t (eg, a rating: a real number between 0-10, # of followers, house price) What do I need in order to predict these outputs? Predicting continuous outputs is called regression ◮ Features (inputs), we’ll call these x (or x if vectors) ◮ Training examples, many x ( i ) for which t ( i ) is known (eg, many movies for which we know the rating) ◮ A model, a function that represents the relationship between x and t Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  13. Regression What do all these problems have in common? ◮ Continuous outputs, we’ll call these t (eg, a rating: a real number between 0-10, # of followers, house price) What do I need in order to predict these outputs? Predicting continuous outputs is called regression ◮ Features (inputs), we’ll call these x (or x if vectors) ◮ Training examples, many x ( i ) for which t ( i ) is known (eg, many movies for which we know the rating) ◮ A model, a function that represents the relationship between x and t ◮ A loss or a cost or an objective function, which tells us how well our model approximates the training examples Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  14. Regression What do all these problems have in common? ◮ Continuous outputs, we’ll call these t (eg, a rating: a real number between 0-10, # of followers, house price) What do I need in order to predict these outputs? Predicting continuous outputs is called regression ◮ Features (inputs), we’ll call these x (or x if vectors) ◮ Training examples, many x ( i ) for which t ( i ) is known (eg, many movies for which we know the rating) ◮ A model, a function that represents the relationship between x and t ◮ A loss or a cost or an objective function, which tells us how well our model approximates the training examples ◮ Optimization, a way of finding the parameters of our model that minimizes the loss function Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  15. Today: Linear Regression Linear regression ◮ continuous outputs ◮ simple model (linear) Introduce key concepts: ◮ loss functions ◮ generalization ◮ optimization ◮ model complexity ◮ regularization Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 4 / 22

  16. Simple 1-D regression Circles are data points (i.e., training examples) that are given to us Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 5 / 22

  17. Simple 1-D regression Circles are data points (i.e., training examples) that are given to us The data points are uniform in x , but may be displaced in y t ( x ) = f ( x ) + ǫ with ǫ some noise Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 5 / 22

  18. Simple 1-D regression Circles are data points (i.e., training examples) that are given to us The data points are uniform in x , but may be displaced in y t ( x ) = f ( x ) + ǫ with ǫ some noise In green is the ”true” curve that we don’t know Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 5 / 22

  19. Simple 1-D regression Circles are data points (i.e., training examples) that are given to us The data points are uniform in x , but may be displaced in y t ( x ) = f ( x ) + ǫ with ǫ some noise In green is the ”true” curve that we don’t know Goal: We want to fit a curve to these points Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 5 / 22

  20. Simple 1-D regression Key Questions: Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 6 / 22

  21. Simple 1-D regression Key Questions: ◮ How do we parametrize the model? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 6 / 22

  22. Simple 1-D regression Key Questions: ◮ How do we parametrize the model? ◮ What loss (objective) function should we use to judge the fit? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 6 / 22

  23. Simple 1-D regression Key Questions: ◮ How do we parametrize the model? ◮ What loss (objective) function should we use to judge the fit? ◮ How do we optimize fit to unseen test data (generalization)? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 6 / 22

  24. Example: Boston Housing data Estimate median house price in a neighborhood based on neighborhood statistics Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 7 / 22

  25. Example: Boston Housing data Estimate median house price in a neighborhood based on neighborhood statistics Look at first possible attribute (feature): per capita crime rate Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 7 / 22

  26. Example: Boston Housing data Estimate median house price in a neighborhood based on neighborhood statistics Look at first possible attribute (feature): per capita crime rate Use this to predict house prices in other neighborhoods Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 7 / 22

  27. Example: Boston Housing data Estimate median house price in a neighborhood based on neighborhood statistics Look at first possible attribute (feature): per capita crime rate Use this to predict house prices in other neighborhoods Is this a good input (attribute) to predict house prices? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 7 / 22

  28. Represent the Data Data is described as pairs D = { ( x (1) , t (1) ) , · · · , ( x ( N ) , t ( N ) ) } ◮ x ∈ R is the input feature (per capita crime rate) ◮ t ∈ R is the target output (median house price) ◮ ( i ) simply indicates the training examples (we have N in this case) Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 8 / 22

  29. Represent the Data Data is described as pairs D = { ( x (1) , t (1) ) , · · · , ( x ( N ) , t ( N ) ) } ◮ x ∈ R is the input feature (per capita crime rate) ◮ t ∈ R is the target output (median house price) ◮ ( i ) simply indicates the training examples (we have N in this case) Here t is continuous, so this is a regression problem Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 8 / 22

  30. Represent the Data Data is described as pairs D = { ( x (1) , t (1) ) , · · · , ( x ( N ) , t ( N ) ) } ◮ x ∈ R is the input feature (per capita crime rate) ◮ t ∈ R is the target output (median house price) ◮ ( i ) simply indicates the training examples (we have N in this case) Here t is continuous, so this is a regression problem Model outputs y , an estimate of t y ( x ) = w 0 + w 1 x Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 8 / 22

Recommend

CS70: Lecture 35. Regression (contd.): Linear and Beyond CS70: Lecture 35. Regression (contd.):

CS70: Lecture 35. Regression (contd.): Linear and Beyond CS70: Lecture 35. Regression (contd.):

CS70: Lecture 35. Regression (contd.): Linear and Beyond CS70: Lecture 35. Regression (contd.): Linear and Beyond 1. Review: Linear Regression (LR), LLSE 2. LR: Examples 3. Beyond LR: Quadratic Regression 4. Conditional Expectation (CE) and

2.12k views • 132 slides
CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear

CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear

CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear Regression 1 / 30 Overview First learning algorithm of the course: linear regression Task: predict scalar-valued targets, e.g. stock prices (hence

506 views • 34 slides
CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear

CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear

CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear Regression 1 / 26 Overview First learning algorithm of the course: linear regression Task: predict scalar-valued targets, e.g. stock prices (hence

328 views • 29 slides
Learning From Data Lecture 8 Linear Classification and Regression Linear Classification Linear

Learning From Data Lecture 8 Linear Classification and Regression Linear Classification Linear

Learning From Data Lecture 8 Linear Classification and Regression Linear Classification Linear Regression M. Magdon-Ismail CSCI 4100/6100 recap: Approximation Versus Generalization VC Analysis Bias-Variance Analysis E out E in + ( d

390 views • 23 slides
Outline The Simple Linear Regression Model (12.1) Fitting the Regression Line (12.2)

Outline The Simple Linear Regression Model (12.1) Fitting the Regression Line (12.2)

2/22/2007 219323 Probability and Statistics for Software Statistics for Software and Knowledge Engineers Lecture 13: Simple Linear Regression Simple Linear Regression and Correlation Monchai Sopitkamon, Ph.D. Outline The Simple Linear

968 views • 19 slides
CSC 411 Lecture 6: Linear Regression Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla

CSC 411 Lecture 6: Linear Regression Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla

CSC 411 Lecture 6: Linear Regression Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla University of Toronto UofT CSC 411: 06-Linear Regression 1 / 37 A Timely XKCD UofT CSC 411: 06-Linear Regression 2 / 37 Overview So far,

482 views • 37 slides
Learning From Data Lecture 9 Logistic Regression and Gradient Descent Logistic Regression

Learning From Data Lecture 9 Logistic Regression and Gradient Descent Logistic Regression

Learning From Data Lecture 9 Logistic Regression and Gradient Descent Logistic Regression Gradient Descent M. Magdon-Ismail CSCI 4100/6100 recap: Linear Classification and Regression The linear signal: s = w t x Good Features are Important

272 views • 23 slides
Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common

Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common

0. Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common property CMU, 2004 fall, Andrew Moore, HW2, pr. 4 2. Linear Regression and Logistic Regression: Definitions Given an input vector X , linear

1.07k views • 22 slides
Lecture 1. From Linear Regression Nan Ye School of Mathematics and Physics University of

Lecture 1. From Linear Regression Nan Ye School of Mathematics and Physics University of

Lecture 1. From Linear Regression Nan Ye School of Mathematics and Physics University of Queensland 1 / 20 Quiz Q1. Which dataset is linear regression of y against x suitable for? y y 15 1

596 views • 21 slides
Notes on the Non-linear Regression The model Non-linear regression models, like ordinary linear

Notes on the Non-linear Regression The model Non-linear regression models, like ordinary linear

Notes on the Non-linear Regression The model Non-linear regression models, like ordinary linear models, assume that the response, Y , has a normal distribution with constant variance. Unlike linear models, however, the mean function may depend

159 views • 4 slides
LINEAR REGRESSION LINEAR REGRESSION - FROM A MACHINE LEARNING POINT OF VIEW 25 SIMPLE LINEAR

LINEAR REGRESSION LINEAR REGRESSION - FROM A MACHINE LEARNING POINT OF VIEW 25 SIMPLE LINEAR

LINEAR REGRESSION LINEAR REGRESSION - FROM A MACHINE LEARNING POINT OF VIEW 25 SIMPLE LINEAR REGRESSION 20 Sales 15 Starting point 10 Simplest parametric function 5 Easy to interpret the parameters: 0 50 100 150 200 250

1.12k views • 13 slides
Lecture 6: Multiple Linear Regression, Polynomial Regression and Model Selection CS109A

Lecture 6: Multiple Linear Regression, Polynomial Regression and Model Selection CS109A

Lecture 6: Multiple Linear Regression, Polynomial Regression and Model Selection CS109A Introduction to Data Science Pavlos Protopapas and Kevin Rader Announcements Section : Friday 1:30-2:45pm : @ MD 123 (only this Friday) A-section: Today:

921 views • 65 slides
CS70: Jean Walrand: Lecture 30. Linear Regression: Preamble Linear Regression: Preamble Thus, if

CS70: Jean Walrand: Lecture 30. Linear Regression: Preamble Linear Regression: Preamble Thus, if

X Y CS70: Jean Walrand: Lecture 30. Linear Regression: Preamble Linear Regression: Preamble Thus, if we want to guess the value of Y , we choose E [ Y ] . The best guess about Y , if we know only the distribution of Y , is E [ Y ] . Now assume

53 views • 4 slides
CS480/680 Machine Learning Lecture 3: May 13, 2019 Linear Regression [RN] Sec. 18.6.1, [HTF]

CS480/680 Machine Learning Lecture 3: May 13, 2019 Linear Regression [RN] Sec. 18.6.1, [HTF]

CS480/680 Machine Learning Lecture 3: May 13, 2019 Linear Regression [RN] Sec. 18.6.1, [HTF] Sec. 2.3.1, [D] Sec. 7.6, [B] Sec. 3.1, [M] Sec. 1.4.5 University of Waterloo CS480/680 Spring 2019 Pascal Poupart 1 Linear model for regression

582 views • 14 slides
Lecture #5: Multiple Linear Regression Data Science 1 CS 109A, STAT 121A, AC 209A, E-109A Pavlos

Lecture #5: Multiple Linear Regression Data Science 1 CS 109A, STAT 121A, AC 209A, E-109A Pavlos

Lecture #5: Multiple Linear Regression Data Science 1 CS 109A, STAT 121A, AC 209A, E-109A Pavlos Protopapas Kevin Rader Margo Levine Rahul Dave Lecture Outline Review More on Model Evaluation Multiple Linear Regression Evaluating

585 views • 32 slides
Lecture 5: Multiple Linear Regression CS109A Introduction to Data Science Pavlos Protopapas and

Lecture 5: Multiple Linear Regression CS109A Introduction to Data Science Pavlos Protopapas and

Lecture 5: Multiple Linear Regression CS109A Introduction to Data Science Pavlos Protopapas and Kevin Rader Lecture Outline Simple Regression: Predictor variables Standard Errors Evaluating Significance of Predictors Hypothesis

540 views • 50 slides
Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied

Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied

Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied linear regression ; the output/label space Y was R . 90 80 delay 70 60 50 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 duration 1 / 68 Linear

607 views • 38 slides
Lecture 11: Interpreting logistic regression models Ani Manichaikul amanicha@jhsph.edu 3 May

Lecture 11: Interpreting logistic regression models Ani Manichaikul amanicha@jhsph.edu 3 May

Lecture 11: Interpreting logistic regression models Ani Manichaikul amanicha@jhsph.edu 3 May 2007 Logistic regression n Framework and ideas of linear modelling similar to linear regression n Still have a systematic and probabilistic part to

1.29k views • 62 slides
Chapter 7 Linear Regression 04/05/2016 Huamei Dong 1. Review Least square regression line 2.

Chapter 7 Linear Regression 04/05/2016 Huamei Dong 1. Review Least square regression line 2.

Chapter 7 Linear Regression 04/05/2016 Huamei Dong 1. Review Least square regression line 2. Example of linear regression analysis with R 1. Review of linear regression analysis In last lecture, we learned correlation coefficient R : We also

240 views • 8 slides
CSC 411 Lecture 19: Bayesian Linear Regression Roger Grosse, Amir-massoud Farahmand, and Juan

CSC 411 Lecture 19: Bayesian Linear Regression Roger Grosse, Amir-massoud Farahmand, and Juan

CSC 411 Lecture 19: Bayesian Linear Regression Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla University of Toronto UofT CSC 411: 19-Bayesian Linear Regression 1 / 36 Overview Weve covered both parametric and nonparametric

543 views • 36 slides
Week 3: Linear Regression Instructor: Sergey Levine 1 Recap In the previous lecture we saw how

Week 3: Linear Regression Instructor: Sergey Levine 1 Recap In the previous lecture we saw how

Week 3: Linear Regression Instructor: Sergey Levine 1 Recap In the previous lecture we saw how linear regression can solve the following problem: given a dataset D = { ( x 1 , y 1 ) , . . . , ( x N , y N ) } , learn to predict y from x . In

319 views • 5 slides
Which models can be fit with linear regression? Simple linear regression in Matlab X = rand(3,3)

Which models can be fit with linear regression? Simple linear regression in Matlab X = rand(3,3)

Which models can be fit with linear regression? Simple linear regression in Matlab X = rand(3,3) b = pinv(X) * y X = b = 0.0467 0.5188 0.5317 14.3966 0.6587 0.3323 0.5070 50.9428 0.7573 0.0428 0.2532 -50.8102 y = rand(3,1) b = X \

643 views • 13 slides
Trendlines Simple Linear Regression Multiple Linear Regression Systematic Model

Trendlines Simple Linear Regression Multiple Linear Regression Systematic Model

Trendlines Simple Linear Regression Multiple Linear Regression Systematic Model Building Practical Issues Overfitting Categorical Variables Interaction Terms Non-linear Terms Linear y = a + bx Logarithmic

945 views • 65 slides
Lecture 3: Kernel Regression Distance Metrics Curse of Dimensionality Linear

Lecture 3: Kernel Regression Distance Metrics Curse of Dimensionality Linear

Lecture 3: Kernel Regression Distance Metrics Curse of Dimensionality Linear Regression Aykut Erdem October 2016 Hacettepe University Administrative Assignment 1 will be out on Wednesday It is due October 26 (i.e. in two

476 views • 46 slides

More recommend