Linear Regression Part 1: Fitting Lines INFO-1301, Quantitative - PowerPoint PPT Presentation

Sep 16, 2022 •26 likes •126 views

Linear Regression Part 1: Fitting Lines INFO-1301, Quantitative Reasoning 1 University of Colorado Boulder April 19, 2017 Prof. Michael Paul Interpreting Linear Functions Fishermen in the Finger Lakes Region have been recording the dead fish

Linear Regression Part 1: Fitting Lines INFO-1301, Quantitative Reasoning 1 University of Colorado Boulder April 19, 2017 Prof. Michael Paul
Interpreting Linear Functions Fishermen in the Finger Lakes Region have been recording the dead fish they encounter while fishing in the region. The Department of Environmental Conservation monitors the pollution index for the Finger Lakes Region. The model for the number of fish deaths y for a given pollution index x is y = 9.607x + 111.958. What can we do with this function? • Estimate fish deaths for pollution values that we’ve never measured
Interpolation and Extrapolation y = 9.607x + 111.958 x is the pollution index Suppose we came up with this formula as an approximation after measuring fish deaths when the pollution index was: 1.1, 1.8, 2.5, 3.0, 3.9, 5.2 • What if we wanted to know deaths at x=3.5? Interpolation is when we use our linear function to estimate a value at a point in between points we have already measured
Interpolation and Extrapolation y = 9.607x + 111.958 x is the pollution index Suppose we came up with this formula as an approximation after measuring fish deaths when the pollution index was: 1.1, 1.8, 2.5, 3.0, 3.9, 5.2 • What if we wanted to know deaths at x=7.0? Extrapolation is when we use our linear function to estimate a value at a point outside of points we have already measured • Why might this fail?
Fitting Linear Functions Where does a linear function such as “y = 9.607x + 111.958” come from? Want to pick slope and y-intercept (y= mx + b ) such that the line is as close as possible to the true data points • Want to minimize distance from each point to the line • We’ll be more concrete next time
Fitting Linear Functions The process of picking the parameters of a function (e.g., m and b ) to make it is close as possible to a set of data points is regression If the function is linear (i.e., a line) then this is linear regression Statistical software such as MiniTab Express can perform linear regression automatically
Practice Regression in MiniTab Express.
Revisiting Correlation The Pearson correlation measures how strongly data points are related linearly A perfect correlation of 1 or -1 occurs the best-fit line exactly matches every data point • No error → perfect linear fit • 1 if slope is positive, -1 if slope is negative The size of the correlation tells you how well the points can be approximated with a line The sign of the correlation tells you the slope
Revisiting Correlation
R squared A common metric for measuring the quality of the fit of a line is called R 2 This is the square of the correlation (sometimes called R) between the true Y values and the Y values that you estimate with the y=mx+b line

Recommend

Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common

0. Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common property CMU, 2004 fall, Andrew Moore, HW2, pr. 4 2. Linear Regression and Logistic Regression: Definitions Given an input vector X , linear

1.07k views • 22 slides

Notes on the Non-linear Regression The model Non-linear regression models, like ordinary linear

Notes on the Non-linear Regression The model Non-linear regression models, like ordinary linear models, assume that the response, Y , has a normal distribution with constant variance. Unlike linear models, however, the mean function may depend

159 views • 4 slides

LINEAR REGRESSION LINEAR REGRESSION - FROM A MACHINE LEARNING POINT OF VIEW 25 SIMPLE LINEAR

LINEAR REGRESSION LINEAR REGRESSION - FROM A MACHINE LEARNING POINT OF VIEW 25 SIMPLE LINEAR REGRESSION 20 Sales 15 Starting point 10 Simplest parametric function 5 Easy to interpret the parameters: 0 50 100 150 200 250

1.12k views • 13 slides

Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied

Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied linear regression ; the output/label space Y was R . 90 80 delay 70 60 50 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 duration 1 / 68 Linear

607 views • 38 slides

Which models can be fit with linear regression? Simple linear regression in Matlab X = rand(3,3)

Which models can be fit with linear regression? Simple linear regression in Matlab X = rand(3,3) b = pinv(X) * y X = b = 0.0467 0.5188 0.5317 14.3966 0.6587 0.3323 0.5070 50.9428 0.7573 0.0428 0.2532 -50.8102 y = rand(3,1) b = X \

643 views • 13 slides

Trendlines Simple Linear Regression Multiple Linear Regression Systematic Model

Trendlines Simple Linear Regression Multiple Linear Regression Systematic Model Building Practical Issues Overfitting Categorical Variables Interaction Terms Non-linear Terms Linear y = a + bx Logarithmic

945 views • 65 slides

Outline The Simple Linear Regression Model (12.1) Fitting the Regression Line (12.2)

2/22/2007 219323 Probability and Statistics for Software Statistics for Software and Knowledge Engineers Lecture 13: Simple Linear Regression Simple Linear Regression and Correlation Monchai Sopitkamon, Ph.D. Outline The Simple Linear

968 views • 19 slides

Unit 7: Multiple linear regression 1. Introduction to multiple linear regression Sta 101 - Fall

Announcements Unit 7: Multiple linear regression 1. Introduction to multiple linear regression Sta 101 - Fall 2018 Project questions? Duke University, Department of Statistical Science Dr. Abrahamsen Slides posted at

537 views • 10 slides

Linear Models for Regression Greg Mori - CMPT 419/726 Bishop PRML Ch. 3 Regression Linear Basis

Regression Linear Basis Function Models Loss Functions for Regression Finding Optimal Weights Regularization Bayesian Linear Linear Models for Regression Greg Mori - CMPT 419/726 Bishop PRML Ch. 3 Regression Linear Basis Function Models

1.19k views • 84 slides

Regression 1: Linear Regression Marco Baroni Practical Statistics in R Outline Classic linear

Regression 1: Linear Regression Marco Baroni Practical Statistics in R Outline Classic linear regression Linear regression in R Outline Classic linear regression Introduction Constructing the model Estimation Looking at the fitted model

1.13k views • 77 slides

Overview IAML: Linear Regression The linear model Fitting the linear model to data

Overview IAML: Linear Regression The linear model Fitting the linear model to data Probabilistic interpretation of the error function Nigel Goddard School of Informatics Examples of regression problems Dealing with multiple

325 views • 10 slides

Learning From Data Lecture 8 Linear Classification and Regression Linear Classification Linear

Learning From Data Lecture 8 Linear Classification and Regression Linear Classification Linear Regression M. Magdon-Ismail CSCI 4100/6100 recap: Approximation Versus Generalization VC Analysis Bias-Variance Analysis E out E in + ( d

390 views • 23 slides

Linear regression Linear regression is a simple approach to supervised learning. It assumes

Linear regression Linear regression is a simple approach to supervised learning. It assumes that the dependence of Y on X 1 , X 2 , . . . X p is linear. True regression functions are never linear! 7 6 f(X) 5 4 3 2 4 6 8 X

704 views • 48 slides

10-601 Machine Learning Regression Outline Regression vs Classification Linear regression

10-601 Machine Learning Regression Outline Regression vs Classification Linear regression another discriminative learning method As optimization Gradient descent As matrix inversion (Ordinary Least Squares) Overfitting

1.52k views • 97 slides

Introduction to Linear Regression Rebecca C. Steorts September 15, 2015 Today

Introduction to Linear Regression Rebecca C. Steorts September 15, 2015 Today (Re-)Introduction to linear models and the model space What is linear regression Basic properties of linear regression Using data frames for statistical

849 views • 37 slides

Secure Linear Regression on Secure Linear Regression on Vertically Partitioned Datasets

Secure Linear Regression on Secure Linear Regression on Vertically Partitioned Datasets Vertically Partitioned Datasets Adria Gascon Phillipp Schoppmann Borja Balle Mariana Raykova Samee Zahur Jack Doerner David Evans Cryptography in the

233 views • 22 slides

Linear Regression Yijun Zhao Northeastern University Fall 2016 Yijun Zhao Linear Regression

Linear Regression Yijun Zhao Northeastern University Fall 2016 Yijun Zhao Linear Regression Regression Examples Any Attributes Continuous Value = x y { age , major , gender , race } GPA { income , credit score , profession }

613 views • 37 slides

CS70: Lecture 35. Regression (contd.): Linear and Beyond CS70: Lecture 35. Regression (contd.):

CS70: Lecture 35. Regression (contd.): Linear and Beyond CS70: Lecture 35. Regression (contd.): Linear and Beyond 1. Review: Linear Regression (LR), LLSE 2. LR: Examples 3. Beyond LR: Quadratic Regression 4. Conditional Expectation (CE) and

2.12k views • 132 slides

Regression Methods 1. Linear Regression with only one parameter, and without offset; MLE and

0. Regression Methods 1. Linear Regression with only one parameter, and without offset; MLE and MAP estimation CMU, 2012 fall, Tom Mitchell, Ziv Bar-Joseph, midterm, pr. 3 2. Consider real-valued variables X and Y . The Y variable is

1.25k views • 86 slides

Multiple Linear Regression The population model In a simple linear regression model, a single

Multiple Linear Regression The population model In a simple linear regression model, a single response measure- ment Y is related to a single predictor (covariate, regressor) X for each observation. The critical assumption of the model is

859 views • 49 slides

Statistics for Applications Chapter 7: Regression 1/43 Heuristics of the linear regression (1)

Statistics for Applications Chapter 7: Regression 1/43 Heuristics of the linear regression (1) Consider a cloud of i.i.d. random points ( X i , Y i ) , i = 1 , . . . , n : 2/43 Heuristics of the linear regression (2) I Idea: Fit the best line

780 views • 44 slides

CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear

CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear Regression 1 / 26 Overview First learning algorithm of the course: linear regression Task: predict scalar-valued targets, e.g. stock prices (hence

328 views • 29 slides

CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear

CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear Regression 1 / 30 Overview First learning algorithm of the course: linear regression Task: predict scalar-valued targets, e.g. stock prices (hence

506 views • 34 slides

Concise Implementation of Linear Regression Concise Implementation of Linear Regression

Concise Implementation of Linear Regression Concise Implementation of Linear Regression Generating Data Sets Generating Data Sets In [1]: from mxnet import autograd, nd num_inputs = 2 num_examples = 1000 true_w = nd.array([2, -3.4]) true_b =

782 views • 9 slides