logistic regression
play

Logistic regression Predict binary outcomes (success/failure) from - PowerPoint PPT Presentation

Mar 12, 2024 •232 likes •543 views

Logistic regression Predict binary outcomes (success/failure) from numerical or categorical predictors. Linear vs. logistic regression Linear regression: y = 0 + 1 x 1 + 2 x 2 + ! + n x n + Linear vs. logistic regression Linear

  1. Logistic regression Predict binary outcomes (success/failure) from numerical or categorical predictors.

  2. Linear vs. logistic regression Linear regression: y = β 0 + β 1 x 1 + β 2 x 2 + ! + β n x n + ε

  3. Linear vs. logistic regression Linear regression: y = β 0 + β 1 x 1 + β 2 x 2 + ! + β n x n + ε Logistic regression: e t Pr( success ) = 1 + e t t = β 0 + β 1 x 1 + β 2 x 2 + ! + β n x n + ε

  4. Linear vs. logistic regression Linear regression: y = β 0 + β 1 x 1 + β 2 x 2 + ! + β n x n + ε Logistic regression: e t Pr( success ) = 1 + e t t = β 0 + β 1 x 1 + β 2 x 2 + ! + β n x n + ε (generalized linear model, GLM)

  5. The logistic equation e t f ( t ) = 1 + e t

  6. Example: Pr(malignant) in biopsy data set

  7. Let’s do this step by step…

  8. Recall the biopsy data set clump_thickness uniform_cell_size uniform_cell_shape marg_adhesion 1 5 1 1 1 2 5 4 4 5 3 3 1 1 1 4 6 8 8 1 5 4 1 1 3 6 8 10 10 8 epithelial_cell_size bare_nuclei bland_chromatin normal_nucleoli mitoses 1 2 1 3 1 1 2 7 10 3 2 1 3 2 2 3 1 1 4 3 4 3 7 1 5 2 1 3 1 1 6 7 10 9 7 1 outcome 1 benign 2 benign 3 benign 4 benign 5 benign 6 malignant

  9. We do logistic regression with the glm() function > glm_out <- glm( outcome ~ clump_thickness + uniform_cell_size + uniform_cell_shape + marg_adhesion + epithelial_cell_size + bare_nuclei + bland_chromatin + normal_nucleoli + mitoses, data = biopsy, family = binomial )

  10. > summary(glm_out) Call: glm(formula = outcome ~ clump_thickness + uniform_cell_size + uniform_cell_shape + marg_adhesion + epithelial_cell_size + bare_nuclei + bland_chromatin + normal_nucleoli + mitoses, family = binomial, data = biopsy) Deviance Residuals: Min 1Q Median 3Q Max -3.4841 -0.1153 -0.0619 0.0222 2.4698 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -10.10394 1.17488 -8.600 < 2e-16 *** clump_thickness 0.53501 0.14202 3.767 0.000165 *** uniform_cell_size -0.00628 0.20908 -0.030 0.976039 uniform_cell_shape 0.32271 0.23060 1.399 0.161688 marg_adhesion 0.33064 0.12345 2.678 0.007400 ** epithelial_cell_size 0.09663 0.15659 0.617 0.537159 bare_nuclei 0.38303 0.09384 4.082 4.47e-05 *** bland_chromatin 0.44719 0.17138 2.609 0.009073 ** normal_nucleoli 0.21303 0.11287 1.887 0.059115 . mitoses 0.53484 0.32877 1.627 0.103788 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  11. > summary(glm_out) Call: glm(formula = outcome ~ clump_thickness + uniform_cell_size + uniform_cell_shape + marg_adhesion + epithelial_cell_size + bare_nuclei + bland_chromatin + normal_nucleoli + mitoses, family = binomial, data = biopsy) Deviance Residuals: Min 1Q Median 3Q Max -3.4841 -0.1153 -0.0619 0.0222 2.4698 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -10.10394 1.17488 -8.600 < 2e-16 *** clump_thickness 0.53501 0.14202 3.767 0.000165 *** uniform_cell_size -0.00628 0.20908 -0.030 0.976039 uniform_cell_shape 0.32271 0.23060 1.399 0.161688 marg_adhesion 0.33064 0.12345 2.678 0.007400 ** epithelial_cell_size 0.09663 0.15659 0.617 0.537159 bare_nuclei 0.38303 0.09384 4.082 4.47e-05 *** bland_chromatin 0.44719 0.17138 2.609 0.009073 ** normal_nucleoli 0.21303 0.11287 1.887 0.059115 . mitoses 0.53484 0.32877 1.627 0.103788 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  12. > glm_out <- glm( outcome ~ clump_thickness + uniform_cell_shape + marg_adhesion + epithelial_cell_size + bare_nuclei + bland_chromatin + normal_nucleoli + mitoses, data = biopsy, family = binomial )

  13. > summary(glm_out) Call: glm(formula = outcome ~ clump_thickness + uniform_cell_shape + marg_adhesion + epithelial_cell_size + bare_nuclei + bland_chromatin + normal_nucleoli + mitoses, family = binomial, data = biopsy) Deviance Residuals: Min 1Q Median 3Q Max -3.4823 -0.1154 -0.0620 0.0222 2.4694 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -10.09765 1.15546 -8.739 < 2e-16 *** clump_thickness 0.53456 0.14125 3.784 0.000154 *** uniform_cell_shape 0.31816 0.17424 1.826 0.067847 . marg_adhesion 0.32993 0.12115 2.723 0.006465 ** epithelial_cell_size 0.09612 0.15564 0.618 0.536876 bare_nuclei 0.38308 0.09384 4.082 4.46e-05 *** bland_chromatin 0.44648 0.16986 2.628 0.008578 ** normal_nucleoli 0.21255 0.11174 1.902 0.057149 . mitoses 0.53406 0.32761 1.630 0.103064 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  14. > summary(glm_out) Call: glm(formula = outcome ~ clump_thickness + uniform_cell_shape + marg_adhesion + epithelial_cell_size + bare_nuclei + bland_chromatin + normal_nucleoli + mitoses, family = binomial, data = biopsy) Deviance Residuals: Min 1Q Median 3Q Max -3.4823 -0.1154 -0.0620 0.0222 2.4694 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -10.09765 1.15546 -8.739 < 2e-16 *** clump_thickness 0.53456 0.14125 3.784 0.000154 *** uniform_cell_shape 0.31816 0.17424 1.826 0.067847 . marg_adhesion 0.32993 0.12115 2.723 0.006465 ** epithelial_cell_size 0.09612 0.15564 0.618 0.536876 bare_nuclei 0.38308 0.09384 4.082 4.46e-05 *** bland_chromatin 0.44648 0.16986 2.628 0.008578 ** normal_nucleoli 0.21255 0.11174 1.902 0.057149 . mitoses 0.53406 0.32761 1.630 0.103064 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  15. > glm_out <- glm( outcome ~ clump_thickness + uniform_cell_shape + marg_adhesion + bare_nuclei + bland_chromatin + normal_nucleoli + mitoses, data = biopsy, family = binomial )

  16. > summary(glm_out) Call: glm(formula = outcome ~ clump_thickness + uniform_cell_shape + marg_adhesion + bare_nuclei + bland_chromatin + normal_nucleoli + mitoses, family = binomial, data = biopsy) Deviance Residuals: Min 1Q Median 3Q Max -3.5235 -0.1149 -0.0627 0.0219 2.4115 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -9.98278 1.12610 -8.865 < 2e-16 *** clump_thickness 0.53400 0.14079 3.793 0.000149 *** uniform_cell_shape 0.34529 0.17164 2.012 0.044255 * marg_adhesion 0.34249 0.11922 2.873 0.004068 ** bare_nuclei 0.38830 0.09356 4.150 3.32e-05 *** bland_chromatin 0.46194 0.16820 2.746 0.006025 ** normal_nucleoli 0.22606 0.11097 2.037 0.041644 * mitoses 0.53119 0.32446 1.637 0.101598 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  17. > summary(glm.out) Call: glm(formula = outcome ~ clump_thickness + uniform_cell_shape + marg_adhesion + bare_nuclei + bland_chromatin + normal_nucleoli + mitoses, family = binomial, data = biopsy) Deviance Residuals: Min 1Q Median 3Q Max -3.5235 -0.1149 -0.0627 0.0219 2.4115 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -9.98278 1.12610 -8.865 < 2e-16 *** clump_thickness 0.53400 0.14079 3.793 0.000149 *** uniform_cell_shape 0.34529 0.17164 2.012 0.044255 * marg_adhesion 0.34249 0.11922 2.873 0.004068 ** bare_nuclei 0.38830 0.09356 4.150 3.32e-05 *** bland_chromatin 0.46194 0.16820 2.746 0.006025 ** normal_nucleoli 0.22606 0.11097 2.037 0.041644 * mitoses 0.53119 0.32446 1.637 0.101598 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  18. > glm_out <- glm( outcome ~ clump_thickness + uniform_cell_shape + marg_adhesion + bare_nuclei + bland_chromatin + normal_nucleoli, data = biopsy, family = binomial )

  19. > summary(glm_out) Call: glm(formula = outcome ~ clump_thickness + uniform_cell_shape + marg_adhesion + bare_nuclei + bland_chromatin + normal_nucleoli, family = binomial, data = biopsy) Deviance Residuals: Min 1Q Median 3Q Max -3.5201 -0.1186 -0.0570 0.0250 2.4055 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -9.76708 1.08506 -9.001 < 2e-16 *** clump_thickness 0.62253 0.13712 4.540 5.62e-06 *** uniform_cell_shape 0.34951 0.16503 2.118 0.03419 * marg_adhesion 0.33753 0.11561 2.920 0.00350 ** bare_nuclei 0.37855 0.09381 4.035 5.45e-05 *** bland_chromatin 0.47134 0.16612 2.837 0.00455 ** normal_nucleoli 0.24317 0.10855 2.240 0.02509 * --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

  20. The fitted logistic model

Recommend

Logistic Regression James H. Steiger Department of Psychology and Human Development Vanderbilt

Logistic Regression James H. Steiger Department of Psychology and Human Development Vanderbilt

Introduction The Logistic Regression Model Binary Logistic Regression Binomial Logistic Regression Interpreting Logistic Regression Parameters Examples Logistic Regression and Retrospective Studies Logistic Regression James H. Steiger

777 views • 54 slides
Todays lecture Logistic regression How can we use logistic regression for reranking? Shay

Todays lecture Logistic regression How can we use logistic regression for reranking? Shay

Todays lecture Logistic regression How can we use logistic regression for reranking? Shay Cohen How do we set the parameters of a logistic regression model? (based on slides by Sharon Goldwater) How is logistic regression related

652 views • 9 slides
w o o o o o o o o x o o o o o x o that represents how aligned the o x x x x

w o o o o o o o o x o o o o o x o that represents how aligned the o x x x x

Outline Logistic function Logistic regression IAML: Logistic Regression Learning logistic regression Optimization Charles Sutton and Victor Lavrenko The power of non-linear basis functions School of Informatics

343 views • 6 slides
w o o o o o o o x o o o x o o o that represents how aligned the x x x x x x

w o o o o o o o x o o o x o o o that represents how aligned the x x x x x x

Outline Logistic function IAML: Logistic Regression Logistic regression Learning logistic regression Optimization Nigel Goddard The power of non-linear basis functions School of Informatics Least-squares classification

622 views • 6 slides
w o o o o o o o x o o o x o o that represents how aligned the o x x x x x x

w o o o o o o o x o o o x o o that represents how aligned the o x x x x x x

Outline Logistic function IAML: Logistic Regression Logistic regression Learning logistic regression Optimization Nigel Goddard and Victor Lavrenko The power of non-linear basis functions School of Informatics

338 views • 6 slides
Regression 3: Logistic Regression Marco Baroni Practical Statistics in R Outline Logistic

Regression 3: Logistic Regression Marco Baroni Practical Statistics in R Outline Logistic

Regression 3: Logistic Regression Marco Baroni Practical Statistics in R Outline Logistic regression Logistic regression in R Outline Logistic regression Introduction The model Looking at and comparing fitted models Logistic regression in

1.09k views • 47 slides
LEARNING Outline Math Behind Logistic Regression Visualizing Logistic Regression Loss

LEARNING Outline Math Behind Logistic Regression Visualizing Logistic Regression Loss

Logistic Regression CSCI 447/547 MACHINE LEARNING Outline Math Behind Logistic Regression Visualizing Logistic Regression Loss Function Minimizing Log Likelihood Function Batch/Full Logistic Regression Gradient Descent for

498 views • 29 slides
Logistic Regression Lecture 6 Logistic Regression Classification Model CS 335

Logistic Regression Lecture 6 Logistic Regression Classification Model CS 335

Logistic Regression Lecture 6 Logistic Regression Classification Model CS 335 Cost function Gradient descent Linear classifiers and decision boundaries Dan Sheldon Classification Example: Hand-Written Digits Input: 20

541 views • 5 slides
From Logistic Regression to Neural Networks CMSC 470 Marine Carpuat Logistic Regression What

From Logistic Regression to Neural Networks CMSC 470 Marine Carpuat Logistic Regression What

From Logistic Regression to Neural Networks CMSC 470 Marine Carpuat Logistic Regression What you should know How to make a prediction with logistic regression classifier How to train a logistic regression classifier Machine learning

502 views • 33 slides
1 Relation Between Multinomial Logistic Regression Nave Bayes and Logistic Regression

1 Relation Between Multinomial Logistic Regression Nave Bayes and Logistic Regression

Logistic Regression Assumes a parametric form for directly estimating P( Y | X ). For binary concepts, this is: CS 391L: Machine Learning: 1 P Y = X = ( 1 | ) n + w + w i X Bayesian Learning: 1 exp( ) = i 0 i 1

308 views • 5 slides
Workshop 10.5a: Logistic regression Murray Logan August 23, 2016 Table of contents 1 Logistic

Workshop 10.5a: Logistic regression Murray Logan August 23, 2016 Table of contents 1 Logistic

-1- Workshop 10.5a: Logistic regression Murray Logan August 23, 2016 Table of contents 1 Logistic regression 1 2 Worked Examples 5 1. Logistic regression 1.1. Logistic regression 1.1.1. Binary data ( ) Link: log 1 Transform

188 views • 7 slides
Workshop 10.5a: Logistic regression Murray Logan 05 Sep 2016 Section 1 Logistic regression

Workshop 10.5a: Logistic regression Murray Logan 05 Sep 2016 Section 1 Logistic regression

Workshop 10.5a: Logistic regression Murray Logan 05 Sep 2016 Section 1 Logistic regression Logistic regression a d a t r y i n a B ( ) Link: log 1 Transform scale of linear predictor ( , ) into that of the

289 views • 16 slides
Logistic regression and Poisson regression Rasmus Waagepetersen Department of Mathematics

Logistic regression and Poisson regression Rasmus Waagepetersen Department of Mathematics

Logistic regression and Poisson regression Rasmus Waagepetersen Department of Mathematics Aalborg University Denmark October 6, 2017 Outline Logistic regression Poisson regression Binary and count data Linear mixed models very

391 views • 28 slides
Learning From Data Lecture 9 Logistic Regression and Gradient Descent Logistic Regression

Learning From Data Lecture 9 Logistic Regression and Gradient Descent Logistic Regression

Learning From Data Lecture 9 Logistic Regression and Gradient Descent Logistic Regression Gradient Descent M. Magdon-Ismail CSCI 4100/6100 recap: Linear Classification and Regression The linear signal: s = w t x Good Features are Important

272 views • 23 slides
Machine Learning Logistic Regression Hamid R. Rabiee Spring 2015

Machine Learning Logistic Regression Hamid R. Rabiee Spring 2015

Machine Learning Logistic Regression Hamid R. Rabiee Spring 2015 http://ce.sharif.edu/courses/93-94/2/ce717-1 / Agenda Agenda Probabilistic Classification Introduction to Logistic regression Binary logistic regression Logistic

362 views • 20 slides
Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common

Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common

0. Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common property CMU, 2004 fall, Andrew Moore, HW2, pr. 4 2. Linear Regression and Logistic Regression: Definitions Given an input vector X , linear

1.07k views • 22 slides
Topics of the day Logistic regression and generalized linear models Rasmus Waagepetersen

Topics of the day Logistic regression and generalized linear models Rasmus Waagepetersen

Topics of the day Logistic regression and generalized linear models Rasmus Waagepetersen Logistic regression Department of Mathematics Overdispersion Aalborg University Logistic regression with random effects Denmark November 5,

422 views • 7 slides
LOGISTIC REGRESSION, GRADIENT LOGISTIC REGRESSION, GRADIENT DESCENT, NEWTON DESCENT, NEWTON

LOGISTIC REGRESSION, GRADIENT LOGISTIC REGRESSION, GRADIENT DESCENT, NEWTON DESCENT, NEWTON

LOGISTIC REGRESSION, GRADIENT LOGISTIC REGRESSION, GRADIENT DESCENT, NEWTON DESCENT, NEWTON Matthieu R Bloch Thursday, January 30, 2020 1 LOGISTICS LOGISTICS TAs and Office hours Monday: Mehrdad (TSRB 523a) - 2pm-3:15pm Tuesday: TJ (VL

299 views • 14 slides
Binary Logistic Regression + Multinomial Logistic Regression Matt Gormley Lecture 10 Feb. 17,

Binary Logistic Regression + Multinomial Logistic Regression Matt Gormley Lecture 10 Feb. 17,

10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Binary Logistic Regression + Multinomial Logistic Regression Matt Gormley Lecture 10 Feb. 17, 2020 1 Reminders

452 views • 43 slides
4. Logistic Regression Simple Logistic Regression Y = 0 or 1 = Pr ( Y = 1 ) ( x )

4. Logistic Regression Simple Logistic Regression Y = 0 or 1 = Pr ( Y = 1 ) ( x )

4. Logistic Regression Simple Logistic Regression Y = 0 or 1 = Pr ( Y = 1 ) ( x ) ( x ) = log = + x logit 1 ( x ) Uses logit link for binomial Y . Equivalently, exp ( + x ) ( x ) = 1 + exp (

896 views • 56 slides
Introduction to Machine Learning Classification: Logistic Regression

Introduction to Machine Learning Classification: Logistic Regression

Introduction to Machine Learning Classification: Logistic Regression compstat-lmu.github.io/lecture_i2ml MOTIVATION A discriminant approach for directly modeling the posterior probabilities ( x | ) of the labels is logistic regression . For

484 views • 9 slides
Logistic regression Shay Cohen (based on slides by Sharon Goldwater) 28 October 2019 Todays

Logistic regression Shay Cohen (based on slides by Sharon Goldwater) 28 October 2019 Todays

Logistic regression Shay Cohen (based on slides by Sharon Goldwater) 28 October 2019 Todays lecture How can we use logistic regression for reranking? How do we set the parameters of a logistic regression model? How is logistic

652 views • 33 slides
Machine Learning - MT 2016 8. Classification: Logistic Regression Varun Kanade University of

Machine Learning - MT 2016 8. Classification: Logistic Regression Varun Kanade University of

Machine Learning - MT 2016 8. Classification: Logistic Regression Varun Kanade University of Oxford November 2, 2016 Logistic Regression Logistic Regression is actually a classification method In its simplest form it is a binary (two classes)

486 views • 14 slides
Day 09 - Logistic Regression Day 09 - Logistic Regression Oct. 6, 2020 Oct. 6, 2020

Day 09 - Logistic Regression Day 09 - Logistic Regression Oct. 6, 2020 Oct. 6, 2020

Day 09 - Logistic Regression Day 09 - Logistic Regression Oct. 6, 2020 Oct. 6, 2020 Administrative Administrative Homework 3 will be assigned Friday 10/9 and due Friday 10/23 Midterm will be given Thursday 10/29 in class From Pre-Class

586 views • 17 slides

More recommend