Logistic Regression: MLE vs. OLS1 in Excel2013 29 Aug 2016 V0B V0B - PDF document

Logistic Regression: MLE vs. OLS1 in Excel2013 29 Aug 2016 V0B V0B V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 1 Schield MLE vs. OLS1-Based Logistic Excel 2013 2 Logistic Regression: Background & Goals MLE vs. OLS1 in Excel 2013 Modeling a binary outcome requires a logistic model. by Doing logistic regression properly requires MLE. Milo Schield Doing MLE in Excel is not easy. See Schield (2015) Member: International Statistical Institute Schield created two logistic OLS models: OLS1+OLS3 US Rep: International Statistical Literacy Project OLS1: Model Ln[Odds(Pnudge)]. See Schield (2014a). Director, W. M. Keck Statistical Literacy Project OLS2: Model Ln[Odds(Pgroup)]. See Schield (2016c) OLS3: Use OLS to estimate logistic parameters. Slides and data at: www.StatLit.org/ See Schield (2014b) pdf/2016-Schield-Logistic-MLE-OLS1-Excel2013-slides.pdf pdf/2016-Schield-Logistic-MLE-OLS1-Excel2013-demo.pdf These slides compare MLE with OLS1 logistic regression Excel/2016-Schield-Logistic-MLE-OLS1-Excel2013.xlsx V0B V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 3 Schield MLE vs. OLS1-Based Logistic Excel 2013 4 Model Gender by Height (OLS) Model using a Logistic Function Must use a logistic function This linear trend-line goes outside the valid range Range of Odds(p): Zero to infinity Range of Ln[Odds(p)]: Minus infinity to infinity Logistic model: Ln[Odds(p)] = a + b*X V0A Logistic P(male|Height) 1 0.8 0.6 0.4 0.2 0 61 63 65 67 69 71 73 75 Height (inches) Pulse data Schield V0B V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 5 Schield MLE vs. OLS1-Based Logistic Excel 2013 6 |Ln[Odds(p)]| = infinity if p = 0 or 1 1a: Logistic P(male|Height) Pnudge: =If(p=0, 0.001, 0.999) MLE Source: Pulse dataset MLE Logistic Excel 2013 Gender by Height 1 Men Probability (Male) MLE Logistic regression 0.75 P(Y|X) = 1 / {1 + Exp[‐(a + bX)]} OLS1 Model : a = ‐53.32 ; b = 0.7905. 0.5 X = Xo if P(Y) = 0.5 Xo = ‐a/b = 67.4529 Ln[Odds(Pnudge)] Slope(X=Xo) = b/4 = 0.1976 = Constant + 0.25 b1*X1 + b2*X2 Women 0 using Ordinary 60 62 64 66 68 70 72 74 76 Least Squares. Height (inches) Pulse.xls Schield 2016-Schield-Logistic-MLE-OLS1-Excel2013-Slides.pdf 1

Logistic Regression: MLE vs. OLS1 in Excel2013 29 Aug 2016 V0B V0B V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 7 Schield MLE vs. OLS1-Based Logistic Excel 2013 8 1b: Logistic P(male|Height) 1c: Logistic P(male|Height) OLS1 MLE vs. OLS1 . V0B Logistic P(male|Height): OLS1 V0B Logistic P(male|Height): MLE vs. OLS1 1 1 OLS1A (dash): OLS1A (dash): 0.8 0.8 Ln[Odds(Male1)] Ln[Odds(Male1)] MLE1A (solid): = ‐88.80 = ‐88.80 Ln[Odds(Male)] 0.6 0.6 + 1.316*Height + 1.316*Height = ‐55.32 + 0.7905*Height 0.4 0.4 0.2 0.2 0 0 61 63 65 67 69 71 73 75 61 63 65 67 69 71 73 75 Height (inches) Height (inches) Pulse data Schield Pulse data Schield V0B V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 9 Schield MLE vs. OLS1-Based Logistic Excel 2013 10 2a: Logistic P(male|Weight) 2b: Logistic P(male|Weight) MLE OLS1 . . V0A Logistic P(male|Weight): MLE V0A Logistic P(male|Weight): OLS1 1 1 0.8 0.8 MLE1B (solid): OLS1B (dash): Ln[Odds(Male)] Ln[Odds(Male)] 0.6 0.6 = ‐21.48 = ‐27.57 + 0.1577*Weight + 0.2013*Weight 0.4 0.4 0.2 0.2 0 0 95 115 135 155 175 195 95 115 135 155 175 195 Weight (pounds) Weight (pounds) Pulse data Schield Pulse data Schield V0B V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 11 Schield MLE vs. OLS1-Based Logistic Excel 2013 12 2c: Logistic P(male|Weight) 3a: Logistic P(male|Ht, Wt=Ave) MLE vs. OLS1 MLE . . V0A V0A Logistic P(male|Ht, Wt=Ave): MLE Logistic P(male|Weight): MLE vs. OLS1 1 1 0.8 0.8 OLS1B (dash): MLE1B (solid): MLE1C (solid): Ln[Odds(Male)] Ln[Odds(Male)] 0.6 0.6 = ‐27.57 = ‐21.48 Ln[Odds(Male)] + 0.2013*Weight = ‐41.40 + 0.1577*Weight 0.4 0.4 + 0.3817*Height + 0.1146*AveWt 0.2 0.2 0 0 61 63 65 67 69 71 73 75 95 115 135 155 175 195 Height (inches) Weight (pounds) Pulse data Schield Pulse data Schield 2016-Schield-Logistic-MLE-OLS1-Excel2013-Slides.pdf 2

Logistic Regression: MLE vs. OLS1 in Excel2013 29 Aug 2016 V0B V0B V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 13 Schield MLE vs. OLS1-Based Logistic Excel 2013 14 3b: Logistic P(male|Ht, Wt=Ave) 3c: Logistic P(male|Ht, Wt=Ave) OLS1 MLE vs. OLS1 . . V0A V0A Logistic P(male|Ht, Wt=Ave): MLE vs. OLS1 Logistic P(male|Ht, Wt=Ave): OLS1 1 1 OLS1C (dash): OLS1B (dash): Ln[Odds(Male)] Ln[Odds(Male)] 0.8 0.8 = ‐66.374 = ‐66.374 MLE1B (solid): + 0.7586*Height + 0.7586*Height 0.6 0.6 Ln[Odds(Male)] + 0.1095*AveWt + 0.1095*AveWt = ‐41.40 0.4 0.4 + 0.3817*Height + 0.1146*AveWt 0.2 0.2 0 0 61 63 65 67 69 71 73 75 61 63 65 67 69 71 73 75 Height (inches) Height (inches) Pulse data Schield Pulse data Schield V0B V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 15 Schield MLE vs. OLS1-Based Logistic Excel 2013 16 Conclusions References 1. Can use OLS with a logistic function to Schield, Milo (2015). Copy at www.statlit.org/pdf/ illustrate logistic regression since logistic- 2015-Schield-Logistic-MLE1A-Excel2013-Slides.pdf OLS is better than linear OLS. 2015-Schield-Logistic-MLE1B-Excel2013-Slides.pdf 2015-Schield-Logistic-MLE1C-Excel2013-Slides.pdf 2. OLS with ‘nudge’, a single predictor and logistic (OLS1A) is quite close to the MLE. Schield, Milo (2014a). Copy at www.statlit.org/pdf/ 2014-Schield-Logistic-OLS1A-Excel2013-Demo.pdf 3. OLS with ‘nudge’ and logistic worsens as 2014-Schield-Logistic-OLS1B-Excel2013-Demo.pdf number of predictors increases (OLS1C) . 2014-Schield-Logistic-OLS1C-Excel2013-Demo.pdf 4. If higher accuracy is needed, use MLE Schield, Milo (2014b). Copy at www.statlit.org/pdf/ (Excel or other) or consult a statistician. 2014-Schield-Logistic-MLE-OLS3-Excel2013-Demo.pdf 2016-Schield-Logistic-MLE-OLS1-Excel2013-Slides.pdf 3

V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 1 Logistic Regression: MLE vs. OLS1 in Excel 2013 by Milo Schield Member: International Statistical Institute US Rep: International Statistical Literacy Project Director, W. M. Keck Statistical Literacy Project Slides and data at: www.StatLit.org/ pdf/2016-Schield-Logistic-MLE-OLS1-Excel2013-slides.pdf pdf/2016-Schield-Logistic-MLE-OLS1-Excel2013-demo.pdf Excel/2016-Schield-Logistic-MLE-OLS1-Excel2013.xlsx

V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 2 Background & Goals Modeling a binary outcome requires a logistic model. Doing logistic regression properly requires MLE. Doing MLE in Excel is not easy. See Schield (2015) Schield created two logistic OLS models: OLS1+OLS3 OLS1: Model Ln[Odds(Pnudge)]. See Schield (2014a). OLS2: Model Ln[Odds(Pgroup)]. See Schield (2016c) OLS3: Use OLS to estimate logistic parameters. See Schield (2014b) These slides compare MLE with OLS1 logistic regression

V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 3 Model Gender by Height (OLS) Must use a logistic function This linear trend-line goes outside the valid range

V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 4 Model using a Logistic Function Range of Odds(p): Zero to infinity Range of Ln[Odds(p)]: Minus infinity to infinity Logistic model: Ln[Odds(p)] = a + b*X V0A Logistic P(male|Height) 1 0.8 0.6 0.4 0.2 0 61 63 65 67 69 71 73 75 Height (inches) Pulse data Schield

V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 5 |Ln[Odds(p)]| = infinity if p = 0 or 1 Pnudge: =If(p=0, 0.001, 0.999) Source: Pulse dataset OLS1 Model : Ln[Odds(Pnudge)] = Constant + b1*X1 + b2*X2 using Ordinary Least Squares.

V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 6 1a: Logistic P(male|Height) MLE MLE Logistic Excel 2013 Gender by Height 1 Men Probability (Male) MLE Logistic regression 0.75 P(Y|X) = 1 / {1 + Exp[-(a + bX)]} a = -53.32 ; b = 0.7905. 0.5 X = Xo if P(Y) = 0.5 Xo = -a/b = 67.4529 Slope(X=Xo) = b/4 = 0.1976 0.25 Women 0 60 62 64 66 68 70 72 74 76 Height (inches) Pulse.xls Schield

V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 7 1b: Logistic P(male|Height) OLS1 . V0B Logistic P(male|Height): OLS1 1 OLS1A (dash): 0.8 Ln[Odds(Male1)] = -88.80 0.6 + 1.316*Height 0.4 0.2 0 61 63 65 67 69 71 73 75 Height (inches) Pulse data Schield

V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 8 1c: Logistic P(male|Height) MLE vs. OLS1 V0B Logistic P(male|Height): MLE vs. OLS1 1 OLS1A (dash): 0.8 Ln[Odds(Male1)] MLE1A (solid): = -88.80 Ln[Odds(Male)] 0.6 + 1.316*Height = -55.32 + 0.7905*Height 0.4 0.2 0 61 63 65 67 69 71 73 75 Height (inches) Pulse data Schield

V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 9 2a: Logistic P(male|Weight) MLE . V0A Logistic P(male|Weight): MLE 1 0.8 MLE1B (solid): Ln[Odds(Male)] 0.6 = -21.48 + 0.1577*Weight 0.4 0.2 0 95 115 135 155 175 195 Weight (pounds) Pulse data Schield

V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 10 2b: Logistic P(male|Weight) OLS1 . V0A Logistic P(male|Weight): OLS1 1 0.8 OLS1B (dash): Ln[Odds(Male)] 0.6 = -27.57 + 0.2013*Weight 0.4 0.2 0 95 115 135 155 175 195 Weight (pounds) Pulse data Schield

Logistic Regression: MLE vs. OLS1 in Excel2013 29 Aug 2016 V0B V0B - PDF document

Logistic Regression: MLE vs. OLS1 in Excel2013 29 Aug 2016 V0B V0B V0B Schield MLE vs. OLS1-Based Logistic Excel 2013 1 Schield MLE vs. OLS1-Based Logistic Excel 2013 2 Logistic Regression: Background & Goals MLE vs. OLS1 in Excel 2013

2015 Schield Logistic MLE1C Excel2013 8/18/2016 V0D V0D V0D 2015 Schield Logistic MLE 1C

Excel2013: Model Logistic MLE 1Y1X Sept 2015 V1A V1A V1A Excel2013 Model Logistic MLE 1Y1X

2015 Schield Logistic MLE1A Excel2013 10/29/2015 V0D V0D V0D 2015 Schield Logistic MLE 1A

Logistic Regression: MLE vs. OLS3 in Excel2013 25 Aug 2016 V0H V0H V0H Schield MLE vs.

Logistic Regression James H. Steiger Department of Psychology and Human Development Vanderbilt

Logistic Regression using OLS1D in Excel 2013 XL4D: V0H XL4D: V0H XL4D: V0H 2015 Schield

XL4B: Logistic Regression using OLS1B in Excel 2013 25 Feb 2018 V0C-2x XL4B: V0C-2x XL4B: V0C-2x

Workshop 10.5a: Logistic regression Murray Logan August 23, 2016 Table of contents 1 Logistic

Workshop 10.5a: Logistic regression Murray Logan 05 Sep 2016 Section 1 Logistic regression

Discriminant Analysis using Logistic Regression OLS1D XL4E: V0D XL4E : OLS1D V0D XL4E : OLS1D V0D

Todays lecture Logistic regression How can we use logistic regression for reranking? Shay

XL4A: Logistic Model using OLS1A in Excel 2013 1 Mar 2017 V0E 2x XL4A: V0E2x XL4A: V0E2x 2015

w o o o o o o o o x o o o o o x o that represents how aligned the o x x x x

w o o o o o o o x o o o x o o that represents how aligned the o x x x x x x

w o o o o o o o x o o o x o o o that represents how aligned the x x x x x x

Regression 3: Logistic Regression Marco Baroni Practical Statistics in R Outline Logistic

LEARNING Outline Math Behind Logistic Regression Visualizing Logistic Regression Loss

Machine Learning - MT 2016 8. Classification: Logistic Regression Varun Kanade University of

Logistic Regression Lecture 6 Logistic Regression Classification Model CS 335

From Logistic Regression to Neural Networks CMSC 470 Marine Carpuat Logistic Regression What

Multiple Regression and Logistic Regression I Dajiang Liu @PHS 525 Apr-14-2016 Multiple

1 Relation Between Multinomial Logistic Regression Nave Bayes and Logistic Regression

Multiple and Logistic Regression IV Dajiang Liu @PHS 525 Apr-21 st -2016 Review of Last Two

Logistic regression and Poisson regression Rasmus Waagepetersen Department of Mathematics