CS147 2015-06-15 CS 147: Computer Systems Performance Analysis Multiple and Categorical Regression CS 147: Computer Systems Performance Analysis Multiple and Categorical Regression 1 / 36
Overview CS147 Overview 2015-06-15 Multiple Linear Regression Basic Formulas Example Quality of the Example Overview Categorical Models Multiple Linear Regression Basic Formulas Example Quality of the Example Categorical Models 2 / 36
Multiple Linear Regression Multiple Linear Regression CS147 Multiple Linear Regression 2015-06-15 Multiple Linear Regression ◮ Develops models with more than one predictor variable ◮ But each predictor variable has linear relationship to response variable ◮ Conceptually, plotting a regression line in n -dimensional Multiple Linear Regression space, instead of 2-dimensional ◮ Develops models with more than one predictor variable ◮ But each predictor variable has linear relationship to response variable ◮ Conceptually, plotting a regression line in n -dimensional space, instead of 2-dimensional 3 / 36
Multiple Linear Regression Basic Formulas Basic Multiple Linear Regression Formula CS147 Basic Multiple Linear Regression Formula 2015-06-15 Multiple Linear Regression Response y is a function of k predictor variables x 1 , x 2 , . . . , x k Basic Formulas y = b 0 + b 1 x 1 + b 2 x 2 + · · · + b k x k + e Basic Multiple Linear Regression Formula Response y is a function of k predictor variables x 1 , x 2 , . . . , x k y = b 0 + b 1 x 1 + b 2 x 2 + · · · + b k x k + e 4 / 36
Multiple Linear Regression Basic Formulas A Multiple Linear Regression Model CS147 A Multiple Linear Regression Model 2015-06-15 Given sample of n observations Multiple Linear Regression { ( x 11 , x 21 , . . . , x k 1 , y 1 ) , . . . , ( x 1 n , x 2 n , . . . , x kn , y n ) } model consists of n equations (note possible + vs. − typo in Basic Formulas book): y 1 = b 0 + b 1 x 11 + b 2 x 21 + · · · + b k x k 1 + e 1 A Multiple Linear Regression Model y 2 = b 0 + b 1 x 12 + b 2 x 22 + · · · + b k x k 2 + e 2 . . . Given sample of n observations y n = b 0 + b 1 x 1 n + b 2 x 2 n + · · · + b k x kn + e n { ( x 11 , x 21 , . . . , x k 1 , y 1 ) , . . . , ( x 1 n , x 2 n , . . . , x kn , y n ) } model consists of n equations (note possible + vs. − typo in book): y 1 = b 0 + b 1 x 11 + b 2 x 21 + · · · + b k x k 1 + e 1 y 2 = b 0 + b 1 x 12 + b 2 x 22 + · · · + b k x k 2 + e 2 . . . y n = b 0 + b 1 x 1 n + b 2 x 2 n + · · · + b k x kn + e n 5 / 36
Multiple Linear Regression Basic Formulas Looks Like It’s Matrix Arithmetic Time CS147 Looks Like It’s Matrix Arithmetic Time 2015-06-15 Multiple Linear Regression y = Xb + e y 1 1 x 11 x 21 . . . x k 1 b 0 e 0 y 2 1 x 12 x 22 . . . x k 2 b 1 e 1 Basic Formulas = + . . . . . . . . . . . . . . . . . . . . . . . . y n 1 x 1 n x 2 n . . . x kn b k e n Looks Like It’s Matrix Arithmetic Time Note that: ◮ y and e have n elements ◮ b has k + 1 ◮ x is k by n y = Xb + e y 1 1 x 11 x 21 . . . x k 1 b 0 e 0 y 2 1 x 12 x 22 . . . x k 2 b 1 e 1 = + . . . . . . . . . . . . . . . . . . . . . . . . y n 1 x 1 n x 2 n . . . x kn b k e n Note that: ◮ y and e have n elements ◮ b has k + 1 ◮ x is k by n 6 / 36
Multiple Linear Regression Basic Formulas Analysis of Multiple Linear Regression CS147 Analysis of Multiple Linear Regression 2015-06-15 Multiple Linear Regression ◮ Listed in box 15.1 of Jain ◮ Not terribly important (for our purposes) how they were Basic Formulas derived ◮ This isn’t a class on statistics ◮ But you need to know how to use them Analysis of Multiple Linear Regression ◮ Mostly matrix analogs to simple linear regression results ◮ Listed in box 15.1 of Jain ◮ Not terribly important (for our purposes) how they were derived ◮ This isn’t a class on statistics ◮ But you need to know how to use them ◮ Mostly matrix analogs to simple linear regression results 7 / 36
Multiple Linear Regression Example Example of Multiple Linear Regression CS147 Example of Multiple Linear Regression 2015-06-15 Multiple Linear Regression ◮ IMDB keeps numerical popularity ratings of movies ◮ Postulate popularity of Academy Award-winning films is based on two factors: Example ◮ Year made ◮ Running time ◮ Produce a regression Example of Multiple Linear Regression rating = b 0 + b 1 ( year ) + b 2 ( length ) ◮ IMDB keeps numerical popularity ratings of movies ◮ Postulate popularity of Academy Award-winning films is based on two factors: ◮ Year made ◮ Running time ◮ Produce a regression rating = b 0 + b 1 ( year ) + b 2 ( length ) 8 / 36
Multiple Linear Regression Example Some Sample Data CS147 Some Sample Data 2015-06-15 Multiple Linear Regression Title Year Length Rating Silence of the Lambs 1991 118 8.1 Terms of Endearment 1983 132 6.8 Example Rocky 1976 119 7.0 Oliver! 1968 153 7.4 Marty 1955 91 7.7 Gentleman’s Agreement 1947 118 7.5 Some Sample Data Mutiny on the Bounty 1935 132 7.6 It Happened One Night 1934 105 8.0 Title Year Length Rating Silence of the Lambs 1991 118 8.1 Terms of Endearment 1983 132 6.8 Rocky 1976 119 7.0 Oliver! 1968 153 7.4 Marty 1955 91 7.7 Gentleman’s Agreement 1947 118 7.5 Mutiny on the Bounty 1935 132 7.6 It Happened One Night 1934 105 8.0 9 / 36
Multiple Linear Regression Example Now for Some Tedious Matrix Arithmetic CS147 Now for Some Tedious Matrix Arithmetic 2015-06-15 Multiple Linear Regression ◮ We need to calculate X , X T , X T X , ( X T X ) − 1 , and X T y ◮ Because b = ( X T X ) − 1 ( X T y ) Example ◮ We will see that b = ( 18 . 5430 , − 0 . 0051 , − 0 . 0086 ) ◮ Meaning the regression predicts: Now for Some Tedious Matrix Arithmetic rating = 18 . 5430 − 0 . 0051 ( year ) − 0 . 0086 ( length ) ◮ We need to calculate X , X T , X T X , ( X T X ) − 1 , and X T y ◮ Because b = ( X T X ) − 1 ( X T y ) ◮ We will see that b = ( 18 . 5430 , − 0 . 0051 , − 0 . 0086 ) ◮ Meaning the regression predicts: rating = 18 . 5430 − 0 . 0051 ( year ) − 0 . 0086 ( length ) 10 / 36
Multiple Linear Regression Example X Matrix for Example CS147 X Matrix for Example 2015-06-15 Multiple Linear Regression 1 1991 118 1 1983 132 Example 1 1976 119 1 1968 153 X = 1 1955 91 1 1947 118 X Matrix for Example 1 1935 132 1 1934 105 1 1991 118 1 1983 132 1 1976 119 1 1968 153 X = 1 1955 91 1 1947 118 1 1935 132 1 1934 105 11 / 36
Multiple Linear Regression Example Transpose to Get X T CS147 Transpose to Get X T 2015-06-15 Multiple Linear Regression Example 1 1 1 1 1 1 1 1 X T = 1991 1983 1976 1968 1955 1947 1935 1934 118 132 119 153 91 118 132 105 Transpose to Get X T 1 1 1 1 1 1 1 1 X T = 1991 1983 1976 1968 1955 1947 1935 1934 118 132 119 153 91 118 132 105 12 / 36
Multiple Linear Regression Example Multiply To Get X T X CS147 Multiply To Get X T X 2015-06-15 Multiple Linear Regression Example 8 15689 968 X T X = 15689 30771385 1899083 968 1899083 119572 Multiply To Get X T X 8 15689 968 X T X = 15689 30771385 1899083 968 1899083 119572 13 / 36
Multiple Linear Regression Example Invert to Get C = ( X T X ) − 1 CS147 Invert to Get C = ( X T X ) − 1 2015-06-15 Multiple Linear Regression Example 1207.7585 -0.6240 0.1328 C = ( X T X ) − 1 = -0.6240 0.0003 -0.0001 0.1328 -0.0001 0.0004 Invert to Get C = ( X T X ) − 1 1207.7585 -0.6240 0.1328 C = ( X T X ) − 1 = -0.6240 0.0003 -0.0001 0.1328 -0.0001 0.0004 14 / 36
Multiple Linear Regression Example Multiply to Get X T y CS147 Multiply to Get X T y 2015-06-15 Multiple Linear Regression Example 60.1 X T y = 117840.7 7247.5 Multiply to Get X T y 60.1 X T y = 117840.7 7247.5 15 / 36
Multiple Linear Regression Example Multiply ( X T X ) − 1 ( X T y ) to Get b CS147 Multiply ( X T X ) − 1 ( X T y ) to Get b 2015-06-15 Multiple Linear Regression Example 18.5430 b = -0.0051 -0.0086 Multiply ( X T X ) − 1 ( X T y ) to Get b 18.5430 b = -0.0051 -0.0086 16 / 36
Recommend
More recommend
