linear regression estimating parameters
play

Linear Regression - Estimating Parameters Bernd Schr oder logo1 - PowerPoint PPT Presentation

Aug 18, 2022 •133 likes •1.16k views

Coefficients Examples Error Sum of Squares Coefficient of Determination Linear Regression - Estimating Parameters Bernd Schr oder logo1 Bernd Schr oder Louisiana Tech University, College of Engineering and Science Linear Regression -

  1. Coefficients Examples Error Sum of Squares Coefficient of Determination n n ∂ f = ∂ [ y i − ( b 0 + b 1 x i )] ! [ y i − ( b 0 + b 1 x i )] 2 ∑ ∑ = − 2 = 0 ∂ b 0 ∂ b 0 i = 1 i = 1 n n ∂ f = ∂ x i [ y i − ( b 0 + b 1 x i )] ! [ y i − ( b 0 + b 1 x i )] 2 ∑ ∑ = = 0 − 2 ∂ b 1 ∂ b 1 i = 1 i = 1 n n ∑ ∑ y i − nb 0 − b 1 x i = 0 i = 1 i = 1 n n n x 2 ∑ ∑ ∑ = x i y i − b 0 x i − b 1 0 i i = 1 i = 1 i = 1 n n ∑ ∑ nb 0 + b 1 x i = y i i = 1 i = 1 n n n x 2 ∑ ∑ ∑ x i + b 1 = b 0 x i y i i i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  2. Coefficients Examples Error Sum of Squares Coefficient of Determination logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  3. Coefficients Examples Error Sum of Squares Coefficient of Determination n n ∑ ∑ nb 0 + b 1 = x i y i i = 1 i = 1 n n n ∑ ∑ x 2 ∑ b 0 x i + b 1 = x i y i i i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  4. Coefficients Examples Error Sum of Squares Coefficient of Determination n n ∑ ∑ nb 0 + b 1 = x i y i i = 1 i = 1 n n n ∑ ∑ x 2 ∑ b 0 x i + b 1 = x i y i i i = 1 i = 1 i = 1 � � n n 1 ∑ ∑ = b 0 y i − b 1 x i n i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  5. Coefficients Examples Error Sum of Squares Coefficient of Determination n n ∑ ∑ nb 0 + b 1 = x i y i i = 1 i = 1 n n n ∑ ∑ x 2 ∑ b 0 x i + b 1 = x i y i i i = 1 i = 1 i = 1 � � n n 1 ∑ ∑ = b 0 y i − b 1 x i n i = 1 i = 1 � � n n n n n 1 x 2 ∑ ∑ ∑ ∑ ∑ x i + b 1 = y i − b 1 x i x i y i i n i = 1 i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  6. Coefficients Examples Error Sum of Squares Coefficient of Determination n n ∑ ∑ nb 0 + b 1 = x i y i i = 1 i = 1 n n n ∑ ∑ x 2 ∑ b 0 x i + b 1 = x i y i i i = 1 i = 1 i = 1 � � n n 1 ∑ ∑ = b 0 y i − b 1 x i n i = 1 i = 1 � � n n n n n 1 x 2 ∑ ∑ ∑ ∑ ∑ x i + b 1 = y i − b 1 x i x i y i i n i = 1 i = 1 i = 1 i = 1 i = 1  � 2  � n n n n n i − 1 x i y i − 1 x 2 ∑ ∑ ∑ ∑ ∑ b 1 x i = y i x i   n n i = 1 i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  7. Coefficients Examples Error Sum of Squares Coefficient of Determination logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  8. Coefficients Examples Error Sum of Squares Coefficient of Determination ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n i = 1 x i = b 1 � i = 1 x i ) 2 � ∑ n i − 1 n ( ∑ n i = 1 x 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  9. Coefficients Examples Error Sum of Squares Coefficient of Determination ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n i = 1 x i = b 1 � i = 1 x i ) 2 � ∑ n i − 1 n ( ∑ n i = 1 x 2 ∑ n i = 1 x i y i − ∑ n i = 1 y i x = � ∑ n i = 1 x 2 i − ∑ n � i = 1 x i x logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  10. Coefficients Examples Error Sum of Squares Coefficient of Determination ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n i = 1 x i = b 1 � i = 1 x i ) 2 � ∑ n i − 1 n ( ∑ n i = 1 x 2 ∑ n i = 1 x i y i − ∑ n i = 1 y i x = � ∑ n i = 1 x 2 i − ∑ n � i = 1 x i x ∑ n i = 1 y i ( x i − x ) − ∑ n i = 1 y ( x i − x ) = ∑ n i = 1 x i ( x i − x ) − ∑ n i = 1 x ( x i − x ) logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  11. Coefficients Examples Error Sum of Squares Coefficient of Determination ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n i = 1 x i = b 1 � i = 1 x i ) 2 � ∑ n i − 1 n ( ∑ n i = 1 x 2 ∑ n i = 1 x i y i − ∑ n i = 1 y i x = � ∑ n i = 1 x 2 i − ∑ n � i = 1 x i x ∑ n i = 1 y i ( x i − x ) − ∑ n i = 1 y ( x i − x ) = ∑ n i = 1 x i ( x i − x ) − ∑ n i = 1 x ( x i − x ) ∑ n i = 1 ( x i − x )( y i − y ) = i = 1 ( x i − x ) 2 ∑ n logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  12. Coefficients Examples Error Sum of Squares Coefficient of Determination ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n i = 1 x i = b 1 � i = 1 x i ) 2 � ∑ n i − 1 n ( ∑ n i = 1 x 2 ∑ n i = 1 x i y i − ∑ n i = 1 y i x = � ∑ n i = 1 x 2 i − ∑ n � i = 1 x i x ∑ n i = 1 y i ( x i − x ) − ∑ n i = 1 y ( x i − x ) = ∑ n i = 1 x i ( x i − x ) − ∑ n i = 1 x ( x i − x ) ∑ n i = 1 ( x i − x )( y i − y ) = i = 1 ( x i − x ) 2 ∑ n = b 0 y − b 1 x logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  13. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  14. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas 1. b 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  15. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas 1. b 1 = ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n i = 1 x i � i = 1 x i ) 2 � ∑ n i − 1 n ( ∑ n i = 1 x 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  16. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas 1. b 1 = ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n = ∑ n i = 1 x i i = 1 ( x i − x )( y i − y ) � i = 1 x i ) 2 � i = 1 ( x i − x ) 2 ∑ n ∑ n i − 1 n ( ∑ n i = 1 x 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  17. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas 1. b 1 = ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n = ∑ n i = 1 x i i = 1 ( x i − x )( y i − y ) = S xy � i = 1 x i ) 2 � i = 1 ( x i − x ) 2 ∑ n S xx ∑ n i − 1 n ( ∑ n i = 1 x 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  18. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas 1. b 1 = ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n = ∑ n i = 1 x i i = 1 ( x i − x )( y i − y ) = S xy � i = 1 x i ) 2 � i = 1 ( x i − x ) 2 ∑ n S xx ∑ n i − 1 n ( ∑ n i = 1 x 2 2. b 0 = y − b 1 x logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  19. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas 1. b 1 = ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n = ∑ n i = 1 x i i = 1 ( x i − x )( y i − y ) = S xy � i = 1 x i ) 2 � i = 1 ( x i − x ) 2 ∑ n S xx ∑ n i − 1 n ( ∑ n i = 1 x 2 2. b 0 = y − b 1 x y i : = ˆ β 0 + ˆ β 1 x i . 3. Fitted (or predicted) values: ˆ logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  20. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas 1. b 1 = ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n = ∑ n i = 1 x i i = 1 ( x i − x )( y i − y ) = S xy � i = 1 x i ) 2 � i = 1 ( x i − x ) 2 ∑ n S xx ∑ n i − 1 n ( ∑ n i = 1 x 2 2. b 0 = y − b 1 x y i : = ˆ β 0 + ˆ β 1 x i . 3. Fitted (or predicted) values: ˆ 4. Residuals: y i − ˆ y i . logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  21. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  22. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  23. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  24. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  25. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  26. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  27. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  28. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  29. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  30. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  31. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  32. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  33. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  34. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  35. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  36. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? 1. Error sum of squares (or residual sum of squares). logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  37. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? 1. Error sum of squares (or residual sum of squares). n n �� 2 y i ) 2 = � � β 0 + ˆ ˆ ∑ ∑ SSE = ( y i − ˆ β 1 x i y i − i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  38. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? 1. Error sum of squares (or residual sum of squares). n n �� 2 y i ) 2 = � � β 0 + ˆ ˆ ∑ ∑ SSE = ( y i − ˆ β 1 x i y i − i = 1 i = 1 Estimate of σ 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  39. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? 1. Error sum of squares (or residual sum of squares). n n �� 2 y i ) 2 = � � β 0 + ˆ ˆ ∑ ∑ SSE = ( y i − ˆ β 1 x i y i − i = 1 i = 1 y i ) 2 n − 2 = ∑ n i = 1 ( y i − ˆ σ 2 = s 2 = SSE Estimate of σ 2 : ˆ . n − 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  40. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? 1. Error sum of squares (or residual sum of squares). n n �� 2 y i ) 2 = � � β 0 + ˆ ˆ ∑ ∑ SSE = ( y i − ˆ β 1 x i y i − i = 1 i = 1 y i ) 2 n − 2 = ∑ n i = 1 ( y i − ˆ σ 2 = s 2 = SSE Estimate of σ 2 : ˆ . n − 2 2. We divide by n − 2 because we have only n − 2 degrees of freedom after determining two parameters ˆ β 0 and ˆ β 1 . logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  41. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? 1. Error sum of squares (or residual sum of squares). n n �� 2 y i ) 2 = � � β 0 + ˆ ˆ ∑ ∑ SSE = ( y i − ˆ β 1 x i y i − i = 1 i = 1 y i ) 2 n − 2 = ∑ n i = 1 ( y i − ˆ σ 2 = s 2 = SSE Estimate of σ 2 : ˆ . n − 2 2. We divide by n − 2 because we have only n − 2 degrees of freedom after determining two parameters ˆ β 0 and ˆ β 1 . (Previously, using an estimate of µ resulted in the loss of one degree of freedom.) logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  42. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? 1. Error sum of squares (or residual sum of squares). n n �� 2 y i ) 2 = � � β 0 + ˆ ˆ ∑ ∑ SSE = ( y i − ˆ β 1 x i y i − i = 1 i = 1 y i ) 2 n − 2 = ∑ n i = 1 ( y i − ˆ σ 2 = s 2 = SSE Estimate of σ 2 : ˆ . n − 2 2. We divide by n − 2 because we have only n − 2 degrees of freedom after determining two parameters ˆ β 0 and ˆ β 1 . (Previously, using an estimate of µ resulted in the loss of one degree of freedom.) 3. It can be shown (we cannot present the proof) that S 2 is an unbiased estimator for σ 2 . logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  43. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  44. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE SSE logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  45. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n y i ) 2 ∑ = ( y i − ˆ SSE i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  46. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n y i ) 2 ∑ = ( y i − ˆ SSE i = 1 n �� 2 � � β 0 + ˆ ˆ ∑ = y i − β 1 x i i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  47. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n y i ) 2 ∑ = ( y i − ˆ SSE i = 1 n �� 2 � � β 0 + ˆ ˆ ∑ = y i − β 1 x i i = 1 n i − 2 ˆ β 0 y i − 2 ˆ β 1 x i y i + ˆ 0 + 2 ˆ β 0 ˆ β 1 x i + ˆ y 2 β 2 β 2 1 x 2 ∑ = i i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  48. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n y i ) 2 ∑ = ( y i − ˆ SSE i = 1 n �� 2 � � β 0 + ˆ ˆ ∑ = y i − β 1 x i i = 1 n i − 2 ˆ β 0 y i − 2 ˆ β 1 x i y i + ˆ 0 + 2 ˆ β 0 ˆ β 1 x i + ˆ y 2 β 2 β 2 1 x 2 ∑ = i i = 1 n n n n n i − 2 ˆ y i − 2 ˆ x i y i + n ˆ 0 + 2 ˆ β 0 ˆ x i + ˆ ∑ y 2 ∑ ∑ β 2 ∑ β 2 ∑ x 2 = β 0 β 1 β 1 1 i i = 1 i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  49. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n n n n n i − 2 ˆ y i − 2 ˆ x i y i + n ˆ 0 + 2 ˆ β 0 ˆ x i + ˆ y 2 β 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE 1 i i = 1 i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  50. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n n n n n i − 2 ˆ y i − 2 ˆ x i y i + n ˆ 0 + 2 ˆ β 0 ˆ x i + ˆ y 2 β 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE 1 i i = 1 i = 1 i = 1 i = 1 i = 1 � � n n n n n y 2 i − 2 ˆ y i − 2 ˆ x i y i + ˆ y i − ˆ ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 0 β 1 x i i = 1 i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  51. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n n n n n i − 2 ˆ y i − 2 ˆ x i y i + n ˆ 0 + 2 ˆ β 0 ˆ x i + ˆ y 2 β 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE 1 i i = 1 i = 1 i = 1 i = 1 i = 1 � � n n n n n y 2 i − 2 ˆ y i − 2 ˆ x i y i + ˆ y i − ˆ ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 0 β 1 x i i = 1 i = 1 i = 1 i = 1 i = 1 n n + 2 ˆ β 0 ˆ x i + ˆ β 2 x 2 ∑ ∑ β 1 1 i i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  52. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n n n n n i − 2 ˆ y i − 2 ˆ x i y i + n ˆ 0 + 2 ˆ β 0 ˆ x i + ˆ y 2 β 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE 1 i i = 1 i = 1 i = 1 i = 1 i = 1 � � n n n n n y 2 i − 2 ˆ y i − 2 ˆ x i y i + ˆ y i − ˆ ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 0 β 1 x i i = 1 i = 1 i = 1 i = 1 i = 1 n n + 2 ˆ β 0 ˆ x i + ˆ β 2 x 2 ∑ ∑ β 1 1 i i = 1 i = 1 n n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ x i + ˆ y 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 1 i i = 1 i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  53. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ x i + ˆ y 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE 1 i i = 1 i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  54. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ x i + ˆ y 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE 1 i i = 1 i = 1 i = 1 i = 1 i = 1 n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ ∑ y 2 ∑ ∑ ∑ = β 0 β 1 β 1 x i i = 1 i = 1 i = 1 i = 1  � 2  � n n i ± 1 + ˆ β 2 x 2 ∑ ∑ x i 1   n i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  55. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ x i + ˆ y 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE 1 i i = 1 i = 1 i = 1 i = 1 i = 1 n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ ∑ y 2 ∑ ∑ ∑ = β 0 β 1 β 1 x i i = 1 i = 1 i = 1 i = 1  � 2  � n n i ± 1 + ˆ β 2 x 2 ∑ ∑ x i 1   n i = 1 i = 1 n n n n y 2 i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 x i i = 1 i = 1 i = 1 i = 1  � 2  � 2 � � n n n i − 1 1 + ˆ β 2 x 2  + ˆ β 2 ∑ ∑ ∑ x i x i 1  1 n n i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  56. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ y 2 ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE x i i = 1 i = 1 i = 1 i = 1  � 2  � 2 � � n n n i − 1 1 + ˆ β 2 x 2  + ˆ β 2 ∑ ∑ ∑ x i x i 1  1 n n i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  57. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ y 2 ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE x i i = 1 i = 1 i = 1 i = 1  � 2  � 2 � � n n n i − 1 1 + ˆ β 2 x 2  + ˆ β 2 ∑ ∑ ∑ x i x i 1  1 n n i = 1 i = 1 i = 1 n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ ∑ y 2 ∑ ∑ ∑ = β 0 β 1 β 1 x i i = 1 i = 1 i = 1 i = 1 � 2 � � � n n n n x i y i − 1 1 + ˆ + ˆ β 2 ∑ ∑ ∑ ∑ β 1 y i x i x i 1 n n i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  58. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ y 2 ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE x i i = 1 i = 1 i = 1 i = 1 � 2 � � � n n n n x i y i − 1 1 + ˆ + ˆ β 2 ∑ ∑ ∑ ∑ β 1 y i x i x i 1 n n i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  59. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ y 2 ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE x i i = 1 i = 1 i = 1 i = 1 � 2 � � � n n n n x i y i − 1 1 + ˆ + ˆ β 2 ∑ ∑ ∑ ∑ β 1 y i x i x i 1 n n i = 1 i = 1 i = 1 i = 1 n n n n i − ˆ y i − ˆ x i y i + ˆ β 0 ˆ y 2 ∑ ∑ ∑ ∑ = β 0 β 1 β 1 x i i = 1 i = 1 i = 1 i = 1 � � n n n 1 1 − ˆ y i − ˆ ∑ ∑ ∑ β 1 β 1 x i x i n n i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  60. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ y 2 ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE x i i = 1 i = 1 i = 1 i = 1 � 2 � � � n n n n x i y i − 1 1 + ˆ + ˆ β 2 ∑ ∑ ∑ ∑ β 1 y i x i x i 1 n n i = 1 i = 1 i = 1 i = 1 n n n n i − ˆ y i − ˆ x i y i + ˆ β 0 ˆ y 2 ∑ ∑ ∑ ∑ = β 0 β 1 β 1 x i i = 1 i = 1 i = 1 i = 1 � � n n n 1 1 − ˆ y i − ˆ ∑ ∑ ∑ β 1 β 1 x i x i n n i = 1 i = 1 i = 1 n n n i − ˆ y i − ˆ ∑ y 2 ∑ ∑ = β 0 β 1 x i y i i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  61. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  62. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  63. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1. SSE ≈ 1 . 1194 and s 2 ≈ 0 . 0622 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  64. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  65. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  66. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2. SSE ≈ 79 . 7958 and s 2 ≈ 4 . 4331 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  67. Coefficients Examples Error Sum of Squares Coefficient of Determination Coefficient of Determination logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  68. Coefficients Examples Error Sum of Squares Coefficient of Determination Coefficient of Determination n ( y i − y ) 2 be the total sum of squares. ∑ 1. Let SST : = i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  69. Coefficients Examples Error Sum of Squares Coefficient of Determination Coefficient of Determination n ( y i − y ) 2 be the total sum of squares. The ∑ 1. Let SST : = i = 1 coefficient of determination is given by r 2 = 1 − SSE SST . logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  70. Coefficients Examples Error Sum of Squares Coefficient of Determination Coefficient of Determination n ( y i − y ) 2 be the total sum of squares. The ∑ 1. Let SST : = i = 1 coefficient of determination is given by r 2 = 1 − SSE SST . 2. SSE measures how much variation is not attributed to the linear relationship. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  71. Coefficients Examples Error Sum of Squares Coefficient of Determination Coefficient of Determination n ( y i − y ) 2 be the total sum of squares. The ∑ 1. Let SST : = i = 1 coefficient of determination is given by r 2 = 1 − SSE SST . 2. SSE measures how much variation is not attributed to the linear relationship. 3. SST measures the total variation in y . logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  72. Coefficients Examples Error Sum of Squares Coefficient of Determination Coefficient of Determination n ( y i − y ) 2 be the total sum of squares. The ∑ 1. Let SST : = i = 1 coefficient of determination is given by r 2 = 1 − SSE SST . 2. SSE measures how much variation is not attributed to the linear relationship. 3. SST measures the total variation in y . 4. The quotient gives the proportion of the variation not attributed to the linear relationship. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  73. Coefficients Examples Error Sum of Squares Coefficient of Determination Coefficient of Determination n ( y i − y ) 2 be the total sum of squares. The ∑ 1. Let SST : = i = 1 coefficient of determination is given by r 2 = 1 − SSE SST . 2. SSE measures how much variation is not attributed to the linear relationship. 3. SST measures the total variation in y . 4. The quotient gives the proportion of the variation not attributed to the linear relationship. 5. r 2 gives the proportion of the variation that is attributed to the linear relationship. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  74. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  75. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  76. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1. r 2 ≈ . 9516 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  77. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

Recommend

Estimating the Error at Given Test Estimating the Error at Given Test Input Points for Linear

Estimating the Error at Given Test Estimating the Error at Given Test Input Points for Linear

IASTED-NCI2004 Feb. 23-25, 2004 Estimating the Error at Given Test Estimating the Error at Given Test Input Points for Linear Regression Input Points for Linear Regression Masashi Sugiyama Fraunhofer FIRST-IDA, Berlin, Germany Tokyo

336 views • 21 slides
LINEAR REGRESSION LINEAR REGRESSION - FROM A MACHINE LEARNING POINT OF VIEW 25 SIMPLE LINEAR

LINEAR REGRESSION LINEAR REGRESSION - FROM A MACHINE LEARNING POINT OF VIEW 25 SIMPLE LINEAR

LINEAR REGRESSION LINEAR REGRESSION - FROM A MACHINE LEARNING POINT OF VIEW 25 SIMPLE LINEAR REGRESSION 20 Sales 15 Starting point 10 Simplest parametric function 5 Easy to interpret the parameters: 0 50 100 150 200 250

1.12k views • 13 slides
Week 3: Linear Regression Instructor: Sergey Levine 1 The regression problem We saw how we can

Week 3: Linear Regression Instructor: Sergey Levine 1 The regression problem We saw how we can

Week 3: Linear Regression Instructor: Sergey Levine 1 The regression problem We saw how we can estimate the parameters of probability distributions over a random variable x . However, in a supervised learning setting, we might be interested in

64 views • 5 slides
Estimating the parameters of some probability distributions: Exemplifications 1. Estimating the

Estimating the parameters of some probability distributions: Exemplifications 1. Estimating the

0. Estimating the parameters of some probability distributions: Exemplifications 1. Estimating the parameter of the Bernoulli distribution: the MLE and MAP approaches CMU, 2015 spring, Tom Mitchell, Nina Balcan, HW2, pr. 2 2. Suppose we

889 views • 54 slides
EC3062 ECONOMETRICS ELEMENTARY REGRESSION ANALYSIS We shall consider three methods for estimating

EC3062 ECONOMETRICS ELEMENTARY REGRESSION ANALYSIS We shall consider three methods for estimating

EC3062 ECONOMETRICS ELEMENTARY REGRESSION ANALYSIS We shall consider three methods for estimating statistical parameters. These are the method of moments, the method of least squares and the principle of maximum likelihood. In the case of the

624 views • 23 slides
Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common

Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common

0. Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common property CMU, 2004 fall, Andrew Moore, HW2, pr. 4 2. Linear Regression and Logistic Regression: Definitions Given an input vector X , linear

1.07k views • 22 slides
Notes on the Non-linear Regression The model Non-linear regression models, like ordinary linear

Notes on the Non-linear Regression The model Non-linear regression models, like ordinary linear

Notes on the Non-linear Regression The model Non-linear regression models, like ordinary linear models, assume that the response, Y , has a normal distribution with constant variance. Unlike linear models, however, the mean function may depend

159 views • 4 slides
Estimating Parameters of Pareto Distribution Under Interval and Fuzzy Uncertainty Nitaya Buntao

Estimating Parameters of Pareto Distribution Under Interval and Fuzzy Uncertainty Nitaya Buntao

Formulation of the Problem First Result: Estimating x 0 Under Interval Uncertainty Estimating Under Interval Uncertainty Algorithm for Computing the Smallest Value of Algorithm for Computing the Largest Value of Estimating Parameters of

977 views • 48 slides
Parametric Methods Steven J Zeil Old Dominion Univ. Fall 2010 1 Distributions Estimating

Parametric Methods Steven J Zeil Old Dominion Univ. Fall 2010 1 Distributions Estimating

Distributions Estimating Distribution Parameters Parametric Classification Regression Parametric Methods Steven J Zeil Old Dominion Univ. Fall 2010 1 Distributions Estimating Distribution Parameters Parametric Classification Regression

889 views • 59 slides
The General Linear Model. April 22, 2008 Multiple regression Data: The Faroese Mercury Study

The General Linear Model. April 22, 2008 Multiple regression Data: The Faroese Mercury Study

The General Linear Model. April 22, 2008 Multiple regression Data: The Faroese Mercury Study Simple linear regression Confounding The multiple linear regression model Interpretation of parameters Model control

564 views • 39 slides
Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied

Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied

Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied linear regression ; the output/label space Y was R . 90 80 delay 70 60 50 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 duration 1 / 68 Linear

607 views • 38 slides
Analysis of variance and regression November 22, 2007 Parametrisations : Choice of parameters

Analysis of variance and regression November 22, 2007 Parametrisations : Choice of parameters

Analysis of variance and regression November 22, 2007 Parametrisations : Choice of parameters Comparison of models Test for linearity Linear splines Lene Theil Skovgaard, Dept. of Biostatistics, Institute of Public Health,

966 views • 71 slides
Which models can be fit with linear regression? Simple linear regression in Matlab X = rand(3,3)

Which models can be fit with linear regression? Simple linear regression in Matlab X = rand(3,3)

Which models can be fit with linear regression? Simple linear regression in Matlab X = rand(3,3) b = pinv(X) * y X = b = 0.0467 0.5188 0.5317 14.3966 0.6587 0.3323 0.5070 50.9428 0.7573 0.0428 0.2532 -50.8102 y = rand(3,1) b = X \

643 views • 13 slides
Trendlines Simple Linear Regression Multiple Linear Regression Systematic Model

Trendlines Simple Linear Regression Multiple Linear Regression Systematic Model

Trendlines Simple Linear Regression Multiple Linear Regression Systematic Model Building Practical Issues Overfitting Categorical Variables Interaction Terms Non-linear Terms Linear y = a + bx Logarithmic

945 views • 65 slides
Lecture 4: Multivariate Regression, Part 2 Gauss-Markov Assumptions Linear in Parameters : 1)

Lecture 4: Multivariate Regression, Part 2 Gauss-Markov Assumptions Linear in Parameters : 1)

Lecture 4: Multivariate Regression, Part 2 Gauss-Markov Assumptions Linear in Parameters : 1) Y X X X 0 1 1 2 2 i k k Random Sampling : we have a random sample from 2) the

776 views • 39 slides
Unit 7: Multiple linear regression 1. Introduction to multiple linear regression Sta 101 - Fall

Unit 7: Multiple linear regression 1. Introduction to multiple linear regression Sta 101 - Fall

Announcements Unit 7: Multiple linear regression 1. Introduction to multiple linear regression Sta 101 - Fall 2018 Project questions? Duke University, Department of Statistical Science Dr. Abrahamsen Slides posted at

537 views • 10 slides
Outline The Simple Linear Regression Model (12.1) Fitting the Regression Line (12.2)

Outline The Simple Linear Regression Model (12.1) Fitting the Regression Line (12.2)

2/22/2007 219323 Probability and Statistics for Software Statistics for Software and Knowledge Engineers Lecture 13: Simple Linear Regression Simple Linear Regression and Correlation Monchai Sopitkamon, Ph.D. Outline The Simple Linear

968 views • 19 slides
Linear Models for Regression Greg Mori - CMPT 419/726 Bishop PRML Ch. 3 Regression Linear Basis

Linear Models for Regression Greg Mori - CMPT 419/726 Bishop PRML Ch. 3 Regression Linear Basis

Regression Linear Basis Function Models Loss Functions for Regression Finding Optimal Weights Regularization Bayesian Linear Linear Models for Regression Greg Mori - CMPT 419/726 Bishop PRML Ch. 3 Regression Linear Basis Function Models

1.19k views • 84 slides
Regression 1: Linear Regression Marco Baroni Practical Statistics in R Outline Classic linear

Regression 1: Linear Regression Marco Baroni Practical Statistics in R Outline Classic linear

Regression 1: Linear Regression Marco Baroni Practical Statistics in R Outline Classic linear regression Linear regression in R Outline Classic linear regression Introduction Constructing the model Estimation Looking at the fitted model

1.13k views • 77 slides
Gov 2000: 7. What is Regression? Matthew Blackwell Fall 2016 1 / 65 1. Relationships between

Gov 2000: 7. What is Regression? Matthew Blackwell Fall 2016 1 / 65 1. Relationships between

Gov 2000: 7. What is Regression? Matthew Blackwell Fall 2016 1 / 65 1. Relationships between Variables 2. Conditional Expectation 3. Estimating the CEF 4. Linear CEFs and Linear Projections 5. Least Squares 2 / 65 Where are we? Where are

888 views • 65 slides
Overview IAML: Linear Regression The linear model Fitting the linear model to data

Overview IAML: Linear Regression The linear model Fitting the linear model to data

Overview IAML: Linear Regression The linear model Fitting the linear model to data Probabilistic interpretation of the error function Nigel Goddard School of Informatics Examples of regression problems Dealing with multiple

325 views • 10 slides
Learning From Data Lecture 8 Linear Classification and Regression Linear Classification Linear

Learning From Data Lecture 8 Linear Classification and Regression Linear Classification Linear

Learning From Data Lecture 8 Linear Classification and Regression Linear Classification Linear Regression M. Magdon-Ismail CSCI 4100/6100 recap: Approximation Versus Generalization VC Analysis Bias-Variance Analysis E out E in + ( d

390 views • 23 slides
Linear regression Linear regression is a simple approach to supervised learning. It assumes

Linear regression Linear regression is a simple approach to supervised learning. It assumes

Linear regression Linear regression is a simple approach to supervised learning. It assumes that the dependence of Y on X 1 , X 2 , . . . X p is linear. True regression functions are never linear! 7 6 f(X) 5 4 3 2 4 6 8 X

704 views • 48 slides
10-601 Machine Learning Regression Outline Regression vs Classification Linear regression

10-601 Machine Learning Regression Outline Regression vs Classification Linear regression

10-601 Machine Learning Regression Outline Regression vs Classification Linear regression another discriminative learning method As optimization Gradient descent As matrix inversion (Ordinary Least Squares) Overfitting

1.52k views • 97 slides

More recommend