a simple bayesian regression model
play

A simple Bayesian regression model Alicia Johnson Associate - PowerPoint PPT Presentation

Jul 02, 2023 •251 likes •858 views

DataCamp Bayesian Modeling with RJAGS BAYESIAN MODELING WITH RJAGS A simple Bayesian regression model Alicia Johnson Associate Professor, Macalester College DataCamp Bayesian Modeling with RJAGS Chapter 3 goals Engineer a simple Bayesian

  1. DataCamp Bayesian Modeling with RJAGS BAYESIAN MODELING WITH RJAGS A simple Bayesian regression model Alicia Johnson Associate Professor, Macalester College

  2. DataCamp Bayesian Modeling with RJAGS Chapter 3 goals Engineer a simple Bayesian regression model Define, compile, and simulate regression models in RJAGS Use Markov chain simulation output for posterior inference & prediction

  3. DataCamp Bayesian Modeling with RJAGS Modeling weight Y = weight of adult i (kg) i Model 2 Y ∼ N ( m , s ) i

  4. DataCamp Bayesian Modeling with RJAGS Modeling weight by height Y = weight of adult i (kg) i Model 2 Y ∼ N ( m , s ) i

  5. DataCamp Bayesian Modeling with RJAGS Modeling weight by height Y = weight of adult i (kg) i X = height of adult i (cm) i Model 2 Y ∼ N ( m , s ) i i

  6. DataCamp Bayesian Modeling with RJAGS Modeling weight by height Y = weight of adult i (kg) i X = height of adult i (cm) i Model 2 Y ∼ N ( m , s ) i i m = a + bX i i

  7. DataCamp Bayesian Modeling with RJAGS Modeling weight by height Y = weight of adult i (kg) i X = height of adult i (cm) i Model 2 Y ∼ N ( m , s ) i i m = a + bX i i

  8. DataCamp Bayesian Modeling with RJAGS Modeling weight by height Y = weight of adult i (kg) i X = height of adult i (cm) i Model 2 Y ∼ N ( m , s ) i i m = a + bX i i

  9. DataCamp Bayesian Modeling with RJAGS Modeling weight by height Y = weight of adult i (kg) i X = height of adult i (cm) i Model 2 Y ∼ N ( m , s ) i i m = a + bX i i

  10. DataCamp Bayesian Modeling with RJAGS Modeling weight by height Y = weight of adult i (kg) i X = height of adult i (cm) i Model 2 Y ∼ N ( m , s ) i i m = a + bX i i

  11. DataCamp Bayesian Modeling with RJAGS Bayesian regression model 2 Y ∼ N ( m , s ) i i m = a + bX i i a = y-intercept value of m when X = 0 i i b = slope rate of change in weight (kg) per 1 cm increase in height s = residual standard deviation individual deviation from trend m i

  12. DataCamp Bayesian Modeling with RJAGS Priors for the intercept & slope

  13. DataCamp Bayesian Modeling with RJAGS Priors for the intercept & slope

  14. DataCamp Bayesian Modeling with RJAGS Priors for the intercept & slope

  15. DataCamp Bayesian Modeling with RJAGS Prior for the residual standard deviation

  16. DataCamp Bayesian Modeling with RJAGS Prior for the residual standard deviation

  17. DataCamp Bayesian Modeling with RJAGS Bayesian regression model 2 Y ∼ N ( m , s ) i i m = a + bX i i 2 a ∼ N (0,200 ) 2 b ∼ N (1,0.5 ) s ∼ Unif (0,20)

  18. DataCamp Bayesian Modeling with RJAGS BAYESIAN MODELING WITH RJAGS Let's practice!

  19. DataCamp Bayesian Modeling with RJAGS BAYESIAN MODELING WITH RJAGS Bayesian regression in RJAGS Alicia Johnson Associate Professor, Macalester College

  20. DataCamp Bayesian Modeling with RJAGS Bayesian regression model Y = weight of adult i (kg) i X = height of adult i (cm) i Model 2 Y ∼ N ( m , s ) i i m = a + bX i i 2 a ∼ N (0,200 ) 2 b ∼ N (1,0.5 ) s ∼ Unif (0,20)

  21. DataCamp Bayesian Modeling with RJAGS Prior insight

  22. DataCamp Bayesian Modeling with RJAGS Insight from the observed weight & height data 2 Y ∼ N ( m , s ) i i m = a + bX i i > wt_mod <- lm(wgt ~ hgt, bdims) > coef(wt_mod) (Intercept) hgt -105.011254 1.017617 > summary(wt_mod)$sigma [1] 9.30804

  23. DataCamp Bayesian Modeling with RJAGS DEFINE the regression model weight_model <- "model{ # Likelihood model for Y[i] # Prior models for a, b, s }"

  24. DataCamp Bayesian Modeling with RJAGS DEFINE the regression model weight_model <- "model{ 2 Y ∼ N ( m , s ) for i from 1 to 507 i i # Likelihood model for Y[i] for(i in 1:length(Y)) { } # Prior models for a, b, s }"

  25. DataCamp Bayesian Modeling with RJAGS DEFINE the regression model weight_model <- "model{ 2 Y ∼ N ( m , s ) for i from 1 to 507 i i # Likelihood model for Y[i] for(i in 1:length(Y)) { Y[i] ~ dnorm(m[i], s^(-2)) } # Prior models for a, b, s }"

  26. DataCamp Bayesian Modeling with RJAGS DEFINE the regression model weight_model <- "model{ # Likelihood model for Y[i] 2 Y ∼ N ( m , s ) for i from 1 to 507 for(i in 1:length(Y)) { i i Y[i] ~ dnorm(m[i], s^(-2)) m = a + bX m[i] <- a + b * X[i] i i } NOTE: use " <- " not " ~ " # Prior models for a, b, s }"

  27. DataCamp Bayesian Modeling with RJAGS DEFINE the regression model weight_model <- "model{ # Likelihood model for Y[i] 2 Y ∼ N ( m , s ) for i from 1 to 507 for(i in 1:length(Y)) { i i Y[i] ~ dnorm(m[i], s^(-2)) m = a + bX m[i] <- a + b * X[i] i i } # Prior models for a, b, s 2 a ∼ N (0,200 ) a ~ dnorm(0, 200^(-2)) b ~ dnorm(1, 0.5^(-2)) 2 b ∼ N (1,0.5 ) s ~ dunif(0, 20) s ∼ Unif (0,20) }"

  28. DataCamp Bayesian Modeling with RJAGS COMPILE the regression model # COMPILE the model weight_jags <- jags.model(textConnection(weight_model), data = list(X = bdims$hgt, Y = bdims$wgt), inits = list(.RNG.name = "base::Wichmann-Hill", .RNG.seed = 2018)) > dim(bdims) [1] 507 25 > head(bdims$hgt) [1] 174.0 175.3 193.5 186.5 187.2 181.5 > head(bdims$wgt) [1] 65.6 71.8 80.7 72.6 78.8 74.8

  29. DataCamp Bayesian Modeling with RJAGS SIMULATE the regression model # COMPILE the model weight_jags <- jags.model(textConnection(weight_model), data = list(X = bdims$hgt, Y = bdims$wgt), inits = list(.RNG.name = "base::Wichmann-Hill", .RNG.seed = 2018)) # SIMULATE the posterior weight_sim <- coda.samples(model = weight_jags, variable.names = c("a", "b", "s"), n.iter = 10000)

  30. DataCamp Bayesian Modeling with RJAGS

  31. DataCamp Bayesian Modeling with RJAGS Addressing Markov chain instability Standardize the height predictor (subtract the mean and divide by the standard deviation). Increase chain length.

  32. DataCamp Bayesian Modeling with RJAGS

  33. DataCamp Bayesian Modeling with RJAGS Posterior insights

  34. DataCamp Bayesian Modeling with RJAGS BAYESIAN MODELING WITH RJAGS Let's practice!

  35. DataCamp Bayesian Modeling with RJAGS BAYESIAN MODELING WITH RJAGS Posterior estimation & inference Alicia Johnson Associate Professor, Macalester College

  36. DataCamp Bayesian Modeling with RJAGS Bayesian regression model Y = weight of adult i (kg) i X = height of adult i (cm) i Model 2 Y ∼ N ( m , s ) i i m = a + bX i i 2 a ∼ N (0,200 ) 2 b ∼ N (1,0.5 ) s ∼ Unif (0,20)

  37. DataCamp Bayesian Modeling with RJAGS Posterior point estimation

  38. DataCamp Bayesian Modeling with RJAGS Posterior point estimation

  39. DataCamp Bayesian Modeling with RJAGS Posterior point estimation > summary(weight_sim_big) 1. Empirical mean and standard deviation for each variable, plus standard error of the mean: Mean SD Naive SE Time-series SE a -104.038 7.85296 0.0248332 0.661515 b 1.012 0.04581 0.0001449 0.003849 s 9.331 0.29495 0.0009327 0.001216 2. Quantiles for each variable: 2.5% 25% 50% 75% 97.5% a -118.6843 -109.5171 -104.365 -99.036 -87.470 b 0.9152 0.9828 1.014 1.044 1.098 s 8.7764 9.1284 9.322 9.524 9.933 posterior mean of a ≈ -104.038 posterior mean of b ≈ 1.012

  40. DataCamp Bayesian Modeling with RJAGS Posterior point estimation Posterior mean trend: m = −104.038 + 1.012 X i i Markov chain output: > head(weight_chains) a b s [1,] -113.9029 1.072505 8.772007 [2,] -115.0644 1.077914 8.986393 [3,] -114.6958 1.077130 9.679812 [4,] -115.0568 1.072668 8.814403 [5,] -114.0782 1.071775 8.895299 [6,] -114.3271 1.069477 9.016185

  41. DataCamp Bayesian Modeling with RJAGS Posterior uncertainty Posterior mean trend: m = −104.038 + 1.012 X i i Markov chain output: > head(weight_chains) a b s [1,] -113.9029 1.072505 8.772007 [2,] -115.0644 1.077914 8.986393 [3,] -114.6958 1.077130 9.679812 [4,] -115.0568 1.072668 8.814403 [5,] -114.0782 1.071775 8.895299 [6,] -114.3271 1.069477 9.016185

  42. DataCamp Bayesian Modeling with RJAGS Posterior credible intervals

  43. DataCamp Bayesian Modeling with RJAGS Posterior credible intervals > summary(weight_sim_big) 1. Empirical mean and standard deviation for each variable, plus standard error of the mean: Mean SD Naive SE Time-series SE a -104.038 7.85296 0.0248332 0.661515 b 1.012 0.04581 0.0001449 0.003849 s 9.331 0.29495 0.0009327 0.001216 2. Quantiles for each variable: 2.5% 25% 50% 75% 97.5% a -118.6843 -109.5171 -104.365 -99.036 -87.470 b 0.9152 0.9828 1.014 1.044 1.098 s 8.7764 9.1284 9.322 9.524 9.933 95% posterior credible interval for a : (-118.6843, -87.470) 95% posterior credible interval for b : (0.9152, 1.098)

  44. DataCamp Bayesian Modeling with RJAGS Posterior credible intervals Interpretation In light of our priors & observed data, there's a 95% (posterior) chance that b is between 0.9152 & 1.098 kg/cm.

  45. DataCamp Bayesian Modeling with RJAGS Posterior probabilities > table(weight_chains$b > 1.1) FALSE TRUE 97835 2165 > mean(weight_chains$b > 1.1) [1] 0.02165 Interpretation: There's a 2.165% posterior chance that b exceeds 1.1 kg/cm.

Recommend

BS2247 Introduction to Econometrics Lecture 3: The simple regression model OLS, Algebraic

BS2247 Introduction to Econometrics Lecture 3: The simple regression model OLS, Algebraic

BS2247 Introduction to Econometrics Lecture 3: The simple regression model OLS, Algebraic properties, Goodness-of-fit Dr. Kai Sun Aston Business School 1 / 28 The Simple Regression Model: y = 0 + 1 x + u In the simple linear regression of

708 views • 54 slides
Multiple Linear Regression The population model In a simple linear regression model, a single

Multiple Linear Regression The population model In a simple linear regression model, a single

Multiple Linear Regression The population model In a simple linear regression model, a single response measure- ment Y is related to a single predictor (covariate, regressor) X for each observation. The critical assumption of the model is

859 views • 49 slides
Outline The Simple Linear Regression Model (12.1) Fitting the Regression Line (12.2)

Outline The Simple Linear Regression Model (12.1) Fitting the Regression Line (12.2)

2/22/2007 219323 Probability and Statistics for Software Statistics for Software and Knowledge Engineers Lecture 13: Simple Linear Regression Simple Linear Regression and Correlation Monchai Sopitkamon, Ph.D. Outline The Simple Linear

968 views • 19 slides
Simple Linear Regression Recall: A regression model describes how a dependent variable (or

Simple Linear Regression Recall: A regression model describes how a dependent variable (or

ST 430/514 Introduction to Regression Analysis/Statistics for Management and the Social Sciences II Simple Linear Regression Recall: A regression model describes how a dependent variable (or response ) Y is affected, on average, by one or more

592 views • 20 slides
The Simple Regression Model Deriving the Ordinary Least Squares Estimates Properties of Caio

The Simple Regression Model Deriving the Ordinary Least Squares Estimates Properties of Caio

Definition of the Simple Regression Model The Simple Regression Model Deriving the Ordinary Least Squares Estimates Properties of Caio Vigo OLS on any Sample of Data The University of Kansas Units of Measurement Department of

1.21k views • 86 slides
The Simple Regression Model Deriving the Ordinary Least Squares Estimates Properties of Caio

The Simple Regression Model Deriving the Ordinary Least Squares Estimates Properties of Caio

Definition of the Simple Regression Model The Simple Regression Model Deriving the Ordinary Least Squares Estimates Properties of Caio Vigo OLS on any Sample of Data The University of Kansas Units of Measurement Department of

999 views • 83 slides
Computational Bayesian data analysis Bruno Nicenboim / Shravan Vasishth 2020-03-11 1 Bayesian

Computational Bayesian data analysis Bruno Nicenboim / Shravan Vasishth 2020-03-11 1 Bayesian

Computational Bayesian data analysis Bruno Nicenboim / Shravan Vasishth 2020-03-11 1 Bayesian Regression Models using Stan: brms Examples 1: A single participant pressing a button repeatedly (A simple linear model) Prior predictive

1.14k views • 71 slides
Bayesian Learning 1 Outline MLE, MAP vs. Bayesian Learning Bayesian Linear Regression

Bayesian Learning 1 Outline MLE, MAP vs. Bayesian Learning Bayesian Linear Regression

Bayesian Learning 1 Outline MLE, MAP vs. Bayesian Learning Bayesian Linear Regression Bayesian Gaussian Mixture Models Non-parametric Bayes 2 Take Away ... 1. Maximum Likelihood Estimate (MLE) = arg max p ( D| )

609 views • 27 slides
Trendlines Simple Linear Regression Multiple Linear Regression Systematic Model

Trendlines Simple Linear Regression Multiple Linear Regression Systematic Model

Trendlines Simple Linear Regression Multiple Linear Regression Systematic Model Building Practical Issues Overfitting Categorical Variables Interaction Terms Non-linear Terms Linear y = a + bx Logarithmic

945 views • 65 slides
STAT 213 Simple Linear Regression I Colin Reimer Dawson Oberlin College 5 October 2016 Outline

STAT 213 Simple Linear Regression I Colin Reimer Dawson Oberlin College 5 October 2016 Outline

Outline Simple Linear Regression Model STAT 213 Simple Linear Regression I Colin Reimer Dawson Oberlin College 5 October 2016 Outline Simple Linear Regression Model Outline Simple Linear Regression Model Outline Simple Linear Regression

399 views • 16 slides
BS2247 Introduction to Econometrics Lecture 4: The simple regression model OLS Unbiasedness, OLS

BS2247 Introduction to Econometrics Lecture 4: The simple regression model OLS Unbiasedness, OLS

BS2247 Introduction to Econometrics Lecture 4: The simple regression model OLS Unbiasedness, OLS Variances, Units of measurement, Nonlinearities Dr. Kai Sun Aston Business School 1 / 26 Unbiasedness of OLS SLR for Simple Linear

797 views • 56 slides
Bayesian regression models Bruno Nicenboim / Shravan Vasishth 2020-03-17 1 A first linear

Bayesian regression models Bruno Nicenboim / Shravan Vasishth 2020-03-17 1 A first linear

Bayesian regression models Bruno Nicenboim / Shravan Vasishth 2020-03-17 1 A first linear model: Does attentional load affect pupil size? Log-normal model: Does trial affect reaction times? Logistic regression: Does set size affect free

796 views • 79 slides
A Simple Bayesian Game Prediction Model By: Jake Flancer The Motivation Quantify Team

A Simple Bayesian Game Prediction Model By: Jake Flancer The Motivation Quantify Team

A Simple Bayesian Game Prediction Model By: Jake Flancer The Motivation Quantify Team Performance With Uncertainty Estimates Using Limited Data 2 Bayesian Statistics Incorporating beliefs not in data Prior padding data +

847 views • 28 slides
Outline Intro to RL and Bayesian Learning History of Bayesian RL Model-based Bayesian

Outline Intro to RL and Bayesian Learning History of Bayesian RL Model-based Bayesian

Outline Intro to RL and Bayesian Learning History of Bayesian RL Model-based Bayesian RL Prior knowledge, policy optimization, discussion, Bayesian approaches for other RL variants Model-free Bayesian RL Gaussian

1.23k views • 63 slides
Multiple and Logistic Regression IV Dajiang Liu @PHS 525 Apr-21 st -2016 Review of Last Two

Multiple and Logistic Regression IV Dajiang Liu @PHS 525 Apr-21 st -2016 Review of Last Two

Multiple and Logistic Regression IV Dajiang Liu @PHS 525 Apr-21 st -2016 Review of Last Two Classes Linear regression model: Logistic regression model Deal with binary outcomes Why is multiple (simple) linear regression model

260 views • 15 slides
Bayesian Linear Regression Seung-Hoon Na Chonbuk National University Bayesian Linear Regression

Bayesian Linear Regression Seung-Hoon Na Chonbuk National University Bayesian Linear Regression

Bayesian Linear Regression Seung-Hoon Na Chonbuk National University Bayesian Linear Regression Compute the full posterior over and 2 Case 1) the noise variance 2 is known Use Gaussian prior Case 2) the noise

327 views • 19 slides
Classification Basic concepts Decision tree Nave Bayesian classifier Model

Classification Basic concepts Decision tree Nave Bayesian classifier Model

Classification Basic concepts Decision tree Nave Bayesian classifier Model evaluation Support Vector Machines Regression Neural Networks and Deep Learning Lazy Learners (k Nearest Neighbors) Bayesian Belief

1.62k views • 137 slides
Simple Linear Regression and Correlation Model for designed experiment: Y i = 0 + 1 x i +

Simple Linear Regression and Correlation Model for designed experiment: Y i = 0 + 1 x i +

Simple Linear Regression and Correlation Model for designed experiment: Y i = 0 + 1 x i + i 1 , . . . , n independent, mean 0, variance 2 . Model for sample of pairs: ( X i , Y i ) , i = 1 , . . . , n sample from

501 views • 11 slides
Business Statistics CONTENTS Ordinary least squares (recap for some) Statistical formulation of

Business Statistics CONTENTS Ordinary least squares (recap for some) Statistical formulation of

SIMPLE REGRESSION ANALYSIS Business Statistics CONTENTS Ordinary least squares (recap for some) Statistical formulation of the regression model Assessing the regression model Testing the regression coefficients The ANOVA table Old exam

554 views • 30 slides
A Bayesian approach to estimate the number and position of knots for linear regression splines

A Bayesian approach to estimate the number and position of knots for linear regression splines

Framework Knots The model Simulation study Real data application Discussion A Bayesian approach to estimate the number and position of knots for linear regression splines Gioia Di Credico, Francesco Pauli and Nicola Torelli Department of

336 views • 16 slides
Poisson Regression Models for Count Data Outline Review Introduction to Poisson

Poisson Regression Models for Count Data Outline Review Introduction to Poisson

Poisson Regression Models for Count Data Outline Review Introduction to Poisson regression A simple model: equiprobable model Pearson and likelihood-ratio test statistics Residual analysis Poisson regression with a

487 views • 17 slides
Case Study: Bayesian Linear Regression and Sparse Bayesian Models Piyush Rai Dept. of CSE, IIT

Case Study: Bayesian Linear Regression and Sparse Bayesian Models Piyush Rai Dept. of CSE, IIT

Case Study: Bayesian Linear Regression and Sparse Bayesian Models Piyush Rai Dept. of CSE, IIT Kanpur (Mini-course: lecture 2) Nov 05, 2015 Piyush Rai (IIT Kanpur) Bayesian Linear Regression and Sparse Bayesian Models 1 Recap Piyush Rai

1.13k views • 72 slides
The General Linear Model. April 22, 2008 Multiple regression Data: The Faroese Mercury Study

The General Linear Model. April 22, 2008 Multiple regression Data: The Faroese Mercury Study

The General Linear Model. April 22, 2008 Multiple regression Data: The Faroese Mercury Study Simple linear regression Confounding The multiple linear regression model Interpretation of parameters Model control

564 views • 39 slides
Bayesian linear regression (cont.) Dr. Jarad Niemi STAT 544 - Iowa State University April 20,

Bayesian linear regression (cont.) Dr. Jarad Niemi STAT 544 - Iowa State University April 20,

Bayesian linear regression (cont.) Dr. Jarad Niemi STAT 544 - Iowa State University April 20, 2017 Jarad Niemi (STAT544@ISU) Bayesian linear regression (cont.) April 20, 2017 1 / 25 Outline Subjective Bayesian regression Ridge regression

563 views • 25 slides

More recommend