Multi-parameter models Applied Bayesian Statistics Dr. Earvin - PowerPoint PPT Presentation

Multi-parameter models Applied Bayesian Statistics Dr. Earvin Balderama Department of Mathematics & Statistics Loyola University Chicago September 26, 2017 Multi-parameter models Last edited September 24, 2017 by Earvin Balderama 1 <ebalderama@luc.edu>

Multi-parameter models Examples Most analyses consist of several parameters. Example Consider the normal model, Y i ∼ Normal ( µ, σ 2 ) . We want to study the joint posterior distribution f ( µ, σ 2 | y 1 , . . . , y n ) . Another Example Consider the simple linear regression model, Y i ∼ Normal ( β 0 + β 1 X 1 , σ 2 ) . We want to study the joint posterior distribution f ( β 0 , β 1 , σ 2 | y 1 , . . . , y n ) . Multi-parameter models Last edited September 24, 2017 by Earvin Balderama 2 <ebalderama@luc.edu>

Multi-parameter models The normal model The joint distribution ( bivariate PDF) of ( µ, σ 2 ) is µ, σ 2 | y 1 , . . . , y n � � f y 1 , . . . , y n | µ, σ 2 � µ, σ 2 � � � ∝ f f � y 1 , . . . , y n | µ, σ 2 � � µ | σ 2 � � σ 2 � ∝ f f f � � � � � − n � ( y i − µ ) 2 � ( µ − θ ) 2 � 1 � � σ 2 � − a − 1 exp − b � ∝ exp − exp − 2 σ 2 2 τ 2 σ 2 σ Multi-parameter models Last edited September 24, 2017 by Earvin Balderama 3 <ebalderama@luc.edu>

Multi-parameter models Summarizing the posterior How can we compute high-dimensional posterior distributions? How to summarize them concisely? Computing the posterior on a grid. 1 Coding MCMC sampling ourselves in R. 2 Using BUGS, JAGS, or Stan for MCMC sampling. 3 Multi-parameter models Last edited September 24, 2017 by Earvin Balderama 4 <ebalderama@luc.edu>

Multi-parameter models Computing the posterior on a grid For models with only a few parameters, it may be simple enough to plot the posterior distribution on a grid. For all combinations of r values of µ and r values of σ 2 , 1 µ, σ 2 | y 1 , . . . , y n � � compute f If there are p parameters in the model, the number of grid points is r p . 2 Multi-parameter models Last edited September 24, 2017 by Earvin Balderama 5 <ebalderama@luc.edu>

Multi-parameter models Summarizing the results Typically, we are interested in the marginal posterior distribution, � ∞ µ, σ 2 | y 1 , . . . , y n d σ 2 , � � f ( µ | y 1 , . . . , y n ) = f 0 which accounts for uncertainty in σ 2 . We can then form inference on this one parameter, summaries credible intervals hypothesis testing For most analyses the marginal posteriors will not be nice distributions, and a grid is impossible if there are many parameters. Instead we will use Monte Carlo methods to draw representative samples from the posterior. Multi-parameter models Last edited September 24, 2017 by Earvin Balderama 6 <ebalderama@luc.edu>

Multi-parameter models Drawing from the joint posterior Recall in the normal model, if either µ or σ 2 is known, we can sample from the other parameter, using its full conditional distribution. µ | y 1 , . . . , y n , σ 2 ∼ Normal � � σ 2 n ¯ y + m θ n + m , n + m � n σ 2 � 2 + a , SSE � � y 1 , . . . , y n , µ ∼ InverseGamma + b � 2 But how do we draw from the joint distribution? Multi-parameter models Last edited September 24, 2017 by Earvin Balderama 7 <ebalderama@luc.edu>