Introduction to Bayesian Analysis in Stata The Method Fundamental - PowerPoint PPT Presentation

Bayesian analysis in Stata Outline The general idea Introduction to Bayesian Analysis in Stata The Method Fundamental equation MCMC Gustavo Sánchez Stata tools bayes: - bayesmh Postestimation StataCorp LLC Examples 1- Linear regression October 24 , 2018 bayesstats ess bayesgraph Barcelona, Spain thinning() bayestestmodel 2- Random effects probit bayesgraph bayestest interval 3- Change point model Gibbs sampling Summary References

Bayesian analysis in Stata Outline Outline 1 Bayesian analysis: Basic Concepts The general idea • The general idea The Method • The Method Fundamental equation MCMC 2 The Stata Tools Stata tools bayes: - bayesmh • The general command bayesmh Postestimation • The bayes prefix Examples • Postestimation Commands 1- Linear regression bayesstats ess 3 A few examples bayesgraph thinning() • Linear regression bayestestmodel • Panel data random effect probit model 2- Random effects probit • Change point model bayesgraph bayestest interval 3- Change point model Gibbs sampling Summary References

Bayesian analysis in Stata The general idea Outline The general idea The Method Fundamental equation MCMC Stata tools bayes: - bayesmh Postestimation Examples 1- Linear regression bayesstats ess bayesgraph thinning() bayestestmodel 2- Random effects probit bayesgraph bayestest interval 3- Change point model Gibbs sampling Summary References

Bayesian analysis in Stata The general idea Outline The general idea The Method Fundamental equation MCMC Stata tools bayes: - bayesmh Postestimation Examples 1- Linear regression bayesstats ess bayesgraph thinning() bayestestmodel 2- Random effects probit bayesgraph bayestest interval 3- Change point model Gibbs sampling Summary References

Bayesian analysis in Stata Bayesian Analysis vs Frequentist Analysis Outline Frequentist Analysis Bayesian Analyis The general idea The Method • Estimate unknown fixed • Probability distributions for Fundamental equation parameters. unknown random parameters MCMC Stata tools • The data is fixed. • Data for a (hypothetical) bayes: - bayesmh Postestimation repeatable random sample. Examples • Combines data with prior • Uses data to estimate 1- Linear beliefs to get probability unknown fixed parameters. regression distributions for the bayesstats ess parameters. bayesgraph thinning() • Posterior distribution is used • Data expected to satisfy the bayestestmodel to make explicit probabilistic assumptions for the specified 2- Random statements. effects probit model. bayesgraph "Conclusions are based on the "Bayesian analysis answers bayestest interval distribution of statistics derived questions based on the distribution 3- Change from random samples, assuming point model of parameters conditional on the unknown but fixed parameters." Gibbs sampling observed sample." Summary References

Bayesian analysis in Stata Stata’s simple syntax: bayes : Outline regress y x1 x2 x3 The general idea The Method bayes: regress y x1 x2 x3 Fundamental equation MCMC Stata tools bayes: - bayesmh logit y x1 x2 x3 Postestimation Examples bayes: logit y x1 x2 x3 1- Linear regression bayesstats ess bayesgraph thinning() bayestestmodel mixed y x1 x2 x3 || region: 2- Random effects probit bayesgraph bayes: mixed y x1 x2 x3 || region: bayestest interval 3- Change point model Gibbs sampling Summary References

Bayesian analysis in Stata Outline The general idea The Method Fundamental equation MCMC Stata tools The Method bayes: - bayesmh Postestimation Examples 1- Linear regression bayesstats ess bayesgraph thinning() bayestestmodel 2- Random effects probit bayesgraph bayestest interval 3- Change point model Gibbs sampling Summary References

Bayesian analysis in Stata The Method Outline • Inverse law of probability (Bayes’ Theorem): The general idea f ( y ; θ ) π ( θ ) The Method f ( θ | y ) = Fundamental equation f ( y ) MCMC Stata tools • Marginal distribution of y, f(y), does not depend on ( θ ) bayes: - bayesmh Postestimation Examples • We can then write the fundamental equation for 1- Linear regression Bayesian analysis: bayesstats ess bayesgraph thinning() bayestestmodel p ( θ | y ) ∝ L ( y | θ ) π ( θ ) 2- Random effects probit bayesgraph bayestest interval 3- Change point model Gibbs sampling Summary References

Bayesian analysis in Stata The Method • Let’s assume that both, the data and the prior beliefs, Outline are normally distributed: The general idea θ, σ 2 • The data : y ∼ N � � The Method d Fundamental equation � µ p , σ 2 � • The prior : θ ∼ N p MCMC Stata tools • Homework...: Doing the algebra with the fundamental bayes: - bayesmh Postestimation equation we find that the posterior distribution would be Examples normal with (see for example Cameron & Trivedi 2005): 1- Linear regression � µ, σ 2 � • The posterior : θ | y ∼ N bayesstats ess bayesgraph thinning() bayestestmodel Where: 2- Random σ 2 � y /σ 2 d + µ p /σ 2 � µ = N ¯ effects probit p bayesgraph � − 1 bayestest interval σ 2 N /σ 2 d + 1 /σ 2 � = p 3- Change point model Gibbs sampling Summary References

Bayesian analysis in Stata Example (Posterior distributions) Outline The general idea The Method Fundamental equation MCMC Stata tools bayes: - bayesmh Postestimation Examples 1- Linear regression bayesstats ess bayesgraph thinning() bayestestmodel 2- Random effects probit bayesgraph bayestest interval 3- Change point model Gibbs sampling Summary References

Bayesian analysis in Stata The Method Outline • The previous example has a closed form solution. The general idea • What about the cases with non-closed solutions, or The Method Fundamental more complex distributions? equation MCMC • Integration is performed via simulation Stata tools • We need to use intensive computational simulation bayes: - bayesmh Postestimation tools to find the posterior distribution in most cases. Examples • Markov chain Monte Carlo (MCMC) methods are the 1- Linear regression current standard in most software. Stata implement two bayesstats ess alternatives: bayesgraph thinning() • Metropolis-Hastings (MH) algorithm bayestestmodel • Gibbs sampling 2- Random effects probit bayesgraph bayestest interval 3- Change point model Gibbs sampling Summary References

Bayesian analysis in Stata Outline The Method The general idea • Links for Bayesian analysis and MCMC on our youtube The Method channel: Fundamental equation • Introduction to Bayesian statistics, part 1: The basic MCMC Stata tools concepts bayes: - bayesmh Postestimation https://www.youtube.com/watch?v=0F0QoMCSKJ4&feature=youtu.be Examples 1- Linear regression • Introduction to Bayesian statistics, part 2: MCMC and bayesstats ess the Metropolis Hastings algorithm. bayesgraph thinning() bayestestmodel 2- Random https://www.youtube.com/watch?v=OTO1DygELpY&feature=youtu.be effects probit bayesgraph bayestest interval 3- Change point model Gibbs sampling Summary References

Bayesian analysis in The Method Stata • Monte Carlo Simulation Outline The general idea The Method Fundamental equation MCMC Stata tools bayes: - bayesmh Postestimation Examples 1- Linear regression bayesstats ess bayesgraph thinning() bayestestmodel 2- Random effects probit bayesgraph bayestest interval 3- Change point model Gibbs sampling Summary References

Bayesian analysis in The Method Stata • Markov Chain Monte Carlo Simulation Outline The general idea The Method Fundamental equation MCMC Stata tools bayes: - bayesmh Postestimation Examples 1- Linear regression bayesstats ess bayesgraph thinning() bayestestmodel 2- Random effects probit bayesgraph bayestest interval 3- Change point model Gibbs sampling Summary References

Bayesian The Method analysis in Stata • Metropolis Hastings intuitive idea • Green points represent accepted proposal states and Outline red points represent rejected proposal states. The general idea The Method Fundamental equation MCMC Stata tools bayes: - bayesmh Postestimation Examples 1- Linear regression bayesstats ess bayesgraph thinning() bayestestmodel 2- Random effects probit bayesgraph bayestest interval 3- Change point model Gibbs sampling Summary References

Bayesian The Method analysis in Stata • Metropolis Hastings simulation • The trace plot illustrates the sequence of accepted Outline proposal states. The general idea The Method Fundamental equation MCMC Stata tools bayes: - bayesmh Postestimation Examples 1- Linear regression bayesstats ess bayesgraph thinning() bayestestmodel 2- Random effects probit bayesgraph bayestest interval 3- Change point model Gibbs sampling Summary References

Bayesian analysis in The Method Stata • We expect to obtain a stationary sequence when Outline convergence is achieved. The general idea The Method Fundamental equation MCMC Stata tools bayes: - bayesmh Postestimation Examples 1- Linear regression bayesstats ess bayesgraph thinning() bayestestmodel 2- Random effects probit bayesgraph bayestest interval 3- Change point model Gibbs sampling Summary References