Introduction to Bayesian Analysis in Stata The Method Bayes rule - PowerPoint PPT Presentation

Bayesian analysis in Stata Outline The general idea Introduction to Bayesian Analysis in Stata The Method Bayes rule Fundamental equation MCMC Gustavo Sánchez Stata tools bayesmh bayesstats ess StataCorp LLC Blocking bayesgraph bayes: prefix bayesstats ic September 15 , 2017 bayestest model Porto, Portugal Random Effects Probit Thinning bayestest interval Change-point model bayesgraph matrix Summary References

Bayesian analysis in Stata Outline Outline The general 1 Bayesian analysis: The general idea idea The Method 2 Basic Concepts Bayes rule Fundamental • The Method equation MCMC • The tools Stata tools • Stata 14: The bayesmh command bayesmh • Stata 15: The bayes prefix bayesstats ess Blocking • Postestimation commands bayesgraph bayes: prefix bayesstats ic 3 A few examples bayestest model Random • Linear regression Effects Probit • Panel data random effect probit model Thinning bayestest interval • Change point model Change-point model bayesgraph matrix Summary References

Bayesian analysis in Stata The general idea Outline The general idea The Method Bayes rule Fundamental equation MCMC Stata tools bayesmh bayesstats ess Blocking bayesgraph bayes: prefix bayesstats ic bayestest model Random Effects Probit Thinning bayestest interval Change-point model bayesgraph matrix Summary References

Bayesian analysis in Stata The general idea Outline The general idea The Method Bayes rule Fundamental equation MCMC Stata tools bayesmh bayesstats ess Blocking bayesgraph bayes: prefix bayesstats ic bayestest model Random Effects Probit Thinning bayestest interval Change-point model bayesgraph matrix Summary References

Bayesian analysis in Stata The general idea Outline The general idea The Method Bayes rule Fundamental equation MCMC Stata tools bayesmh bayesstats ess Blocking bayesgraph bayes: prefix bayesstats ic bayestest model Random Effects Probit Thinning bayestest interval Change-point model bayesgraph matrix Summary References

Bayesian analysis in Stata Bayesian Analysis vs Frequentist Analysis Outline Frequentist Analysis Bayesian Analyis The general idea The Method • Results are based on • Results are based on Bayes rule Fundamental estimations for unknown fixed probability distributions about equation parameters. unknown random parameters MCMC • The data are considered to • The data are considered to Stata tools be a (hypothetical) be fixed. bayesmh bayesstats ess repeatable random sample. • The results are produced by Blocking • Uses the data to obtain bayesgraph combining the data with prior bayes: prefix estimates about the unknown beliefs about the parameters. bayesstats ic fixed parameters. • The posterior distribution is bayestest model • Depends on whether the data used to make explicit Random Effects Probit satisfies the assumptions for probabilistic statements Thinning the specified model. bayestest interval "Bayesian analysis answers "Frequentists base their Change-point questions based on the distribution model conclusions on the distribution of of parameters conditional on the bayesgraph matrix statistics derived from random observed sample." samples, assuming that the Summary parameters are unknown but References fixed."

Bayesian analysis in Stata Outline The general Some Advantages idea The Method Bayes rule • Based on the Bayes rule, which applies to all Fundamental equation MCMC parametric models. Stata tools • Inference is exact, estimation and prediction are based bayesmh bayesstats ess on posterior distribution. Blocking bayesgraph bayes: prefix • Provides more intuitive interpretation in terms of bayesstats ic bayestest model probabilities (e.g Credible intervals). Random Effects Probit • It is not limited by the sample size. Thinning bayestest interval Change-point model bayesgraph matrix Summary References

Bayesian analysis in Stata Outline The general idea The Method Some Disadvantages Bayes rule Fundamental equation MCMC • Subjectivity in specifying prior beliefs. Stata tools bayesmh • Computationally challenging. bayesstats ess Blocking • Setting up a model and performing analysis could be bayesgraph bayes: prefix an involving task. bayesstats ic bayestest model Random Effects Probit Thinning bayestest interval Change-point model bayesgraph matrix Summary References

Bayesian analysis in Stata Outline Some Examples (Taken from Hahn, 2014) The general idea The Method • TranScan Medical use small dataset and priors based Bayes rule on previous studies to determine the efficacy of its Fundamental equation 2000 device for mammografy (FDA 1999). MCMC Stata tools • homeprice.com.hk used Bayesian analysis for pricing bayesmh bayesstats ess information on over a million real state properties in Blocking bayesgraph Hong Kong and surrounding areas (Shamdasany, bayes: prefix bayesstats ic 2011). bayestest model Random • Researchers in the energy industry have used Effects Probit Thinning Bayesian analysis to understand petroleum reservoir bayestest interval parameters (Glinsky and Gunning, 2011). Change-point model bayesgraph matrix Summary References

Bayesian analysis in Stata Outline The general idea The Method Bayes rule Fundamental equation MCMC The Method Stata tools bayesmh bayesstats ess Blocking bayesgraph bayes: prefix bayesstats ic bayestest model Random Effects Probit Thinning bayestest interval Change-point model bayesgraph matrix Summary References

Bayesian analysis in Stata The Method Outline The general idea The Method • Let’s start by writing the Bayes’ Rule: Bayes rule Fundamental equation MCMC p ( A | B ) p ( B ) p ( B | A ) = Stata tools p ( A ) bayesmh bayesstats ess Blocking Where: bayesgraph bayes: prefix p ( A | B ) : conditional probability of A given B bayesstats ic bayestest model p ( B | A ) : conditional probability of B given A Random Effects Probit p ( B ) : marginal probability of B Thinning p ( A ) : marginal probability of A bayestest interval Change-point model bayesgraph matrix Summary References

Bayesian analysis in The Method Stata • If we have a probability model for a vector of Outline observations y and a vector of unknown parameters θ , The general idea we can represent the model with a likelihood function: The Method Bayes rule n Fundamental � equation L ( θ ; y ) = f ( y ; θ ) = f ( y i | θ ) MCMC i = 1 Stata tools bayesmh bayesstats ess Where: Blocking bayesgraph f ( y ; θ ) : conditional probability of y give θ bayes: prefix bayesstats ic • Let’s assume that θ has a probability distribution π ( θ ) , bayestest model Random and that denote m(y) denote the marginal distribution Effects Probit of y, such that: Thinning bayestest interval Change-point � model m ( y ) = f ( y ; θ ) π ( θ ) d θ bayesgraph matrix Summary References

Bayesian analysis in Stata The Method • Let’s now write the inverse law of probability (Bayes’ Outline Theorem): The general idea The Method f ( y ; θ ) π ( θ ) Bayes rule f ( θ | y ) = Fundamental f ( y ) equation MCMC Stata tools • But notice that the marginal distribution of y, f(y), does bayesmh not depend on ( θ ) bayesstats ess Blocking bayesgraph bayes: prefix bayesstats ic • Then, we can write the fundamental equation for bayestest model Random Bayesian analysis: Effects Probit Thinning bayestest interval p ( θ | y ) ∝ L ( y | θ ) π ( θ ) Change-point model bayesgraph matrix Summary References

Bayesian analysis in Stata Let’s go back to our initial example Outline The general idea The Method Bayes rule Fundamental equation MCMC Stata tools bayesmh bayesstats ess Blocking bayesgraph bayes: prefix bayesstats ic bayestest model Random Effects Probit Thinning bayestest interval Change-point model bayesgraph matrix Summary References

Bayesian analysis in Stata Outline The Method The general idea • In the example we have the data (the likelihood The Method component) Bayes rule Fundamental equation • We also have the experts belief (the prior component) MCMC Stata tools • Then, how do we get the posterior distribution? bayesmh bayesstats ess • We use the fundamental equation Blocking bayesgraph bayes: prefix bayesstats ic p ( θ | y ) ∝ L ( y | θ ) π ( θ ) bayestest model Random Effects Probit Thinning bayestest interval Change-point model bayesgraph matrix Summary References

Bayesian analysis in Stata The Method • Let’s assume that both, the data and the prior beliefs, Outline are normally distributed: The general idea θ, σ 2 • The data : y ∼ N � � The Method d Bayes rule � µ p , σ 2 � • The prior : θ ∼ N Fundamental p equation MCMC • Homework...: Doing the algebra with the fundamental Stata tools bayesmh equation we find that the posterior distribution would be bayesstats ess normal with: Blocking bayesgraph bayes: prefix bayesstats ic � µ, σ 2 � • The posterior : θ | y ∼ N bayestest model Random Effects Probit Where: Thinning bayestest interval σ 2 � y /σ 2 d + µ p /σ 2 N ¯ � µ = p Change-point model � − 1 σ 2 N /σ 2 d + 1 /σ 2 � = bayesgraph matrix p Summary References