ABC Methods for Bayesian Model Choice ABC Methods for Bayesian Model Choice Christian P. Robert Universit´ e Paris-Dauphine, IuF, & CREST http://www.ceremade.dauphine.fr/~xian Bayes-250, Edinburgh, September 6, 2011
ABC Methods for Bayesian Model Choice Approximate Bayesian computation Approximate Bayesian computation Approximate Bayesian computation ABC for model choice Gibbs random fields Generic ABC model choice
ABC Methods for Bayesian Model Choice Approximate Bayesian computation Regular Bayesian computation issues When faced with a non-standard posterior distribution π ( θ | y ) ∝ π ( θ ) L ( θ | y ) the standard solution is to use simulation (Monte Carlo) to produce a sample θ 1 , . . . , θ T from π ( θ | y ) (or approximately by Markov chain Monte Carlo methods) [Robert & Casella, 2004]
ABC Methods for Bayesian Model Choice Approximate Bayesian computation Untractable likelihoods Cases when the likelihood function f ( y | θ ) is unavailable and when the completion step � f ( y | θ ) = f ( y , z | θ ) d z Z is impossible or too costly because of the dimension of z c � MCMC cannot be implemented!
ABC Methods for Bayesian Model Choice Approximate Bayesian computation Untractable likelihoods � MCMC cannot be implemented! c
ABC Methods for Bayesian Model Choice Approximate Bayesian computation The ABC method Bayesian setting: target is π ( θ ) f ( x | θ )
ABC Methods for Bayesian Model Choice Approximate Bayesian computation The ABC method Bayesian setting: target is π ( θ ) f ( x | θ ) When likelihood f ( x | θ ) not in closed form, likelihood-free rejection technique:
ABC Methods for Bayesian Model Choice Approximate Bayesian computation The ABC method Bayesian setting: target is π ( θ ) f ( x | θ ) When likelihood f ( x | θ ) not in closed form, likelihood-free rejection technique: ABC algorithm For an observation y ∼ f ( y | θ ) , under the prior π ( θ ) , keep jointly simulating θ ′ ∼ π ( θ ) , z ∼ f ( z | θ ′ ) , until the auxiliary variable z is equal to the observed value, z = y . [Tavar´ e et al., 1997]
ABC Methods for Bayesian Model Choice Approximate Bayesian computation A as approximative When y is a continuous random variable, equality z = y is replaced with a tolerance condition, ̺ ( y , z ) ≤ ǫ where ̺ is a distance
ABC Methods for Bayesian Model Choice Approximate Bayesian computation A as approximative When y is a continuous random variable, equality z = y is replaced with a tolerance condition, ̺ ( y , z ) ≤ ǫ where ̺ is a distance Output distributed from π ( θ ) P θ { ̺ ( y , z ) < ǫ } ∝ π ( θ | ̺ ( y , z ) < ǫ )
ABC Methods for Bayesian Model Choice Approximate Bayesian computation ABC algorithm Algorithm 1 Likelihood-free rejection sampler for i = 1 to N do repeat generate θ ′ from the prior distribution π ( · ) generate z from the likelihood f ( ·| θ ′ ) until ρ { η ( z ) , η ( y ) } ≤ ǫ set θ i = θ ′ end for where η ( y ) defines a (maybe in-sufficient) statistic
ABC Methods for Bayesian Model Choice Approximate Bayesian computation Output The likelihood-free algorithm samples from the marginal in z of: π ( θ ) f ( z | θ ) I A ǫ, y ( z ) � π ǫ ( θ, z | y ) = A ǫ, y × Θ π ( θ ) f ( z | θ ) d z d θ , where A ǫ, y = { z ∈ D| ρ ( η ( z ) , η ( y )) < ǫ } .
ABC Methods for Bayesian Model Choice Approximate Bayesian computation Output The likelihood-free algorithm samples from the marginal in z of: π ( θ ) f ( z | θ ) I A ǫ, y ( z ) � π ǫ ( θ, z | y ) = A ǫ, y × Θ π ( θ ) f ( z | θ ) d z d θ , where A ǫ, y = { z ∈ D| ρ ( η ( z ) , η ( y )) < ǫ } . The idea behind ABC is that the summary statistics coupled with a small tolerance should provide a good approximation of the posterior distribution: � π ǫ ( θ | y ) = π ǫ ( θ, z | y ) d z ≈ π ( θ | y ) .
ABC Methods for Bayesian Model Choice Approximate Bayesian computation MA example Consider the MA ( q ) model q � x t = ǫ t + ϑ i ǫ t − i i =1 Simple prior: uniform prior over the identifiability zone, e.g. triangle for MA (2)
ABC Methods for Bayesian Model Choice Approximate Bayesian computation MA example (2) ABC algorithm thus made of 1. picking a new value ( ϑ 1 , ϑ 2 ) in the triangle 2. generating an iid sequence ( ǫ t ) − q<t ≤ T 3. producing a simulated series ( x ′ t ) 1 ≤ t ≤ T
ABC Methods for Bayesian Model Choice Approximate Bayesian computation MA example (2) ABC algorithm thus made of 1. picking a new value ( ϑ 1 , ϑ 2 ) in the triangle 2. generating an iid sequence ( ǫ t ) − q<t ≤ T 3. producing a simulated series ( x ′ t ) 1 ≤ t ≤ T Distance: basic distance between the series � T ρ (( x ′ ( x t − x ′ t ) 2 t ) 1 ≤ t ≤ T , ( x t ) 1 ≤ t ≤ T ) = t =1 or between summary statistics like the first q autocorrelations T � τ j = x t x t − j t = j +1
ABC Methods for Bayesian Model Choice Approximate Bayesian computation Comparison of distance impact Evaluation of the tolerance on the ABC sample against both distances ( ǫ = 100% , 10% , 1% , 0 . 1% ) for an MA (2) model
ABC Methods for Bayesian Model Choice Approximate Bayesian computation Comparison of distance impact 4 1.5 3 1.0 2 0.5 1 0.0 0 0.0 0.2 0.4 0.6 0.8 −2.0 −1.0 0.0 0.5 1.0 1.5 θ 1 θ 2 Evaluation of the tolerance on the ABC sample against both distances ( ǫ = 100% , 10% , 1% , 0 . 1% ) for an MA (2) model
ABC Methods for Bayesian Model Choice Approximate Bayesian computation Comparison of distance impact 4 1.5 3 1.0 2 0.5 1 0.0 0 0.0 0.2 0.4 0.6 0.8 −2.0 −1.0 0.0 0.5 1.0 1.5 θ 1 θ 2 Evaluation of the tolerance on the ABC sample against both distances ( ǫ = 100% , 10% , 1% , 0 . 1% ) for an MA (2) model
ABC Methods for Bayesian Model Choice ABC for model choice ABC for model choice Approximate Bayesian computation ABC for model choice Gibbs random fields Generic ABC model choice
ABC Methods for Bayesian Model Choice ABC for model choice Bayesian model choice Principle Several models M 1 , M 2 , . . . are considered simultaneously for dataset y and model index M central to inference. Use of a prior π ( M = m ) , plus a prior distribution on the parameter conditional on the value m of the model index, π m ( θ m ) Goal is to derive the posterior distribution of M , a challenging computational target when models are complex.
ABC Methods for Bayesian Model Choice ABC for model choice Generic ABC for model choice Algorithm 2 Likelihood-free model choice sampler (ABC-MC) for t = 1 to T do repeat Generate m from the prior π ( M = m ) Generate θ m from the prior π m ( θ m ) Generate z from the model f m ( z | θ m ) until ρ { η ( z ) , η ( y ) } < ǫ Set m ( t ) = m and θ ( t ) = θ m end for [Toni, Welch, Strelkowa, Ipsen & Stumpf, 2009]
ABC Methods for Bayesian Model Choice ABC for model choice ABC estimates Posterior probability π ( M = m | y ) approximated by the frequency of acceptances from model m T � 1 I m ( t ) = m . T t =1 Early issues with implementation: ◮ should tolerances ǫ be the same for all models? ◮ should summary statistics vary across models? incl. their dimension? ◮ should the distance measure ρ vary across models?
ABC Methods for Bayesian Model Choice ABC for model choice ABC estimates Posterior probability π ( M = m | y ) approximated by the frequency of acceptances from model m T � 1 I m ( t ) = m . T t =1 Early issues with implementation: ◮ ǫ then needs to become part of the model ◮ Varying statistics incompatible with Bayesian model choice proper ◮ ρ then part of the model Extension to a weighted polychotomous logistic regression estimate of π ( M = m | y ) , with non-parametric kernel weights [Cornuet et al., DIYABC, 2009]
ABC Methods for Bayesian Model Choice ABC for model choice The great ABC controversy On-going controvery in phylogeographic genetics about the validity of using ABC for testing Against: Templeton, 2008, 2009, 2010a, 2010b, 2010c, &tc argues that nested hypotheses cannot have higher probabilities than nesting hypotheses (!)
ABC Methods for Bayesian Model Choice ABC for model choice The great ABC controversy On-going controvery in phylogeographic genetics about the validity of using ABC for testing Replies: Fagundes et al., 2008, Against: Templeton, 2008, Beaumont et al., 2010, Berger et 2009, 2010a, 2010b, 2010c, &tc al., 2010, Csill` ery et al., 2010 argues that nested hypotheses point out that the criticisms are cannot have higher probabilities addressed at [Bayesian] than nesting hypotheses (!) model-based inference and have nothing to do with ABC...
ABC Methods for Bayesian Model Choice Gibbs random fields Potts model Potts model � c ∈ C V c ( y ) is of the form � � V c ( y ) = θS ( y ) = θ δ y l = y i c ∈ C l ∼ i where l ∼ i denotes a neighbourhood structure
ABC Methods for Bayesian Model Choice Gibbs random fields Potts model Potts model � c ∈ C V c ( y ) is of the form � � V c ( y ) = θS ( y ) = θ δ y l = y i c ∈ C l ∼ i where l ∼ i denotes a neighbourhood structure In most realistic settings, summation � exp { θ T S ( x ) } Z θ = x ∈X involves too many terms to be manageable and numerical approximations cannot always be trusted [Cucala et al., 2009]
Recommend
More recommend
