Approximate Bayesian Computation using Auxiliary Models Tony Pettitt Co-authors Chris Drovandi, Malcolm Faddy Queensland University of Technology Brisbane MCQMC February 2012 Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 1 / 32
Outline Motivating Problem 1 Application to modelling Macroparasite Immunity Approximate Bayesian Computation 2 Introduction to ABC Three ABC Algorithms Sequential Monte Carlo ABC Macroparasite Population Evolution 3 Summary Statistics Auxiliary models Results 4 Posterior results ABC fits to Data Conclusions 5 Macroparasite model ABC Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 2 / 32
Motivating Problem Motivating Problem 1 Application to modelling Macroparasite Immunity Approximate Bayesian Computation 2 Introduction to ABC Three ABC Algorithms Sequential Monte Carlo ABC Macroparasite Population Evolution 3 Summary Statistics Auxiliary models Results 4 Posterior results ABC fits to Data Conclusions 5 Macroparasite model ABC Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 3 / 32
Motivating Problem Application to modelling Macroparasite Immunity Macroparasite Immunity Estimate parameters of a Markov process model explaining macroparasite population development with host immunity 212 hosts (cats) i = 1 , . . . , 212. Each cat injected with l i juvenile Brugia pahangi larvae (approximately 100 or 200). At time t i host is sacrificed and the number of mature worms are recorded Host assumed to develop an immunity Three discrete variables: M ( t ) matures, L ( t ) juveniles, I ( t ) immunity. Only L (0) and M at sacrifice time are observed for each host Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 4 / 32
Motivating Problem Application to modelling Macroparasite Immunity Macroparasite Immunity data, proportion of mature vs sacrifice time 1 0.9 0.8 0.7 Proporton of Matures 0.6 0.5 0.4 0.3 0.2 0.1 0 0 200 400 600 800 1000 1200 Time Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 5 / 32
Motivating Problem Application to modelling Macroparasite Immunity Trivariate Markov Process of Riley et al (2003) Invisible Invisible Mature Parasites Juvenile Parasites Maturation M(t) L(t) γL ( t ) Natural death Death due to immunity Natural death µ M M ( t ) µ L L ( t ) βI ( t ) L ( t ) Immunity Gain of immunity Loss of immunity I(t) νL ( t ) µ I I ( t ) Invisible Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 6 / 32
Motivating Problem Application to modelling Macroparasite Immunity The Model and Intractable Likelihood L , M , I are discrete counts, I hypothesised variable Deterministic form of the model dL dt = − µ L L − β IL − γ L , dM dt = γ L − µ M M , dI dt = ν L − µ I I , µ m , γ fixed. ν, µ L , µ I , β require estimation Likelihood based on Markov process is intractable Simulation of process L , M , I using Gillespie’s algorithm (Gillespie, 1977) A common mathematical model: epidemics, chemical kinetics, systems biology.... Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 7 / 32
Approximate Bayesian Computation Introduction to ABC Motivating Problem 1 Application to modelling Macroparasite Immunity Approximate Bayesian Computation 2 Introduction to ABC Three ABC Algorithms Sequential Monte Carlo ABC Macroparasite Population Evolution 3 Summary Statistics Auxiliary models Results 4 Posterior results ABC fits to Data Conclusions 5 Macroparasite model ABC Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 8 / 32
Approximate Bayesian Computation Introduction to ABC ABC and the Approximate Posterior I Bayesian statistics involves inference based on the posterior distribution p ( θ | y ) ∝ p ( y | θ ) p ( θ ) . and if the likelihood cannot be evaluated? but easy to simulate, x , from the likelihood Applications... genetics, biology, finance, ... Involves a joint posterior distribution for θ and simulated data x p ( θ, x | y ) ∝ g ( y | x ) p ( x | θ ) p ( θ ) where g ( y | x ) measures closeness of observed data y to simulated data x . (Reeves and Pettitt, 2005) Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 9 / 32
Approximate Bayesian Computation Introduction to ABC ABC and the Approximate Posterior II Compare simulated values, x , and observed data, y through summary statistics S ( . ) = S 1 ( . ) , . . . , S p ( . ) ρ ( y , x ) = � S ( y ) − S ( x ) � One choice, set g ( y | x ) = 1( ρ ( y , x ) ≤ ǫ ) Choice of ǫ trade off between accuracy and computational time Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 10 / 32
Approximate Bayesian Computation Three ABC Algorithms Rejection Sampling ABC - RS-ABC Rejection Sampling (RS-ABC) (Beaumont et al, 2002) Sample θ ∗ ∼ p ( θ ) Simulate x ∼ p ( . | θ ∗ ) Accept θ ∗ if ρ ( y , x ) ≤ ǫ Repeat the above until we have N values, θ 1 , . . . , θ N Advantages: Simplicity, Independent values, parallelizable Disadvantage: Inefficient. Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 11 / 32
Approximate Bayesian Computation Three ABC Algorithms MCMC ABC Majoram et al, 2003 Advantages: theoretical understanding, use in SMC Disadvantages: Dependent Samples, Markov chain convergence, sampler can get stuck, inefficient, needs tuning Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 12 / 32
Approximate Bayesian Computation Sequential Monte Carlo ABC Sequential Monte Carlo for ABC Chopin(2002), Del Moral et al (2006),Sisson et al (2007, 2009), Beaumont et al (2009) Approximate posterior by weighted sample { θ i , W i } N i =1 , N particles Define sequence of joint targets p t ( θ, x | y ) ∝ p ( x | θ ) p ( θ )1( ρ ( x , y ) ≤ ǫ t ) , for t = 1 , . . . , T , and a sequence of decreasing tolerances ǫ 1 ≥ ǫ 2 ≥ · · · ≥ ǫ T . At t = 1 draw particles from prior, set ǫ 1 = ∞ Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 13 / 32
Approximate Bayesian Computation Sequential Monte Carlo ABC A Fully Adaptive SMC ABC Algorithm I Drovandi and Pettitt (2011), Del Moral et al (2011) Reweight particles, either zero or proportional to 1, 1( ρ ( x i t , y ) ≤ ǫ t ) W i t ∝ t , y ) ≤ ǫ t − 1 ) , 1( ρ ( x i Choose ǫ t so that M have zero weights and N − M have non-zero weights Replenish population by resampling M from N − M ‘alive’ particles. Diversify the particles with an MCMC kernel, q t ( . | . ), that is stationary for the current target. q t ( . | . ) determined adaptively using the ‘alive’ set. Apply MCMC kernel R t times so that overall acceptance close to 1. Learn R t adaptively . Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 14 / 32
Macroparasite Population Evolution Motivating Problem 1 Application to modelling Macroparasite Immunity Approximate Bayesian Computation 2 Introduction to ABC Three ABC Algorithms Sequential Monte Carlo ABC Macroparasite Population Evolution 3 Summary Statistics Auxiliary models Results 4 Posterior results ABC fits to Data Conclusions 5 Macroparasite model ABC Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 15 / 32
Macroparasite Population Evolution Macroparasite Population Evolution 1 0.9 0.8 0.7 Proporton of Matures 0.6 0.5 0.4 0.3 0.2 0.1 0 0 200 400 600 800 1000 1200 Time Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 16 / 32
Macroparasite Population Evolution Summary Statistics Developing Summary Statistics Nunes and Balding (2009) For all sets and subsets of summary statistics, carry out AS-ABC for a fixed acceptance rate Compare ABC approximations in terms of concentration of posterior distribution. Use a non-parametric measure of entropy Fearnhead and Prangle (2012) suitable for iid cases Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 17 / 32
Macroparasite Population Evolution Summary Statistics Developing Summary Statistics using Indirect Inference Summary statistics that efficiently summarize data! Different numbers of juveniles L , different sacrifice times. An approach based on indirect inference (Gouri´ eroux and Ronchetti, 1993) Propose an auxiliary model p a ( y | θ a ) where parameter θ a is easily estimated (eg easy MLE) Auxiliary model is flexible enough to provide a good description of the data Simulate x θ from target intractable likelihood p ( •| θ ) and find ˆ θ a ( x θ ) Estimate θ using ˆ θ a ( x θ ) closest to ˆ θ a ( y ) Alternative to ABC Estimates of parameters of the auxiliary model fitted to the data become the summary statistics in ABC. Models based on Beta-Binomial to capture variability Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 18 / 32
Macroparasite Population Evolution Summary Statistics ABC Summary Statistics from auxiliary models How to choose between different auxiliary models? Either use data analytical tools for the original data set, eg AIC Or use the Nunes and Balding approach based on ABC approximations for each model/ summary statistic choice Former is far less computer intensive but does not consider the ABC approximation Compare and contrast different Beta Binomial models Models fitted using MLE and AIC used to the rank models The models range over about 33 units of AIC How are the different fits (AIC) reflected in the ABC posteriors? Tony Pettitt () ABC using Auxiliary Models MCQMC February 2012 19 / 32
Recommend
More recommend
