Sequential Monte Carlo Dr. Jarad Niemi STAT 615 - Iowa State - PowerPoint PPT Presentation

Sequential Monte Carlo Dr. Jarad Niemi STAT 615 - Iowa State University October 20, 2017 Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 1 / 72

Overview Outline Outline 1. State-space models 3. State and parameter inference p ( y | θ, ψ ) p ( θ | ψ ) p ( θ, ψ | y ) Definition Bootstrap filter Terminology Kernel density Notation Sufficient statistics 2. State inference p ( θ | y, ψ ) 4. Advanced SMC Exact inference SMC-MCMC Importance sampling Sequential importance Fixed parameter sampling SMC for marginal likelihood Bootstrap filter - resampling calculations Auxiliary particle filter Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 2 / 72

Overview Bayesian inference Definition Bayes’ rule is P ( A | B ) = P ( B | A ) P ( A ) . P ( B ) In this rule, B represents what we know about the world and A represents what we don’t. Suppose p ( θ t , ψ | y 1: t − 1 ) is our current knowledge about the state of the world. We observe datum y t then p ( θ t , ψ | y 1: t ) = p ( y t | θ t , ψ ) p ( θ t , ψ | y 1: t − 1 ) . p ( y t | y 1: t − 1 ) where y 1: t = ( y 1 , y 2 , . . . , y t ) . Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 3 / 72

State-space models Definition Definition A state-space model can be described by these conditional distributions: an observation equation: p o ( y t | θ t , ψ ) , an evolution equation: p e ( θ t | θ t − 1 , ψ ) , and a prior p ( θ 0 , ψ ) . where y t : an observation vector of length m θ t : a latent state vector of length p ψ : a fixed parameter vector of length q Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 4 / 72

State-space models Graphical representation Graphical representation y t − 1 y t − 1 y t − 1 y t y t y t y t + 1 y t + 1 y t + 1 θ t − 1 θ t − 1 θ t − 1 θ t θ t θ t θ t + 1 θ t + 1 θ t + 1 p ( θ t | θ t − 1 , ψ ) p ( y t | θ t , ψ ) Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 5 / 72

State-space models Interpretation Interpretation Model State interpretation Local level model True level Linear growth model True level and slope Seasonal factor model Seasonal effect Dynamic regression Time-varying regression coefficients Stochastic volatility Underlying volatility in the market Markov switching model Influenza epidemic on/off Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 6 / 72

State-space models Examples Stochastic volatility 10 5 observation 0 −5 −10 value 10 5 volatility 0 −5 −10 0 250 500 750 1000 time Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 7 / 72

State-space models Examples Markov switching model 3 2 observation 1 0 −1 value 3 2 1 state 0 −1 0 250 500 750 1000 time Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 8 / 72

State-space models Inference Inference Definition The state filtering distribution is the distribution for the state conditional on all observations up to and including time t , i.e. p ( θ t | y 1: t , ψ ) = p ( θ t | y 1 , y 2 , . . . , y t , ψ ) . Definition The state smoothing distribution is the distribution for the state conditional on all observed data, i.e. p ( θ t | y 1: T , ψ ) = p ( θ t | y 1 , y 2 , . . . , y T , ψ ) where t < T . Definition The state forecasting distribution is the distribution for future states conditional on all observed data, i.e. p ( θ T + k | y 1: T , ψ ) = p ( θ T + k | y 1 , y 2 , . . . , y T , ψ ) where k > 0 . Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 9 / 72

State-space models Inference y t − 1 y t y t + 1 θ t − 1 θ t θ t + 1 Filtering Smoothing Forecasting Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 10 / 72

State-space models Filtering Filtering Goal: p ( θ t | y 1: t ) (filtered distribution) Recursive procedure: Assume p ( θ t − 1 | y 1: t − 1 ) Prior for θ t � p ( θ t | y 1: t − 1 ) = p ( θ t , θ t − 1 | y 1: t − 1 ) dθ t − 1 � = p ( θ t | θ t − 1 , y 1: t − 1 ) p ( θ t − 1 | y 1: t − 1 ) dθ t − 1 � = p ( θ t | θ t − 1 ) p ( θ t − 1 | y 1: t − 1 ) dθ t − 1 One-step ahead predictive distribution for y t � p ( y t | y 1: t − 1 ) = p ( y t , θ t | y 1: t − 1 ) dθ t � = p ( y t | θ t , y 1: t − 1 ) p ( θ t | y 1: t − 1 ) dθ t � = p ( y t | θ t ) p ( θ t | y 1: t − 1 ) dθ t Filtered distribution for θ t p ( y t | θ t , y 1: t − 1 ) p ( θ t | y 1: t − 1 ) p ( y t | θ t ) p ( θ t | y 1: t − 1 ) p ( θ t | y 1: t ) = = p ( y t | y 1: t − 1 ) p ( y t | y 1: t − 1 ) Start from p ( θ 0 ) . Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 11 / 72

State-space models Smoothing Smoothing Goal: p ( θ t | y 1: T ) for t < T Backward transition probability p ( θ t | θ t +1 , y 1: T ) p ( θ t | θ t +1 , y 1: T ) = p ( θ t | θ t +1 , y 1: t ) p ( θ t +1 | θ t , y 1: t ) p ( θ t | y 1: t ) = p ( θ t +1 | y 1: t ) p ( θ t +1 | θ t ) p ( θ t | y 1: t ) = p ( θ t +1 | y 1: t ) Recursive smoothing distributions p ( θ t | y 1: T ) assuming we know p ( θ t +1 | y 1: T ) � p ( θ t | y 1: T ) = p ( θ t , θ t +1 | y 1: T ) dθ t +1 � = p ( θ t +1 | y 1: T ) p ( θ t | θ t +1 , y 1: T ) dθ t +1 � p ( θ t +1 | y 1: T ) p ( θ t +1 | θ t ) p ( θ t | y 1: t ) = dθ t +1 p ( θ t +1 | y 1: t ) � p ( θ t +1 | θ t ) = p ( θ t | y 1: t ) p ( θ t +1 | y 1: t ) p ( θ t +1 | y 1: T ) dθ t +1 Start from p ( θ T | y 1: T ) . Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 12 / 72

State-space models Forecasting Forecasting Goal: p ( y T + k , θ T + k | y 1: T ) p ( y T + k , θ T + k | y 1: T ) = p ( y T + k | θ T + k ) p ( θ T + k | y 1: T ) Recursively, given p ( θ T +( k − 1) | y 1: T ) � p ( θ T + k | y 1: T ) = p ( θ T + k , θ T +( k − 1) | y 1: T ) dθ T +( k − 1) � = p ( θ T + k | θ T +( k − 1) , y 1: T ) p ( θ T +( k − 1) | y 1: T ) dθ T +( k − 1) � = p ( θ T + k | θ T +( k − 1) ) p ( θ T +( k − 1) | y 1: T ) dθ T +( k − 1) Start with k = 1 . Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 13 / 72

State inference Outline 1. State-space models 3. State and parameter inference p ( y | θ, ψ ) p ( θ | ψ ) p ( θ, ψ | y ) Definition Bootstrap filter Terminology Kernel density Notation Sufficient statistics 2. State inference p ( θ | y, ψ ) 4. Advanced SMC Exact inference SMC-MCMC Importance sampling Sequential importance Fixed parameter sampling SMC for marginal likelihood Bootstrap filter - resampling calculations Auxiliary particle filter Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 14 / 72

State inference Exact inference Exact inference Our goal for most of today is to find filtering methods. We assume p ( θ t − 1 | y 1: t − 1 ) is known and try to obtain p ( θ t | y 1: t ) using p ( θ t | θ t − 1 ) and p ( y t | θ t ) . Then, starting with p ( θ 0 | y 0 ) = p ( θ 0 ) we can find p ( θ t | y 1: t ) for all t . Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 15 / 72

State inference Exact inference There are two important state-space models when the filtering updating is availably analytically: Hidden Markov models Dynamic linear models Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 16 / 72

State inference Exact inference Hidden Markov models Definition A hidden Markov model (HMM) is a state-space model with an arbitrary observation equation and an evolution equation that can be represented by a transition probability matrix, i.e. p ( θ t = j | θ t − 1 = i ) = p ij . Filtering in HMMs Suppose we have a HMM with p states. Let q i = p ( θ t − 1 = i | y 1: t − 1 ) , then p � p ( θ t = j | y 1: t − 1 ) = q i p ij i =1 p ( θ t = j | y 1: t ) ∝ p ( y t | θ t = j ) p ( θ t = j | y 1: t − 1 ) . a i If p i ∝ a i for i ∈ { 1 , 2 , . . . , p } , then p i = i =1 a i . � p Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 17 / 72

State inference Exact inference Definition A dynamic linear model (DLM) is a state-space model where both the observation and evolution equations are linear in the states and have additive Gaussian errors and the prior is Gaussian, i.e. y t = F t θ t + v t v t ∼ N (0 , V t ) θ t = G t θ t − 1 + w t w t ∼ N (0 , W t ) θ 0 ∼ N ( m 0 , C 0 ) where v t , w t , and θ 0 are independent of each other and mutually independent through time. Kalman filter Kalman smoother Jarad Niemi (STAT615@ISU) Sequential Monte Carlo October 20, 2017 18 / 72