Hybrid Monte Carlo: Geometric Integration and Statistics Andrew - PowerPoint PPT Presentation

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Hybrid Monte Carlo: Geometric Integration and Statistics Andrew Stuart 1 1 Mathematics Institute and Centre for Scientific Computing University of Warwick SCMS2010 Heriot-Watt, September 6 th 2010 Funded by EPSRC, ONR

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT “Stable Periodic Bifurcations of an Explicit Discretization of a Nonlinear Partial Differential Equation in Reaction Diffusion”. D.F . Griffiths and A.R.Mitchell IMA J Numerical Analysis 8 (1988), 435-454.

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Outline INVARIANT MEASURES AND DYNAMICAL SYSTEMS 1 MARKOV CHAIN MONTE CARLO 2 3 EXPLICIT DISCRETIZATIONS IMPLICIT DISCRETIZATIONS 4 CONCLUSIONS 5

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Outline INVARIANT MEASURES AND DYNAMICAL SYSTEMS 1 MARKOV CHAIN MONTE CARLO 2 3 EXPLICIT DISCRETIZATIONS IMPLICIT DISCRETIZATIONS 4 CONCLUSIONS 5

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Goals of Work To find numerical methods to sample a probability density function (pdf) π : R n → R + . To analyze and develop methods in the cases of high dimensions n ≫ 1 . Baisc building blocks are π − invariant dynamical systems and MCMC methodology.

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Langevin Stochastic Dynamics Let A be a positive-definite symmetric matrix. The Langevin SDE is √ 2 A ˙ x = A ∇ log π ( x ) + ˙ W . This equation is π − invariant: if x ( 0 ) ∼ π then x ( t ) ∼ π for all t > 0 .

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Hybrid Monte Carlo (Duane et al 1987) Let A be a positive-definite symmetric matrix. Define the Hamiltonian H ( x , p ) = 1 2 � p , Ap � − log π ( x ) . Hamiltons equations are ˙ x = Ap , p = ∇ ˙ � � log π ( x ) . Assume that p ( 0 ) ∼ N ( 0 , A − 1 ) . This equation is π − invariant: if x ( 0 ) ∼ π, then x ( t ) ∼ π for all t > 0 .

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Outline INVARIANT MEASURES AND DYNAMICAL SYSTEMS 1 MARKOV CHAIN MONTE CARLO 2 3 EXPLICIT DISCRETIZATIONS IMPLICIT DISCRETIZATIONS 4 CONCLUSIONS 5

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Metropolis-Hastings Algorithm Enforcing π − invariant dynamics via accept-reject: 1. Set k = 0 and choose x ( 0 ) ∈ R n . ξ ( k ) ∼ N ( 0 , 1 ) . 2. Propose y = G ( x ( k ) , ξ ( k ) , ∆ t ) , 3. Set x ( k + 1 ) = y with probability α ; else x ( k + 1 ) = x ( k ) . 4. Set k → k + 1 and goto 2. Step 3. is the proposal: how do we choose it?

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Choice of Parameters We choose G to be a time-discretization of one of the invariant dynamical systems. We want largest ∆ t compatible with O ( 1 ) average acceptance probability for n ≫ 1 . Choose ∆ t = n − γ Courant condition. � � γ 0 = min γ ≥ 0 γ : lim inf n →∞ E α > 0 . Number of steps required to adequately sample π is then M ( n ) = O (∆ t ) − 1 = O ( n γ 0 ) .

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Structure of the Target IID Product in R n π 0 ( x ) = Π n i = 1 f ( x i ) . Change of Measure From Gaussian in R n � � − Φ n ( x ) π ( x ) = exp π 0 ( x ) − 1 � � 2 � x , C − 1 π 0 ( x ) ∝ exp 0 x � . The covariance matrix C 0 is assumed to have condition number O ( n 2 k ) . The resulting Gaussian part is assumed to dominate Φ n , uniformly in n .

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Outline INVARIANT MEASURES AND DYNAMICAL SYSTEMS 1 MARKOV CHAIN MONTE CARLO 2 3 EXPLICIT DISCRETIZATIONS IMPLICIT DISCRETIZATIONS 4 CONCLUSIONS 5

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Langevin 1 π 0 ( x ) = Π n i = 1 f ( x i ) . Recall that the SDE √ 2 ˙ ˙ x = ∇ log π 0 ( x ) + W is π 0 − invariant. We use the following discretization as proposal: � y − x 2 Proposal = β ∇ log π 0 ( x ) + ∆ t ξ, ξ ∼ N ( 0 , I ) . ∆ t Theorem 1. (Roberts et al 97, Roberts/Rosenthal 98) β = 0 then M ( n ) = O ( n 1 ) . β = 1 then M ( n ) = O ( n 1 / 3 ) . Steepest Descents Impacts Cost

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Langevin 2 − Φ n ( x ) − 1 � � � � 2 � x , C − 1 π ( x ) = exp − Φ n ( x ) π 0 ( x ) = exp 0 x � . � y − x 2 A Proposal = A ∇ log π 0 ( x ) + ∆ t ξ, ξ ∼ N ( 0 , I ) . ∆ t Theorem 2. (Beskos, Roberts, Stuart 2009) A = I then M ( n ) = O ( n ( 2 k + 1 / 3 ) ) . A = C 0 then M ( n ) = O ( n 1 / 3 ) . Preconditioning Impacts Cost

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Hybrid Monte Carlo 1 The key dynamical system is: ˙ x = Ap , p = ∇ ˙ � � log π ( x ) . Volume preserving reversible integration is required to ensure that the acceptance probability α is tractable. This can be achieved by operator splitting (eg Verlet) based on the two dynamical systems � ˙ ˙ x = Ap , x = 0 , � � � ˙ p = ∇ ˙ � � p = 0 . log π ( x ) . � �

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Hybrid Monte Carlo 2 π 0 ( x ) = Π n i = 1 f ( x i ) . Theorem 3. (Beskos, Pillai, Roberts, Sanz-Serna and Stuart 2010) For Verlet integration within Hybrid Monte Carlo we have M ( n ) = O ( n 1 / 4 ) . Hamiltonian Formulation Impacts Cost

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Outline INVARIANT MEASURES AND DYNAMICAL SYSTEMS 1 MARKOV CHAIN MONTE CARLO 2 3 EXPLICIT DISCRETIZATIONS IMPLICIT DISCRETIZATIONS 4 CONCLUSIONS 5

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Langevin − Φ n ( x ) − 1 � � 2 � x , C − 1 π ( x ) = exp n x � . � y − x 2 A � � θ C − 1 n y + ( 1 − θ ) C − 1 + A n x = ∆ t ξ, ξ ∼ N ( 0 , I ) . ∆ t Theorem 4. (Beskos, Roberts, Stuart 2009) θ � = 1 2 and A = C n then M ( n ) = O ( n 1 / 3 ) . θ = 1 2 and A = I , C n then M ( n ) = O ( 1 ) . Implicitness Impacts Cost

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Hybrid Monte Carlo 1 The key dynamical system is: ˙ x = Ap , p = −C − 1 ˙ n x − ∇ Φ n ( x ) . Volume preserving reversible integration is required to ensure that the acceptance probability α is tractable. This can be achieved by operator splitting based on the two dynamical systems � ˙ ˙ x = Ap , x = 0 , � � � p = −C − 1 ˙ ˙ n x . p = −∇ Φ n ( x ) . � �

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Hybrid Monte Carlo 2 If we use a second-order (Strang-splitting) for this operator-split then we obtain: Theorem 4. (Beskos, Pinski, Sanz-Serna and Stuart 2010) If A = C n then M ( n ) = O ( 1 ) . Implicitness Impacts Cost

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT Outline INVARIANT MEASURES AND DYNAMICAL SYSTEMS 1 MARKOV CHAIN MONTE CARLO 2 3 EXPLICIT DISCRETIZATIONS IMPLICIT DISCRETIZATIONS 4 CONCLUSIONS 5

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT What We Have Shown We have shown that the following ideas from numerical analysis have direct impact on MCMC based statistical sampling methods in high dimensions: Steepest descents Preconditioning Implicit integration for dissipative systems Geometric integration

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT References For all papers see: http : // www . maths . warwick . ac . uk / ∼ masdr / sample . html A. Beskos, N. Pillai, G.O. Roberts, J.-M. Sanz-Serna and A.M. Stuart. “Optimal tuning of hybrid Monte Carlo”. Submitted. A. Beskos, F . Pinski, J.-M. Sanz-Serna and A.M. Stuart. “Hybrid Monte Carlo on Hilbert spaces”. Submitted. A. Beskos, G.O. Roberts and A.M. Stuart. ”Optimal scalings for local Metropolis-Hastings chains on non-product targets in high dimensions.” Ann. Appl. Prob. 19(2009), 863–898.

INVARIANT MEASURES AND DYNAMICAL SYSTEMS MARKOV CHAIN MONTE CARLO EXPLICIT DISCRETIZATIONS IMPLICIT References (Continued) A. Beskos and A.M. Stuart. ”MCMC Methods for Sampling Function Space”. To appear, proceedings of ICIAM 2007. M. Hairer, A.M.Stuart and J. Voss. ”Sampling the posterior: an approach to non-Gaussian data assimilation.” PhysicaD, 230 (2007), 50–64. S. Duane, A.D. Kennedy, B.J. Pendelton and D. Roweth. “Hybrid Monte Carlo.” Physics Letters B, 195 (1987), 216-222.