Fluid Limits for some MCMC samplers Gersende FORT, CNRS, Paris, - PowerPoint PPT Presentation

Fluid Limits for some MCMC samplers ◮ Gersende FORT, CNRS, Paris, France. Joint work with ◮ Sean MEYN (Univ. of Illinois, Urbana, USA), ◮ Eric MOULINES (GET, France), ◮ Pierre PRIOURET (University Paris VI, France).

Outline of the talk We are interested in ◮ the existence + stability of the fluid limits for skip free Markov Chains. ◮ their use in the study of (some) MCMC samplers.

Outline of the talk We are interested in ◮ the existence + stability of the fluid limits for skip free Markov Chains. ◮ their use in the study of (some) MCMC samplers. We will discuss 1. Fluid Limits for skip-free Markov Chains. ◮ the existence of fluid limits ◮ their characterization ◮ their stability and the stability of the Markov Chain. 2. Applications to Metropolis-Hastings Markov Chains ◮ Convergence of the samplers ◮ How to tune the parameters ?

MCMC samplers / Hastings-Metropolis Sample from a (complex, unnormalized) distribution π on R d when exact sampling is not possible : Define a Markov Chain (Φ n , n ≥ 0) , with unique stationary distribution ∝ π and ergodic .

MCMC samplers / Hastings-Metropolis Sample from a (complex, unnormalized) distribution π on R d when exact sampling is not possible : Define a Markov Chain (Φ n , n ≥ 0) , with unique stationary distribution ∝ π and ergodic . Ex. Hastings-Metropolis algorithm Given Φ t , define Φ t +1 by · Φ t +1 / 2 ∼ Q (Φ t , · ) . � Φ t +1 / 2 with prob. α (Φ t , Φ t +1 / 2 ) · Φ t +1 = with prob. 1 − α (Φ t , Φ t +1 / 2 ) , Φ t α ( x, z ) = 1 ∧ π ( z ) Q ( z,x ) where π ( x ) Q ( x,z ) .

MCMC samplers / Hastings-Metropolis Problems : ◮ ( ⋆ ) Convergence ? (ergodicity) κ ( n ) | E x [ g (Φ n )] − π ( g ) | → 0 ∀ x, g ∈ ? ◮ Limit Theorems n n 1 � � n − 1 { g (Φ k ) − π ( g ) } → d N (0 , σ 2 g (Φ k ) → a . s . π ( g ) √ n g ) . k =1 k =1 ◮ ( ⋆ ) How to tune the parameters i.e. (here) the proposal kernel Q ( x, y )

MCMC samplers / Hastings-Metropolis Problems : ◮ ( ⋆ ) Convergence ? (ergodicity) κ ( n ) | E x [ g (Φ n )] − π ( g ) | → 0 ∀ x, g ∈ ? ◮ Limit Theorems n n 1 � � n − 1 { g (Φ k ) − π ( g ) } → d N (0 , σ 2 g (Φ k ) → a . s . π ( g ) √ n g ) . k =1 k =1 ◮ ( ⋆ ) How to tune the parameters i.e. (here) the proposal kernel Q ( x, y ) Hereafter, illustrations in the case · symmetric HM : Q ( x, y ) = q ( | x − y | ) · q ( z ) ∼ σ N d (0 , I )[ z ]

Existence of fluid limits (a) ֒ → Define a normalized process (i) in the initial point η r (0; x ) = 1 r Φ 0 = x, Φ 0 = rx. (ii) in time and space η r ( t ; x ) = 1 r Φ ⌊ tr ⌋ , � k � η r ( t ; x ) = 1 r ; ( k + 1) r Φ k on . r

Existence of fluid limits (a) ֒ → Define a normalized process (i) in the initial point η r (0; x ) = 1 r Φ 0 = x, Φ 0 = rx. (ii) in time and space η r ( t ; x ) = 1 r Φ ⌊ tr ⌋ , � k � η r ( t ; x ) = 1 r ; ( k + 1) r Φ k on . r ◮ Distributions · P x : distribution of the Markov Chain with initial distribution δ x . · Q r ; x : image prob. of P x by η r ( · ; x ) prob. on the space of ag functions R + → X. c` ad-l`

Existence of fluid limits (b) efinition : Q x is a fluid limitif there exists { r n } n → + ∞ , { x n } n → x ◮ D´ s.t. ⇒ Q x Q r n ; x n = ag functions R + → X. on the space of the c` ad-l`

Existence of fluid limits (b) efinition : Q x is a fluid limitif there exists { r n } n → + ∞ , { x n } n → x ◮ D´ s.t. ⇒ Q x Q r n ; x n = ag functions R + → X. on the space of the c` ad-l` Φ k +1 = Φ k + E [Φ k +1 |F k ] − Φ k + Φ k +1 − E [Φ k +1 |F k ]

Existence of fluid limits (b) efinition : Q x is a fluid limitif there exists { r n } n → + ∞ , { x n } n → x ◮ D´ s.t. ⇒ Q x Q r n ; x n = ag functions R + → X. on the space of the c` ad-l` Φ k +1 = Φ k + E [Φ k +1 |F k ] − Φ k + Φ k +1 − E [Φ k +1 |F k ] = Φ k + E x [Φ k +1 − Φ k |F k ] + (Φ k +1 − E x [Φ k +1 |F k ]) . � �� � � �� � ǫ k +1 ∆(Φ k ) martingale increment

Existence of fluid limits (b) efinition : Q x is a fluid limitif there exists { r n } n → + ∞ , { x n } n → x ◮ D´ s.t. ⇒ Q x Q r n ; x n = ag functions R + → X. on the space of the c` ad-l` Φ k +1 = Φ k + E [Φ k +1 |F k ] − Φ k + Φ k +1 − E [Φ k +1 |F k ] = Φ k + E x [Φ k +1 − Φ k |F k ] + (Φ k +1 − E x [Φ k +1 |F k ]) . � �� � � �� � ǫ k +1 ∆(Φ k ) martingale increment ◮ Result if � � | ǫ 1 | p 1 · ∃ p > 1 , lim K → + ∞ sup x ∈ X E x I | ǫ 1 | >K → 0 . · sup x ∈ X | ∆( x ) | < ∞ . Then fluid limits exist, prob. on the space of continuous functions ( whatever the initial point on the unit sphere )

Example 1 : (regular case) 1 0.8 Level curves of the target density 15 0.6 0.4 10 0.2 5 0 0 −0.2 −0.4 −5 −0.6 −10 −0.8 −1 −15 −15 −10 −5 0 5 10 15 −0.2 0 0.2 0.4 0.6 0.8 1 1.2 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 −0.2 −0.2 −0.4 −0.4 −0.6 −0.6 −0.8 −0.8 −1 −1 −0.2 0 0.2 0.4 0.6 0.8 1 1.2 −0.2 0 0.2 0.4 0.6 0.8 1 1.2 π ( x, y ) ∝ (1 + x 2 + y 2 + x 8 y 2) exp( − ( x 2 + y 2)) , q ∼ N (0 , 4) , r=100, r=1000, r=5000

Example 2 : (irregular case) 1.2 1 Level curves of the target density 15 0.8 10 0.6 5 0.4 0 0.2 −5 0 −10 −0.2 −15 −15 −10 −5 0 5 10 15 −0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.2 1.2 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 −0.2 −0.2 −0.2 0 0.2 0.4 0.6 0.8 1 1.2 −0.2 0 0.2 0.4 0.6 0.8 1 1.2 π ( x, y ) ∝ N (0 , Γ − 1 ) + N (0 , Γ − 1 ) , q ∼ N (0 , 1) , r=100, r=1000, r=5000 1 2

Characterisation of the fluid limits ֒ → Can we describe the distributions Q x ? 1.2 1 0.8 0.6 0.4 0.2 0 −0.2 −0.2 0 0.2 0.4 0.6 0.8 1 1.2 π ( x, y ) ∝ mixture of Gaussian, q ∼ N (0 , I ) , r=5000 T=5

Characterization (b) Φ k +1 = Φ k + ( E x [Φ k +1 |F k ] − Φ k ) + (Φ k +1 − E x [Φ k +1 |F k ]) � �� � � �� � ∆(Φ k ) ǫ k +1 martingale increment ◮ For the normalized process � k + 1 � = 1 η r , x r Φ k +1 r � k � � � k �� + 1 + 1 = η r r , x r ∆ r η r r , x r ǫ k +1 � k � � � k �� + 1 + 1 = η r r , x h η r r , x r ( ξ k + ǫ k +1 ) r where h ( x ) = r → + ∞ ∆( r x ) . lim

Characterization (b) Φ k +1 = Φ k + ( E x [Φ k +1 |F k ] − Φ k ) + (Φ k +1 − E x [Φ k +1 |F k ]) � �� � � �� � ∆(Φ k ) ǫ k +1 martingale increment ◮ For the normalized process � k + 1 � = 1 η r , x r Φ k +1 r � k � � � k �� + 1 + 1 = η r r , x r ∆ r η r r , x r ǫ k +1 � k � � � k �� + 1 + 1 = η r r , x h η r r , x r ( ξ k + ǫ k +1 ) r where h ( x ) = r → + ∞ ∆( r x ) . lim ◮ Thus the dynamic � k + 1 � � k � � k � + 1 µ = µ r h ← → ODE : ˙ µ ( t ) = h ( µ ( t )) r r r in an additive noise.

Characterisation (c) ◮ Theorem If · Existence of the fluid limit. · there exists an open cone O de X \ { 0 } , · h : O → X s.t. � � � → 0 , � r β ∆( rx ) − | x | − β h ( x ) sup r → + ∞ , x ∈ H for any compact H ⊆ O

Characterisation (c) ◮ Theorem If · Existence of the fluid limit. · there exists an open cone O de X \ { 0 } , · h : O → X s.t. � � � → 0 , � r β ∆( rx ) − | x | − β h ( x ) sup r → + ∞ , x ∈ H for any compact H ⊆ O Then for all 0 ≤ s ≤ t , on { η, η ( u ) ∈ O , s ≤ u ≤ t } , � u � � � � Q β sup � η ( u ) − η ( s ) − h ◦ η ( v ) dv � = 0 , x − a.s. � � s ≤ u ≤ t s

Characterisation (c) ◮ Theorem If · Existence of the fluid limit. · there exists an open cone O de X \ { 0 } , · h : O → X s.t. � � � → 0 , � r β ∆( rx ) − | x | − β h ( x ) sup r → + ∞ , x ∈ H for any compact H ⊆ O Then for all 0 ≤ s ≤ t , on { η, η ( u ) ∈ O , s ≤ u ≤ t } , � u � � � � Q β sup � η ( u ) − η ( s ) − h ◦ η ( v ) dv � = 0 , x − a.s. � � s ≤ u ≤ t s ◮ i.e. the fluid limit Q β x is a Dirac mass at the point η satisfying � u η ( u ) = η ( s ) + h ◦ η ( v ) dv, s ≤ u ≤ t, s whenever η ([ s, t ]) ⊂ O.

Example 3 : Super-exponential case, O = X \ { 0 } Level curves of the target density Courbes de niveau de la densite 15 40 35 10 30 25 5 20 0 15 10 −5 5 0 −10 −5 −15 −10 −15 −10 −5 0 5 10 15 0 5 10 15 20 25 30 35 40 10 1.2 8 1 6 4 0.8 2 0.6 0 0.4 −2 −4 0.2 −6 0 −8 −10 −0.2 −4 −2 0 2 4 6 8 10 12 14 −0.2 0 0.2 0.4 0.6 0.8 1 1.2 UpperLeft- Level curves of π UpperRight- Rejection area LowerRight- Process ηβ LowerLeft- Level curves, ∆ and h x and flow of the ODE.

Example 4 : Super-exponential case, O � X \ { 0 } 6 1.2 Level curves of the target density 5.5 15 1 5 10 0.8 4.5 5 0.6 4 0 0.4 3.5 −5 0.2 3 −10 0 2.5 2 −15 −0.2 −15 −10 −5 0 5 10 15 2 2.5 3 3.5 4 4.5 5 5.5 6 −0.2 0 0.2 0.4 0.6 0.8 1 1.2 Process ηβ Level curves of π Level curves, ∆ and h x and flow of the ODE.