Stability of Markov Chains based on fluid limit techniques. - PowerPoint PPT Presentation

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC Stability of Markov Chains based on fluid limit techniques. Applications to MCMC Gersende FORT LTCI CNRS - TELECOM ParisTech In collaboration with Sean MEYN (Univ. Illinois) , Eric MOULINES (TELECOM ParisTech) and Pierre PRIOURET (Univ. Paris 6) .

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC We introduce ◮ a transformation of the Markov Chain − → family of time-continuous processes − → a limiting time-continuous process ◮ such that the stability of this process, is related to the ergodicity of the Markov chain. ⇒ characterization of the ergodicity ; ⇒ identification of the factors that play a role in the dynamic of the Markov chain.

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC We introduce ◮ a transformation of the Markov Chain − → family of time-continuous processes − → a limiting time-continuous process ◮ such that the stability of this process, is related to the ergodicity of the Markov chain. ⇒ characterization of the ergodicity ; ⇒ identification of the factors that play a role in the dynamic of the Markov chain. The Markov Chain Monte Carlo (MCMC) algorithms ◮ are iterative algorithms that draw path of a Markov chain with given stationary distribution ; ◮ the performances of which are related (among other factors) to some parameters of implementation (design parameters) . ◮ ⇒ find the role of the parameters in the definition of the fluid limit and propose an “optimal choice” of these parameters.

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC ֒ → Outline of the talk I. A MCMC sampler : the Metropolis-within-Gibbs (MwG), and its design parameters. II. Fluid limits. III. Applications : guidelines on the choice of the design parameters for the MwG.

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC I. MCMC samplers I-a General presentation MCMC samplers : Given a probability π , sample a Markov chain { Φ n , n ≥ 0 } with unique stationary distribution π . → Allow ֒ ◮ to explore the target density π . ◮ to approximate quantities of the form E π [ g (Φ)] as soon as a LLN exists (and other limit theorems). ֒ → Algorithms : Hastings-Metropolis, Gibbs, Metropolis-within-Gibbs, · · ·

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC I. MCMC samplers I-b Metropolis-within-Gibbs samplers Metropolis-within-Gibbs samplers in R d ◮ Choose a selection probability : ω = { ω i , i ∈ { 1 , · · · , d }} ◮ Choose a family of transition kernels on R , q i ( x, y ) ex. qi ( x, y ) = N ( x, σ 2 i )[ y ] ◮ Repeat : • select a direction I with prob. P ( I = k ) = ω k . • draw a candidate Y ∼ q I (Φ n,I , · ) . • accept or reject the candidate : all the components are unchanged except the I -th � α (Φ n , Y ) = 1 ∧ π ( Y, Φ n, − I ) q I ( Y, Φ n,I ) Y with proba Φ n +1 ,I = π (Φ n ) q I (Φ n,I ,Y ) Φ n,I otherwise .

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC I. MCMC samplers I-b Metropolis-within-Gibbs samplers Example : Metropolis-within-Gibbs (MwG) ◮ Explore on R 2 a Gaussian distribution π with diagonal dispersion matrix ◮ and in each direction, the move is Gaussian. Initial value (and level curves of π ) 3 2 1 0 −1 −2 −3 −3 −2 −1 0 1 2 3

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC I. MCMC samplers I-b Metropolis-within-Gibbs samplers Example : Metropolis-within-Gibbs (MwG) ◮ Explore on R 2 a Gaussian distribution π with diagonal dispersion matrix ◮ and in each direction, the move is Gaussian. Initial value (and level curves of π ), Propose 3 2 1 0 −1 −2 −3 −3 −2 −1 0 1 2 3

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC I. MCMC samplers I-b Metropolis-within-Gibbs samplers Example : Metropolis-within-Gibbs (MwG) ◮ Explore on R 2 a Gaussian distribution π with diagonal dispersion matrix ◮ and in each direction, the move is Gaussian. Initial value (and level curves of π ), Propose , Accepted 3 2 1 0 −1 −2 −3 −3 −2 −1 0 1 2 3

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC I. MCMC samplers I-b Metropolis-within-Gibbs samplers Example : Metropolis-within-Gibbs (MwG) ◮ Explore on R 2 a Gaussian distribution π with diagonal dispersion matrix ◮ and in each direction, the move is Gaussian. Initial value (and level curves of π ), Propose , Accepted , Propose 3 2 1 0 −1 −2 −3 −3 −2 −1 0 1 2 3

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC I. MCMC samplers I-b Metropolis-within-Gibbs samplers Example : Metropolis-within-Gibbs (MwG) ◮ Explore on R 2 a Gaussian distribution π with diagonal dispersion matrix ◮ and in each direction, the move is Gaussian. Initial value (and level curves of π ), Propose , Accepted , Propose , Rejected 3 2 1 0 −1 −2 −3 −3 −2 −1 0 1 2 3

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC I. MCMC samplers I-b Metropolis-within-Gibbs samplers Example : Metropolis-within-Gibbs (MwG) ◮ Explore on R 2 a Gaussian distribution π with diagonal dispersion matrix ◮ and in each direction, the move is Gaussian. Initial value (and level curves of π ), Propose , Accepted , Propose , Rejected , Propose 3 2 1 0 −1 −2 −3 −3 −2 −1 0 1 2 3

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC I. MCMC samplers I-b Metropolis-within-Gibbs samplers Example : Metropolis-within-Gibbs (MwG) ◮ Explore on R 2 a Gaussian distribution π with diagonal dispersion matrix ◮ and in each direction, the move is Gaussian. Initial value (and level curves of π ), Propose , Accepted , Propose , Rejected , Propose , Accepted 3 2 1 0 −1 −2 −3 −3 −2 −1 0 1 2 3

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC I. MCMC samplers I-b Metropolis-within-Gibbs samplers Example : Metropolis-within-Gibbs (MwG) ◮ Explore on R 2 a Gaussian distribution π with diagonal dispersion matrix ◮ and in each direction, the move is Gaussian. Initial value (and level curves of π ), Propose , Accepted , Propose , Rejected , Propose , Accepted , After 10000 iterations. 5 4 3 2 1 0 −1 −2 −3 −4 −5 −10 −5 0 5 10

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC I. MCMC samplers I-d Design parameters for the MwG Design parameters for the MwG · Selection { ω i , i ≤ d } , · Gaussian proposal distributions in each direction, with std σ i . ֒ → Efficiency of the algorithm π ∼ N 2 (0 , ∆) with diagonal dispersion matrix ∆ such that ∆ 1 , 1 >> ∆ 2 , 2 , 3 3 2 2 1 1 0 0 −1 −1 −2 −2 −3 −3 −15 −10 −5 0 5 10 15 −15 −10 −5 0 5 10 15 (left) ω 1 = ω 2 , σ 1 = σ 2 (right) ω 1 = ω 2 , σ 1 >> σ 2 .

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC I. MCMC samplers I-d Design parameters for the MwG ֒ → Questions ◮ Optimal value of the design parameters. ◮ Adaptive methods : modify “on line” these parameters based on the past behavior of the algorithm. → Hereafter, ֒ ◮ characterization of the role of these parameters on the dynamic of the chain. ◮ guidelines to fix / adapt the value of these parameters.

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC II. Fluid limits II. Fluid Limits

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC II. Fluid limits II-a Definition Normalized processes (X = R d ). Let { Φ k , k ≥ 0 } be a Markov chain on X A set of transformations : normalized process η r , for r > 0 (i) in the initial value : η r (0; x ) = 1 r Φ 0 = x ∈ R d , Φ 0 = rx (ii) in time and space : η r ( t ; x ) = 1 r Φ ⌊ tr ⌋ .

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC II. Fluid limits II-a Definition Normalized processes (X = R d ). Let { Φ k , k ≥ 0 } be a Markov chain on X A set of transformations : normalized process η r , for r > 0 (i) in the initial value : η r (0; x ) = 1 r Φ 0 = x ∈ R d , Φ 0 = rx (ii) in time and space : η r ( t ; x ) = 1 r Φ ⌊ tr ⌋ . � k � η r ( · ; x ) = 1 r ; ( k + 1) Hence r Φ k on the time interval . r By definition, cad-lag paths.

Stability of Markov Chains based on fluid limit techniques. Applications to MCMC II. Fluid limits II-a Definition Definition ֒ → Distributions · P x : law of the canonical chain { Φ k , k ≥ 0 } with initial value δ x . · Q r ; x : distribution image of P rx by η r ( · ; x ) , distribution on D ( R + , X ) of cadlag functions R + → X