Nonequilibrium Markov processes conditioned on large deviations - PDF document

Nonequilibrium Markov processes conditioned on large deviations Hugo Touchette National Institute for Theoretical Physics (NITheP) Stellenbosch, South Africa Advances in Nonequilibrium Statistical Mechanics Galileo Galilei Institute for Theoretical Physics Florence, Italy • Work with Rapha¨ el Chetrite, Universit´ e de Nice, France Hugo Touchette (NITheP) Conditioned processes June 2014 1 / 21 Problem • Markov process: { X t } T t =0 • Observable (rv): A T • Conditioned process: X t | A T = a x H t L Questions 1 Conditional process Markov? 2 Generator? t 3 Relation with X t ? Connections P H A T = a L • Markov conditioning (Doob) • Nonequilibrium systems • Rare event simulations a • Quasi-stationary distributions • Stochastic control (Fleming) Hugo Touchette (NITheP) Conditioned processes June 2014 2 / 21

Markov conditioning Doob conditioning (1957) X t | X T ∈ A target point or set • Brownian bridge: W t | W 1 = 0 Schr¨ odinger bridge (1931) X t | p ( x , T ) = q ( x ) target distribution • Classical (Markov) representation of QM • Nelson’s mechanics Here • X t |A T with A T defined on [0 , T ] • Requires generalization of Doob’s transform • Asymptotic equivalence Hugo Touchette (NITheP) Conditioned processes June 2014 3 / 21 Comparison with optimal paths Arbitrary noise Low noise limit x H t L x H t L t t • Concentration in path space • No concentration • Prob dominated by single path • Prob coming from many paths • Dominant path, most • No dominating path probable path, instanton Fluctuation path Fluctuating dynamics Hugo Touchette (NITheP) Conditioned processes June 2014 4 / 21

Process • Markov process: X t ∈ E • State space: E x H t L • Time interval (horizon): t ∈ [0 , T ] • Generator: t ∂ t E x [ f ( X t )] = E x [ Lf ( X t )] • Master (Fokker-Planck) equation: ∂ t p ( x , t ) = L † p ( x , t ) x H t L • Path measure: t P [ x ] = P ( { x t } T t =0 ) Hugo Touchette (NITheP) Conditioned processes June 2014 5 / 21 Examples of Markov processes Pure jump process • Transition rates: W ( x , y ) = P ( x → y in dt ) / dt • Escape rates: x H t L � λ ( x ) = W ( x , y ) = ( W 1)( x ) y • Generator: L = W − λ t ���� ���� off-diag diag Pure diffusion • SDE: dX t = F ( X t ) dt + σ dW t • Generator: x H t L L = F · ∇ + D 2 ∇ 2 , D = σσ T t Hugo Touchette (NITheP) Conditioned processes June 2014 6 / 21

Conditioning observable • Observable: A T [ x ] • Jump processes: x H t L � T A T = 1 f ( X t ) dt + 1 � g ( X t − , X t + ) T T 0 ∆ X t � =0 t • Diffusions: � T � T x H t L A T = 1 f ( X t ) dt + 1 g ( X t ) ◦ dX t T T 0 0 t Examples • Occupation time X t ∈ ∆ • Mean number jumps (activity), current • Work, heat, entropy production,... Hugo Touchette (NITheP) Conditioned processes June 2014 7 / 21 Rare event conditioning Large deviation principle P ( A T = a ) ≍ e − TI ( a ) • Meaning of ≍ : T →∞ − 1 P ( A T = a ) = e − TI ( a )+ o ( T ) lim T ln P ( A T = a ) = I ( a ) , • Rate function: I ( a ) P ( A T = a ) • Zero of I = Law of Large Numbers T = 100 • Concentration point(s): I ( a ∗ ) = 0 I ( a ) • Small fluctuations = T = 50 Central Limit Theorem T = 10 s µ Hugo Touchette (NITheP) Conditioned processes June 2014 8 / 21

Conditioned process P H A T = a L x H t L t a • Conditioned process: X t | A T = a • Path measure: P a [ x ] = P [ x | A T = a ] = P [ x , A T = a ] P ( A T = a ) = P [ x ] δ ( A T [ x ] − a ) P ( A T = a ) • Path microcanonical ensemble • Not Markov for T < ∞ • Becomes equivalent to Markov process as T → ∞ • Non-conditioned process realizing conditioning Hugo Touchette (NITheP) Conditioned processes June 2014 9 / 21 Spectral elements Scaled cumulant function G¨ artner-Ellis Theorem Λ k differentiable, then 1 T ln E [ e TkA T ] Λ k = lim 1 LDP for A T T →∞ 2 I ( a ) = sup { ka − Λ k } • k ∈ R k Feynman-Kac-Perron-Frobenius L k r k = Λ k r k • Tilted (twisted) operator: L k • Dominant eigenvalue: Λ k • Dominant eigenfunction: r k Diffusions Jump processes L k = F · ( ∇ + kg ) + D L k = We kg − λ + kf 2 ( ∇ + kg ) 2 + kf Hugo Touchette (NITheP) Conditioned processes June 2014 10 / 21

Driven process Definition • Process Y t • Generator: L k = r − 1 k L k r k − r − 1 k ( L k r k ) • Generalized Doob transform • Positive, Markov operator: ( L k 1) = 0 • Path measure: P driven [ x ] k ( X 0 ) e T ( kA T − Λ k ) r k ( X T ) = r − 1 k P [ x ] • Radon-Nikodym derivative Hugo Touchette (NITheP) Conditioned processes June 2014 11 / 21 Main result Hypotheses • A T satisfies LDP • Rate function I ( a ) convex • Other properties of spectral elements (gap, regular r k ) Result T →∞ ∼ k ( a ) = I ′ ( a ) X t | A T = a = Y t P a [ x ] P driven ≍ k ( a ) [ x ] almost everywhere B T → b ∗ B T → b ∗ ⇒ in probability A T = a A T → a • Same typical states • Different fluctuations (LDPs) in general Hugo Touchette (NITheP) Conditioned processes June 2014 12 / 21

Idea of the proof Microcanonical Canonical Driven X t | A T = a Y t [ x ] = e kTA T [ x ] P cano E [ e kTA T ] P [ x ] k P driven P a [ x ] = P [ x | A T = a ] [ x ] k Driven → canonical • P driven [ x ] ≍ P cano [ x ] k k • Same large deviations Microcanonical → canonical • P a [ x ] ≍ P cano [ x ] if I ( a ) convex k • Same typical states • General result about conditioning vs tilting Hugo Touchette (NITheP) Conditioned processes June 2014 13 / 21 Driven process: Explicit form Jump process • Original process: W ( x , y ) • Driven process: W k ( x , y ) = r k − 1 ( x ) W ( x , y ) e kg ( x , y ) r k ( y ) , k = I ′ ( a ) • Evans PRL 2004, Jack and Sollich PTPS 2010 Diffusion • Reference SDE: dX t = F ( X t ) dt + σ dW t • Driven SDE: dY t = F k ( Y t ) dt + σ dW t • Modified drift: k = I ′ ( a ) F k ( y ) = F ( y ) + D ( kg + ∇ ln r k ) , Hugo Touchette (NITheP) Conditioned processes June 2014 14 / 21

Application: Langevin equation dX t = − γ X t dt + σ dW t − → X t | A T = a Empirical variance Area under path � T � T A T = 1 A T = 1 X 2 t dt X t dt T T 0 0 • f ( x ) = x 2 , g = 0 • f ( x ) = x , g = 0 • Rate function: I ( a ) = γ 2 a 2 • Modified drift: 2 σ 2 • Eigenfunction: r k ( x ) = e kx /γ F k ( a ) = − σ 2 2 ax • Modified drift: F k ( a ) ( x ) = − γ x + a • Modified friction γ • k ( a ) = I ′ ( a ) Hugo Touchette (NITheP) Conditioned processes June 2014 15 / 21 Other applications • Sheared fluids • R.M.L. Evans PRL 2004; JPA 2005 • Baule & Evans PRL 2008; PRE 2008 • Diffusion on circle • Conditioning on current • Chetrite & HT PRL 2013 • Nemoto & Sasa PRE 2011, PRL 2014 • Interacting particles on lattices • Conditioning on current • TASEP: Sch¨ utz et al . JSTAT 2010; JSP 2011 • Zero-range: Harris et al . 2013 • Glauber-Ising: Jack & Sollich PTPS 2010 a b • East model: Jack & Sollich JPA 2014 • Rotators: Knezevic & Evans PRE 2014 Conditioning typically induces long-range interaction Hugo Touchette (NITheP) Conditioned processes June 2014 16 / 21

Nonequilibrium systems Nonequilibrium Equilibrium J > 0 J = 0 T a T b T b T b • Microscopic dynamics: • Microscopic dynamics known • Detailed balance: W noneq ( x → y )? W eq ( x → y ) W eq ( y → x ) = e β ∆ E • Many models possible Mike Evans’s hypothesis PRL 2004; JPA 2005 W noneq ( x → y ) = W eq ( x → y | J ) • Nonequilibrium = conditioning of equilibrium • True? Approximation? Hugo Touchette (NITheP) Conditioned processes June 2014 17 / 21 Other connections Conditional limit theorems • Sequence of rvs: X 1 , X 2 , . . . , X n , X i ∼ P ( x ) n • Sample mean: S n = 1 � f ( X i ) n i =1 • Conditional marginal: e kf ( x ) n →∞ P ( X i = x | S n = s ) = lim E [ e kf ( X ) ] P ( x ) Control representations of PDEs I = − ln φ PDE → Hamilton-Jacobi equation (Hopf-Cole) φ ( x , t ) ↓ ∂ t φ = L φ Dynamic programming ↓ Optimal stochastic control = Doob transform • Fleming, Sheu, Dupuis 1980’s, 1990’s Hugo Touchette (NITheP) Conditioned processes June 2014 18 / 21

Large deviation simulations • A T = a exponentially rare • Direct sampling: sample size ∼ e T P H A T = a L • Importance sampling (reweighting) • Change process • Make A T = a typical a � dP X � P ( A T = a ) = E X [ δ ( A T − a )] = E Y δ ( A T − a ) dP Y Driven process Y t • Makes A T = a typical • Good (optimal) change of process • Problem: Y t based on r k , Λ k and I ( a ) Learning algorithm [Borkar 2008] 1 Direct sampling + feedback → iterative estimation of r k 2 Control leading to driven process Hugo Touchette (NITheP) Conditioned processes June 2014 19 / 21 Conclusions T →∞ ∼ X t | A T = a = Y t ���� � �� � driven conditioned • Effective Markov dynamics for rare events • Explicit interpretation of asymptotic equivalence • Similar to equivalence of equilibrium ensembles • Generalization of Markov conditioning and bridges • Links: QSD, stochastic control, conditional limit theorems Future work • Large deviation simulations • Consequences for nonequilibrium systems Hugo Touchette (NITheP) Conditioned processes June 2014 20 / 21