Large deviation simulations: Equilibrium vs nonequilibrium systems - PDF document

Large deviation simulations: Equilibrium vs nonequilibrium systems Hugo Touchette National Institute for Theoretical Physics (NITheP) Stellenbosch, South Africa Numerical Aspects of Nonequilibrium Dynamics Institut Henri Poincar´ e, Paris 25 April 2017 Hugo Touchette (NITheP) IHP, Paris April 2017 1 / 18 Problems Large deviation problem P H A T = a L P ( A n ∈ B ) ≈ e − nI Dual problem a E [ e kA n ] ≈ e n λ ( k ) • Generating function x ( t ) • Spectral problem (Kac) Prediction problem t How is the fluctuation created? • Reaction or optimal path x ( t ) • Fluctuation process • Conditioning t Hugo Touchette (NITheP) IHP, Paris April 2017 2 / 18

Methods Milestone Forward flux Cross Questions MUCA entropy WL • Equilibrium or Splitting nonequilibrium Cloning Prob Tilting Importance method? Meta- sampling dynamics • Equilibrium vs nonequilibrium GF , Simulation Adaptive fluctuations Reaction & numerical path methods • Nonequilibrium Empirical GF more difficult? String Control DMRG Methods covered Optimization Spectral • Spectral Dyn MFT prog HFT • Optimization Monte Carlo Adaptive • Importance sampling Exact diag Hugo Touchette (NITheP) IHP, Paris April 2017 3 / 18 Low-noise large deviations • Process: ∂ D t ) dt + √ ε dW t , dX ε t = F ( X ε X 0 ∈ O • Transition rare event: D t P ( X ε τ ∈ B | X 0 ∈ O ) ≈ e − V /ε , ε → 0 Freidlin-Wentzell-Graham � t 1 x s − F ( x s )) 2 ds V = inf V ( x ) = inf inf ( ˙ 2 x ∈ B x ∈ B x 0 ∈ O , x t = x � �� � 0 � �� � quasi-potential FW action, Lagrangian J [ x ] • Deterministic control problem • Optimal path { x ∗ t } • V ( x ) solves Hamilton-Jacobi-Bellman equation (1st order) Hugo Touchette (NITheP) IHP, Paris April 2017 4 / 18

Low-noise large deviations (cont’d) Fluctuation created by optimal path Conditioning • P [ x ] ≈ e − J [ x ] /ε max • P [ x | escape event] concentrates on optimal path Equilibrium • Optimal path = relaxation path R • V ( x ) = J [ x R relax ] Nonequilibrium • Optimal path � = relaxation path R • Current loops (b) • Transversal decomposition [Graham 80’s] [Luchinsky et al 1997] Hugo Touchette (NITheP) IHP, Paris April 2017 5 / 18 Applications SDEs dX t = F ( X t ) dt + √ ε noise • Escape, transition paths, escape time [Onsager-Machlup 1953] [Freidlin-Wentzell 70s] [Graham 80-90s] ... • Experiments [Luchinsky and McClintock 90s] • Metastability [Olivieri-Vares 2005] SPDEs d ρ ( x , t ) = F [ ρ ] dt + √ ε noise • Heat equation [Faris, Jona-Lasinio 1982] • Ginzburg equation [Graham 1990s] • Reaction-diffusion [Vanden-Eijnden 2000s] • 2D fluid equations [Laurie-Bouchet 2014] • MFT/HFT [Bertini et al 2000s] Hugo Touchette (NITheP) IHP, Paris April 2017 6 / 18

Long-time large deviations • Process: dX t = F ( X t ) dt + σ dW t x H t L • Observable: � T � T t A T = 1 f ( X t ) dt + 1 g ( X t ) ◦ dX t T T P ( A T = a ) 0 0 T = 100 I ( a ) • Large deviation principle (LDP): T = 50 P ( A T = a ) ≈ e − TI ( a ) , T → ∞ T = 10 s µ Examples • Occupation time, empirical density • Current, mean speed, activity • Work, heat, entropy production (stochastic thermo) Hugo Touchette (NITheP) IHP, Paris April 2017 7 / 18 Dual problem Scaled cumulant function G¨ artner-Ellis Theorem λ ( k ) differentiable, then 1 T ln E [ e TkA T ] 1 LDP for A T λ ( k ) = lim T →∞ 2 I ( a ) = sup { ka − λ ( k ) } k Feynman-Kac-Perron-Frobenius L k r k = λ ( k ) r k • Tilted (twisted) operator: L k • Dominant eigenvalue: λ ( k ) • Dominant eigenfunction: r k Diffusions Jump processes L k = F · ( ∇ + kg ) + D L k = We kg − λ + kf 2 ( ∇ + kg ) 2 + kf Hugo Touchette (NITheP) IHP, Paris April 2017 8 / 18

Spectral problem Equilibrium • X t reversible, g gradient, f arbitrary • L k non-Hermitian but conjugated to Hermitian: H k = ρ 1 / 2 L k ρ − 1 / 2 , ψ 2 k = r k l k • Real spectrum (quantum problem) Nonequilibrium • X t nonreversible OR g non-gradient, f arbitrary • L k non-Hermitian, not conjugated to Hermitian • Complex spectrum • Full spectral problem: | x |→∞ L k r k = λ ( k ) r k r k ( x ) l k ( x ) − → 0 L † k l k = λ ( k ) l k Hugo Touchette (NITheP) IHP, Paris April 2017 9 / 18 Algorithms Markov chains • Non-symmetric positive matrices • Direct diagonalization • Power method • DMRG [Gorissen-Vanderzande 2011] Diffusions • Equilibrium: (Quantum) spectral methods • Nonequilibrium: Fourier decomposition, basis functions Cloning/splitting • Particle/trajectory simulation • Yields SCGF • No eigenvector Hugo Touchette (NITheP) IHP, Paris April 2017 10 / 18

Other approach: Optimization and control � � A T = ˜ A ( ρ T , J T ) = f ( x ) ρ T ( x ) dx + g ( x ) J T ( x ) dx � �� � � �� � current dist. Drift optimization � 1 ( u ( x ) − F ( x )) 2 ρ u I ( a ) = inf inv ( x ) dx 2 σ 2 u : E u [ A T ]= a • Requires stationary distribution ρ u inv Stochastic optimal control � T 1 ( u ( X u t ) − F ( X u t )) 2 dt I ( a ) = lim inf 2 σ 2 T u : A u T →∞ T → a 0 • Controlled process: X u t • Constrained control (dual for SCGF is unconstrained) • Solves Hamilton-Jacobi-Bellman equation (2nd order) Hugo Touchette (NITheP) IHP, Paris April 2017 11 / 18 Fluctuation process Optimal control process d ˆ X t = F k ( ˆ X t ) dt + σ dW t x H t L • Optimal drift: I ′ ( a ) = k F k ( x ) = F ( x )+ D ( kg + ∇ ln r k ) , t Conditioning P H A T = a L T →∞ ∼ ˆ X t | A T = a = X t ���� � �� � driven conditioned canonical microcanonical a • Effective process creating the fluctuation • Process generalization of optimal path [Chetrite HT, PRL 2013, AHP 2015, JSTAT 2015] Hugo Touchette (NITheP) IHP, Paris April 2017 12 / 18

Equilibrium vs nonequilibrium � � A T = ˜ A ( ρ T , J T ) = f ( x ) ρ T ( x ) dx + g ( x ) J T ( x ) dx � �� � � �� � current dist. ˆ X t g X t 0 Reversible Reversible Same spectrum Rayleigh-Ritz variational principle Gradient Reversible Rayleigh-Ritz variational principle Non-gradient Non-reversible 0 Non-reversible Non-reversible Donsker-Varadhan principle Other Non-reversible • Process closest to X t that creates fluctuation inv or 1 • Distance: � u − F � ρ u T S ( P ˆ X || P X ) Hugo Touchette (NITheP) IHP, Paris April 2017 13 / 18 Simulations: Importance sampling • P ( A T = a ) ≈ e − TI ( a ) P H A T = a L • Direct sampling: sample size = L ∼ e T a Importance sampling • Change process: X t → ˆ X t • Make A T = a typical • Change of measure: � dP X � P ( A T = a ) = E X [1 1 a ( A T )] = E ˆ 1 1 a ( A T ) X dP ˆ X • Estimator: L P L ( a ) = 1 � 1 a ( A ( j ) ˆ 1 T ) R L j =1 Hugo Touchette (NITheP) IHP, Paris April 2017 14 / 18

Efficiency Zero-variance process • Conditioned process: X t | A T = a • Estimator variance: X [ R 2 1 1 a ( A T )] − p 2 P L ) = E ˆ X ( ˆ var ˆ = 0 L Effective process • Not zero variance • Asymptotic optimality: T →∞ − 1 X [ R 2 1 lim T ln E ˆ 1 a ( A T )] = 2 I ( a ) • Variance goes to 0 with largest rate • Exponential tilting: P driven [ x ] ≈ P k [ x ] = e TkA T [ x ] P [ x ] E [ e TkA T ] Hugo Touchette (NITheP) IHP, Paris April 2017 15 / 18 Connections σ > 0 • Stochastic optimal control • Quadratic cost function (log RND) x ( t ) • Nonlinear HJB equation (2nd order) • Similar to cross-entropy minimization t σ → 0 • Deterministic optimal control • Quadratic cost function (log RND) x ( t ) • Nonlinear HJB (1st order, viscosity) t Optimal controller Value function ← → ← → Exponential tilting Dyn programming HJB equations ˆ LD functions X t Hugo Touchette (NITheP) IHP, Paris April 2017 16 / 18

Conclusions LD ≡ control/optimization ≡ spectral Type of fluctuations determines class of control designs Equilibrium fluctuations Nonequilibrium fluctuations Reversible control Non-reversible control Hermitian spectral problem Non-hermitian spectral problem Future work • Adapt methods from quantum mechanics • Methods for non-hermitian operators • HJB-based methods • Adaptive control methods • Error bars • Benchmarking Hugo Touchette (NITheP) IHP, Paris April 2017 17 / 18 References H. Touchette Introduction to large deviations: Theory, applications, simulations 2011 Oldenburg School Lecture Notes, arxiv:1106.4146 R. Chetrite, H. Touchette Variational and optimal control representations of conditioned and driven processes JSTAT P12001, 2015 J. Bucklew Introduction to Rare Event Simulation Springer, 2004 More pointers at www.physics.sun.ac.za/~htouchette/ldt Hugo Touchette (NITheP) IHP, Paris April 2017 18 / 18