Stochastic Processes MATH5835, P. Del Moral UNSW, School of - PowerPoint PPT Presentation

Stochastic Processes MATH5835, P. Del Moral UNSW, School of Mathematics & Statistics Lectures Notes, No. 8 Consultations (RC 5112): Wednesday 3.30 pm � 4.30 pm & Thursday 3.30 pm � 4.30 pm 1/21

Reminder + Information References in the slides ◮ Material for research projects � Moodle ( Stochastic Processes and Applications ∋ variety of applications) ◮ Important results ⊂ Assessment/Final exam = LOGO = 2/21

– Albert Einstein (1879-1955) 3/21

– Albert Einstein (1879-1955) 3/21

Plan of the lecture ◮ Markov chain models ◮ Elementary transitions ◮ Random dynamical systems 4/21

Plan of the lecture ◮ Markov chain models ◮ Elementary transitions ◮ Random dynamical systems ◮ Stability properties ◮ 2 states model ◮ Perron Frobenius theorem ◮ Spectral analysis ◮ Total variation norms 4/21

Plan of the lecture ◮ Markov chain models ◮ Elementary transitions ◮ Random dynamical systems ◮ Stability properties ◮ 2 states model ◮ Perron Frobenius theorem ◮ Spectral analysis ◮ Total variation norms ◮ Quantitative rates ◮ Spectral Gaps ◮ Dobrushin contraction/ergodic coef. ◮ Poisson equation 4/21

Three objectives ◮ Formalize/Recognize a Markov chain model 5/21

Three objectives ◮ Formalize/Recognize a Markov chain model ◮ Analyze the stability properties ◮ Analysis on reduced and toy models ◮ L 2 techniques and spectral tools ◮ Total variation norms and Dobrushin contractions 5/21

Three objectives ◮ Formalize/Recognize a Markov chain model ◮ Analyze the stability properties ◮ Analysis on reduced and toy models ◮ L 2 techniques and spectral tools ◮ Total variation norms and Dobrushin contractions ◮ Open/Ask questions [ ∼ continuous/discrete time models?] 5/21

Markov transitions P ( X n ∈ dx n | X 0 , . . . , X n − 2 , X n − 1 ) = P ( X n ∈ dx n | X n − 1 ) ⇓ P ( X n ∈ dx n | X n − 1 = x n − 1 ) = M n ( x n − 1 , dx n ) 6/21

Markov transitions P ( X n ∈ dx n | X 0 , . . . , X n − 2 , X n − 1 ) = P ( X n ∈ dx n | X n − 1 ) ⇓ P ( X n ∈ dx n | X n − 1 = x n − 1 ) = M n ( x n − 1 , dx n ) ◮ S = R 2 ( x n − a ( x n − 1 )) 2 dx n M n ( x n − 1 , dx n ) = 1 2 δ 0 ( dx n ) + 1 1 e − 1 √ 2 2 π 6/21

Markov transitions P ( X n ∈ dx n | X 0 , . . . , X n − 2 , X n − 1 ) = P ( X n ∈ dx n | X n − 1 ) ⇓ P ( X n ∈ dx n | X n − 1 = x n − 1 ) = M n ( x n − 1 , dx n ) ◮ S = R 2 ( x n − a ( x n − 1 )) 2 dx n M n ( x n − 1 , dx n ) = 1 2 δ 0 ( dx n ) + 1 1 e − 1 √ 2 2 π ◮ S = { e 1 , . . . , e d }   M n ( e 1 , e 1 ) . . . M n ( e 1 , e d ) . . .   . . . M n = . . .   M n ( e d , e 1 ) . . . M n ( e d , e d ) 6/21

Markov transitions P ( X n ∈ dx n | X 0 , . . . , X n − 2 , X n − 1 ) = P ( X n ∈ dx n | X n − 1 ) ⇓ P ( X n ∈ dx n | X n − 1 = x n − 1 ) = M n ( x n − 1 , dx n ) ◮ S = R 2 ( x n − a ( x n − 1 )) 2 dx n M n ( x n − 1 , dx n ) = 1 2 δ 0 ( dx n ) + 1 1 e − 1 √ 2 2 π ◮ S = { e 1 , . . . , e d }   M n ( e 1 , e 1 ) . . . M n ( e 1 , e d ) . . .   . . . M n = . . .   M n ( e d , e 1 ) . . . M n ( e d , e d ) ◮ { e 1 , . . . , e d } ⊂ S = R d � ∀ x n − 1 = e i M n ( x n − 1 , dx n ) = M n ( e i , e j ) δ e j ( dx n ) 1 ≤ j ≤ d 6/21

Advantages (Chapman-Kolmogorov) Transport equation M n ( x n − 1 , dx n ) = η n − 1 ( dx n − 1 ) � � �� � � �� � P ( X n ∈ dx n ) P ( X n ∈ dx n | X n − 1 = x n − 1 ) P ( X n − 1 ∈ dx n − 1 ) = � �� � x n − 1 � η n ( dx n ) = η n − 1 ( dx n − 1 ) M n ( x n − 1 , dx n ) x n − 1 7/21

Advantages (Chapman-Kolmogorov) Transport equation M n ( x n − 1 , dx n ) = η n − 1 ( dx n − 1 ) � � �� � � �� � P ( X n ∈ dx n ) P ( X n ∈ dx n | X n − 1 = x n − 1 ) P ( X n − 1 ∈ dx n − 1 ) = � �� � x n − 1 � η n ( dx n ) = η n − 1 ( dx n − 1 ) M n ( x n − 1 , dx n ) x n − 1 � Dynamical system representation η n = η n − 1 M n = . . . = η 0 M 1 . . . M n with � ( M 1 . . . M n )( x 0 , dx n ) = M 1 ( x 0 , dx 1 ) . . . M n ( x n − 1 , dx n ) x 1 ,..., x n − 1 = P ( X n ∈ dx n | X 0 = x 0 ) 7/21

Advantages (Chapman-Kolmogorov) Transport equation M n ( x n − 1 , dx n ) = η n − 1 ( dx n − 1 ) � � �� � � �� � P ( X n ∈ dx n ) P ( X n ∈ dx n | X n − 1 = x n − 1 ) P ( X n − 1 ∈ dx n − 1 ) = � �� � x n − 1 � η n ( dx n ) = η n − 1 ( dx n − 1 ) M n ( x n − 1 , dx n ) x n − 1 � Dynamical system representation η n = η n − 1 M n = . . . = η 0 M 1 . . . M n with � ( M 1 . . . M n )( x 0 , dx n ) = M 1 ( x 0 , dx 1 ) . . . M n ( x n − 1 , dx n ) x 1 ,..., x n − 1 = P ( X n ∈ dx n | X 0 = x 0 ) Note: S = { e 1 , . . . , e d } ≃ { 1 , . . . , d } � matrix/vector operations 7/21

Random dynamical systems State space models X n = F n ( X n − 1 , W n ) with i.i.d. W n and some initial r.v. X 0 8/21

Random dynamical systems State space models X n = F n ( X n − 1 , W n ) with i.i.d. W n and some initial r.v. X 0 Ex.: Linear Gaussian models X n = A n X n − 1 + B n W n 8/21

Random dynamical systems State space models X n = F n ( X n − 1 , W n ) with i.i.d. W n and some initial r.v. X 0 Ex.: Linear Gaussian models X n = A n X n − 1 + B n W n ⇓ � X n = ( A n . . . A 1 ) X 0 + ( A n . . . A p +1 ) B p W p 1 ≤ p ≤ n 8/21

Random dynamical systems State space models X n = F n ( X n − 1 , W n ) with i.i.d. W n and some initial r.v. X 0 Ex.: Linear Gaussian models X n = A n X n − 1 + B n W n ⇓ � X n = ( A n . . . A 1 ) X 0 + ( A n . . . A p +1 ) B p W p 1 ≤ p ≤ n Dimension 1 : [ A n = a n = a ∈ [0 , 1[ and X ′ n a copy of X n starting at X ′ 0 (same W n )] ⇓ n = a n ( X 0 − X ′ X n − X ′ 0 ) − → n ↑∞ 0 8/21

Stability properties Limit random states X n = F ( X n − 1 , W n ) − → n ↑∞ X ∞ ?? 9/21

Stability properties Limit random states X n = F ( X n − 1 , W n ) − → n ↑∞ X ∞ ?? or ( η n = η n − 1 M ) Law ( X n ) − → n ↑∞ Law ( X ∞ ) := η ∞ ?? = = = = = = = = ⇒ η ∞ = η ∞ M 9/21

Stability properties Limit random states X n = F ( X n − 1 , W n ) − → n ↑∞ X ∞ ?? or ( η n = η n − 1 M ) Law ( X n ) − → n ↑∞ Law ( X ∞ ) := η ∞ ?? = = = = = = = = ⇒ η ∞ = η ∞ M Limiting occupation measures 1 � δ X p − → n ↑∞ Law ( X ∞ )?? n 0 ≤ p < n 9/21

Stability properties Limit random states X n = F ( X n − 1 , W n ) − → n ↑∞ X ∞ ?? or ( η n = η n − 1 M ) Law ( X n ) − → n ↑∞ Law ( X ∞ ) := η ∞ ?? = = = = = = = = ⇒ η ∞ = η ∞ M Limiting occupation measures 1 � δ X p − → n ↑∞ Law ( X ∞ )?? n 0 ≤ p < n � ∀ f : S �→ R (a.k.a. observable [physics literature])   �  1 1 � �  ( dx ) f ( x ) δ X p = f ( X p ) n n 0 ≤ p < n 0 ≤ p < n � − → n ↑∞ f ( x ) P ( X ∞ ∈ dx ) E ( f ( X ∞ )) = 9/21

� � � 2 states model p � 1 0 1 − p 1 − q q � � � 1 − p p M = q 1 − q 10/21

� � � 2 states model p � 1 0 1 − p 1 − q q � � � 1 − p p M = q 1 − q Invariant measure � 1 − p � � � q p p π = p + q , = ⇒ π M ∝ [ q , p ] = [ q , p ] ∝ π q 1 − q p + q 10/21

� � � 2 states model p � 1 0 1 − p 1 − q q � � � 1 − p p M = q 1 − q Invariant measure � 1 − p � � � q p p π = p + q , = ⇒ π M ∝ [ q , p ] = [ q , p ] ∝ π q 1 − q p + q Some question 1 δ X p (1 0 ) = 1 q � � 1 X p =0 ≃ n ↑∞ π (0) = p + q ?? n n 0 ≤ p < n 0 ≤ p < n 10/21

Perron-Frobenius theo � 1 − p � p M = q 1 − q Exercise: ◮ Find eigenvalues λ 1 , λ 2 and eigenvectors ϕ 1 , ϕ 2 . 11/21

Perron-Frobenius theo � 1 − p � p M = q 1 − q Exercise: ◮ Find eigenvalues λ 1 , λ 2 and eigenvectors ϕ 1 , ϕ 2 . ◮ Using the change of variable matrix � λ 1 � 0 P := ( ϕ 1 , ϕ 2 ) � M = PDP − 1 with D = 0 λ 2 11/21

Perron-Frobenius theo � 1 − p � p M = q 1 − q Exercise: ◮ Find eigenvalues λ 1 , λ 2 and eigenvectors ϕ 1 , ϕ 2 . ◮ Using the change of variable matrix � λ 1 � 0 P := ( ϕ 1 , ϕ 2 ) � M = PDP − 1 with D = 0 λ 2 ◮ Elementary matrix operations M 2 = PDP − 1 PDP − 1 = PD 2 P − 1 ⇒ . . . ⇒ M n = PD n P − 1 11/21

Perron-Frobenius theo � 1 − p � p M = q 1 − q Exercise: ◮ Find eigenvalues λ 1 , λ 2 and eigenvectors ϕ 1 , ϕ 2 . ◮ Using the change of variable matrix � λ 1 � 0 P := ( ϕ 1 , ϕ 2 ) � M = PDP − 1 with D = 0 λ 2 ◮ Elementary matrix operations M 2 = PDP − 1 PDP − 1 = PD 2 P − 1 ⇒ . . . ⇒ M n = PD n P − 1 ◮ Key decomposition � � � � π (0) π (1) π (1) − π (1) M n = + λ n 2 π (0) π (1) − π (0) π (0) 11/21