Semimartingale methods for Markov chains, interacting particle - PowerPoint PPT Presentation

Semimartingale methods for Markov chains, interacting particle systems and random growth models A series of 8 live-streamed lectures Chak Hei Lo (Lectures 1,2 and 4) University of Edinburgh LMS PiNE Lectures 2020

Course outline Foster–Lyapunov methods for Markov chains Chak Hei Lo (3 lectures on 10–11 September). Interacting particle systems and martingales Conrado da Costa (3 lectures on 10–11 September). Planar random growth and scaling limits George Liddle and Frankie Higgs (2 lectures on 14 September).

What am I going to cover? Course outline Martingale background Recurrence and transience criteria Positive recurrence criterion Example: random walks on half-strips Example: voter model

References and Acknowledgements Non-homogeneous Random Walks Cambridge University Press 2016 Mikhail Menshikov Serguei Popov Andrew Wade

References and Acknowledgements Topics in the Constructive Theory of Countable Markov Chains Cambridge University Press 1995 Guy Fayolle Vadim Malyshev Mikhail Menshikov

References and Acknowledgements I am grateful to Andrew Wade for fruitful discussions on the topic of these lectures. I am thankful to Nicholas Georgiou for the template of the slides. More references at the end.

Outline Course outline Martingale background Recurrence and transience criteria Positive recurrence criterion Example: random walks on half-strips Example: voter model

Outline Course outline Martingale background Recurrence and transience criteria Positive recurrence criterion Example: random walks on half-strips Example: voter model

Notation Suppose that Z = ( Z n ; n ∈ Z + ) is a real-valued, discrete-time stochastic process adapted to a filtration ( F n ; n ∈ Z + ) . The process Z n is a martingale (with respect to the given filtration) if, for all n ≥ 0, (i) E [ | Z n | ] < ∞ , and (ii) E [ Z n + 1 − Z n | F n ] = 0. If in (ii) ‘ = ’ is replaced by ‘ ≥ ’ (respectively, ‘ ≤ ’), then Z n is called a submartingale (respectively, supermartingale ).

Martingale background Theorem 1 (Convergence of non-negative supermartingales) Suppose Z n ≥ 0 is a supermartingale. Then there is an integrable random variable Z such that Z n → Z a.s. as n → ∞ , and E [ Z ] ≤ E [ Z 0 ] . Theorem 2 (Optional stopping for supermartingales) Suppose Z n ≥ 0 is a supermartingale and σ ≤ τ are stopping times. Then E [ Z τ ] ≤ E [ Z σ ] < ∞ and E [ Z τ | F σ ] ≤ Z σ a.s.

Displacement and exit estimates Theorem 3 Let Z n be an integrable F n -adapted process on R + . Suppose that for some B ∈ R + , E [ Z n + 1 − Z n | F n ] ≤ B a.s. Then for any step n and any x > 0 , � � ≤ Bn + E [ Z 0 ] 0 ≤ m ≤ n Z m ≥ z max . P x

Displacement and exit estimates Proof of Theorem 3 Let τ be a stopping time. Then � � E Z ( m + 1 ) ∧ τ − Z m ∧ τ | F m ≤ B 1 { τ > m } . Taking expectations on both sides we get � � E Z ( m + 1 ) ∧ τ − E [ Z m ∧ τ ] ≤ B P ( τ > m ) . Then summing from m = 0 to m = n − 1 gives n − 1 � E [ Z n ∧ τ ] − E [ Z 0 ] ≤ B P ( τ > m ) ≤ B E [ τ ] . m = 0

Displacement and exit estimates Take τ = n ∧ σ x . Then Bn ≥ B E [ τ ] ≥ E [ Z n ∧ σ x ] − E [ Z 0 ] . But since Z n ≥ 0 we have � � Z n ∧ σ x ≥ x 1 { σ x ≤ n } = x 1 0 ≤ m ≤ n Z m ≥ x max and the result follows. �

Example Let S n = � n k = 1 θ k be simple symmetric random walk on Z . Let Z n = S 2 n . Then n = ( S n + θ n + 1 ) 2 − S 2 Z n + 1 − Z n = S 2 n + 1 − S 2 n = 2 S n θ n + 1 + θ 2 n + 1 . So E [ Z n + 1 − Z n | F n ] = 2 S n E [ θ n + 1 ] + E [ θ 2 n + 1 ] = 1 . Hence we have � � � � 0 ≤ m ≤ n Z m ≥ x 2 P 0 ≤ m ≤ n | S n | ≥ x max = P max ≤ n for x > 0 . x 2 In this case, Z n is a submartingale, so one could use the Doob’s inequality to get the same result.

Example 2 + ε for ε > 0. Then 1 1 2 (log n ) Let u ( n ) = n � � ≤ (log n ) − 1 − 2 ε ) . 0 ≤ m ≤ n | S m | ≥ u ( n ) max P Although this seems a rather weak bound, we can still extract a reasonable result by considering the subsequence n = 2 k , k ≥ 0. Borel-Cantelli shows that max 0 ≤ m ≤ 2 k | S m | ≤ u ( 2 k ) for all but finitely many k , a.s.

Example Any n ∈ N has 2 k n ≤ n ≤ 2 k n + 1 with k n → ∞ as n → ∞ . Hence for all but finitely many n , 0 ≤ m ≤ 2 kn + 1 | S m | ≤ u ( 2 · 2 k n ) ≤ 2 u ( n ) . 0 ≤ m ≤ n | S m | ≤ max max So we have show that for any ε > 0, for all but finitely many n , 1 1 2 + ε . 2 (log n ) 0 ≤ m ≤ n | S m | ≤ n max

Outline Course outline Martingale background Recurrence and transience criteria Positive recurrence criterion Example: random walks on half-strips Example: voter model

Outline Course outline Martingale background Recurrence and transience criteria Positive recurrence criterion Example: random walks on half-strips Example: voter model

Recurrence classification Suppose that X n is an irreducible Markov chain on a countable state space Σ . Recurrent : With probability 1, for every x ∈ Σ , X n = x infinitely often. Transient : With probability 1, for every x ∈ Σ , X n = x only finitely often. Positive recurrent : There exists a probability distribution π on Σ such that n 1 � lim 1 { X k = x } = π ( x ) , a.s., n n →∞ k = 0 for all x ∈ Σ . Necessarily π is a stationary distribution. ( P ( X n = x ) → π ( x ) with some additional aperiodicity. )

Recurrence classification Equivalent definitions (uses irreducibility and strong Markov heavily): For a fixed A ⊆ Σ , we define that τ A = min { n ≥ 0 : X n ∈ A } (stopping / hitting time). We call: X n recurrent if for some finite A , P ( τ A < ∞ | X n = x ) = 1 for all x . X n transient if for some non-empty A , P ( τ A = ∞ | X n = x ) > 0 for all x / ∈ A . X n positive recurrent if for some finite A , E [ τ A | X n = x ] < ∞ for all x .

Recurrence classification Theorem 4 (P´ olya’s Recurrence Theorem) The simple symmetric random walk on Z d is recurrent in one or two dimensions, but transient in three or more dimensions. A quote by Shizuo Kakutani, somewhat ‘equivalent’ to the theorem. ‘A drunken man will find his way home, but a drunken bird may get lost forever.’

Random walk in 2-dimensions 3 simulations on 2-dimensional simple symmetric random walk with 10 5 steps

Random walk in 3-dimensions 3 simulations on 3-dimensional simple symmetric random walk with 10 5 steps

Recurrence and transience criteria Theorem 5 (Recurrence criterion) An irreducible Markov chain X n on a countably infinite state space Σ is recurrent if and only if there exist a function f : Σ → R + and a finite non-empty set A ⊂ Σ such that E [ f ( X n + 1 ) − f ( X n ) | X n = x ] ≤ 0 for all x ∈ Σ \ A , and f ( x ) → ∞ as x → ∞ . A weaker version of the ‘if’ part of this theorem is due to Foster (1953), then improved by Pakes (1969), and the ‘only if’ part by Mertens et al. (1978).

Example Let S n be simple symmetric random walk on Z 2 , and consider � γ � log( 1 + || x || 2 ) f ( x ) = for γ ∈ ( 0 , 1 ) . A Taylor’s theorem computation gives E [ f ( S n + 1 ) − f ( S n ) | S n = x ] = 1 + || x || 2 �� γ − 1 γ ( γ − 1 ) || x || − 2 � � log ( 1 + o ( 1 )) which is < 0 for || x || sufficiently large. Hence S n is recurrent.

Recurrence and transience criteria Proof of Theorem 5 (‘if’ part) Take X 0 = x ∈ Σ . Set Y n = f ( X n ∧ τ A ) . Then Y n is a non-negative supermartingale. Hence Y n → Y ∞ a.s. for some Y ∞ , and E [ Y ∞ | X 0 = x ] ≤ E [ Y 0 | X 0 = x ] = f ( x ) . (1) On the other hand, since f → ∞ , it holds that the set { y ∈ Σ : f ( y ) ≤ M } is finite for any M ∈ R + , so irreducibility implies that lim sup n →∞ f ( X n ) = + ∞ a.s. on { τ A = ∞} . Hence on { τ A = ∞} we must have Y ∞ = lim n →∞ Y n = + ∞ . This would contradict the inequality (1) if we assume P ( τ A = ∞ | X 0 = x ) > 0, because then E [ Y ∞ | X 0 = x ] = ∞ . Hence P ( τ A = ∞ | X 0 = x ) = 0 for all x ∈ Σ , which implies recurrence. �

Recurrence and transience criteria Theorem 6 (Transience criterion) An irreducible Markov chain X n on a countably infinite state space Σ is transient if and only if there exist a function f : Σ → R + and a non-empty set A ⊂ Σ such that E [ f ( X n + 1 ) − f ( X n ) | X n = x ] ≤ 0 for all x ∈ Σ \ A , and f ( y ) < inf x ∈ A f ( x ) for at least one y ∈ Σ \ A. A weaker version of this theorem is due to Foster(1953), then improved by Mertens et al. (1978).

Example Let S n be simple symmetric random walk on Z d . Let α > 0 and consider the function f : Z d → ( 0 , 1 ] defined by f ( 0 ) = 1 and f ( x ) = || x || − 2 α for x � = 0. A Taylor’s theorem computation gives E [ f ( S n + 1 ) − f ( S n ) | S n = x ] = α d || x || − 2 − 2 α ( 2 ( α + 1 ) − d + o ( 1 )) which is < 0 for || x || sufficiently large provided we choose � � 0 , d − 2 α ∈ , which we may do for any d ≥ 3. 2 Thus the simple symmetric random walk is transient if d ≥ 3.