Modern Discrete Probability III - Stopping times and martingales - PowerPoint PPT Presentation

Conditioning Stopping times Martingales Modern Discrete Probability III - Stopping times and martingales Review S´ ebastien Roch UW–Madison Mathematics October 15, 2014 S´ ebastien Roch, UW–Madison Modern Discrete Probability – Martingales

Conditioning Stopping times Martingales Conditioning 1 Stopping times 2 Definitions and examples Some useful results Application: Hitting times and cover times Martingales 3 Definitions and examples Some useful results Application: critical percolation on trees S´ ebastien Roch, UW–Madison Modern Discrete Probability – Martingales

Conditioning Stopping times Martingales Conditioning I Theorem (Conditional expectation) Let X ∈ L 1 (Ω , F , P ) and G ⊆ F a sub σ -field. Then there exists a (a.s.) unique Y ∈ L 1 (Ω , G , P ) (note the G -measurability) s.t. E [ Y ; G ] = E [ X ; G ] , ∀ G ∈ G . Such a Y is called a version of the conditional expectation of X given G and is denoted by E [ X | G ] . Theorem (Conditional expectation: L 2 case) Let � U , V � = E [ UV ] . Let X ∈ L 2 (Ω , F , P ) and G ⊆ F a sub σ -field. Then there exists a (a.s.) unique Y ∈ L 2 (Ω , G , P ) s.t. � X − Y � 2 = inf {� X − W � 2 : W ∈ L 2 (Ω , G , P ) } , and, moreover, � Z , X − Y � = 0 , ∀ Z ∈ L 2 (Ω , G , P ) . S´ ebastien Roch, UW–Madison Modern Discrete Probability – Martingales

Conditioning Stopping times Martingales Conditioning II In addition to linearity and the usual inequalities (e.g. Jensen’s inequality, etc.) and convergence theorems (e.g. dominated convergence, etc.). We highlight the following three properties: Lemma (Taking out what is known) If Z ∈ G is bounded then E [ ZX | G ] = Z E [ X | G ] . Lemma (Role of independence) If H is independent of σ ( σ ( X ) , G ) , then E [ X | σ ( G , H )] = E [ X | G ] . Lemma (Tower property (or law of total probability)) We have E [ E [ X | G ]] = E [ X ] . In fact, if H ⊆ G is a σ -field E [ E [ X | G ] | H ] = E [ X | H ] . S´ ebastien Roch, UW–Madison Modern Discrete Probability – Martingales

Conditioning Definitions and examples Stopping times Some useful results Martingales Application: Hitting times and cover times Conditioning 1 Stopping times 2 Definitions and examples Some useful results Application: Hitting times and cover times Martingales 3 Definitions and examples Some useful results Application: critical percolation on trees S´ ebastien Roch, UW–Madison Modern Discrete Probability – Martingales

Conditioning Definitions and examples Stopping times Some useful results Martingales Application: Hitting times and cover times Filtrations I Definition A filtered space is a tuple (Ω , F , ( F t ) t ∈ Z + , P ) where: (Ω , F , P ) is a probability space ( F t ) t ∈ Z + is a filtration , i.e., F 0 ⊆ F 1 ⊆ · · · ⊆ F ∞ := σ ( ∪F t ) ⊆ F . where each F t is a σ -field. Example Let X 0 , X 1 , . . . be i.i.d. random variables. Then a filtration is given by F t = σ ( X 0 , . . . , X t ) , ∀ t ≥ 0 . S´ ebastien Roch, UW–Madison Modern Discrete Probability – Martingales

Conditioning Definitions and examples Stopping times Some useful results Martingales Application: Hitting times and cover times Filtrations II Fix (Ω , F , ( F t ) t ∈ Z + , P ) . Definition (Adapted process) A process ( W t ) t is adapted if W t ∈ F t for all t . Example (Continued) Let ( S t ) t where S t = � i ≤ t X i is adapted. S´ ebastien Roch, UW–Madison Modern Discrete Probability – Martingales

Conditioning Definitions and examples Stopping times Some useful results Martingales Application: Hitting times and cover times Stopping times I Definition A random variable τ : Ω → Z + := { 0 , 1 , . . . , + ∞} is called a stopping time if { τ ≤ t } ∈ F t , ∀ t ∈ Z + , or, equivalently, { τ = t } ∈ F t , ∀ t ∈ Z + . (To see the equivalence, note { τ = t } = { τ ≤ t } \ { τ ≤ t − 1 } , and { τ ≤ t } = ∪ i ≤ t { τ = i } . ) Example Let ( A t ) t ∈ Z + , with values in ( E , E ) , be adapted and B ∈ E . Then τ = inf { t ≥ 0 : A t ∈ B } , is a stopping time. S´ ebastien Roch, UW–Madison Modern Discrete Probability – Martingales

Conditioning Definitions and examples Stopping times Some useful results Martingales Application: Hitting times and cover times Stopping times II Definition (The σ -field F τ ) Let τ be a stopping time. Denote by F τ the set of all events F such that ∀ t ∈ Z + F ∩ { τ = t } ∈ F t . Lemma F τ = F t if τ ≡ t, F τ = F ∞ if τ ≡ ∞ and F τ ⊆ F ∞ for any τ . Lemma If ( X t ) is adapted and τ is a stopping time then X τ ∈ F τ . Lemma If σ, τ are stopping times then F σ ∧ τ ⊆ F τ . S´ ebastien Roch, UW–Madison Modern Discrete Probability – Martingales

Conditioning Definitions and examples Stopping times Some useful results Martingales Application: Hitting times and cover times Examples Let ( X t ) be a Markov chain on a countable space V . Example (Hitting time) The first visit time and first return time to x ∈ V are τ + τ x := inf { t ≥ 0 : X t = x } and x := inf { t ≥ 1 : X t = x } . Similarly, τ B and τ + B are the first visit and first return to B ⊆ V . Example (Cover time) Assume V is finite. The cover time of ( X t ) is the first time that all states have been visited, i.e., τ cov := inf { t ≥ 0 : { X 0 , . . . , X t } = V } . S´ ebastien Roch, UW–Madison Modern Discrete Probability – Martingales

Conditioning Definitions and examples Stopping times Some useful results Martingales Application: Hitting times and cover times Strong Markov property Let ( X t ) be a Markov chain and let F t = σ ( X 0 , . . . , X t ) . The Markov property extends to stopping times. Let τ be a stopping time with P [ τ < + ∞ ] > 0 and let f t : V ∞ → R be a sequence of measurable functions, uniformly bounded in t and let F t ( x ) := E x [ f t (( X t ) t ≥ 0 )] , then (see [D, Thm 6.3.4]): Theorem (Strong Markov property) E [ f τ (( X τ + t ) t ≥ 0 ) | F τ ] = F τ ( X τ ) on { τ < + ∞} Proof: Let A ∈ F τ . Summing over the value of τ and using Markov � E [ f τ (( X τ + t ) t ≥ 0 ); A ∩ { τ < + ∞} ] = E [ f s (( X s + t ) t ≥ 0 ); A ∩ { τ = s } ] s ≥ 0 � = E [ F s ( X s ); A ∩ { τ = s } ] = E [ F τ ( X τ ); A ∩ { τ < + ∞} ] . s ≥ 0 S´ ebastien Roch, UW–Madison Modern Discrete Probability – Martingales

Conditioning Definitions and examples Stopping times Some useful results Martingales Application: Hitting times and cover times Reflection principle I Theorem Let X 1 , X 2 , . . . be i.i.d. with a distribution symmetric about 0 and let S t = � i ≤ t X i . Then, for b > 0 , � � sup S i ≥ b ≤ 2 P [ S t ≥ b ] . P i ≤ t Proof: Let τ := inf { i ≤ t : S i ≥ b } . By the strong Markov property, on { τ < t } , S t − S τ is independent on F τ and is symmetric about 0. In particular, it has probability at least 1 / 2 of being greater or equal to 0 (which implies that S t is greater or equal to b ). Hence P [ S t ≥ b ] ≥ P [ τ = t ] + 1 2 P [ τ < t ] ≥ 1 2 P [ τ ≤ t ] . S´ ebastien Roch, UW–Madison Modern Discrete Probability – Martingales

Conditioning Definitions and examples Stopping times Some useful results Martingales Application: Hitting times and cover times Reflection principle II Theorem Let ( S t ) be simple random walk on Z . Then, ∀ a , b , t > 0 , � � P 0 [ S t = b + a ] = P 0 S t = b − a , sup S i ≥ b . i ≤ t Theorem (Ballot theorem) In an election with n voters, candidate A gets α votes and candidate B gets β < α votes. The probability that A leads B throughout the counting is α − β n . S´ ebastien Roch, UW–Madison Modern Discrete Probability – Martingales

Conditioning Definitions and examples Stopping times Some useful results Martingales Application: Hitting times and cover times Recurrence I Let ( X t ) be a Markov chain on a countable state space V . The time of k-th return to y is (letting τ 0 y := 0) τ k y := inf { t > τ k − 1 : X t = y } . y In particular, τ 1 y ≡ τ + y . Define ρ xy := P x [ τ + y < + ∞ ] . Then by the strong Markov property P x [ τ k y < + ∞ ] = ρ xy ρ k − 1 yy . Letting N y := � ρ xy t > 0 ✶ { X t = y } , by linearity E x [ N y ] = 1 − ρ yy . So either ρ yy < 1 and E y [ N y ] < + ∞ or ρ yy = 1 and τ k y < + ∞ a.s. for all k . S´ ebastien Roch, UW–Madison Modern Discrete Probability – Martingales

Conditioning Definitions and examples Stopping times Some useful results Martingales Application: Hitting times and cover times Recurrence II Definition (Recurrent state) A state x is recurrent if ρ xx = 1. Otherwise it is transient . A chain is recurrent or transient if all its states are. If x is recurrent and E x [ τ + x ] < + ∞ , we say that x is positive recurrent . Lemma: If x is recurrent and ρ xy > 0 then y is recurrent and ρ yx = ρ xy = 1. A subset C ⊆ V is closed if x ∈ C and ρ xy > 0 implies y ∈ C . A subset D ⊆ V is irreducible if x , y ∈ D implies ρ xy > 0. Theorem (Decomposition theorem) Let R := { x : ρ xx = 1 } be the recurrent states of the chain. Then R can be written as a disjoint union ∪ j R j where each R j is closed and irreducible. S´ ebastien Roch, UW–Madison Modern Discrete Probability – Martingales