continuous time markov chains construction and basic tools
play

Continuous time Markov Chains: construction and basic tools Conrado - PowerPoint PPT Presentation

Feb 20, 2023 •109 likes •1.29k views

Continuous time Markov Chains: construction and basic tools Conrado da Costa Department of Mathematical Sciences (Durham University) email: conrado.da-costa@durham.ac.uk September, 2020 1 / 21 Outline Construction Finite Infinite

  1. Infinite state spaces Challenges of infinite state spaces: sup x r ( x ) = ∞ . Explosion, Implosions Solutions: Construction up to explosion, Construction via limits. 7 / 21

  2. Infinite state spaces Challenges of infinite state spaces: sup x r ( x ) = ∞ . Explosion, Implosions Solutions: Construction up to explosion, Construction via limits. This is the case of Interacting Particle Systems (IPS) 7 / 21

  3. Interacting Particle Systems S = X V , η ∈ S , x , y ∈ V 8 / 21

  4. Interacting Particle Systems S = X V , η ∈ S , x , y ∈ V Simple cases { 0 , 1 } N - not infinite, but very big and sparse. 8 / 21

  5. Interacting Particle Systems S = X V , η ∈ S , x , y ∈ V Simple cases { 0 , 1 } N - not infinite, but very big and sparse. Local interaction. Local transformation maps: Γ m : S → S , m ∈ M . 8 / 21

  6. Interacting Particle Systems S = X V , η ∈ S , x , y ∈ V Simple cases { 0 , 1 } N - not infinite, but very big and sparse. Local interaction. Local transformation maps: Γ m : S → S , m ∈ M . rates { r ( η, m ) , m ∈ S } . 8 / 21

  7. Interacting Particle Systems S = X V , η ∈ S , x , y ∈ V Simple cases { 0 , 1 } N - not infinite, but very big and sparse. Local interaction. Local transformation maps: Γ m : S → S , m ∈ M . rates { r ( η, m ) , m ∈ S } . Formal generators � Lf ( η ) = r ( η, m )[ f (Γ m η ) − f ( η )] m 8 / 21

  8. Interacting Particle Systems S = X V , η ∈ S , x , y ∈ V Simple cases { 0 , 1 } N - not infinite, but very big and sparse. Local interaction. Local transformation maps: Γ m : S → S , m ∈ M . rates { r ( η, m ) , m ∈ S } . Formal generators � Lf ( η ) = r ( η, m )[ f (Γ m η ) − f ( η )] m V = Z : 8 / 21

  9. Interacting Particle Systems S = X V , η ∈ S , x , y ∈ V Simple cases { 0 , 1 } N - not infinite, but very big and sparse. Local interaction. Local transformation maps: Γ m : S → S , m ∈ M . rates { r ( η, m ) , m ∈ S } . Formal generators � Lf ( η ) = r ( η, m )[ f (Γ m η ) − f ( η )] m V = Z : uncountable state space 8 / 21

  10. Overview What is the object of interest here? 9 / 21

  11. Overview What is the object of interest here? � d = L ∗ µ t dt µ t = δ x µ 0 9 / 21

  12. Overview What is the object of interest here? � d = L ∗ µ t dt µ t = δ x µ 0 Goal: To understand the behavior of µ t in the relevant scales. 9 / 21

  13. Questions The type of questions we look: 10 / 21

  14. Questions The type of questions we look: ◮ evolution of S t ( f ) for test functions f . 10 / 21

  15. Questions The type of questions we look: ◮ evolution of S t ( f ) for test functions f . ◮ invariant measures 10 / 21

  16. Questions The type of questions we look: ◮ evolution of S t ( f ) for test functions f . ◮ invariant measures ◮ convergence 10 / 21

  17. Questions The type of questions we look: ◮ evolution of S t ( f ) for test functions f . ◮ invariant measures ◮ convergence ◮ effects of space-time rescaling for S n · 10 / 21

  18. Questions The type of questions we look: ◮ evolution of S t ( f ) for test functions f . ◮ invariant measures ◮ convergence ◮ effects of space-time rescaling for S n · ◮ An attempt of explaining physical phenomena: 10 / 21

  19. Questions The type of questions we look: ◮ evolution of S t ( f ) for test functions f . ◮ invariant measures ◮ convergence ◮ effects of space-time rescaling for S n · ◮ An attempt of explaining physical phenomena: ◮ fluid equation 10 / 21

  20. Questions The type of questions we look: ◮ evolution of S t ( f ) for test functions f . ◮ invariant measures ◮ convergence ◮ effects of space-time rescaling for S n · ◮ An attempt of explaining physical phenomena: ◮ fluid equation ◮ state of matter 10 / 21

  21. Questions The type of questions we look: ◮ evolution of S t ( f ) for test functions f . ◮ invariant measures ◮ convergence ◮ effects of space-time rescaling for S n · ◮ An attempt of explaining physical phenomena: ◮ fluid equation ◮ state of matter ◮ phase transition 10 / 21

  22. Tools 11 / 21

  23. Tools ◮ Kolmogorov equations 11 / 21

  24. Tools ◮ Kolmogorov equations ◮ Dynkin Martingales 11 / 21

  25. Tools ◮ Kolmogorov equations ◮ Dynkin Martingales ◮ Tightness criteria 11 / 21

  26. Tools ◮ Kolmogorov equations ◮ Dynkin Martingales ◮ Tightness criteria ◮ Martingale problems 11 / 21

  27. Tools ◮ Kolmogorov equations ◮ Dynkin Martingales ◮ Tightness criteria ◮ Martingale problems ◮ Limits of IPS 11 / 21

  28. Forward Kolmogorov equations 12 / 21

  29. Forward Kolmogorov equations � ∂ t P ( s , x ; t , • ) = L ∗ P ( s , x ; t , • ) lim t ↓ s P ( s , x ; t , • ) = δ x ( • ) 12 / 21

  30. Forward Kolmogorov equations � ∂ t P ( s , x ; t , • ) = L ∗ P ( s , x ; t , • ) lim t ↓ s P ( s , x ; t , • ) = δ x ( • ) � Let v ( t , x ) := E [ ϕ ( X t ) | X s = x ] = P ( s , x ; t , dy ) ϕ ( y ) 12 / 21

  31. Forward Kolmogorov equations � ∂ t P ( s , x ; t , • ) = L ∗ P ( s , x ; t , • ) lim t ↓ s P ( s , x ; t , • ) = δ x ( • ) � Let v ( t , x ) := E [ ϕ ( X t ) | X s = x ] = P ( s , x ; t , dy ) ϕ ( y ) � ∂ t v ( t , x ) = ∂ t P ( s , x ; t , dy ) ϕ ( y ) 12 / 21

  32. Forward Kolmogorov equations � ∂ t P ( s , x ; t , • ) = L ∗ P ( s , x ; t , • ) lim t ↓ s P ( s , x ; t , • ) = δ x ( • ) � Let v ( t , x ) := E [ ϕ ( X t ) | X s = x ] = P ( s , x ; t , dy ) ϕ ( y ) � ∂ t v ( t , x ) = ∂ t P ( s , x ; t , dy ) ϕ ( y ) � � P ( s , x ; t , dy ) h − 1 ∂ t v ( t , x ) = lim P ( t , y ; t + h , dz )( ϕ ( z ) − ϕ ( y )) h 12 / 21

  33. Forward Kolmogorov equations � ∂ t P ( s , x ; t , • ) = L ∗ P ( s , x ; t , • ) lim t ↓ s P ( s , x ; t , • ) = δ x ( • ) � Let v ( t , x ) := E [ ϕ ( X t ) | X s = x ] = P ( s , x ; t , dy ) ϕ ( y ) � ∂ t v ( t , x ) = ∂ t P ( s , x ; t , dy ) ϕ ( y ) � � P ( s , x ; t , dy ) h − 1 ∂ t v ( t , x ) = lim P ( t , y ; t + h , dz )( ϕ ( z ) − ϕ ( y )) h � = P ( s , x ; t , dy ) L ϕ ( y ) 12 / 21

  34. Forward Kolmogorov equations � ∂ t P ( s , x ; t , • ) = L ∗ P ( s , x ; t , • ) lim t ↓ s P ( s , x ; t , • ) = δ x ( • ) � Let v ( t , x ) := E [ ϕ ( X t ) | X s = x ] = P ( s , x ; t , dy ) ϕ ( y ) � ∂ t v ( t , x ) = ∂ t P ( s , x ; t , dy ) ϕ ( y ) � � P ( s , x ; t , dy ) h − 1 ∂ t v ( t , x ) = lim P ( t , y ; t + h , dz )( ϕ ( z ) − ϕ ( y )) h � � = P ( s , x ; t , dy ) L ϕ ( y ) = L ∗ P ( s , x ; t , dy ) ϕ ( y ) 12 / 21

  35. Backward Kolmogorov equations � − ∂ s P ( s , x ; t , • ) = LP ( s , x ; t , • ) lim s ↑ t P ( s , x ; t , • ) = δ x ( • ) 13 / 21

  36. Backward Kolmogorov equations � − ∂ s P ( s , x ; t , • ) = LP ( s , x ; t , • ) lim s ↑ t P ( s , x ; t , • ) = δ x ( • ) � Let w ( s , x ) := E [ ϕ ( X t ) | X s = x ] = P ( s , x ; t , dy ) ϕ ( y ) 13 / 21

  37. Backward Kolmogorov equations � − ∂ s P ( s , x ; t , • ) = LP ( s , x ; t , • ) lim s ↑ t P ( s , x ; t , • ) = δ x ( • ) � Let w ( s , x ) := E [ ϕ ( X t ) | X s = x ] = P ( s , x ; t , dy ) ϕ ( y ) � ∂ s w ( s , x ) = − ∂ s P ( s , x ; t , dy ) ϕ ( y ) 13 / 21

  38. Backward Kolmogorov equations � − ∂ s P ( s , x ; t , • ) = LP ( s , x ; t , • ) lim s ↑ t P ( s , x ; t , • ) = δ x ( • ) � Let w ( s , x ) := E [ ϕ ( X t ) | X s = x ] = P ( s , x ; t , dy ) ϕ ( y ) � ∂ s w ( s , x ) = − ∂ s P ( s , x ; t , dy ) ϕ ( y ) � h h − 1 − ∂ s w ( s , x ) = lim P ( s − h , x ; s , dy )[ w ( s , y ) − w ( s , x )] 13 / 21

  39. Backward Kolmogorov equations � − ∂ s P ( s , x ; t , • ) = LP ( s , x ; t , • ) lim s ↑ t P ( s , x ; t , • ) = δ x ( • ) � Let w ( s , x ) := E [ ϕ ( X t ) | X s = x ] = P ( s , x ; t , dy ) ϕ ( y ) � ∂ s w ( s , x ) = − ∂ s P ( s , x ; t , dy ) ϕ ( y ) � h h − 1 − ∂ s w ( s , x ) = lim P ( s − h , x ; s , dy )[ w ( s , y ) − w ( s , x )] h h − 1 [ Lw ( s − h , x ) h + o ( h )] = lim 13 / 21

  40. Backward Kolmogorov equations � − ∂ s P ( s , x ; t , • ) = LP ( s , x ; t , • ) lim s ↑ t P ( s , x ; t , • ) = δ x ( • ) � Let w ( s , x ) := E [ ϕ ( X t ) | X s = x ] = P ( s , x ; t , dy ) ϕ ( y ) � ∂ s w ( s , x ) = − ∂ s P ( s , x ; t , dy ) ϕ ( y ) � h h − 1 − ∂ s w ( s , x ) = lim P ( s − h , x ; s , dy )[ w ( s , y ) − w ( s , x )] h h − 1 [ Lw ( s − h , x ) h + o ( h )] = Lw ( s , x ) = lim 13 / 21

  41. Dynkin Martingales If � < C � � ∂ j � F : R + × S → R sup s F ( s , x ) ( s , x ) 14 / 21

  42. Dynkin Martingales If � < C � � ∂ j � F : R + × S → R sup s F ( s , x ) ( s , x ) then � t � M F ( t ) := F ( t , X t ) − F (0 , X 0 ) − 0 ( ∂ s + L ) F ( s , X s ) ds � t N F ( t ) := [ M F ( t )] 2 − 0 ( QF )( s , X s ) ds are martingales, 14 / 21

  43. Dynkin Martingales If � < C � � ∂ j � F : R + × S → R sup s F ( s , x ) ( s , x ) then � t � M F ( t ) := F ( t , X t ) − F (0 , X 0 ) − 0 ( ∂ s + L ) F ( s , X s ) ds � t N F ( t ) := [ M F ( t )] 2 − 0 ( QF )( s , X s ) ds are martingales, where QF ( s , x ) := LF 2 ( s , x ) − 2 F ( s , x ) LF ( s , x ) 14 / 21

  44. Dynkin Martingales If � < C � � ∂ j � F : R + × S → R sup s F ( s , x ) ( s , x ) then � t � M F ( t ) := F ( t , X t ) − F (0 , X 0 ) − 0 ( ∂ s + L ) F ( s , X s ) ds � t N F ( t ) := [ M F ( t )] 2 − 0 ( QF )( s , X s ) ds are martingales, where QF ( s , x ) := LF 2 ( s , x ) − 2 F ( s , x ) LF ( s , x ) = � r xy [ F ( s , y ) − F ( s , x )] 2 y 14 / 21

  45. Proof (Dynkin) � t M F ( t ) − M F ( s ) = F ( t , X t ) − F ( s , X s ) − ( ∂ r + L ) F ( r , X r ) dr s 15 / 21

  46. Proof (Dynkin) � t M F ( t ) − M F ( s ) = F ( t , X t ) − F ( s , X s ) − ( ∂ r + L ) F ( r , X r ) dr s E [ M F ( t ) − M F ( s ) |F s ] = 0 15 / 21

  47. Proof (Dynkin) � t M F ( t ) − M F ( s ) = F ( t , X t ) − F ( s , X s ) − ( ∂ r + L ) F ( r , X r ) dr s E [ M F ( t ) − M F ( s ) |F s ] = 0 ϕ ( r ) := E [ M F ( r ) − M F ( s ) |F s ] ϕ ′ ( r ) = 0 ϕ ( s ) = 0 15 / 21

  48. Proof (Dynkin) � t M F ( t ) − M F ( s ) = F ( t , X t ) − F ( s , X s ) − ( ∂ r + L ) F ( r , X r ) dr s E [ M F ( t ) − M F ( s ) |F s ] = 0 ϕ ( r ) := E [ M F ( r ) − M F ( s ) |F s ] ϕ ′ ( r ) = 0 ϕ ( s ) = 0 � r + h h − 1 E [ ( ∂ u + L ) F ( u , X u ) du |F s ] → E [( ∂ r + L ) F ( r , X r ) |F s ] r 15 / 21

  49. Proof (Dynkin) � t M F ( t ) − M F ( s ) = F ( t , X t ) − F ( s , X s ) − ( ∂ r + L ) F ( r , X r ) dr s E [ M F ( t ) − M F ( s ) |F s ] = 0 ϕ ( r ) := E [ M F ( r ) − M F ( s ) |F s ] ϕ ′ ( r ) = 0 ϕ ( s ) = 0 � r + h h − 1 E [ ( ∂ u + L ) F ( u , X u ) du |F s ] → E [( ∂ r + L ) F ( r , X r ) |F s ] r h − 1 E [ F ( r + h , X r + h ) − F ( r , X r ) |F s ] = 15 / 21

  50. Proof (Dynkin) � t M F ( t ) − M F ( s ) = F ( t , X t ) − F ( s , X s ) − ( ∂ r + L ) F ( r , X r ) dr s E [ M F ( t ) − M F ( s ) |F s ] = 0 ϕ ( r ) := E [ M F ( r ) − M F ( s ) |F s ] ϕ ′ ( r ) = 0 ϕ ( s ) = 0 � r + h h − 1 E [ ( ∂ u + L ) F ( u , X u ) du |F s ] → E [( ∂ r + L ) F ( r , X r ) |F s ] r h − 1 E [ F ( r + h , X r + h ) − F ( r , X r ) |F s ] = h − 1 E [ F ( r + h , X r + h ) − F ( r , X r ) |F s ] + h − 1 E [ F ( r , X r + h ) − F ( r , X r ) |F s ] 15 / 21

  51. Proof (Dynkin) � t M F ( t ) − M F ( s ) = F ( t , X t ) − F ( s , X s ) − ( ∂ r + L ) F ( r , X r ) dr s E [ M F ( t ) − M F ( s ) |F s ] = 0 ϕ ( r ) := E [ M F ( r ) − M F ( s ) |F s ] ϕ ′ ( r ) = 0 ϕ ( s ) = 0 � r + h h − 1 E [ ( ∂ u + L ) F ( u , X u ) du |F s ] → E [( ∂ r + L ) F ( r , X r ) |F s ] r h − 1 E [ F ( r + h , X r + h ) − F ( r , X r ) |F s ] = h − 1 E [ F ( r + h , X r + h ) − F ( r , X r ) |F s ] + h − 1 E [ F ( r , X r + h ) − F ( r , X r ) |F s ] → E [ ∂ r F ( r , X r ) |F s ] + E [ LF ( r , X r ) |F s ] 15 / 21

  52. Proof (Dynkin) � t � t N F ( t ) := [ M F ( t )] 2 − M F ( t ) = F ( t , X t ) − F (0 , X 0 ) − ( QF )( s , X s ) ds ( ∂ s + L ) F ( s , X s ) ds 0 0 16 / 21

  53. Proof (Dynkin) � t � t N F ( t ) := [ M F ( t )] 2 − M F ( t ) = F ( t , X t ) − F (0 , X 0 ) − ( QF )( s , X s ) ds ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t 0 ( t ) := M F ( t ) + F (0 , X 0 ) = F ( t , X t ) − F 2 ( t , X t ) ∼ ( ∂ s + L ) F 2 ( s , X s ) ds M F ( ∂ s + L ) F ( s , X s ) ds 0 0 16 / 21

  54. Proof (Dynkin) � t � t N F ( t ) := [ M F ( t )] 2 − M F ( t ) = F ( t , X t ) − F (0 , X 0 ) − ( QF )( s , X s ) ds ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t 0 ( t ) := M F ( t ) + F (0 , X 0 ) = F ( t , X t ) − F 2 ( t , X t ) ∼ ( ∂ s + L ) F 2 ( s , X s ) ds M F ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t Integration by parts: G t M t = 0 G s dM s + 0 M s dG s 16 / 21

  55. Proof (Dynkin) � t � t N F ( t ) := [ M F ( t )] 2 − M F ( t ) = F ( t , X t ) − F (0 , X 0 ) − ( QF )( s , X s ) ds ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t 0 ( t ) := M F ( t ) + F (0 , X 0 ) = F ( t , X t ) − F 2 ( t , X t ) ∼ ( ∂ s + L ) F 2 ( s , X s ) ds M F ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t Integration by parts: G t M t = 0 G s dM s + 0 M s dG s � t � t � s M F � � 0 ( t ) ( ∂ s + L ) F ( s , X s ) ds ∼ F ( s , X s ) − ( ∂ r + L ) F ( r , X r ) dr ( ∂ s + L ) F ( s , X s ) ds 0 0 0 16 / 21

  56. Proof (Dynkin) � t � t N F ( t ) := [ M F ( t )] 2 − M F ( t ) = F ( t , X t ) − F (0 , X 0 ) − ( QF )( s , X s ) ds ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t 0 ( t ) := M F ( t ) + F (0 , X 0 ) = F ( t , X t ) − F 2 ( t , X t ) ∼ ( ∂ s + L ) F 2 ( s , X s ) ds M F ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t Integration by parts: G t M t = 0 G s dM s + 0 M s dG s � t � t � s M F � � 0 ( t ) ( ∂ s + L ) F ( s , X s ) ds ∼ F ( s , X s ) − ( ∂ r + L ) F ( r , X r ) dr ( ∂ s + L ) F ( s , X s ) ds 0 0 0 � t �� t � 2 [ M F ( t )] 2 = F 2 ( t , X t ) − 2 F ( t , X t ) ( ∂ s + L ) F ( s , X s ) ds + ( ∂ s + L ) F ( s , X s ) ds 0 0 16 / 21

  57. Proof (Dynkin) � t � t N F ( t ) := [ M F ( t )] 2 − M F ( t ) = F ( t , X t ) − F (0 , X 0 ) − ( QF )( s , X s ) ds ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t 0 ( t ) := M F ( t ) + F (0 , X 0 ) = F ( t , X t ) − F 2 ( t , X t ) ∼ ( ∂ s + L ) F 2 ( s , X s ) ds M F ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t Integration by parts: G t M t = 0 G s dM s + 0 M s dG s � t � t � s M F � � 0 ( t ) ( ∂ s + L ) F ( s , X s ) ds ∼ F ( s , X s ) − ( ∂ r + L ) F ( r , X r ) dr ( ∂ s + L ) F ( s , X s ) ds 0 0 0 � t �� t � 2 [ M F ( t )] 2 = F 2 ( t , X t ) − 2 F ( t , X t ) ( ∂ s + L ) F ( s , X s ) ds + ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t �� t � 2 ( ∂ s + L ) F 2 ( s , X s ) ds − 2 M F ∼ 0 ( t ) ( ∂ s + L ) F ( s , X s ) ds − ( ∂ s + L ) F ( s , X s ) ds 0 0 0 16 / 21

  58. Proof (Dynkin) � t � t N F ( t ) := [ M F ( t )] 2 − M F ( t ) = F ( t , X t ) − F (0 , X 0 ) − ( QF )( s , X s ) ds ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t 0 ( t ) := M F ( t ) + F (0 , X 0 ) = F ( t , X t ) − F 2 ( t , X t ) ∼ ( ∂ s + L ) F 2 ( s , X s ) ds M F ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t Integration by parts: G t M t = 0 G s dM s + 0 M s dG s � t � t � s M F � � 0 ( t ) ( ∂ s + L ) F ( s , X s ) ds ∼ F ( s , X s ) − ( ∂ r + L ) F ( r , X r ) dr ( ∂ s + L ) F ( s , X s ) ds 0 0 0 � t �� t � 2 [ M F ( t )] 2 = F 2 ( t , X t ) − 2 F ( t , X t ) ( ∂ s + L ) F ( s , X s ) ds + ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t �� t � 2 ( ∂ s + L ) F 2 ( s , X s ) ds − 2 M F ∼ 0 ( t ) ( ∂ s + L ) F ( s , X s ) ds − ( ∂ s + L ) F ( s , X s ) ds 0 0 0 � t � t ( ∂ s + L ) F 2 ( s , X s ) ds − 2 F ( s , X s )( ∂ s + L ) F ( s , X s ) ds ∼ 0 0 � t � s �� t � 2 + 2 ( ∂ r + L ) F ( r , X r ) dr ( ∂ s + L ) F ( s , X s ) ds − ( ∂ s + L ) F ( s , X s ) ds 0 0 0 16 / 21

  59. Proof (Dynkin) � t � t N F ( t ) := [ M F ( t )] 2 − M F ( t ) = F ( t , X t ) − F (0 , X 0 ) − ( QF )( s , X s ) ds ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t 0 ( t ) := M F ( t ) + F (0 , X 0 ) = F ( t , X t ) − F 2 ( t , X t ) ∼ ( ∂ s + L ) F 2 ( s , X s ) ds M F ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t Integration by parts: G t M t = 0 G s dM s + 0 M s dG s � t � t � s M F � � 0 ( t ) ( ∂ s + L ) F ( s , X s ) ds ∼ F ( s , X s ) − ( ∂ r + L ) F ( r , X r ) dr ( ∂ s + L ) F ( s , X s ) ds 0 0 0 � t �� t � 2 [ M F ( t )] 2 = F 2 ( t , X t ) − 2 F ( t , X t ) ( ∂ s + L ) F ( s , X s ) ds + ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t �� t � 2 ( ∂ s + L ) F 2 ( s , X s ) ds − 2 M F ∼ 0 ( t ) ( ∂ s + L ) F ( s , X s ) ds − ( ∂ s + L ) F ( s , X s ) ds 0 0 0 � t � t ( ∂ s + L ) F 2 ( s , X s ) ds − 2 F ( s , X s )( ∂ s + L ) F ( s , X s ) ds ∼ 0 0 16 / 21

  60. Proof (Dynkin) � t � t N F ( t ) := [ M F ( t )] 2 − M F ( t ) = F ( t , X t ) − F (0 , X 0 ) − ( QF )( s , X s ) ds ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t 0 ( t ) := M F ( t ) + F (0 , X 0 ) = F ( t , X t ) − F 2 ( t , X t ) ∼ ( ∂ s + L ) F 2 ( s , X s ) ds M F ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t Integration by parts: G t M t = 0 G s dM s + 0 M s dG s � t � t � s M F � � 0 ( t ) ( ∂ s + L ) F ( s , X s ) ds ∼ F ( s , X s ) − ( ∂ r + L ) F ( r , X r ) dr ( ∂ s + L ) F ( s , X s ) ds 0 0 0 � t �� t � 2 [ M F ( t )] 2 = F 2 ( t , X t ) − 2 F ( t , X t ) ( ∂ s + L ) F ( s , X s ) ds + ( ∂ s + L ) F ( s , X s ) ds 0 0 � t � t �� t � 2 ( ∂ s + L ) F 2 ( s , X s ) ds − 2 M F ∼ 0 ( t ) ( ∂ s + L ) F ( s , X s ) ds − ( ∂ s + L ) F ( s , X s ) ds 0 0 0 � t � t � t ( ∂ s + L ) F 2 ( s , X s ) ds − 2 F ( s , X s )( ∂ s + L ) F ( s , X s ) ds ∼ ( QF )( s , X s ) ds ∼ 0 0 0 16 / 21

  61. Tightness criterion Family of processes ( X N · , N ∈ N ) 17 / 21

  62. Tightness criterion Family of processes ( X N · , N ∈ N ) is tight if N P ( X N ∈ K ( ε )) > 1 − ε. ∀ ε > 0 , ∃K ( ε ); inf 17 / 21

  63. Tightness criterion Family of processes ( X N · , N ∈ N ) is tight if N P ( X N ∈ K ( ε )) > 1 − ε. ∀ ε > 0 , ∃K ( ε ); inf Aldous’s criterion. 17 / 21

  64. Tightness criterion Family of processes ( X N · , N ∈ N ) is tight if N P ( X N ∈ K ( ε )) > 1 − ε. ∀ ε > 0 , ∃K ( ε ); inf Aldous’s criterion. Probabilistic version of Arzel´ a-Ascoli: 17 / 21

  65. Tightness criterion Family of processes ( X N · , N ∈ N ) is tight if N P ( X N ∈ K ( ε )) > 1 − ε. ∀ ε > 0 , ∃K ( ε ); inf Aldous’s criterion. Probabilistic version of Arzel´ a-Ascoli: P [ X N 1) sup ∈ K ( t , ǫ )] ≤ ǫ / t N P [ X N : ω ′ ( X N , γ ) > ǫ ] = 0 2) γ → 0 lim sup lim N 17 / 21

Recommend

Markov Chains Markov Processes Discrete-time Markov Chains Continuous-time Markov Chains Dr

Markov Chains Markov Processes Discrete-time Markov Chains Continuous-time Markov Chains Dr

Markov Chains Markov Processes Discrete-time Markov Chains Continuous-time Markov Chains Dr Conor McArdle EE414 - Markov Chains 1/30 Markov Processes A Markov Process is a stochastic process X t with the Markov property : Pr ( X t n x n |

491 views • 30 slides
Imprecise Markov chains From basic theory to applications II prof. Jasper De Bock Imprecise

Imprecise Markov chains From basic theory to applications II prof. Jasper De Bock Imprecise

Imprecise Markov chains From basic theory to applications II prof. Jasper De Bock Imprecise continuous-time Markov chains Imprecise continuous-time Markov chains Continuous-time Markov chains Continuous-time Markov chains Markov assumption

807 views • 54 slides
Discrete Time Markov Chains Discrete-Time Markov Chains Books - Introduction to Stochastic

Discrete Time Markov Chains Discrete-Time Markov Chains Books - Introduction to Stochastic

Markov Processes and Applications Markov Processes and Applications Discrete-Time Markov Chains D k h Continuous Time Markov Chains Continuous-Time Markov Chains Applications Applications Queuing theory Performance

977 views • 41 slides
Chapter 5. Continuous-Time Markov Chains Prof. Shun-Ren Yang Department of Computer Science,

Chapter 5. Continuous-Time Markov Chains Prof. Shun-Ren Yang Department of Computer Science,

Chapter 5. Continuous-Time Markov Chains Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan Continuous-Time Markov Chains Consider a continuous-time stochastic process { X ( t ) , t 0 } taking on

577 views • 29 slides
Overview Motivation Verifying Continuous-Time Markov Chains 1 Lecture 1+2: Discrete-Time Markov

Overview Motivation Verifying Continuous-Time Markov Chains 1 Lecture 1+2: Discrete-Time Markov

Verifying Continuous-Time Markov Chains Verifying Continuous-Time Markov Chains Overview Motivation Verifying Continuous-Time Markov Chains 1 Lecture 1+2: Discrete-Time Markov Chains What are discrete-time Markov chains? 2 Reachability

567 views • 34 slides
Overview Verifying Continuous-Time Markov Chains Negative exponential distributions 1 Lecture

Overview Verifying Continuous-Time Markov Chains Negative exponential distributions 1 Lecture

Verifying Continuous-Time Markov Chains Verifying Continuous-Time Markov Chains Overview Verifying Continuous-Time Markov Chains Negative exponential distributions 1 Lecture 3+4: Continuous-Time Markov Chains What are continuous-time Markov

621 views • 30 slides
Model Checking Continuous-Time Markov Chains Joost-Pieter Katoen Software Modeling and

Model Checking Continuous-Time Markov Chains Joost-Pieter Katoen Software Modeling and

Model Checking Continuous-Time Markov Chains Joost-Pieter Katoen Software Modeling and Verification Group RWTH Aachen University associated to University of Twente, Formal Methods and Tools Lecture at MOVEP Summerschool, July 1, 2010 c

1.69k views • 137 slides
STA 331 2.0 Stochastic Processes 5. Continuous Parameter Markov Chains Dr Thiyanga S. Talagala

STA 331 2.0 Stochastic Processes 5. Continuous Parameter Markov Chains Dr Thiyanga S. Talagala

STA 331 2.0 Stochastic Processes 5. Continuous Parameter Markov Chains Dr Thiyanga S. Talagala September 08, 2020 Department of Statistics, University of Sri Jayewardenepura Goals 1. Explain the Markov property in the continuous-time

358 views • 12 slides
Model Checking Continuous-Time Markov Chains Joost-Pieter Katoen Software Modeling and

Model Checking Continuous-Time Markov Chains Joost-Pieter Katoen Software Modeling and

Model Checking Continuous-Time Markov Chains Joost-Pieter Katoen Software Modeling and Verification Group RWTH Aachen University associated to University of Twente, Formal Methods and Tools Lecture at Quantitative Model Checking School, March

672 views • 66 slides
Model Checking Continuous-Time Markov Chains Joost-Pieter Katoen Software Modeling and

Model Checking Continuous-Time Markov Chains Joost-Pieter Katoen Software Modeling and

Model Checking Continuous-Time Markov Chains Joost-Pieter Katoen Software Modeling and Verification Group RWTH Aachen University associated to University of Twente, Formal Methods and Tools Lecture at Quantitative Model Checking School, March

638 views • 45 slides
Rate-Based Stochastic Fusion Calculus and Angelo Troina Continuous Time Markov Chains Fusion

Rate-Based Stochastic Fusion Calculus and Angelo Troina Continuous Time Markov Chains Fusion

Rate-Based Stochastic Fusion Calculus and Continuous Time Markov Chains Gabriel Ciobanu, Rate-Based Stochastic Fusion Calculus and Angelo Troina Continuous Time Markov Chains Fusion Calculus Features Syntax and Semantics Continuous Time

480 views • 33 slides
Lecture 9 Continuous-time Markov chains Dr. Dave Parker Department of Computer Science

Lecture 9 Continuous-time Markov chains Dr. Dave Parker Department of Computer Science

Probabilistic Model Checking Michaelmas Term 2011 Lecture 9 Continuous-time Markov chains Dr. Dave Parker Department of Computer Science University of Oxford Overview Transient probabilities uniformisation

398 views • 27 slides
Computing limit expectations of imprecise continuous-time Markov chains Alexander Erreygers

Computing limit expectations of imprecise continuous-time Markov chains Alexander Erreygers

Computing limit expectations of imprecise continuous-time Markov chains Alexander Erreygers Jasper De Bock WPMSIIP 2018 Ghent University, ELIS, SYSTeMS ? without explicitly evaluating 1 Can we determine E ( f ) ( ) ) n f I + t ( t

525 views • 39 slides
Draft Conditioning by Permutation Monte Carlo for Continuous-Time Markov Chains Pierre

Draft Conditioning by Permutation Monte Carlo for Continuous-Time Markov Chains Pierre

1 Draft Conditioning by Permutation Monte Carlo for Continuous-Time Markov Chains Pierre LEcuyer Universit e de Montr eal, Canada joint work with Zdravko Botev , New South Wales University, Australia Rohan Shah , The University of

486 views • 26 slides
Mean-Payoff Optimization in Continuous-Time Markov Chains with Parametric Alarms Christel Baier

Mean-Payoff Optimization in Continuous-Time Markov Chains with Parametric Alarms Christel Baier

Mean-Payoff Optimization in Continuous-Time Markov Chains with Parametric Alarms Christel Baier Clemens Dubslaff L ubo s Koren ciak ech Anton n Ku cera Vojt Reh ak IFIP WG 2.2 2017 Meeting, Bordeaux Anton n

284 views • 5 slides
Geometric fluid approximation for general continuous-time Markov chains Michalis Michaelides

Geometric fluid approximation for general continuous-time Markov chains Michalis Michaelides

Geometric fluid approximation for general continuous-time Markov chains Michalis Michaelides mic.michaelides@ed.ac.uk Guido Sanguinetti Jane Hillston School of Informatics, University of Edinburgh April 2019 Michalis Michaelides

951 views • 45 slides
Discrete time Markov chains Today: Discrete Time Markov Chains, Limiting Discrete time Markov

Discrete time Markov chains Today: Discrete Time Markov Chains, Limiting Discrete time Markov

Discrete time Markov chains Today: Discrete Time Markov Chains, Limiting Discrete time Markov chains - invariant probability Distribution and Classification distribution Classification of states Classification of chains Bo Friis

250 views • 11 slides
Simulation of Discrete-Time Markov Chains Discrete-Time Markov Chains (DTMCs) Numerical Solution

Simulation of Discrete-Time Markov Chains Discrete-Time Markov Chains (DTMCs) Numerical Solution

Simulation of Discrete-Time Markov Chains Simulation of Discrete-Time Markov Chains Discrete-Time Markov Chains (DTMCs) Numerical Solution of DTMCs Peter J. Haas Simulation of DTMCs Recursive Definition of a DTMC Stationary Distribution of a

394 views • 4 slides
Discrete time Markov chains Today: Short recap of probability theory Markov chain

Discrete time Markov chains Today: Short recap of probability theory Markov chain

Discrete time Markov chains Today: Short recap of probability theory Markov chain introduction (Markov property) Discrete Time Markov Chains, Definition Chapmann-Kolmogorov equations and classification First step analysis Next

216 views • 9 slides
Markov chains and Hidden Markov Models 9000 Markov chains and HMMs We will discuss: Markov

Markov chains and Hidden Markov Models 9000 Markov chains and HMMs We will discuss: Markov

Markov chains and Hidden Markov Models 9000 Markov chains and HMMs We will discuss: Markov chains Hidden Markov Models (HMMs) Algorithms: Viterbi, forward, backward, posterior decoding Profile HMMs Baum-Welch algorithm 9001

1.16k views • 87 slides
Continuous-time Markov Chains Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science

Continuous-time Markov Chains Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science

Continuous-time Markov Chains Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ October 14, 2020 Introduction to Random Processes

834 views • 40 slides
Continuous-time Markov Chains Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science

Continuous-time Markov Chains Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science

Continuous-time Markov Chains Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ October 31, 2016 Introduction to Random Processes

868 views • 51 slides
Applications to Markov chains From Lay, 4.9 (This section is not examinable on the mid-semester

Applications to Markov chains From Lay, 4.9 (This section is not examinable on the mid-semester

Applications to Markov chains From Lay, 4.9 (This section is not examinable on the mid-semester exam.) Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 1 / 34 Theory and definitions Markov chains are useful tools in certain

721 views • 37 slides
Stationary Distributions of Markov Chains Will Perkins April 4, 2013 Back to Markov Chains

Stationary Distributions of Markov Chains Will Perkins April 4, 2013 Back to Markov Chains

Stationary Distributions of Markov Chains Will Perkins April 4, 2013 Back to Markov Chains Recall a discrete time, discrete space Markov Chain, is a process X n so that Pr[ X n = x n | X n 1 = x n 1 , . . . X 1 = x 1 ] = Pr[ X n = x n |

665 views • 20 slides

More recommend