Introduction Establishing drift Using drift Applications References State-dependent Foster-Lyapunov criteria Stephen Connor stephen.connor@york.ac.uk Joint work with Gersende Fort, CNRS-TELECOM ParisTech; supported by CRiSM and the French National Research Agency March 2012
Introduction Establishing drift Using drift Applications References Outline Introduction 1 The general problem Drift conditions Establishing a subsampled drift condition 2 Examples Using a subsampled drift condition 3 Interplay of subsampling rates and moments Applications 4
Introduction Establishing drift Using drift Applications References The general problem Let Φ = { Φ n , n ≥ 0 } be a time-homogeneous Markov chain on a state space X. (Assume that Φ is phi-irreducible and aperiodic, for simplicity.) We’re interested in what can be said about the long-term behaviour of Φ. For example: does Φ converge to an equilibrium distribution? if so, in what norm does this convergence take place, and how fast? and how is this related to the average time spent between successive visits to certain sets?
Introduction Establishing drift Using drift Applications References Notation P n : n -step transition kernel of Φ; for a non-negative function f , and measure µ : � P n f ( x ) = E x [ f (Φ n )] , µ ( f ) = f ( y ) µ ( dy ) ; norm � µ � g : � µ � g = sup | µ ( f ) | , � · � TV ≡ � · � 1 ; f : | f |≤ g first return time to a set: τ A = inf { n ≥ 1 : Φ n ∈ A} ; C is a small set if ∃ ε > 0 and a measure ν s.t. P ( x , · ) ≥ εν ( · ) for all x ∈ C .
Introduction Establishing drift Using drift Applications References Notation P n : n -step transition kernel of Φ; for a non-negative function f , and measure µ : � P n f ( x ) = E x [ f (Φ n )] , µ ( f ) = f ( y ) µ ( dy ) ; norm � µ � g : � µ � g = sup | µ ( f ) | , � · � TV ≡ � · � 1 ; f : | f |≤ g first return time to a set: τ A = inf { n ≥ 1 : Φ n ∈ A} ; C is a small set if ∃ ε > 0 and a measure ν s.t. P ( x , · ) ≥ εν ( · ) for all x ∈ C .
Introduction Establishing drift Using drift Applications References Notation P n : n -step transition kernel of Φ; for a non-negative function f , and measure µ : � P n f ( x ) = E x [ f (Φ n )] , µ ( f ) = f ( y ) µ ( dy ) ; norm � µ � g : � µ � g = sup | µ ( f ) | , � · � TV ≡ � · � 1 ; f : | f |≤ g first return time to a set: τ A = inf { n ≥ 1 : Φ n ∈ A} ; C is a small set if ∃ ε > 0 and a measure ν s.t. P ( x , · ) ≥ εν ( · ) for all x ∈ C .
Introduction Establishing drift Using drift Applications References Notation P n : n -step transition kernel of Φ; for a non-negative function f , and measure µ : � P n f ( x ) = E x [ f (Φ n )] , µ ( f ) = f ( y ) µ ( dy ) ; norm � µ � g : � µ � g = sup | µ ( f ) | , � · � TV ≡ � · � 1 ; f : | f |≤ g first return time to a set: τ A = inf { n ≥ 1 : Φ n ∈ A} ; C is a small set if ∃ ε > 0 and a measure ν s.t. P ( x , · ) ≥ εν ( · ) for all x ∈ C .
Introduction Establishing drift Using drift Applications References Notation P n : n -step transition kernel of Φ; for a non-negative function f , and measure µ : � P n f ( x ) = E x [ f (Φ n )] , µ ( f ) = f ( y ) µ ( dy ) ; norm � µ � g : � µ � g = sup | µ ( f ) | , � · � TV ≡ � · � 1 ; f : | f |≤ g first return time to a set: τ A = inf { n ≥ 1 : Φ n ∈ A} ; C is a small set if ∃ ε > 0 and a measure ν s.t. P ( x , · ) ≥ εν ( · ) for all x ∈ C .
Introduction Establishing drift Using drift Applications References Ergodicity Φ is called ergodic if it has a finite invariant measure π ( π = π P ). In this case, for any x , � P n ( x , · ) − π � TV → 0 as n → ∞ . (1) Equivalently, we can find a small set C with sup E x [ τ C ] < ∞ . x ∈C
Introduction Establishing drift Using drift Applications References Ergodicity Φ is called ergodic if it has a finite invariant measure π ( π = π P ). In this case, for any x , � P n ( x , · ) − π � TV → 0 as n → ∞ . (1) Equivalently, we can find a small set C with sup E x [ τ C ] < ∞ . x ∈C But how fast does the convergence in (1) take place?
Introduction Establishing drift Using drift Applications References Definition Φ is geometrically ergodic if there exists r > 1 with � P n ( x , · ) − π ( · ) � TV ≤ M x r − n .
Introduction Establishing drift Using drift Applications References Definition Φ is geometrically ergodic if there exists r > 1 with � P n ( x , · ) − π ( · ) � TV ≤ M x r − n . Equivalently: there exists a scale function V : X → [1 , ∞ ), a small set C , and constants β ∈ (0 , 1), b < ∞ , with E x [ V (Φ 1 )] = PV ( x ) ≤ β V ( x ) + b 1 C ( x ); sup x ∈C E x [ β − τ C ] < ∞ .
Introduction Establishing drift Using drift Applications References Drift conditions The inequality PV ( x ) ≤ β V ( x ) + b 1 C ( x ); is called a Foster-Lyapunov drift condition . often easiest way of showing that Φ is geometrically ergodic; if V is bounded then Φ is uniformly ergodic .
Introduction Establishing drift Using drift Applications References Drift conditions The inequality PV ( x ) ≤ β V ( x ) + b 1 C ( x ); is called a Foster-Lyapunov drift condition . often easiest way of showing that Φ is geometrically ergodic; if V is bounded then Φ is uniformly ergodic . Subgeometric ergodicity is implied by a weaker drift condition: PV ( x ) ≤ V ( x ) − φ ◦ V ( x ) + b 1 C ( x ) for some concave non-negative function φ .
Introduction Establishing drift Using drift Applications References Questions: Drift conditions tend to look at only one step of Φ though: sometimes more convenient to work with a subsampled chain . . .
Introduction Establishing drift Using drift Applications References Questions: Drift conditions tend to look at only one step of Φ though: sometimes more convenient to work with a subsampled chain . . . 1 When can we find a function n : X → N such that P n ( x ) V ( x ) ≤ β V ( x ) + b 1 C ( x ) ? (2)
Introduction Establishing drift Using drift Applications References Questions: Drift conditions tend to look at only one step of Φ though: sometimes more convenient to work with a subsampled chain . . . 1 When can we find a function n : X → N such that P n ( x ) V ( x ) ≤ β V ( x ) + b 1 C ( x ) ? (2) 2 Alternatively, if (2) holds for some n and V , what can be said about moments of the return time to C ?
Introduction Establishing drift Using drift Applications References Questions: Drift conditions tend to look at only one step of Φ though: sometimes more convenient to work with a subsampled chain . . . 1 When can we find a function n : X → N such that P n ( x ) V ( x ) ≤ β V ( x ) + b 1 C ( x ) ? (2) 2 Alternatively, if (2) holds for some n and V , what can be said about moments of the return time to C ? 3 And when is this useful?!
Introduction Establishing drift Using drift Applications References Establishing a subsampled drift condition Theorem (1) Assume that there exist a small set D , a function V : X → [1 , ∞ ) and a continuously differentiable increasing concave function φ : [1 , ∞ ) → (0 , ∞ ) , such that sup D V < ∞ , inf [1 , ∞ ) φ > 0 , and PV ≤ V − φ ◦ V + b 1 D . � � Fix β ∈ (0 , 1) and let n : X → N satisfy n ( x ) ∼ 1 V ( x ) . β φ ◦ V Then for any β < β ′ < 1 , P n ( x ) W ≤ β ′ W + b ′ 1 C , where W = φ ◦ V .
Introduction Establishing drift Using drift Applications References Comments Theorem (1) says that we can deduce a (state-dependent) subsampled geometric drift condition from a one-step drift, but on a different scale; More general results (not requiring a drift condition for V ) can be stated.
Introduction Establishing drift Using drift Applications References Examples P n ( x ) W ≤ β ′ W + b ′ 1 C PV ≤ V − φ ◦ V + b 1 D ⇒ Polynomially ergodic: if φ ( t ) ∼ ct 1 − α for some α ∈ (0 , 1), then n ∼ V α and W = V 1 − α .
Introduction Establishing drift Using drift Applications References Examples P n ( x ) W ≤ β ′ W + b ′ 1 C PV ≤ V − φ ◦ V + b 1 D ⇒ Polynomially ergodic: if φ ( t ) ∼ ct 1 − α for some α ∈ (0 , 1), then n ∼ V α and W = V 1 − α . Motivation: Deterministic chain: Φ n +1 = Φ n − Φ 0 . 4 (so V ( x ) = x ) n 50 � 800 40 � 600 30 � 400 � 20 � 200 10 � � 100 200 300 400 500 20 30 40 50 60 70 Decay of W (Φ n ) = Φ 0 . 4 n : W (Φ n ) ≤ 0 . 2 W (Φ 0 ) n (blue line ∝ Φ 0 . 6 0 )
Introduction Establishing drift Using drift Applications References Examples P n ( x ) W ≤ β ′ W + b ′ 1 C PV ≤ V − φ ◦ V + b 1 D ⇒ Polynomially ergodic: if φ ( t ) ∼ ct 1 − α for some α ∈ (0 , 1), then n ∼ V α and W = V 1 − α . Subgeometrically ergodic: if φ ( t ) ∼ t [ln t ] − α for some α > 0, then n ∼ [ln V ( x )] α and W = V [ln V ] − α .
Recommend
More recommend
