Stationary Tries and Renewal Theorem Philippe Robert - PowerPoint PPT Presentation

Stationary Tries and Renewal Theorem Philippe Robert INRIA-Rocquencourt

Contents 1 Renewal Theorems 2 2 Tree Algorithms 10 3 A Probabilistic Point of View 19 4 Stationary tries 35

1. Renewal Theorems

Some History Blackwell (1948): — A light bulb last two years in average. — How many are necessary for ten years ? L1 L2 L3 t 0 Breiman, Feller, Lindvall, . . . Babillot (Habilitation).

General Framework L 1 L 2 L 3 L 4 L 5 t 0 R t F t ( L i ) i.i.d. non-negative random variables. — U ( a, b ) : average number of points in [ a, b ] , U : Renewal measure, for h > 0 t → + ∞ U ( t, t + h )? lim — Behavior of ( R t , F t ) as t → + ∞ ?

Renewal Theorem Non-Lattice Case: ∀ δ > 0 , P ( L ∈ δ N ) < 1 h t → + ∞ U ( t, t + h ) = lim E ( L 1 ); dist. ( R t , F t ) → ( R ∞ , F ∞ ) : Z L 1 ! 1 E ( f ( R ∞ , F ∞ ))= E ( L 1 ) E f ( u, L 1 − u ) du 0 F t has density ∼ x → P ( L 1 ≥ x ) / E ( L 1 )

Proofs — Renewal Equation: Z t U (0 , t ) = 1 + U (0 , t − u ) L ( du ) 0 ⇒ Fourier Analysis (Feller). — Coupling: Lindvall, Athreya and Ney, . . .

A Point Process Point of View L n−1 L L n+1 n t t t t t t t τ τ τ τ τ τ −3 −2 −1 0 1 2 Renewal Theorem: dist. ( τ t → ( L ∗ i , i ∈ Z ) − i , i ∈ Z ) Stationary renewal process. 1. ( L ∗ i , i ≤ − 1) , ( L ∗ i , i > 1) , i.i.d. dist. as L 1 ; dist. 2. ( L ∗ 0 , L ∗ 1 ) = ( R ∞ , F ∞ ) .

Lattice Case If P ( L 1 ∈ δ N ) = 1 and P ( L 1 = δ ) > 0 : — ( τ t i , i ∈ Z )) does not converge as t → + ∞ — for h > 0 , n → + ∞ ( τ h + nδ ( L ∗ h , i ∈ Z )) − → i , i ∈ Z ) i — Periodic Behavior dist. ( L ∗ h = ( L ∗ ( h + δ ) i , i ∈ Z ) , i ∈ Z ) i

The Poisson Process ( L n ) exponential r.v. with parameter 1 . P ( L 1 ≥ x ) = exp( − x ) . L n−1 L L n+1 n t E E E E E E −3 −2 −1 0 1 2 ( E n ) also exp. with parameter 1 . Invariant by translation. Renewal Theorem holds at time t = 0 .

2. Tree Algorithms

XYZTU — Each item of root node flips a coin X X = 0 ⇒ left node X = 1 ⇒ right node

XYZTU XZT YU 1 0 Each item of root node flips a coin 0 ⇒ left node 1 ⇒ right node Continue until at most one item per node. Independent coin tossings.

A Non-Symmetrical Tree Algorithm XYZTU p q XZT YU q p p q YU X ZT * p q q p T Z Y U P ( X = 0) = p, P ( X = 1) = q

Applications — Communication protocols; — Search and Sort Algorithms; — Leader election problems; — Statistical tests, — . . .

Asymptotic behavior Cost Function. Starting with n items at root R n : nb. of nodes of the tree E ( R n ) : Average cost to process 1 item. n E ( R n ) Law of Large Numbers: lim ? n n → + ∞ the limit does not always exist !

Simulation n → E ( R n ) /n Simulation of n → R n /n × 106

The sequence n → E ( R n ) /n 2.885393 2.885392 2.885391 2.885390 2.885389 2.885388 2.885387 5000 10000 20000 30000 40000 50000

Asymptotic behavior 8 if log p converges log q �∈ Q > > „ E ( R n ) > « < : n > > > oscillates otherwise : Literature: Complex analysis methods FLajolet and co-authors.

3. A Probabilistic Point of View

Recurrence Relation R 0 = R 1 = 1 . For n ≥ 2 , dist. R n = 1 + R X n + R n − X n with X n = B 1 + B 2 + · · · + B n . ( B i ) ∈ { 0 , 1 } i.i.d. Bernoulli parameter p . ( R n ) same dist. as ( R n ) independent of ( R n )

Equation for the Poisson Transform dist. R n = 1 + R X n + R n − X n − 2 × 1 { n ≤ 1 } If ( N x , x ≥ 0) Poisson process with rate 1 E ( R N x ) = E ( R N px ) + E ( R N qx ) + 1 − 2 P ( t 2 ≥ x ) t 2 second point of Poisson process ( N x ) If r ( x ) = E ( R N x ) − 1 r ( x ) = r ( px ) + r ( qx ) + 2 P ( t 2 < x )

Poisson Transform (II) r ( x ) = r ( px ) + r ( qx ) + 2 P ( t 2 < x ) If A 1 ∈ { p, q } r.v. such that P ( A 1 = p ) = p „ r ( A 1 x ) « ` ´ r ( x ) = E + 2 E 1 { t 2 <x } A 1

Poisson Transform (II) r ( x ) = r ( px ) + r ( qx ) + 2 P ( t 2 < x ) If A 1 ∈ { p, q } r.v. such that P ( A 1 = p ) = p „ r ( A 1 x ) « ` ´ r ( x ) = E + 2 E 1 { t 2 <x } A 1 „ 1 „ r ( A 2 A 1 x ) « « = E +2 E 1 { t 2 <xA 1 } A 2 A 1 A 1 ` ´ +2 E 1 { t 2 <x } + ∞ ! 1 X = 2 1 { t 2 <x Q k E 1 A i } Q k 1 A i k =0

Poisson Transform (III) + ∞ ! 1 X E ( R N x ) = 1 + 2 1 { t 2 <x Q k E Q k 1 A i } 1 A i k =0 + ∞ + ∞ ! 1 X X = 1 + 2 E 1 { t 2 <x Q k Q k 1 A i ,N x = n } 1 A i k =0 n =2

Given { N x = n } the n points of Poisson proc.: n i.i.d. uniformly dist. r.v. on [0 , x ] U 2 ,n : second smallest value of n i.i.d. uniform r.v. on [0 , 1] t 2 ≤ x „ « A , N x = n P = P ( t 2 ≤ x/ A| N x = n ) x n n ! e − x P ( t 2 ≤ x/ A| N x = n ) = P ( U 2 ,n ≤ 1 / A )

Probabilistic de-Poissonization E ( R n ) x n X n ! e − x E ( R N x ) = n ≥ 0 + ∞ + ∞ ! 1 X X = 1 + 2 1 { t 2 <x Q k E 1 A i ,N x = n } Q k 1 A i k =0 n =2 x n X n ! e − x E ( R N x ) = n ≥ 0 + ∞ + ∞ ! x n 1 X X n ! e − x +2 1 { U 2 ,n < Q k E Q k 1 A i } 1 A i n =2 k =0

An Associated Random Walk U 2 ,n : 2 th min. of n ind. uniform r.v. on [0 , 1] 0 1 1 @X A , n ≥ 2 . E ( R n ) = 1+2 E 1 { U 2 ,n < Q k 1 A i } Q k 1 A i k ≥ 0 X E ( R n ) − 1 e − log n + P k i =1 − log( A i ) = E 2 n k 1 × 1 n o A − P k 1 log( A i ) ≤− log U 2 ,n

The Use of Renewal Theorem L i = − log A i U 2 ,n ∼ 1 /n 0 1 e − ( log n − P k i =1 L i )1 n @X ∆ n = E o A P k 1 L i ≤ log n k 0 1 0 1 i = k τ log n e − P 0 e − P 0 i = k L ∗ @X @X A ∼ E ∆ n ∼ E i i A k ≤ 0 k ≤ 0 if L 0 non-lattice.

Renewal Theorems — Dist. of − log A non-lattice: log p/ log q �∈ Q . Continuous Renewal Theorem. Convergence of E ( R n ) /n . — Dist. of − log A lattice: log p/ log q ∈ Q . Discrete Renewal Theorem. Periodic Fluctuations of E ( R n ) /n .

General Splitting Scheme Algorithm A ( n ) — n < D ⇒ STOP. Otherwise: — Take a r.v. G ∈ N ; Branching variable — Take a random probability vector V = ( V 1 , . . . , V G ) , V 1 + · · · + V G = 1 ; Weights on arcs — Split n into G subgroups ( n 1 , . . . n G ) randomly, according to vector V . — Apply A ( n 1 ) , . . . , A ( n G ) .

General Splitting Algorithms ABCDEF V V 1 3 V 2 ABC ABC DE F AB C * D E AB * AB *

Splitting Measure Probability Measure on [0 , 1] : Z 1 G ! X f ( x ) W ( dx ) = E V i,G f ( V i,G ) . 0 i =1

Theorem. [Mohamed and R. (2005)] If Z 1 | log( y ) | W ( dy ) < + ∞ , y 0 — if log W non-lattice, E ( G ) n → + ∞ E ( R n ) /n = lim . R 1 ( D − 1) 0 | log( y ) | W ( dy ) — log W lattice: E ( R n ) /n ∼ F (log n/λ ) Z + ∞ y D − 2 x − log y „  ff« ( D − 1)! e − y dy F ( x )= C exp − λ λ 0

Some connections — Fragmentation processes; — Random Fractal subsets of [0 , 1] ; — Branching processes: Multiplicative martingales; — Dynamical systems: counting problems.

4. Stationary tries

Stationary sequences — X = ( X k , k ∈ Z ) ∈ { 0 , 1 } Z a stationary sequence; — Each item draws a sequence dist. as X ; X i : “coin” (possibly) used at level i ; — R n size of the tree with n items. Asymptotic behavior of E ( R n ) /n ?

X = 10 , Y = 010 , Z = 1110 , T = 1111 , U = 011 . XYZTU XZT YU YU X ZT * T Z Y U

Functional Analysis Approach Context — A function φ [0 , 1] → [0 , 1] ; — A Partition [0 , 1] = ∪ i I i ; — X n = p if φ ( n ) ( X 0 ) ∈ I p . Methods — Ruelle’s Transfer Operator; — Functional Transforms. Baladi, Bourdon, Cl´ ement, Flajolet, Vall´ ee, . . .

Cost Function X C n ( f ) = f ( np α ) α ∈ Σ ∗ Σ ∗ finite vectors in { 0 , 1 , . . . , C } p α proba. of vector α ∈ Σ ∗ ; f (0) = 0 . Exemple: Cl´ ement, Flajolet and Vall´ ee (2001) X 1 − (1 + np α )(1 − p α ) n ´ ` R n = α ∈ Σ ∗ Z np α ue − u du X ∼ 0 α ∈ Σ ∗

Probabilistic rewriting of cost function X C n ( f ) = f ( np α ) α ∈ Σ ∗ f ( np α ) X X = p α p α α ∈ Σ ∗ k ≥ 1 | α | = k „ f ( np κ k ) « X = E p κ k k ≥ 1 κ k projection on k first coord.

Rewriting of cost function: Fubini’s Theorem „ f ( np κ k ) « X C n ( f ) = E p κ k k ≥ 1 0 1 Z + ∞ 1 @X 1 { u ≤ np κk } f ′ ( u ) du = E A p κ k 0 k ≥ 1 Z + ∞ E ( G n ( u )) f ′ ( u ) du, = 0 with 1 X G n ( u ) = 1 { u ≤ np κk } . p κ k k ≥ 1

A rough estimation Shannon’s Theorem: − log p κ k ∼ kH , a.s. H entropy. 1 X G n ( u ) = 1 { u ≤ np κk } p κ k k ≥ 1 e − log p κk 1 {− log p κk ≤ log( n/u ) } X = k ≥ 1 ′′ ∼ ′′ X e kH 1 { k ≤ log( n/u ) /H } k ≥ 1 = e ⌈ log( n/u ) /H ⌉ H − 1 n ∼ u ( e H − 1) . e H − 1 G n of order n .

More Rigor p κ n ( ω ) = P ( X n = ω n , . . . , X 0 = ω 0 | X k = w k , k < 0) n Y = P ( X i = ω i | X i = ω i , . . . , X 0 = ω 0 , X k = ω k , k < 0) i =1 n h ◦ θ i ( ω ) X − log p κ n = i =1 θ : shift on sequences; h ( ω ) = − log P ( X 0 = ω 0 | X k = ω k , k < 0) . h entropy function, H = E ( h )