Minicourse 3: Limiting Distributions in Combinatorics Michael - PowerPoint PPT Presentation

Minicourse 3: Limiting Distributions in Combinatorics Michael Drmota Institute of Discrete Mathematics and Geometry Vienna University of Technology A 1040 Wien, Austria michael.drmota@tuwien.ac.at www.dmg.tuwien.ac.at/drmota/ International Conference on Analysis of Algorithms Maresias, Brazil, April 12–18, 2008

Contents • Sums of independent random variables and powers of generating functions • A central limit theorem • Bivariate generating functions • Functions equations • Non-normal limit laws • Method of moments • Admissible functions and central limit theorems

Standard Reference Philippe Flajolet and Robert Sedgewick, Analytic Combinatorics , Cambridge University Press, to appear 2008. (http://algo.inria.fr/flajolet/Publications/books.html) + special reference for last part: M. Drmota, B. Gittenberger and T. Klausner, Extended admissible functions and Gaussian limiting distributions, Math. Comput. 74 (2005), 1953–1966.

Sums of independent random variables and powers of generating functions Coin tossing • P { ct = head } = P { ct = tail } = 1 2 . � 1 if tail • random variable ξ = I { ct =tail } = 0 if head • n independent runs: ξ 1 , ξ 2 , . . . , ξ n , P { ξ j = 1 } = P { ξ j = 0 } = 1 2 . • X n = ξ 1 + ξ 2 + · · · + ξ n ... the number of tails within n runs � n � k P { X n = k } = 2 n

Sums of independent random variables and powers of generating functions Counting generating function a n = 2 n ... total number of possible n -runs � n � a n,k = ... the number of n -runs with k tails k � n a n,k u k = u k = (1 + u ) n ... counting gen. func. � � � A n ( u ) = k k ≥ 0 k ≥ 0 a n,k = a n = (1 + 1) n = 2 n � A n (1) = k ≥ 0

Sums of independent random variables and powers of generating functions Probability generating function E u X n = P { X n = k } · u k � k ≥ 0 1 � n � · u k � = 2 n k k ≥ 0 = (1 + u ) n = A n ( u ) 2 n A n (1) P { X n = k } = a n,k E u X n = A n ( u ) ⇒ = a n A n (1)

Sums of independent random variables and powers of generating functions Powers of probability generating functions E u ξ = 1 2 + 1 2 u = 1 + u 2 E u X n = E u ξ 1 + ··· + ξ n ⇒ = � u ξ 1 · · · u ξ n � = E � u ξ 1 � � u ξ n � · · · E = E ξ j independent !!! � n � 1 + u = 2

Sums of independent random variables and powers of generating functions General fact X n = ξ 1 + ξ 2 + · · · + ξ n , where the r.v.’s ξ j are iid ∗ � n E u X n = � E u ξ 1 ⇒ = ∗ Notation. “iid” ... independently and identically distributed

Sums of independent random variables and powers of generating functions Relation to moment generating function m Z ( v ) = E e vZ E ( Z r ) ... r -th moment of Z   E ( Z r ) v r Z r v r  = E e vZ = E u Z with u = e v . �  � r ! = E ⇒ = r ! r ≥ 0 r ≥ 0

A central limit theorem Binomial coefficients − ( k − n 2 ) 2 2 n � � � n n ! � + O (2 n /n ) = k !( n − k )! = exp � k n/ 2 πn/ 2

A central limit theorem Standard normal distribution 1 2 πe − 1 2 t 2 . √ density: f ( t ) = � x 2 t 2 dt 1 −∞ e − 1 √ normal distribution function: Φ( x ) = 2 π

A central limit theorem Normally distributed random variable Definition A random variable Z has standard nomal distribution N (0 , 1) if P { Z ≤ x } = Φ( x ) . A random variable Z is normally distributed (or Gaussian ) with mean µ and variance σ 2 if its distribution function is given by � x − µ � P { Z ≤ x } = Φ , σ L ( Z ) = N ( µ, σ 2 ) . Notation.

A central limit theorem Moment generating function of N ( µ, σ 2 ): 2 σ 2 v 2 . m Z ( v ) = E e vZ = e µv − 1 Characteristic function of N ( µ, σ 2 ): 2 σ 2 t 2 . ϕ Z ( t ) = E e itZ = e iµt − 1 Standard normal distribution : µ = 0, σ 2 = 1 2 v 2 , 1 E e itZ = e − 1 2 t 2 E e vZ = e

A central limit theorem Definition We say, that a sequence of random variables X n satisfies a central limit theorem with (scaling) mean µ n and (scaling) variance σ 2 n if P { X n ≤ µ n + x · σ n } = Φ( x ) + o (1) as n → ∞ . Example. X n = number of tails in n runs of coin tossing: 1 � n � � � P { X n ≤ n/ 2 + x · n/ 4 } = 2 n k ≤ n/ 2+ x · √ k n/ 4 − ( k − n 2 ) 2 � � 1 � ∼ ∼ Φ( x ) . exp k ≤ n/ 2+ x · √ � n/ 2 πn/ 2 n/ 4 X n satisfies a central limit theorem with mean n 2 and variance n 4 .

Central Limit Theorem Definition Weak convergence: d X n − → X : ⇐ ⇒ n →∞ P { X n ≤ x } = P { X ≤ x } lim for all points of continuity of F X ( x ) = P { X ≤ x } Reformulation: X n satisfies a central limit theorem with (scaling) mean µ n and (scaling) variance σ 2 n is the same as X n − µ n d − → N (0 , 1) . σ n

A central limit theorem Weak convergence via moment generating functions d n →∞ E e vX n = E e vX lim ( v ∈ R ) X n − → X ⇒ = Moreover, we have convergence of all moments: E ( X r n ) → E ( X r ). Recall: E e vX n = E (( e v ) X n ) = E u X n for u = e v .

A central limit theorem Weak convergence via characteristic functions (Levy’s Criterion) d n →∞ E e itX n = E e itX lim ( t ∈ R ) ⇐ ⇒ X n − → X Moreover, if for all t ∈ R n →∞ E e itX n ψ ( t ) := lim exists and ψ ( t ) is continous at t = 0 then ψ ( t ) is the characteristic d − → X . function of a random variable X for which we have X n

Central Limit Theorem Theorem ξ 1 , ξ 2 , . . . iid, E ξ 2 i < ∞ , X n = ξ 1 + ξ 2 + . . . + ξ n X n − E X n d √ V X n − → N (0 , 1) ⇒ = ⇒ P { X n ≤ E X n + x √ V X n } = Φ( x ) + o (1). Remark . ⇐ Proof µ = E ξ i , σ 2 = V ξ i = E ( ξ 2 i ) − ( E ξ i ) 2 ⇒ E X n = nµ , V X n = nσ 2 . =

Central Limit Theorem 2 σ 2 t 2 (1+ o (1)) ϕ ξ i ( t ) = E e itξ i = e itµ − 1 ( t → 0) ϕ X n ( t ) = ϕ ξ i ( t ) n � σ 2 n Z n := ( X n − µn ) / ⇒ ϕ Z n ( t ) = E e itZ n = = e − it √ nµ/σ · E e ( it/ ( √ nσ ))( ξ 1 + ··· + ξ n ) � � = e − it √ nµ/σ · E e ( it/ ( √ nσ ) ξ 1 � n � = e − it √ nµ/σ · e it √ nµ/σ − 1 2 t 2 (1+ o (1)) 2 t 2 . 2 t 2 (1+ o (1)) → e − 1 = e − 1 + Levy’s criterion.

A central limit theorem Quasi-Power Theorem (Hwang) Let X n be a sequence of random variables with the property that � � �� 1 E u X n = A ( u ) · B ( u ) λ n · 1 + O φ n holds uniformly in a complex neighborhood of u = 1, λ n → ∞ and φ n → ∞ , and A ( u ) and B ( u ) are analytic functions in a neighborhood of u = 1 with A (1) = B (1) = 1. Set σ 2 = B ′′ (1) + B ′ (1) − B ′ (1) 2 . µ = B ′ (1) and V X n = σ 2 λ n + O (1 + λ n /φ n ) , E X n = µλ n + O (1 + λ n /φ n ) , ⇒ = X n − E X n d ( σ 2 � = 0) . √ V X n − → N (0 , 1)

Bivariate generating functions Bivariate counting generating function 1 � n u k x n = (1 + u ) n x n = � � � A ( x, u ) = 1 − x (1 + u ) . k n ≥ 0 n,k ≥ 0 Observation: this is a rational function !

Bivariate generating functions Rational functions P ( x, u ) , Q ( x, u ) polynomials: a n,k u k x n = P ( x, u ) � A ( x, u ) = Q ( x, u ) n,k ≥ 0 Assumption: factorization of denominator r � � x � Q ( x, u ) = 1 − ρ j ( u ) j =1 with | ρ 1 ( u ) | < max 2 ≤ j ≤ r | ρ j ( u ) | for | u − 1 | < ε.

Bivariate generating functions Central limit theorem for rational functions Suppose that A ( x, u ) = � a n,k u k x n with a n,k ≥ 0 is rational and satis- fies the assumptions from above. Let X n be a sequence of random variables with P { X n = k } = a n,k a n with a n = � k a n,k . Then X n satisfies a central limit theorem with µ n = − nρ ′ − ρ ′′ ρ 1 (1) − ρ ′ ρ 1 (1) + ρ ′ 1 (1) 2 � � 1 (1) 1 (1) 1 (1) σ 2 and n = n . ρ 1 (1) 2 ρ 1 (1)

Bivariate generating functions Proof Partial fraction decomposition: C 1 ( u ) C r ( u ) A ( x, u ) = 1 − x/ρ 1 ( u ) + · · · + 1 − x/ρ r ( u ) a n,k u k = C 1 ( u ) ρ 1 ( u ) − n + · · · + C r ( u ) ρ r ( u ) − n ∼ C 1 ( u ) ρ 1 ( u ) − n � A n ( u ) = ⇒ = k ≥ 0 � n � E u X n = A n ( u ) A n (1) ∼ C 1 ( u ) ρ 1 (1) ⇒ = C 1 (1) ρ 1 ( u ) ⇒ central limit theorem . =

Bivariate generating functions Integer compositions 3 = 1 + 1 + 1 = 2 + 1 = 1 + 2 = 3 ... 4 compositions of 3. a n = number of compositions of n , A ( x ) = � a n x n : x A ( x ) = 1 + A ( x )( x + x 2 + x 3 + · · · ) = 1 + A ( x ) 1 − x . = 1 − x 1 A ( x ) = ⇒ = x 1 − 1 − 2 x 1 − x a n = 2 n − 1 ⇒ =

Bivariate generating functions Integer compositions a n,k = number of integer composition of n with k summands A ( x, u ) = � a n,k u k x n : A ( x, u ) = 1 + uA ( x, u )( x + x 2 + x 3 + · · · ) = 1 + A ( x, u ) xu 1 − x . 1 1 − x A ( x, u ) = = ⇒ = 1 − xu 1 − x (1 + u ) 1 − x 2 and σ 2 = n ⇒ central limit theorem with µ n = n 4 . =

Bivariate generating functions Systems of linear equations Suppose, that several generating functions a 1; n,k u k x n , . . . , A r ( x, u ) = a r ; n,k u k x n � � A 1 ( x, u ) = n,k n,k satisfy a linear system of equations . Then all generating functions A j ( x, u ) are rational and a central limit theorem for corresponding random variables is expected .