Expectation Continued: Tail Sum, Coupon Collector, and Functions of - PowerPoint PPT Presentation

Expectation Continued: Tail Sum, Coupon Collector, and Functions of RVs CS 70, Summer 2019 Lecture 20, 7/29/19 1 / 26

Last Time... I Expectation describes the weighted average of a RV. I For more complicated RVs, use linearity Today: I Proof of linearity of expectation I The tail sum formula I Expectations of Geometric and Poisson I Expectation of a function of an RV 2 / 26

Sanity Check Let X be a RV that takes on values in A . Let Y be a RV that takes on values in B . Let c ∈ R be a constant. Both c · X and X + Y are also RVs! Xty , AEA ,bEB } Values :{ Values atb a a , probs : Probe : Atb ]=P[X=a,7=b ] ca ]=p[X=a ] ipfxty - IPCC X = 3 / 26

Proof of Linearity of Expectation I Recall linearity of expectation: E [ X 1 + . . . + X n ] = E [ X 1 ] + . . . + E [ X n ] For constant c , E [ cX i ] = c · E [ X i ] in A Xi values . First, we show E [ cX i ] = c · E [ X i ] : I a) IPCX a ] Efc Xi ) ¢ = = A E A c ÷ , c IE [ X i ] a ] . IP EX a - = = - EC Xi ] 4 / 26

Proof of Linearity of Expectation II Next, we show E [ X + Y ] = E [ X ] + E [ Y ] . values MA X has B values has y in , . at b) p[X=a , Y b ] I ( Efxty ]= - AEA , BEB PCx=aY=b ] . b) " 4*94 ,£⇒b ate . rewrite # - AFA FEB , 't b) tfepbff.PK/--a.Y--b ] = ¥aaq⇒pCx=a of ¥=b] 7¥ EE - PLY b ] BEBAEA b qfaa.ipcx-ay-qf.gg - = TEE TEXT Two variables to n variables? Induction . 5 / 26

The Tail Sum Formula - neg A non Let X be a RV with values in { 0 , 1 , 2 , . . . , n } . integers We use “tail” to describe P [ X ≥ i ] . team What does P ∞ i = 1 P [ X ≥ i ] look like? Small example: X only takes values { 0 , 1 , 2 } : Z i ] 22 ] €7 21 ] PEX PEX PEX t = ¥ I PIX / \ = 2) = 2 ] EX ] t = p Cx t - . PCX 2 ] +1 - I ] IPCX -12 - - - . Or PEX EF 6 / 26

The Tail Sum Formula The tail sum formula states that: x Ronning ∞ ers X E [ X ] = P [ X ≥ i ] = i = 1 Proof: Let p i = P [ X = i ] . ( p , t¥pz P EX Z 2) t PEX 233 ? i ] PEX Z IT €,9P[ x t Z = . ) ! ¥713 t ( Pat . ) t t = . . . . . ) t I p , Pg + t . . ' P O ' p a . Pz . p , 2 3 t t t I . - - E EXT = 7 / 26

Expectation of a Geometric I Let X ∼ Geometric( p ) . I i - - p ) H P [ X ≥ i ] = Apply the tail sum formula : - pit - p ) th - ] ( I ÷ pfxzi It = . . . - , series Geometric Esma =L first - P ) H r - - , Ifpi Fr - - - - 8 / 26

Expectation of a Geometric II - aeomcp ) X Use memorylessness : the fact that the succeed geometric RV “resets” after each trial. trials art failed ft trial Two Cases: P first . in 1€E[X trial 1st fall . " ] days " " win 2day " in win identical . reset day looks " " 2 , day I to . - p ) f ] ITEM (1) ( I ECX ] t =p Http Solve PIECX I ECX ] ' for . Efxtipt 9 / 26

Expectation of a Geometric III Lastly, an intuitive but non-rigorous idea. Cp ) X Geom - Let X i be an indicator variable for success in a single trial. Recall trials are i.i.d. Ber ( p ) X i ∼ expectation of linearity use . .tECXn ] E [ X 1 + X 2 + . . . + X k ] = ECX.IT Efx , It . , ] KEEX = Kp = = pt Xx ) I ECX . t = Need K t order for in . . , - of successes # . 10 / 26

Coupon Collector I (Note 19.) I’m out collecting trading cards. There are n types total. I get a random trading card every time I buy a cereal box. What is the expected number of boxes I need to buy in order to get all n trading cards? High level picture: Time =L T , Tz -13 X4 Xz xz get get get 3rd 2nd 1St card card ! card ! . 11 / 26

Coupon Collector II Dth time / Ci card " # " between - boxes Let X i = ith and the CL ) Geom =L always X , What is the dist. of X 1 ? . ( ht ) What is the dist. of X 2 ? Xz Geom ~ ( MT ) What is the dist. of X 3 ? Xz Geom ~ In general, what is the dist. of X i ? Geom ( nifty Xi - 12 / 26

Coupon Collector III cards all get n total boxes to # Let X = t Xn Xzt X , t X = - . IE [ linearity txn ] E [ X ] = Xitxzt . . . d - TECXN ] Efx , ] HECK ) t = . n-ci Geom I , Xi n . thy # IT t It t = nEi . . % ? ? e- 13 / 26

Aside: (Partial) Harmonic Series Harmonic Series: P ∞ 1 Diverges k = 1 k Approximation for P n 1 k in terms of n ? k = 1 E.it#fIdx=lnx/7--lnn-In1/ " ↳ x inn . E EXT log N F N collector : ⇒ coupon . EE ¥¥IE¥ . 14 / 26

Break A Bad Harmonic Series Joke... A countably infinite number of mathematicians walk into a bar. The first one orders a pint of beer, the second one orders a half pint, the third one orders a third of a pint, the fourth one orders a fourth of a pint, and so on. The bartender says ... 15 / 26

Expectation of a Poisson I Recall the Poisson distribution: values 0 , 1 , 2 , . . . , poi CX ) X - P [ X = i ] = λ i i ! e − λ Eto if We can use the definition to find E [ X ] ! te ¥1 a ) touswrwiesfor ECXI e- - - . ⇒ = - DD Het = - 16 / 26

Expectation of a Poisson II Optional but intuitive / non-rigorous approach: Think of a Poisson( λ ) as a Bin( n , λ n ) distribution, taken as n → ∞ . Let X ∼ Bin( n , λ n ) . * I 71¥ X = 17 / 26

Rest of Today: Functions of RVs! Recall X from Lecture 19: 8 1 wp 0.4 > < 1 X = wp 0.25 2 − 1 > wp 0.35 : 2 Refresh your memory: What is X 2 ? ft 0.4 = { ¥ Wp 0.4 WP . XI 0.25 Wp -4 0.6 Wp 0.35 Wp ¥ 18 / 26

Example: Functions of RVs ( 1 wp 0.4 X 2 = 1 wp 0.6 4 What is E [ X 2 ] ? t 4 PIX - I ] . PC XIII ' Efx 21 I - - - 0.4 . t 0.6=0.55 I t = . What is E [ 3 X 2 − 5 ] ? linearity of exp . 3 EHF 5 - ECS ) - IE [3×2] ) 5 310.55 - 19 / 26

In General: Functions of RVs Let X be a RV with values in A . Distribution of f ( X ) : ! f ! = { ) f Cx ) wppxf-ajac.CA pfX=a ] E [ f ( X )] = Ha ) 20 / 26

Square of a Bernoulli Let X ∼ Bernoulli( p ) . Write out the distribution of X . - { f wpp X - Wp tp What is X 2 ? E [ X 2 ] ? - { to If Ip ECM =p ' X . - 21 / 26

Product of RVs Exercise Let X be a RV with values in A . Let Y be a RV with values in B . XY is also a RV! What is its distribution? (Use the joint distribution! ) 22 / 26

Product of Two Bernoullis Exercise Let X ∼ Bernoulli( p 1 ) , and Y ∼ Bernoulli( p 2 ) . X and Y are independent . What is the distribution of XY ? What is E [ XY ] ? 23 / 26

Square of a Binomial I Let X ∼ Bin( n , p ) . Decompose into X i ∼ Bernoulli( p ) . t X n t X z X , t X = . . . n ] IE [ X , . t X t X at E [ X ] = . . t Ef X n ] t IEC X a It Efx . I = . . . n p = . 24 / 26

Square of a Binomial II Recall, E [ X 2 i ] = 1, and E [ X i X j ] = p 2 . P ECXZ ]=Ef( ,tXzt X . . . ) ) .tt/n)2J=tEfCXi2tXz2t...tXn4t(XiXztXiXst I . . - - cross terms terms square - 1) MIN them of of n terms ] them TEAMS terms ] = Ef square . : :* :* : : " :i÷i " :* . ECXIZ ] ,Xz3=PZ P Efx - - 1) P2 nptnln > = 25 / 26

Summary Today: I Proof of linearity of expectation: did not use independence, but did use joint distribution I Tail sum for non-negative int.-valued RVs! I Coupon Collector: break problem down into a sum of geometrics. I Expectation of a function of an RV: can apply definition and linearity of expectation (after expanding) as well!! 26 / 26