Probability*and*Statistics* ! ! for*Computer*Science** - PowerPoint PPT Presentation

Probability*and*Statistics* ! ! for*Computer*Science** “I!have!now!used!each!of!the! terms!mean,!variance,! covariance!and!standard! devia5on!in!two!slightly! different!ways.”!<<<Prof.! Forsythe!! ! Credit:!wikipedia! Hongye!Liu,!Teaching!Assistant!Prof,!CS361,!UIUC,!2.13.2020!

Last*time* � Random!Variable!! � The!Defini5on!of!Random!Variable! � Probability*distribu.on !of!random! variable! � Joint*&*Condi.onal*probability* distribu.on !of!random!variable!

Content* � Random!Variable!! � Expected*value* � Variance*&*covariance ! � Markov’s*inequality !

Three*important*facts*of* Random*variables* � Random!variables!have! probability*func/ons* � Random!variables!can!be! condi/oned !on!events!or!other! random!variables! � Random!variables!have! averages *

Content* � Random!Variable!! � Expected(value( � Variance*&*covariance ! � Markov’s*inequality !

Expected*value* � The! expected*value !(or! expecta/on )! of!a!random!variable! X !is! pc X=k ) ← - � E [ X ] = xP ( x ) " possible value x - The!expected!value!is!a!weighted!sum! of!the!values! X !can!take! !

Expected*value* � The! expected*value !of!a!random! variable! X !is! <=!1! � E [ X ] = xP ( x ) Ep # =L x N The!expected!value!is!a!weighted!sum! of!the!values! X !can!take! !

Expected(value:(profit(( � A"company"has"a"project"that"has" p " probability"of"earning"10"million"and" 1#p " probability"of"losing"10"million." � Let" X "be"the"return"of"the"project." to Pt f - lo ) ( t - P ) E- EX ) = - to > o cop = " co profit P S 0.5 ,

Expected*value:*profit** � A!company!has!a!project!that!has! p ! probability!of!earning!10!million!and! 1.p ! probability!of!losing!10!million.! � Let! X !be!the!return!of!the!project.! E [ X ] = 10 p + ( − 10)(1 − p ) = 20 p − 10 ! For E [ X ] > 0 we need p > 0 . 5

wrhies 5 ME TE FE ¥7 = ? I Mean t - 92 each $1 each value = ? Expected A) random draw I = Ig 0.75 × 11-0.25 × 2 in deputy 1 twice replacement with draw D) random You draws the the game . are ( if two the prize , 3 get w -4 % $1 Expected value ? = w 3- It is o 4 / 16 4 Tbxltgxz = It g TE \t - Z o eh 4h , oh } To → $2

Expected*value*as*mean** � Suppose!we!have!a!data!set!{ x i }!of! N !data! points.!Let’s!define!a!random!variable! X ! taking!on!each!of!the!data!points!with! equal!probability!1/ N .!! x i P ( x i ) = 1 � � x i = mean ( { x i } ) E [ X ] = N i i � The!expected!value!is!also!called!the! mean. ! !

Linearity*of*Expectation* � For!random!variables! X !and! Y ! and!constants!k,c! � Scaling!property! F ! E [ kX ] = kE [ X ] � Addi5vity! E [ X + Y ] = E [ X ] + E [ Y ] � And!! E [ kX + c ] = kE [ X ] + c

Linearity*of*Expectation* � Proof!of !the!addi5ve!property! E [ X + Y ] = E [ X ] + E [ Y ] Ht 2) PHo=xo=y ) E- Cx t 'll = I 5g ! ! next pg

E Ext Y ) = I -2g - x , F- y , ( key ) P CX - = I -2g cut y , pox . y ) TZ Fy apex , y ) t § I y poky , = y Pl x. y l t Ey § Ee k Ey y pi x. y , = t Ey 2Epcx = I az d p ck ) ptcy , = Ty xp ex ) t Tg y pay , E- Cx ) t ECT ) =

Linearity*of*Expectation* � Proof!of !the!addi5ve!property! E [ X + Y ] = E [ X ] + E [ Y ] � � E [ X + Y ] = ( x + y ) P ( x, y ) ! x y ! P ( { X = x } ∩ { Y = y } )

Linearity*of*Expectation* � � E [ X + Y ] = ( x + y ) P ( x, y ) x y ! � � � � = xP ( x, y ) + yP ( x, y ) ! x y x y � � � � = xP ( x, y ) + yP ( x, y ) x y y x � � � � = P ( x, y ) + P ( x, y ) x y x y y x � � = xP ( x ) + yP ( y ) x y

Linearity*of*Expectation* � � E [ X + Y ] = ( x + y ) P ( x, y ) x y ! � � � � = xP ( x, y ) + yP ( x, y ) ! x y x y � � � � = xP ( x, y ) + yP ( x, y ) x y y x � � � � = P ( x, y ) + P ( x, y ) x y x y y x � � = xP ( x ) + yP ( y ) x y = E [ X ] + E [ Y ]

Q.*What’s*the*value?* � What!is!E[E[X] +1 ]?! e !!!!! A.!E[X]+1!!!!!!!B.!1!!!!!!!!C.!0!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Expected*value*of*a*function*of* X � If! f !is!a!func5on!of!a!random! variable! X !,!then! Y !=! f !( X )!is!a! random!variable!too! � The!expected!value!of! Y !=! f !( X )!is! - Eff Gi ) = E y = -2 fix spc ! !

Expected*value*of*a*function*of* X � If! f !is!a!func5on!of!a!random! variable! X !,!then! Y !=! f !( X )!is!a! random!variable!too! � The!expected!value!of! Y !=! f !( X )!is! � E [ Y ] = E [ f ( X )] = f ( x ) P ( x ) ! x !

Expected*time*of*cat* � A!cat!moves!with!random!constant! EW ) = WE speed! V ,!either!5mile/hr!or!20mile/hr! with!equal!probability,!what’s!the! EF¥¥j4 expected!5me!for!it!to!travel!50!miles?! T=fwt=¥=E ECT ) -_ El I ! . plV=Y ) t upcv=Vu , = xtz-E-xtz-6.vn . = Z !

Expected*time*of*cat* � A!cat!moves!with!random!constant! speed! V ,!either!5mile/hr!or!20mile/hr! with!equal!probability,!what’s!the! expected!5me!for!it!to!travel!50!miles?! t ( V ) = 50 50 � E [ t ( V )] = v P ( v ) V ! v = 50 5 × 1 2 + 50 20 × 1 2 = 6 . 25 hours !

Q:* Is*this*statement*true?** If!there!exists!a!constant!such!that! P ( X !≥!a)!=!!1,!then!E[ X ]!≥!a!.!It!is:! ! I n A. True! Fas B. False! - I xp ex - x , E Cx ) - - + Ea E = Esa xp ' ?FE'a a

Q:* Is*this*statement*true?** If!there!exists!a!constant!such!that! P ( X !≥!a)!=!!1,!then!E[ X ]!≥!a!.!It!is:! ! A. True! B. False! ≥a! 0! x<a ∞ � � E [ X ] = xP ( X = x ) + xP ( X = x ) a −∞

Content* � Random!Variable!! � Expected*value* � Variance(&(covariance * � Towards*the*week*law*of*large* numbers !

Variance*and*standard*deviation � The!variance!of!a!random! mean Cl ki 35 l Xi - var ( f Xi ) > = I variable! X !is! IN var [ X ] = E [( X − E [ X ]) 2 ] o ga � The!standard!devia5on!of!a! random!variable! X !is! ! � std [ X ] = var [ X ] -

Properties*of*variance � For!random!variable! X !and! constant!k! ! non - negative ! var [ X ] ≥ 0 ! var [ kX ] = k 2 var [ X ] with scaling E p ! squared K

A*neater*expression*for*variance � Variance!of!Random!Variable!X!is! defined!as:!! var [ X ] = E [( X − E [ X ]) 2 ] ! - ! � It’s!the!same!as:! var [ X ] = E [ X 2 ] − E [ X ] 2

A*neater*expression*for*variance var [ X ] = E [( X − E [ X ]) 2 ] n -_ Efx ] tu ' ] = El XII MX Efx 'T - 2E -4 × 1 + Elm ' ) = ! = ECI ) - 2h ECXJTNZ ECI ) - ZEEXIETXITETX ] ! = - E' Cx ) ECW ) =

A*neater*expression*for*variance var [ X ] = E [( X − E [ X ]) 2 ] var [ X ] = E [( X − µ ) 2 ] where µ = E [ X ] ! = E [ X 2 − 2 Xµ + µ 2 ] ! = E [ X 2 ] − 2 µE [ X ] + µ 2 By!linearity!of!expecta5on!

A*neater*expression*for*variance var [ X ] = E [( X − E [ X ]) 2 ] var [ X ] = E [( X − µ ) 2 ] where µ = E [ X ] ! = E [ X 2 − 2 Xµ + µ 2 ] ! = E [ X 2 ] − 2 µE [ X ] + µ 2 = E [ X 2 ] − 2 E [ X ] E [ X ] + ( E [ X ]) 2

A*neater*expression*for*variance var [ X ] = E [( X − E [ X ]) 2 ] var [ X ] = E [( X − µ ) 2 ] where µ = E [ X ] ! = E [ X 2 − 2 Xµ + µ 2 ] ! = E [ X 2 ] − 2 µE [ X ] + µ 2 = E [ X 2 ] − 2 E [ X ] E [ X ] + ( E [ X ]) 2 = E [ X 2 ] − E [ X ] 2

Variance:*the*profit*example* � For!the!profit!example,!what!is!the! variance!of!the!return?!We!know!E[ X ]=! 20p<10! - PCX=k)=fB var [ X ] = E [ X 2 ] − ( E [ X ]) 2 x' to Solve infgroup ' TP = (10 2 p + ( − 10) 2 (1 − p )) − (20 p − 10) 2 x ! - - to y ! - cop = 100 − (400 p 2 − 400 p + 100) ! ktpck ) Cup -2 ! ! = 400 p (1 − p ) 2C C - lo ) 'll - :p ) 10 ? - ! p +

Variance:*the*profit*example* � For!the!profit!example,!what!is!the! variance!of!net!return?!We!know!E[ X ]=! 20p<10! var [ X ] = E [ X 2 ] − ( E [ X ]) 2 = (10 2 p + ( − 10) 2 (1 − p )) − (20 p − 10) 2 = 100 − (400 p 2 − 400 p + 100) ! ! = 400 p (1 − p ) ! ! ! !

Variance:*the*profit*example* � For!the!profit!example,!what!is!the! variance!of!net!return?!We!know!E[ X ]=! 20p<10! var [ X ] = E [ X 2 ] − ( E [ X ]) 2 = (10 2 p + ( − 10) 2 (1 − p )) − (20 p − 10) 2 = 100 − (400 p 2 − 400 p + 100) = 400 p (1 − p ) ! ! ! ! !

Variance:*the*profit*example* � For!the!profit!example,!what!is!the! variance!of!net!return?!We!know!E[ X ]=! 20p<10! var [ X ] = E [ X 2 ] − ( E [ X ]) 2 = (10 2 p + ( − 10) 2 (1 − p )) − (20 p − 10) 2 = 100 − (400 p 2 − 400 p + 100) = 400 p (1 − p ) -

Motivation*for*covariance* � Study!the!rela5onship!between! random!variables * � Note!that!it’s!the!un<normalized! correla5on! � Applica5ons!include!the!fire!control! of!radar,!communica5ng!in!the! presence!of!noise.!!

Covariance* � The *covariance !of!random! variables! X !and! Y !is! cov ( X, Y ) = E [( X − E [ X ])( Y − E [ Y ])] a- . * Iii j Eam ix. Es ' � Note!that! - cov ( X, X ) = E [( X − E [ X ]) 2 ] = var [ X ] -

A*neater*form*for*covariance* � A!neater!expression!for * covariance !(similar!deriva5on!as! for!variance)! cov ( X, Y ) = E [ XY ] − E [ X ] E [ Y ]