Continuous Probability CS70 Summer 2016 - Lecture 6A David Dinh 25 - PowerPoint PPT Presentation

Continuous Probability CS70 Summer 2016 - Lecture 6A David Dinh 25 July 2016 UC Berkeley

Logistics Tutoring Sections - M/W 5-8PM in 540 Cory. • Conceptual discussions of material • No homework discussion (take that to OH/HW party, please) Midterm is this Friday - 11:30-1:30, same rooms as last time. • Covers material from MT1 to this Wednesday... • ...but we will expect you to know everything we’ve covered from the start of class. • One double -sided sheet of notes allowed (our advice: reuse sheet from MT1 and add MT2 topics to the other side). • Students with time conflicts and DSP students should have been contacted by us - if you are one and you haven’t heard from us, get in touch ASAP. 1

Today • What is continuous probability? • Expectation and variance in the continuous setting. • Some common distributions. 2

Continuous Probability

Motivation I Sometimes you can’t model things discretely. Random real numbers. Points on a map. Time. Probability space is continuous . What is an event in continuous probability? 3 What is probability? Function mapping events to [ 0 , 1 ] .

Motivation II Class starts at 14:10. You take your seat at some ”uniform” random time between 14:00 and 14:10. What’s an event here? Probability of coming in at exactly 14:03:47.32? Sample space: all times between 14:00 and 14:10. Size of sample space? How many numbers are there between 0 and Chance of getting one event in an infinite sized uniform sample space? 0 Not so simple to define events in continuous probability! 4 10? infinite

Motivation III Look at intervals instead of specific times. 1/k. Probability that you come between 14:03 and 14:04? 1/10. 5 1.0 1.0 1.0 0.8 0.8 0.8 0.6 0.6 0.6 0.4 0.4 0.4 0.2 0.2 0.2 0.0 0.0 0.0 Probability that you come in between 14:00 and 14:10? 1. Probability that you come in between 14:00 and 14:05? 1/2. Probability that you come in some time interval of 10 / k minutes?

PDF (no, not the file format) (PDF). 6 space into k pieces - multiply each one by k . What happens when you take k → ∞ ? Probability goes to 0. What do we do so that this doesn’t disappear? If we split our sample 1.0 1.0 1.0 0.8 0.8 0.8 0.6 0.6 0.6 0.4 0.4 0.4 0.2 0.2 0.2 0.0 0.0 0.0 The resulting curve as k → ∞ is the probability density function

Formally speaking... Another way of looking at it: Total probability is 1: f is nonnegative (negative probability doesn’t make much sense). a 7 PDF f X ( t ) of a random variable X is defined so that the probability of X taking on a value in [ t , t + δ ] is δ f ( t ) for infinitesimally small δ . Pr [ X ∈ [ t , t + δ ]] f X ( t ) = lim δ δ → 0 ∫ b Pr [ X ∈ [ a , b ]] = f X ( t ) dt ∫ ∞ −∞ f X ( t ) dt = 1

CDF lim lim 8 Or, in terms of PDF... Cumulative distribution function (CDF): F X ( t ) = Pr [ X ≤ t ] . ∫ t F X ( t ) = f X ( z ) dz −∞ Pr [ X ∈ ( a , b ]] = Pr [ X ≤ b ] − Pr [ X ≤ a ] = F X ( b ) − F X ( a ) F X ( t ) ∈ [ 0 , 1 ] t →−∞ F X ( t ) = 0 t →∞ F X ( t ) = 1

In Pictures 9

Expectation Expectation of a function? Exercise: try proving these yourself. Proof: also similar to discrete case. Proof: similar to discrete case. Linearity of expectation: 10 Discrete case: E [ X ] = ∑ ∞ t = −∞ ( Pr [ X = t ] t ) Continuous case? Sum → integral. ∫ ∞ E [ X ] = tf X ( t ) dt −∞ ∫ ∞ E [ g ( X )] = g ( t ) f X ( t ) dt −∞ E [ aX + bY ] = aE [ X ] + bE [ Y ] If X , Y , Z are mutually independent, then E [ XYZ ] = E [ X ] E [ Y ] E [ Z ] .

Variance Variance is defined exactly like it is for the discrete case. The standard properties for variance hold in the continuous case as well. For independent r.v. X , Y : . 11 Var ( X ) = E [( X − E [ X ]) 2 ] = E [ X 2 ] − E [ X ] 2 Var ( aX ) = a 2 Var ( X ) Var ( X + Y ) = Var ( X ) + Var ( Y )

Target shooting Suppose an archer always hits a circular target with 1 meter radius, and the exact point that he hits is distributed uniformly across the target. What is the distribution the distance between his arrow and the center (call this r.v. X )? t 1 Probability that arrow is closer than t to the center? area of small circle area of dartboard 12 Pr [ X ≤ t ] = π t 2 = π = t 2 .

Target shooting II t 2 otherwise 0 2 t PDF? 1 CDF: 13 0  for t < 0   F Y ( t ) = Pr [ Y ≤ t ] = for 0 ≤ t ≤ 1  for t > 1  { for 0 ≤ t ≤ 1 f Y ( t ) = F Y ( t ) ′ =

Target shooting III Another way of attacking the same problem: what’s the probability t Area of ring: 14 of hitting some ring with inner radius t and outer radius t + δ for small δ ? t + δ Area of circle: π π (( t + δ ) 2 − t 2 ) = π ( t 2 + 2 t δ + δ 2 − t 2 ) = π ( 2 t δ + δ 2 ) ≈ π 2 t δ Probability of hitting the ring: 2 t δ . PDF for t ≤ 1: 2 t

Shifting & Scaling b b b b b 15 b Let f X ( x ) be the pdf of X and Y = a + bX where b > 0. Then Pr [ Y ∈ ( y , y + δ )] = Pr [ a + bX ∈ ( y , y + δ )] Pr [ X ∈ ( y − a , y + δ − a = )] Pr [ X ∈ ( y − a , y − a + δ = b )] f X ( y − a ) δ = b . Left-hand side is f Y ( y ) δ . Hence, bf X ( y − a f Y ( y ) = 1 ) .

Continuous Distributions

Uniform Distribution: CDF and PDF What’s the value of the constant in the interval? a CDF? 16 PDF is constant over some interval [ a , b ] , zero outside the interval. ∫ ∞ ∫ b kdt = kdt = b − a = 1 −∞ so PDF is 1 / ( b − a ) in [ a , b ] and 0 otherwise. ∫ t 1 / ( b − a ) dz −∞ 0 for t < a , ( t − a ) / ( b − a ) for a < t < b , and 1 for t > b .

Uniform Distribution: CDF and PDF, Graphically otherwise b<t 1 0 17 0  t < a {  1 / ( b − a ) a < t < b   f X ( t ) = F X ( t ) = ( t − a ) / ( b − a ) a < t < b   

Uniform Distribution: Expectation and Variance 2 12 t 3 Expectation? a Variance? t 2 18 t 2 a b 2 − a 2 b − a = b + a ∫ b E [ X ] = b − adt = 1 Var [ X ] = E [ X 2 ] − E [ X ] 2 ( b + a ∫ b ) 2 = b − a dt − ‘ 2 � ( b + a ) 2 = a − � b 3 ( b − a ) ‘ 2 = ( a − b ) 2

Exponential Distribution: Motivation Continuous-time analogue of the geometric distribution. How long until a server fails? How long does it take you to run into pokemon? Can’t “continuously flip a coin”. What do we do? Look at geometric distributions representing processes with higher and higher granularity. 19

Exponential Distribution: Motivation II Probability that server fails on the same day as time t : More precision! What’s the probability that it fails in a 12-hour fail than another. period. n n 20 Suppose a server fails with probability λ every day. ( 1 − λ ) ⌈ t ⌉− 1 λ period? λ/ 2 if we assume that there is no time that it’s more likely to Generally: server fails with probability λ/ n during any 1 / n -day time Probability that server fails on the same 1 / n -day time period as t : ) ⌈ tn ⌉− 1 λ ( 1 − λ

Exponential Distribution: Motivation III to get: This is the PDF of the exponential distribution ! n n lim 21 Probability goes to zero...but we can make a PDF out of this! n n ) ⌈ tn ⌉− 1 λ ( 1 − λ What happens when we try to take n to ∞ ? Remove the width of the interval (1 / n ) and take the limit as n → ∞ ) ⌈ tn ⌉− 1 ( 1 − λ ) tn − 1 1 − λ ( λ = λ lim n →∞ n →∞ λ e − λ t =

Exponential Distribution: PDF and CDF 22 The exponential distribution with parameter λ > 0 is defined by { { 0 , if t < 0 0 , if t < 0 f X ( t ) = F X ( t ) = λ e − λ t , 1 − e − λ t , if t ≥ 0 . if t ≥ 0 . Note that Pr [ X > t ] = e − λ t for t > 0.

Expectation & Variance of the Exponential Distribution 0 0 0 0 Integration by parts: 23 0 X = Expo ( λ ) . Then, f X ( x ) = λ e − λ x for 0 ≤ x ≤ 1. Thus, ∫ ∞ ∫ ∞ x λ e − λ x dx = − xde − λ x . E [ X ] = ∫ ∞ ∫ ∞ xde − λ x [ xe − λ x ] ∞ e − λ x dx = 0 − ∫ ∞ de − λ x = − 1 = 0 − 0 + 1 λ. λ So: expectation is E [ X ] = 1 λ . Variance: 1 /λ 2

Properties of the Exponential Distribution: Memorylessness Similar to memorylessness for geometric distributions. “If your server doesn’t fail today, it’s in the same state as it was before today.” 24 Let X = Expo ( λ ) . Then, for s , t > 0, Pr [ X > t + s ] Pr [ X > t + s | X > s ] = Pr [ X > s ] e − λ ( t + s ) = e − λ t = e − λ s = Pr [ X > t ] .

Properties of the Exponential Distribution: Scaling 25 Let X = Expo ( λ ) and Y = aX for some a > 0. Then Pr [ Y > t ] = Pr [ aX > t ] = Pr [ X > t / a ] e − λ ( t / a ) = e − ( λ/ a ) t = Pr [ Z > t ] for Z = Expo ( λ/ a ) . = Thus, a × Expo ( λ ) = Expo ( λ/ a ) . Also, Expo ( λ ) = 1 λ Expo ( 1 ) .

Normal Distribution 1 normal”. Continuous counterpart to Binomial dist. (more on this later) 26 Normal (or Gaussian) distribution with parameters µ , σ 2 , denoted N ( µ, σ 2 ) : 2 πσ 2 e − ( t − µ ) 2 √ f X ( t ) = 2 σ 2 0.4 0.3 0.2 0.1 - 4 - 2 2 4 Sometimes called a ”bell curve”. Above: N ( 0 , 1 ) , the ”standard