Chapter 3 General Random Variables Peng-Hua Wang Graduate - PowerPoint PPT Presentation

Chapter 3 General Random Variables Peng-Hua Wang Graduate Institute of Communication Engineering National Taipei University

Chapter Contents 3.1 Continuous Random Variables and PDFs 3.2 Cumulative Distribution Functions 3.3 Normal Random Variables 3.4 Joint PDFs of Multiple Random Variables 3.5 Conditioning 3.6 The Continuous Bayes’ Rule 3.7 Summary and Discussion Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 2/37

3.1 Continuous Random Variables and PDFs Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 3/37

Concepts ■ Let X be the arrival time of a bus. X is a continuous random variable taking value between p and q . ■ The sample space Ω = [ p , q ] . For any point c ∈ Ω , P ( X = a ) = 0. ■ We can find P ( x ≤ a ) , P ( a < X ≤ b ) , or P ( a < X ≤ a + δ ) . In general, we can find P ( X ∈ B ) for B ⊂ Ω . ■ Let F X ( x ) = P ( X ≤ x ) . We know that F X ( − ∞ ) = P ( X ≤ − ∞ ) = 0, F X ( ∞ ) = P ( X ≤ ∞ ) = 1 Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 4/37

Concepts ■ Let F X ( x + δ ) − F X ( x ) = dF X ( x ) f X ( x ) � lim δ dx δ → 0 ■ We have � x P ( X ≤ x ) = F X ( x ) = − ∞ f X ( t ) dt ■ Therefore, P ( X ∈ B ) can be evaluated in terms of integral of f X ( x ) . For example, � b P ( a < X ≤ b ) = a f X ( x ) dx Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 5/37

PDF ■ f X ( x ) is called the probability density function (PDF) of continuous random variable X . ■ F X ( x ) = P ( X ≤ x ) is called the cumulative distribution function (CDF). ◆ f X ( x ) ≥ 0 ◆ � ∞ − ∞ f X ( x ) dx = P ( − ∞ < X ≤ ∞ ) = 1 ◆ P ( x < X < x + δ ) ≈ f X ( x ) · δ if δ is small. Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 6/37

Example 3.1. Uniform distribution. � a ≤ x ≤ b c , f X ( x ) = 0, otherwise. Find c . Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 7/37

Example 3.3. Uniform distribution. c √ x , f X ( x ) = 0 < x ≤ 1 Find c . ■ A PDF can take arbitrarily large values. Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 8/37

Expectation ■ The mean or expectation of a continuous random variable X is defined by E [ X ] = � ∞ − ∞ x f X ( x ) dx . ■ The k th moment is E [ X k ] = � ∞ − ∞ x k f X ( x ) dx . ■ The variance of X is Var ( X ) = E [( X − E [ X ]) 2 ] = E [ X 2 ] − ( E [ X ]) 2 ■ The mean of new RV Y = g ( X ) is � ∞ E [ g ( X )] = − ∞ g ( x ) f X ( x ) dx . Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 9/37

Expectation ■ The expectation is well-defined if � ∞ E [ | X | ] = − ∞ | x | f X ( x ) dx < ∞ . ■ A not-well-defined random variable: Cauchy RV. Its PDF is c f X ( x ) = − ∞ < x < ∞ . 1 + x 2 , (Please find c ). E [ X ] is not well-defined. Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 10/37

Example 3.4. Uniformly RV 1 f X ( x ) = a ≤ x ≤ b . b − a , Find E [ X ] and Var ( X ) . Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 11/37

Exponential RV f X ( x ) = ke − λ x , 0 ≤ x < ∞ . Find k , E [ X ] and Var ( X ) . Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 12/37

3.2 Cumulative Distribution Functions Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 13/37

Cumulative Distribution Functions ■ The cumulative distribution function (CDF) of a rv X , denoted by F X ( x ) , is defined by  ∑ k ≤ x p X ( k ) , if X is discrete,  F X ( x ) � P ( X ≤ x ) = � x − ∞ f X ( x ) dx , if X is continuous.  ■ CDF is exactly probability. PDF is NOT probability. ■ Note the “ ≤ ” in the definition. ■ “Any random variable associated with a given probability model has a CDF, regardless of whether it is discrete or continuous.” Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 14/37

Fig 3.6 CDFs of some discrete random variables Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 15/37

Fig 3.7 CDFs of some continuous random variables Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 16/37

Properties of a CDF ■ Definition: F X ( x ) � P ( X ≤ x ) ■ Monotonically nondecreasing: If x ≤ y , then F X ( x ) ≤ F X ( y ) . ■ F X ( − ∞ ) = 0, F X (+ ∞ ) = 1 ■ If X is discrete, F X ( x ) is piecewise constant. If X is continuous, F X ( x ) is continuous. ■ If X is discrete, p X ( k ) = F X ( k ) − F X ( k − 1 ) ■ If X is continuous, f X ( x ) = d dx F X ( x ) Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 17/37

Example 3.6 ■ Let X 1 , X 2 and X 3 be 3 independent discrete random variables with identical PMFs. X = max { X 1 , X 2 , X 3 } Find PMF of X . ■ Let X 1 , X 2 and X 3 be 3 independent continuous random variables with identical PDFs. X = max { X 1 , X 2 , X 3 } Find PDF of X . Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 18/37

3.3 Normal Random Variables Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 19/37

Definition A continuous random variable X is said to be normal or Gaussian if it has a PDF of the form f X ( x ) = ce − ( x − µ ) 2 / ( 2 σ 2 ) , − ∞ < x < ∞ . 1 ■ c = √ 2 πσ ■ E [ X ] = µ ■ Var ( X ) = σ 2 Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 20/37

Standard Normal ■ A standard normal rv Y is the normal rv with µ = 0 and σ = 1. Its CDF is denoted by Φ ( y ) � y 1 e − t 2 /2 dt √ Φ ( y ) = 2 π − ∞ ■ Φ ( − y ) = 1 − Φ ( y ) because The PDF of standard normal is even. ■ For normal rv X with mean µ and variance σ 2 , we know that X = σ Y + µ . Thus, � x − µ � Y ≤ x − µ � � P ( X ≤ x ) = P ( σ Y + µ ≤ x ) = P = Φ σ σ Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 21/37

Example 3.7 The annual snowfall at a particular geographic location is modeled as a normal random variable with a mean of µ = 60 inches and a standard deviation of σ = 20. What is the probability that this year’s snowfall will be at least 80 inches? Y = 20 X + 60 P ( Y ≥ 80 ) = ? Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 22/37

Example 3.8 A binary message is transmitted as a signal s , which is either − l or + 1. The communication channel corrupts the transmission with additive normal noise N with mean µ = 0 and variance σ 2 . The receiver concludes that the signal − 1 (or + 1) was transmitted if the value received is < 0 (or ≥ 0, respectively). What is the probability of error? Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 23/37

3.4 Joint PDFs Of Multiple Random Variables Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 24/37

Definitions ■ X and Y are two continuous random variables. Their joint CDF is F X , Y ( x , y ) � P ( X ≤ x , Y ≤ Y ) ■ Joint PDF is ∂ 2 f X , Y ( x , y ) � ∂ x ∂ yF X , Y ( x , y ) Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 25/37

Properties �� 1. P (( X , Y ) ∈ B ) = ( x , y ) ∈ B f X , Y ( x , y ) dxdy � b � d 2. P ( a < X ≤ b , c < Y ≤ d ) = y = c f X , Y ( x , y ) dydx x = a � x � y 3. F X , Y ( x , y ) = t = − ∞ f X , Y ( s , t ) dtds s = − ∞ � ∞ � ∞ y = − ∞ f X , Y ( x , y ) dydx = 1 4. x = − ∞ 5. P ( x < X ≤ x + δ , y < Y ≤ y + delta ) ≈ f X , Y ( x , y ) δ 2 Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 26/37

Marginal PDF F X ( x ) = P ( X ≤ x ) = P ( X ≤ x , − ∞ < Y < ∞ ) � ∞ � x = y = − ∞ f X , Y ( s , y ) dyds s = − ∞ � ∞ ⇒ f X ( x ) = dF X ( x ) = y = − ∞ f X , Y ( x , y ) dy dx Similarly, � ∞ f Y ( y ) = − ∞ f X , Y ( x , y ) dx Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 27/37

Example 3.9 Jointly Uniform PDF f X , Y ( x , y ) = c , ■ If a < x < b and c < y < d , find c . ■ If | x | + | y | ≤ r , find c . x 2 + y 2 ≤ r , find c . � ■ If Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 28/37

Expectation � ∞ � ∞ E [ g ( X , Y )] = y = − ∞ g ( x , y ) f X , Y ( x , y ) dydx x = − ∞ ■ E [ aX + bY + c ] = aE [ X ] + bE [ Y ] + c ■ You can easily extend the results of two random variables to more joint random variables. Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 29/37

3.5 Conditioning Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 30/37

Conditioning an RV on an Event ■ Conditioning CDF and PDF F X | A ( x ) = P ( X ≤ x | A ) = P ( { X ≤ x } ∩ A ) P ( A ) f X | A ( x ) = d dx F X | A ( x ) ■ Special case: A = { X ∈ B } F X | X ∈ B ( x ) = P ( X ≤ x | X ∈ B ) = P ( { X ≤ x } ∩ { X ∈ B } ) P ( X ∈ B ) � t ≤ x , t ∈ B f X ( t ) dt = P ( X ∈ B ) f X ( x ) f X | X ∈ B ( x ) = d dx F X | X ∈ B ( x ) = x ∈ B . P ( X ∈ B ) , Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 31/37

Example 3.13. The Exponential RV The time T until a new light bulb burns out is an exponential rv with parameter λ . You turn the light on, leaves the room, and when you returns, t time units later, finds that the light bulb is still on, which corresponds to the event A = { T > t } . Let X be the additional time until the light bulb burns out. What is the conditional CDF of X , given the event A ? Hint. P ( X > x | A ) = P ( T > T + x | T > t ) = ? Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 32/37