Random Variables Overview Taylor Series Expansion Probability - PowerPoint PPT Presentation

Random Variables Overview Taylor Series Expansion • Probability Taylor series expansion about t = a , • Random variables ( t − a ) + d 2 x ( t ) ( t − a ) 2 x ( t ) = x ( a ) + d x ( t ) + · · · + d n x ( t ) ( t − a ) n + r n d t d 2 t 2! d n t n ! • Transforms of pdfs where • Moments and cumulants r n = d n +1 x ( t ) � � ( t − a ) n +1 • Useful distributions � d t � t = τ ∈ [ t,a ] • Random vectors • Linear transformations of random vectors • The multivariate normal distribution • Sums of independent random variables • Central limit theorem J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 1 J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 2 Introduction Probability Space • Random signals evolve in time in an unpredictable manner • Let Ω denote all possible outcomes, ζ , of an experiment • We must assume something doesn’t change in order to use them • Event : A subset of outcomes • Usually this is their average properties • The event is said to occur if the outcome of the experiment is one of the members of the subset – It’s an “or” (union) – Not an “and” (intersection) • A collection of subsets with certain properties is called a field and will be denoted as F • The probability of each event in the field is denoted Pr {·} for k = 1 , 2 , . . . • The collection (Ω , F , Pr {·} ) is called a probability space J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 3 J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 4

Random Variables Random Variable Properties • The sample space Ω is the domain of the random variable x ( ζ ) Ω • The set of all values that x ( · ) can have is the range of the Real random variable ζ Line • For now the range is the real line, R 1 Range • Later we will generalize to allow it to be a vector or a sequence • This is a many to one mapping. That is, a set of points, ζ 1 , ζ 2 , . . . • Random variable: a function that assigns a real number, x ( ζ ) , to may take on the same value of the random variable each outcome ζ in the sample space of a random experiment • Will sometimes abbreviate RV • Conditions • The RV may be real- or complex-valued – { ζ : ζ ∈ Ω and − ∞ < x ( ζ ) ≤ x 0 } ∈ F for all x 0 ∈ R 1 • Random variables are actually a deterministic function of any – Pr { x ( ζ ) = ∞} = 0 event in the field: { ζ 1 , ζ 2 , . . . , ζ k } – Pr { x ( ζ ) = −∞} = 0 J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 5 J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 6 Random Variable Properties Continued Cumulative Distribution Function (cdf) • Notation: x ( ζ ) = x • Recall that probability is defined on the field of possible events – x ( ζ ) denotes a random variable (function of the experimental • Some of these events (subsets of outcomes) can be defined as outcome) { ζ : x ( ζ ) ≤ x } – x denotes its value • Cumulative Distribution Function (cdf) : The probability of • A random variable may be Discrete-Valued , these events, Pr { x ( ζ ) ≤ x } Continuous-Valued , or Mixed • Note the cdf is a function of x ∈ R 1 : • Similar to spectral densities F x ( x ) � Pr { x ( ζ ) ≤ x } – If x ( t ) is nearly periodic, it has discrete spectra – Most stationary random signals x ( t ) have continuous spectra • The cdf is continuous from the right – A combination of the two types has mixed spectra J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 7 J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 8

Properties of the cdf Probability Density Function (pdf) • 0 ≤ F x ( x ) ≤ 1 • Probability Density Function (pdf) : When it exists, is defined as • lim x → + ∞ F x ( x ) = 1 f x ( x ) � d F x ( x ) • lim x →−∞ F x ( x ) = 0 d x • F x ( x ) is a nondecreasing function of x • Thus, we also have – Thus, if a < b , then F x ( a ) ≤ F x ( b ) � x F x ( x ) = f x ( u ) d u • F x ( x ) is continuous from the right −∞ – That is, for h > 0 , F x ( b ) = lim h → 0 F x ( b + h ) = F x ( b + ) • For a small interval △ x , can be thought of as • Pr { a < x ( ζ ) ≤ b } = F x ( b ) − F x ( a ) f x ( x ) △ x ≈ F x ( x + △ x ) − F x ( x ) = Pr { x < x ( ζ ) ≤ x + △ x } • Pr { x ( ζ ) = b } = F x ( b ) − F x ( b − ) J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 9 J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 10 Properties of the pdf Point Statistics, Averages, and Moments • In general, we will often not have a complete description of an RV • f x ( x ) ≥ 0 (the pdf or cdf) � b • Pr { a ≤ x ( ζ ) ≤ b } = a f x ( u ) d u • Estimating a pdf or cdf is difficult in general, especially if x ( ζ ) is a � x • F x ( x ) = −∞ f x ( u ) d u sequence or vector � + ∞ • −∞ f x ( u ) d u = 1 • However, we can often estimate some properties of the distribution without estimating the distribution itself • A valid pdf can be formed from any nonnegative, piecewise continuous function g ( x ) that has a finite integral • These are called point statistics • The pdf must be defined for all real values of x • We will only discuss a subset of point statistics: statistical averages or moments • If x does not take on some values, this implies f x ( x ) = 0 for those values • These are useful because – In many cases, we can estimate them accurately from data – They give us useful information about the distribution – We don’t have to know the distribution to estimate them J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 11 J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 12

Expected Values Defined Expectation Properties � + ∞ Expected value : defined for a random variable x ( ζ ) as E [ x ( ζ )] � µ x = xf x ( x ) d x � + ∞ −∞ E[ x ( ζ )] � µ x = xf x ( x ) d x −∞ • Also called the mean of x ( ζ ) If x ( ζ ) is discrete-valued, the pdf consists only of impulses and can be • The mean can be regarded as the “center of gravity” of f x ( x ) alternatively written in terms of the pmf • If f x ( x ) is symmetric about a , f ( x − a ) = f ( a − x ) , then µ x = a � + ∞ �� � • If f x ( x ) is even, µ x = 0 E [ x ( ζ )] = x p k δ ( x − x k ) d x • This is a linear operation −∞ k � + ∞ E [ αx ( ζ ) + β ] = αµ x + β � xδ ( x − x k ) d x = p k −∞ • If y ( ζ ) = g ( x ( ζ )) , a function of the random variable x ( ζ ) , y ( ζ ) is k also a random variable such that � = p k x k � ∞ k E [ y ( ζ )] � E [ g ( x ( ζ ))] = g ( x ) f x ( x ) d x −∞ J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 13 J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 14 Moments Defined Moments and Definitions m th-order Moment of x ( ζ ) is defined as � ∞ r ( m ) � E [ x m ( ζ )] = x m f x ( x ) d x � ∞ x −∞ r ( m ) � E [ x m ( ζ )] = x m f x ( x ) d x � ∞ x −∞ γ ( m ) E [( x ( ζ ) − µ x ) m ] = ( x − µ x ) m f x ( x ) d x � x −∞ m th-order Central Moment of x ( ζ ) is defined as � ∞ • Variance of x ( ζ ) is defined as γ ( m ) � E [( x ( ζ ) − µ x ) m ] = ( x − µ x ) m f x ( x ) d x x −∞ var[ x ( ζ )] � σ 2 x � γ (2) ( x ( ζ ) − µ x ) 2 � � = E x • Mean-Squared Value : The second-order moment, r (2) � • Standard deviation of x ( ζ ) is defined as σ x = var[ x ( ζ )] x . • Obvious moments • Note that � = E 2 [ x ( ζ )] x 2 ( ζ ) � � E r (0) r (1) = 1 = µ x x x • Trivial central moments γ (0) γ (1) γ (2) = σ 2 = 1 = 0 x x x x J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 15 J. McNames Portland State University ECE 538/638 Random Variables Ver. 1.07 16