Statistical Inference Lecture 2: Transformations and Expectations - PowerPoint PPT Presentation

Statistical Inference Lecture 2: Transformations and Expectations MING GAO DASE @ ECNU (for course related communications) mgao@dase.ecnu.edu.cn Mar. 17, 2020

Outline Transformation 1 Functions of a r.v. Monotone Transformations Expectation 2 Properties of Expectations Moment Moment Generating Functions Differentiating Under an Integral Sign 3 Take-aways 4 MING GAO (DaSE@ECNU) Statistical Inference Mar. 17, 2020 2 / 54

Transformation Functions of a r.v. Outline Transformation 1 Functions of a r.v. Monotone Transformations Expectation 2 Properties of Expectations Moment Moment Generating Functions Differentiating Under an Integral Sign 3 Take-aways 4 MING GAO (DaSE@ECNU) Statistical Inference Mar. 17, 2020 3 / 54

Transformation Functions of a r.v. Functions of a r.v. If X is a r.v., then any function of X , say g ( X ), is also a r.v..

Transformation Functions of a r.v. Functions of a r.v. If X is a r.v., then any function of X , say g ( X ), is also a r.v.. A natural question is whether we can describe the probabilistic behavior of Y in terms of that of X .

Transformation Functions of a r.v. Functions of a r.v. If X is a r.v., then any function of X , say g ( X ), is also a r.v.. A natural question is whether we can describe the probabilistic behavior of Y in terms of that of X . That is, for any set A , P ( Y ∈ A ) = P ( g ( X ) ∈ A ) .

Transformation Functions of a r.v. Functions of a r.v. If X is a r.v., then any function of X , say g ( X ), is also a r.v.. A natural question is whether we can describe the probabilistic behavior of Y in terms of that of X . That is, for any set A , P ( Y ∈ A ) = P ( g ( X ) ∈ A ) . We associate with g an inverse mapping, denoted as g − 1 , which is a mapping from subsets of Y to subsets of X , and is defined by g − 1 ( A ) = { x ∈ X | g ( x ) ∈ A } .

Transformation Functions of a r.v. Functions of a r.v. If X is a r.v., then any function of X , say g ( X ), is also a r.v.. A natural question is whether we can describe the probabilistic behavior of Y in terms of that of X . That is, for any set A , P ( Y ∈ A ) = P ( g ( X ) ∈ A ) . We associate with g an inverse mapping, denoted as g − 1 , which is a mapping from subsets of Y to subsets of X , and is defined by g − 1 ( A ) = { x ∈ X | g ( x ) ∈ A } . For any set A ⊂ Y , P ( Y ∈ A ) = P ( g ( X ) ∈ A ) = P ( X ∈ g − 1 ( A )) . MING GAO (DaSE@ECNU) Statistical Inference Mar. 17, 2020 4 / 54

Transformation Functions of a r.v. Example of discrete transformation Binomial transformation A discrete r.v. X has a binomial distribution if its pmf is of the form � n � p x (1 − p ) n − x , x = 0 , 1 , · · · , n . f X ( x ) = P ( X = x ) = x

Transformation Functions of a r.v. Example of discrete transformation Binomial transformation A discrete r.v. X has a binomial distribution if its pmf is of the form � n � p x (1 − p ) n − x , x = 0 , 1 , · · · , n . f X ( x ) = P ( X = x ) = x Consider the r.v. Y = g ( X ) = n − X , and Y = { y | y = g ( x ) , x ∈ X} = { 0 , 1 , · · · , n } .

Transformation Functions of a r.v. Example of discrete transformation Binomial transformation A discrete r.v. X has a binomial distribution if its pmf is of the form � n � p x (1 − p ) n − x , x = 0 , 1 , · · · , n . f X ( x ) = P ( X = x ) = x Consider the r.v. Y = g ( X ) = n − X , and Y = { y | y = g ( x ) , x ∈ X} = { 0 , 1 , · · · , n } . � f Y ( y ) = f X ( x ) = f X ( n − y ) x ∈ g − 1 ( y ) � n � � n � p n − y (1 − p ) n − ( n − y ) = (1 − p ) y p n − y . = n − y y MING GAO (DaSE@ECNU) Statistical Inference Mar. 17, 2020 5 / 54

Transformation Functions of a r.v. Example of continuous transformation Uniform transformation Suppose X has a uniform distribution on the interval (0 , 2 π ), that is 1 � 2 π , 0 < x < 2 π ; f X ( x ) = 0 , otherwise.

Transformation Functions of a r.v. Example of continuous transformation Uniform transformation Suppose X has a uniform distribution on the interval (0 , 2 π ), that is 1 � 2 π , 0 < x < 2 π ; f X ( x ) = 0 , otherwise. Consider the r.v. Y = sin 2 ( X ), and Y ∈ [0 , 1] .

Transformation Functions of a r.v. Example of continuous transformation Uniform transformation Suppose X has a uniform distribution on the interval (0 , 2 π ), that is 1 � 2 π , 0 < x < 2 π ; f X ( x ) = 0 , otherwise. Consider the r.v. Y = sin 2 ( X ), and Y ∈ [0 , 1] . f Y ( y ) = P ( Y ≤ y ) = P ( X ≤ x 1 ) + P ( x 2 ≤ X ≤ x 3 ) + P ( X ≥ x 4 ) = 2 P ( X ≤ x 1 ) + 2 P ( x 2 ≤ X ≤ π ) , where x 1 and x 2 are the two solutions to sin 2 ( x ) = y for 0 < x < π. MING GAO (DaSE@ECNU) Statistical Inference Mar. 17, 2020 6 / 54

Transformation Monotone Transformations Outline Transformation 1 Functions of a r.v. Monotone Transformations Expectation 2 Properties of Expectations Moment Moment Generating Functions Differentiating Under an Integral Sign 3 Take-aways 4 MING GAO (DaSE@ECNU) Statistical Inference Mar. 17, 2020 7 / 54

Transformation Monotone Transformations Theorem for monotone transformations Let X have cdf F X ( x ), let Y = g ( X ), and let X and Y be defined as X = { x | f X ( x ) > 0 } and Y = { y | y = g ( x ) for x ∈ X} .

Transformation Monotone Transformations Theorem for monotone transformations Let X have cdf F X ( x ), let Y = g ( X ), and let X and Y be defined as X = { x | f X ( x ) > 0 } and Y = { y | y = g ( x ) for x ∈ X} . a. If g is increasing on X , F Y ( y ) = F X ( g − 1 ( y )) for y ∈ Y ;

Transformation Monotone Transformations Theorem for monotone transformations Let X have cdf F X ( x ), let Y = g ( X ), and let X and Y be defined as X = { x | f X ( x ) > 0 } and Y = { y | y = g ( x ) for x ∈ X} . a. If g is increasing on X , F Y ( y ) = F X ( g − 1 ( y )) for y ∈ Y ; b. If g is decreasing on X and X is a continuous r.v., F Y ( y ) = 1 − F X ( g − 1 ( y )) for y ∈ Y ;

Transformation Monotone Transformations Theorem for monotone transformations Let X have cdf F X ( x ), let Y = g ( X ), and let X and Y be defined as X = { x | f X ( x ) > 0 } and Y = { y | y = g ( x ) for x ∈ X} . a. If g is increasing on X , F Y ( y ) = F X ( g − 1 ( y )) for y ∈ Y ; b. If g is decreasing on X and X is a continuous r.v., F Y ( y ) = 1 − F X ( g − 1 ( y )) for y ∈ Y ; Proof. [a.] { x ∈ X| g ( x ) ≤ y } = { x ∈ X| x ≤ g − 1 ( y ) } since If g is increasing. Furthermore, we have � g − 1 ( y ) � f X ( x ) dx = F X ( g − 1 ( y )) . F Y ( y ) = f X ( x ) dx = x ∈X : x ≤ g − 1 ( y ) −∞ MING GAO (DaSE@ECNU) Statistical Inference Mar. 17, 2020 8 / 54

Transformation Monotone Transformations Uniform exponential relationship Suppose X ∼ f X ( x ) = 1 if 0 < x < 1 and 0 otherwise, the uniform (0 , 1) distribution. It is straightforward to check that F X ( x ) = x , 0 < x < 1. Let Y = g ( X ) = − log X . dx g ( x ) = d d dx ( − log x ) = − 1 < 0 , for 0 < x < 1 . x

Transformation Monotone Transformations Uniform exponential relationship Suppose X ∼ f X ( x ) = 1 if 0 < x < 1 and 0 otherwise, the uniform (0 , 1) distribution. It is straightforward to check that F X ( x ) = x , 0 < x < 1. Let Y = g ( X ) = − log X . dx g ( x ) = d d dx ( − log x ) = − 1 < 0 , for 0 < x < 1 . x That is g ( x ) is a decreasing function. As X ∈ [0 , 1] and − log x ∈ [0 , ∞ ].

Transformation Monotone Transformations Uniform exponential relationship Suppose X ∼ f X ( x ) = 1 if 0 < x < 1 and 0 otherwise, the uniform (0 , 1) distribution. It is straightforward to check that F X ( x ) = x , 0 < x < 1. Let Y = g ( X ) = − log X . dx g ( x ) = d d dx ( − log x ) = − 1 < 0 , for 0 < x < 1 . x That is g ( x ) is a decreasing function. As X ∈ [0 , 1] and − log x ∈ [0 , ∞ ]. For y > 0, y = − log x implies x = e − y , therefore, F Y ( y ) = 1 − F X ( g − 1 ( y )) = 1 − F X ( e − y ) = 1 − e − y . Of course, F Y ( y ) = 0 for y ≤ 0. MING GAO (DaSE@ECNU) Statistical Inference Mar. 17, 2020 9 / 54

Transformation Monotone Transformations Theorem for continuous r.v. Let X have pdf f X ( x ) and let Y = g ( X ), where g is a monotone function. Suppose that f X ( x ) is continuous on X that g − 1 ( y ) has a continuous derivative on Y . Then the pdf of Y is given by � f X ( g − 1 ( y )) | d dx g − 1 ( y ) | , y ∈ Y ; f Y ( y ) = 0 , otherwise . Proof.

Transformation Monotone Transformations Theorem for continuous r.v. Let X have pdf f X ( x ) and let Y = g ( X ), where g is a monotone function. Suppose that f X ( x ) is continuous on X that g − 1 ( y ) has a continuous derivative on Y . Then the pdf of Y is given by � f X ( g − 1 ( y )) | d dx g − 1 ( y ) | , y ∈ Y ; f Y ( y ) = 0 , otherwise . Proof. By the chain rule,  f X ( g − 1 ( y )) d dy g − 1 ( y ) , if g is increasing;  f Y ( y ) = d  dx F Y ( y ) = − f X ( g − 1 ( y )) d dy g − 1 ( y ) , if g is decreasing .   MING GAO (DaSE@ECNU) Statistical Inference Mar. 17, 2020 10 / 54

Transformation Monotone Transformations Example Inverted Gamma pdf Question: Suppose X has the Gamma pdf 1 ( n − 1)! β n x n − 1 e − x /β , 0 < x < ∞ . f ( x ) = Suppose we want to find the pdf of g ( X ) = 1 X .