19-11-2019 Department of Large Animal Sciences The multivariate - PDF document

19-11-2019 Department of Large Animal Sciences The multivariate normal distribution Anders Ringgaard Kristensen Department of Large Animal Sciences Outline Covariance and correlation Random vectors and multivariate distributions The multinomial distribution Slide 2 Department of Large Animal Sciences Covariance and correlation Let X and Y be two random variables having expected values µ x , µ y and standard deviations σ x and σ y the covariance between X and Y is defined as • Cov( X , Y ) = σ xy = E(( X − µ x )( Y − µ y )) = E( XY ) - µ x µ y The correlation beween X and Y is In particular we have Cov( X , X ) = σ x 2 and Corr( X , X ) = 1 If X and Y are independent, then E( XY ) = µ x µ y and therefore: • Cov( X , Y ) = 0 • Corr( X , Y ) = 0 Slide 3 1

19-11-2019 Department of Large Animal Sciences Random vectors I Some experiments produce outcomes that are vectors. Such a vector X is called a random vector. We write X = ( X 1 X 2 … X n )’. Each element X i in X is a random variable having an expected value E( X i ) = µ i and a variance Var( X i ) = σ i 2 . The covariance between two elements X i and X j is denoted σ ij For convenience we may use the notation σ ii = σ i 2 Slide 4 Department of Large Animal Sciences Random vectors II A random vector X = ( X 1 X 2 … X k )’ has an expected value, which is also a vector. It has a ”variance”, Σ , which is a matrix: Σ is also called the variance-covariance matrix or just the covariance matrix. Since Cov( X i , X j ) = Cov( X j , X i ), we conclude that Σ is symmetric, i.e σ ij = σ ji Slide 5 Department of Large Animal Sciences Random vectors III Let X be a random vector of dimension n. Assume that E( X ) = µ µ µ µ , and let Σ Σ Σ be the covariance matrix of Σ X . Define Y = AX + b , where A is an m × n matrix and b is an m dimensional vector. Then Y is an m dimensional random vector with E( Y ) = A µ µ µ + b , and covariance matrix A Σ µ Σ Σ Σ A ’ (compare with corresponding rule for ordinary random variables). Slide 6 2

19-11-2019 Department of Large Animal Sciences Multivariate distributions The distribution of a random vector is called a multivariate distribution. Some multivariate distributions may be expressed by a certain function over the sample space. We shall consider t he multivariate normal distribution (continuous) Slide 7 Department of Large Animal Sciences The multivariate normal distribution I A k dimensional random vector X with sample space S = R k has a multivariate normal distribution if it has a density function given as The expected value is E( X ) = µ , and the covariance matrix is Σ . Slide 8 Department of Large Animal Sciences The multivariate normal distribution II The density function of the 2 dimensional random vector Z = ( Z 1 Z 2 )’. What is the sign of Cov( Z 1 Z 2 )? Slide 9 3

19-11-2019 Department of Large Animal Sciences The multivariate normal distribution III Conditional distribution of subset: • Suppose that X = ( X 1 … X k )’ is N( µ , Σ ) and we partition X into two sub-vectors X a = ( X 1 … X j )’ and X b = ( X j+1 … X k )’. We partition the mean vector µ and the covariance matrix Σ accordingly and write � � � � • � � � � Σ � Σ �� Σ �� � � � � Σ �� Σ �� • Then X a ~ N( µ a , Σ aa ) and X b ~ N( µ b , Σ bb ) Slide 10 Department of Large Animal Sciences The multivariate normal distribution IV Conditional distribution, continued: • The matrix Σ Σ ab = Σ Σ ’ ba contains the co- Σ Σ Σ Σ variances between elements of the sub-vector X a and the sub-vector X b . • Moreover, for X a = x a the conditional distribution ( X b | x a ) is N( ν ν ν ν , C ) where -1 ( x a − µ • ν ν = µ ν ν µ µ µ b + Σ Σ Σ Σ ba Σ Σ Σ Σ aa µ a ) µ µ • C = Σ Σ Σ Σ bb − Σ Σ Σ ba Σ Σ Σ Σ Σ aa -1 Σ Σ Σ Σ ab Slide 11 Department of Large Animal Sciences The multivariate normal distribution V Example: • Let X 1 , X 2 , … X 5 denote the first five lactations of a dairy cow. • It is then reasonable to assume that X = ( X 1 X 2 … X 5 )’ has a 5 dimensional normal distribution. • Having observed e.g. X 1 , X 2 and X 3 we can predict X 4 and X 5 according to the conditional formulas on previous slide. Slide 12 4