eigenvectors of some large sample covariance matrix
play

Eigenvectors of some large sample covariance matrix ensembles - PowerPoint PPT Presentation

Eigenvectors of some large sample covariance matrix ensembles Random Matrix Workshop, Tlcom ParisTech Monday, October 11 th 2010 Olivier Ledoit Sandrine P ech e oledoit@iew.uzh.ch - sandrine.peche@ujf-grenoble.fr University of Z


  1. Generalization of MP67/S95 Equation � ( S N − zI ) − 1 � 1 m F N ( z ) = N Tr Eigenvectors of some large sample covariance matrix ensembles – p. 10/23

  2. Generalization of MP67/S95 Equation � ( S N − zI ) − 1 � 1 m F N ( z ) = N Tr � � 1 ( S N − zI ) − 1 g (Σ N ) Θ g N ( z ) = N Tr Eigenvectors of some large sample covariance matrix ensembles – p. 10/23

  3. Generalization of MP67/S95 Equation Eigenvectors of some large sample covariance matrix ensembles – p. 11/23

  4. Generalization of MP67/S95 Equation ∃ Θ g : a . s . Θ g ∀ z ∈ C + → Θ g ( z ) N ( z ) − Eigenvectors of some large sample covariance matrix ensembles – p. 11/23

  5. Generalization of MP67/S95 Equation ∃ Θ g : a . s . Θ g ∀ z ∈ C + → Θ g ( z ) N ( z ) − � + ∞ 1 Θ g ( z ) = g ( τ ) × τ [1 − c − czm F ( z )] − zdH ( τ ) −∞ Eigenvectors of some large sample covariance matrix ensembles – p. 11/23

  6. Generalization of MP67/S95 Equation ∃ Θ g : a . s . Θ g ∀ z ∈ C + → Θ g ( z ) N ( z ) − � + ∞ 1 Θ g ( z ) = g ( τ ) × τ [1 − c − czm F ( z )] − zdH ( τ ) −∞ Same integration kernel! Eigenvectors of some large sample covariance matrix ensembles – p. 11/23

  7. Extension to the Real Line Eigenvectors of some large sample covariance matrix ensembles – p. 12/23

  8. Extension to the Real Line N N � � 1 1 i v j | 2 × g ( τ j ) Θ g | u ∗ N ( z ) = N λ i − z i =1 j =1 Eigenvectors of some large sample covariance matrix ensembles – p. 12/23

  9. Extension to the Real Line N N � � 1 1 i v j | 2 × g ( τ j ) Θ g | u ∗ N ( z ) = N λ i − z i =1 j =1 N N � � 1 i v j | 2 × g ( τ j ) Ω g | u ∗ N ( λ ) = 1 [ λ i , + ∞ ) ( λ ) N i =1 j =1 Eigenvectors of some large sample covariance matrix ensembles – p. 12/23

  10. Extension to the Real Line N N � � 1 1 i v j | 2 × g ( τ j ) Θ g | u ∗ N ( z ) = N λ i − z i =1 j =1 N N � � 1 i v j | 2 × g ( τ j ) Ω g | u ∗ N ( λ ) = 1 [ λ i , + ∞ ) ( λ ) N i =1 j =1 � λ 1 Im [Θ g ( l + iη )] dl a . s . Ω g → Ω g ( λ ) = lim N ( λ ) π η → 0 + −∞ wherever Ω g is continuous Eigenvectors of some large sample covariance matrix ensembles – p. 12/23

  11. Sample Eigenvectors Eigenvectors of some large sample covariance matrix ensembles – p. 13/23

  12. Sample Eigenvectors Fix τ and take g = 1 ( −∞ ,τ ) Eigenvectors of some large sample covariance matrix ensembles – p. 13/23

  13. Sample Eigenvectors Fix τ and take g = 1 ( −∞ ,τ ) N N � � N ( λ ) = 1 i v j | 2 1 [ λ i , + ∞ ) ( λ ) × 1 [ τ j , + ∞ ) ( τ ) Ω g | u ∗ N i =1 j =1 � λ � τ clt → | t [1 − c − clm F ( l )] − l | 2 dH ( t ) dF ( l ) −∞ −∞ Eigenvectors of some large sample covariance matrix ensembles – p. 13/23

  14. Sample Eigenvectors Fix τ and take g = 1 ( −∞ ,τ ) N N � � N ( λ ) = 1 i v j | 2 1 [ λ i , + ∞ ) ( λ ) × 1 [ τ j , + ∞ ) ( τ ) Ω g | u ∗ N i =1 j =1 � λ � τ clt → | t [1 − c − clm F ( l )] − l | 2 dH ( t ) dF ( l ) −∞ −∞ cλ i τ j i v j | 2 ≈ N | u ∗ m F ( λ i )] − λ i | 2 | τ j [1 − c − cλ i ˘ Eigenvectors of some large sample covariance matrix ensembles – p. 13/23

  15. Eigenvalues with Multiplicity Eigenvectors of some large sample covariance matrix ensembles – p. 14/23

  16. Eigenvalues with Multiplicity Σ N has K distinct eigenvalues t 1 , . . . , t K with multiplicities n 1 , . . . , n K Eigenvectors of some large sample covariance matrix ensembles – p. 14/23

  17. Eigenvalues with Multiplicity Σ N has K distinct eigenvalues t 1 , . . . , t K with multiplicities n 1 , . . . , n K P k = projection onto k th eigenspace Eigenvectors of some large sample covariance matrix ensembles – p. 14/23

  18. Eigenvalues with Multiplicity Σ N has K distinct eigenvalues t 1 , . . . , t K with multiplicities n 1 , . . . , n K P k = projection onto k th eigenspace n k cλ i t k | P k u i | 2 ≈ m F ( λ i )] − λ i | 2 N | t k [1 − c − cλ i ˘ Eigenvectors of some large sample covariance matrix ensembles – p. 14/23

  19. Estimating the Covariance Matrix (1) Eigenvectors of some large sample covariance matrix ensembles – p. 15/23

  20. Estimating the Covariance Matrix (1) � Tr ( AA ∗ ) Frobenius norm: � A � = Eigenvectors of some large sample covariance matrix ensembles – p. 15/23

  21. Estimating the Covariance Matrix (1) � Tr ( AA ∗ ) Frobenius norm: � A � = U N : matrix of eigenvectors of S N Eigenvectors of some large sample covariance matrix ensembles – p. 15/23

  22. Estimating the Covariance Matrix (1) � Tr ( AA ∗ ) Frobenius norm: � A � = U N : matrix of eigenvectors of S N Find matrix closest to Σ N among those that have eigenvectors U N Eigenvectors of some large sample covariance matrix ensembles – p. 15/23

  23. Estimating the Covariance Matrix (1) � Tr ( AA ∗ ) Frobenius norm: � A � = U N : matrix of eigenvectors of S N Find matrix closest to Σ N among those that have eigenvectors U N � U N D N U ∗ min N − Σ N � D N diagonal Eigenvectors of some large sample covariance matrix ensembles – p. 15/23

  24. Estimating the Covariance Matrix (1) � Tr ( AA ∗ ) Frobenius norm: � A � = U N : matrix of eigenvectors of S N Find matrix closest to Σ N among those that have eigenvectors U N � U N D N U ∗ min N − Σ N � D N diagonal Solution: D N = Diag ( � � d 1 , . . . , � � d i = u ∗ d N ) where i Σ N u i Eigenvectors of some large sample covariance matrix ensembles – p. 15/23

  25. Estimating the Covariance Matrix (2) Eigenvectors of some large sample covariance matrix ensembles – p. 16/23

  26. Estimating the Covariance Matrix (2) Take g ( τ ) = τ Eigenvectors of some large sample covariance matrix ensembles – p. 16/23

  27. Estimating the Covariance Matrix (2) Take g ( τ ) = τ N � N ( λ ) = 1 Ω g u ∗ i Σ N u i 1 [ λ i , + ∞ ) ( λ ) N i =1 � λ l → | 1 − c − clm F ( l ) | 2 dF ( l ) −∞ Eigenvectors of some large sample covariance matrix ensembles – p. 16/23

  28. Estimating the Covariance Matrix (2) Take g ( τ ) = τ N � N ( λ ) = 1 Ω g u ∗ i Σ N u i 1 [ λ i , + ∞ ) ( λ ) N i =1 � λ l → | 1 − c − clm F ( l ) | 2 dF ( l ) −∞ λ i u ∗ i Σ N u i ≈ m F ( λ i ) | 2 | 1 − c − cλ i ˘ Eigenvectors of some large sample covariance matrix ensembles – p. 16/23

  29. Oracle Estimator Eigenvectors of some large sample covariance matrix ensembles – p. 17/23

  30. Oracle Estimator Keep same eigenvectors as those of S n , Eigenvectors of some large sample covariance matrix ensembles – p. 17/23

  31. Oracle Estimator Keep same eigenvectors as those of S n , divide i th sample eigenvalue by | 1 − c − cλ i ˘ m F ( λ i ) | 2 Eigenvectors of some large sample covariance matrix ensembles – p. 17/23

  32. Oracle Estimator Keep same eigenvectors as those of S n , divide i th sample eigenvalue by | 1 − c − cλ i ˘ m F ( λ i ) | 2 → oracle estimator � − S N Eigenvectors of some large sample covariance matrix ensembles – p. 17/23

  33. Oracle Estimator Keep same eigenvectors as those of S n , divide i th sample eigenvalue by | 1 − c − cλ i ˘ m F ( λ i ) | 2 → oracle estimator � − S N Percentage Relative Improvement in Average Loss:   � � 2 � � �� S N − U N � D N U ∗ E � N   PRIAL = 100 ×  1 − � �  2 � � � S N − U N � D N U ∗ E � N Eigenvectors of some large sample covariance matrix ensembles – p. 17/23

  34. Monte-Carlo Simulations Eigenvectors of some large sample covariance matrix ensembles – p. 18/23

  35. Monte-Carlo Simulations 10,000 simulations Eigenvectors of some large sample covariance matrix ensembles – p. 18/23

  36. Monte-Carlo Simulations 10,000 simulations c=1/2 Eigenvectors of some large sample covariance matrix ensembles – p. 18/23

  37. Monte-Carlo Simulations 10,000 simulations c=1/2 Population eigenvalues: 20% equal to 1 40% equal to 3 40% equal to 10 Eigenvectors of some large sample covariance matrix ensembles – p. 18/23

  38. Monte-Carlo Simulations 10,000 simulations c=1/2 Population eigenvalues: 20% equal to 1 40% equal to 3 40% equal to 10 Compare with Ledoit-Wolf (2004) linear shrinkage estimator Eigenvectors of some large sample covariance matrix ensembles – p. 18/23

  39. Simulation Results Eigenvectors of some large sample covariance matrix ensembles – p. 19/23

  40. Simulation Results γ=2 100% Relative Improvement in Average Loss 90% 80% 70% 60% Optimal Nonlinear Shrinkage Optimal Linear Shrinkage 50% 10 20 40 60 80 100 120 140 160 180 200 Sample Size Eigenvectors of some large sample covariance matrix ensembles – p. 19/23

  41. Inverse of the Covariance Matrix (1) Eigenvectors of some large sample covariance matrix ensembles – p. 20/23

  42. Inverse of the Covariance Matrix (1) Find matrix closest to Σ − 1 N among those that have eigenvectors U N Eigenvectors of some large sample covariance matrix ensembles – p. 20/23

  43. Inverse of the Covariance Matrix (1) Find matrix closest to Σ − 1 N among those that have eigenvectors U N � U N ∆ N U ∗ N − Σ − 1 min N � ∆ N diagonal Eigenvectors of some large sample covariance matrix ensembles – p. 20/23

  44. Inverse of the Covariance Matrix (1) Find matrix closest to Σ − 1 N among those that have eigenvectors U N � U N ∆ N U ∗ N − Σ − 1 min N � ∆ N diagonal Solution: ∆ N = Diag ( � � δ 1 , . . . , � � δ i = u ∗ i Σ − 1 δ N ) where N u i Eigenvectors of some large sample covariance matrix ensembles – p. 20/23

  45. Inverse of the Covariance Matrix (1) Find matrix closest to Σ − 1 N among those that have eigenvectors U N � U N ∆ N U ∗ N − Σ − 1 min N � ∆ N diagonal Solution: ∆ N = Diag ( � � δ 1 , . . . , � � δ i = u ∗ i Σ − 1 δ N ) where N u i i Σ N u i ) − 1 u ∗ i Σ − 1 N u i ≥ ( u ∗ Eigenvectors of some large sample covariance matrix ensembles – p. 20/23

  46. Inverse of the Covariance Matrix (2) Eigenvectors of some large sample covariance matrix ensembles – p. 21/23

  47. Inverse of the Covariance Matrix (2) Take g ( τ ) = 1 τ Eigenvectors of some large sample covariance matrix ensembles – p. 21/23

  48. Inverse of the Covariance Matrix (2) Take g ( τ ) = 1 τ N � N ( λ ) = 1 Ω g u ∗ i Σ − 1 N u i 1 [ λ i , + ∞ ) ( λ ) N i =1 � λ 1 − c − 2 cl Re [ m F ( l )] → dF ( l ) l −∞ Eigenvectors of some large sample covariance matrix ensembles – p. 21/23

  49. Inverse of the Covariance Matrix (2) Take g ( τ ) = 1 τ N � N ( λ ) = 1 Ω g u ∗ i Σ − 1 N u i 1 [ λ i , + ∞ ) ( λ ) N i =1 � λ 1 − c − 2 cl Re [ m F ( l )] → dF ( l ) l −∞ N u i ≈ 1 − c − 2 cλ i Re [ ˘ m F ( λ i )] u ∗ i Σ − 1 λ i Eigenvectors of some large sample covariance matrix ensembles – p. 21/23

  50. Conclusion Eigenvectors of some large sample covariance matrix ensembles – p. 22/23

  51. Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Eigenvectors of some large sample covariance matrix ensembles – p. 22/23

  52. Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Gives location of sample eigenvectors relative to: Eigenvectors of some large sample covariance matrix ensembles – p. 22/23

  53. Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Gives location of sample eigenvectors relative to: Population eigenvectors Eigenvectors of some large sample covariance matrix ensembles – p. 22/23

  54. Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Gives location of sample eigenvectors relative to: Population eigenvectors Population covariance matrix as a whole Eigenvectors of some large sample covariance matrix ensembles – p. 22/23

  55. Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Gives location of sample eigenvectors relative to: Population eigenvectors Population covariance matrix as a whole Inverse of population covariance matrix Eigenvectors of some large sample covariance matrix ensembles – p. 22/23

  56. Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Gives location of sample eigenvectors relative to: Population eigenvectors Population covariance matrix as a whole Inverse of population covariance matrix We do for sample eigenvectors what MP67/S95 did for sample eigenvalues Eigenvectors of some large sample covariance matrix ensembles – p. 22/23

  57. Directions for Future Research Eigenvectors of some large sample covariance matrix ensembles – p. 23/23

  58. Directions for Future Research 1. Construct bona fide nonlinear shrinkage estimator of the covariance matrix Eigenvectors of some large sample covariance matrix ensembles – p. 23/23

Recommend


More recommend