Generalization of MP67/S95 Equation � ( S N − zI ) − 1 � 1 m F N ( z ) = N Tr Eigenvectors of some large sample covariance matrix ensembles – p. 10/23
Generalization of MP67/S95 Equation � ( S N − zI ) − 1 � 1 m F N ( z ) = N Tr � � 1 ( S N − zI ) − 1 g (Σ N ) Θ g N ( z ) = N Tr Eigenvectors of some large sample covariance matrix ensembles – p. 10/23
Generalization of MP67/S95 Equation Eigenvectors of some large sample covariance matrix ensembles – p. 11/23
Generalization of MP67/S95 Equation ∃ Θ g : a . s . Θ g ∀ z ∈ C + → Θ g ( z ) N ( z ) − Eigenvectors of some large sample covariance matrix ensembles – p. 11/23
Generalization of MP67/S95 Equation ∃ Θ g : a . s . Θ g ∀ z ∈ C + → Θ g ( z ) N ( z ) − � + ∞ 1 Θ g ( z ) = g ( τ ) × τ [1 − c − czm F ( z )] − zdH ( τ ) −∞ Eigenvectors of some large sample covariance matrix ensembles – p. 11/23
Generalization of MP67/S95 Equation ∃ Θ g : a . s . Θ g ∀ z ∈ C + → Θ g ( z ) N ( z ) − � + ∞ 1 Θ g ( z ) = g ( τ ) × τ [1 − c − czm F ( z )] − zdH ( τ ) −∞ Same integration kernel! Eigenvectors of some large sample covariance matrix ensembles – p. 11/23
Extension to the Real Line Eigenvectors of some large sample covariance matrix ensembles – p. 12/23
Extension to the Real Line N N � � 1 1 i v j | 2 × g ( τ j ) Θ g | u ∗ N ( z ) = N λ i − z i =1 j =1 Eigenvectors of some large sample covariance matrix ensembles – p. 12/23
Extension to the Real Line N N � � 1 1 i v j | 2 × g ( τ j ) Θ g | u ∗ N ( z ) = N λ i − z i =1 j =1 N N � � 1 i v j | 2 × g ( τ j ) Ω g | u ∗ N ( λ ) = 1 [ λ i , + ∞ ) ( λ ) N i =1 j =1 Eigenvectors of some large sample covariance matrix ensembles – p. 12/23
Extension to the Real Line N N � � 1 1 i v j | 2 × g ( τ j ) Θ g | u ∗ N ( z ) = N λ i − z i =1 j =1 N N � � 1 i v j | 2 × g ( τ j ) Ω g | u ∗ N ( λ ) = 1 [ λ i , + ∞ ) ( λ ) N i =1 j =1 � λ 1 Im [Θ g ( l + iη )] dl a . s . Ω g → Ω g ( λ ) = lim N ( λ ) π η → 0 + −∞ wherever Ω g is continuous Eigenvectors of some large sample covariance matrix ensembles – p. 12/23
Sample Eigenvectors Eigenvectors of some large sample covariance matrix ensembles – p. 13/23
Sample Eigenvectors Fix τ and take g = 1 ( −∞ ,τ ) Eigenvectors of some large sample covariance matrix ensembles – p. 13/23
Sample Eigenvectors Fix τ and take g = 1 ( −∞ ,τ ) N N � � N ( λ ) = 1 i v j | 2 1 [ λ i , + ∞ ) ( λ ) × 1 [ τ j , + ∞ ) ( τ ) Ω g | u ∗ N i =1 j =1 � λ � τ clt → | t [1 − c − clm F ( l )] − l | 2 dH ( t ) dF ( l ) −∞ −∞ Eigenvectors of some large sample covariance matrix ensembles – p. 13/23
Sample Eigenvectors Fix τ and take g = 1 ( −∞ ,τ ) N N � � N ( λ ) = 1 i v j | 2 1 [ λ i , + ∞ ) ( λ ) × 1 [ τ j , + ∞ ) ( τ ) Ω g | u ∗ N i =1 j =1 � λ � τ clt → | t [1 − c − clm F ( l )] − l | 2 dH ( t ) dF ( l ) −∞ −∞ cλ i τ j i v j | 2 ≈ N | u ∗ m F ( λ i )] − λ i | 2 | τ j [1 − c − cλ i ˘ Eigenvectors of some large sample covariance matrix ensembles – p. 13/23
Eigenvalues with Multiplicity Eigenvectors of some large sample covariance matrix ensembles – p. 14/23
Eigenvalues with Multiplicity Σ N has K distinct eigenvalues t 1 , . . . , t K with multiplicities n 1 , . . . , n K Eigenvectors of some large sample covariance matrix ensembles – p. 14/23
Eigenvalues with Multiplicity Σ N has K distinct eigenvalues t 1 , . . . , t K with multiplicities n 1 , . . . , n K P k = projection onto k th eigenspace Eigenvectors of some large sample covariance matrix ensembles – p. 14/23
Eigenvalues with Multiplicity Σ N has K distinct eigenvalues t 1 , . . . , t K with multiplicities n 1 , . . . , n K P k = projection onto k th eigenspace n k cλ i t k | P k u i | 2 ≈ m F ( λ i )] − λ i | 2 N | t k [1 − c − cλ i ˘ Eigenvectors of some large sample covariance matrix ensembles – p. 14/23
Estimating the Covariance Matrix (1) Eigenvectors of some large sample covariance matrix ensembles – p. 15/23
Estimating the Covariance Matrix (1) � Tr ( AA ∗ ) Frobenius norm: � A � = Eigenvectors of some large sample covariance matrix ensembles – p. 15/23
Estimating the Covariance Matrix (1) � Tr ( AA ∗ ) Frobenius norm: � A � = U N : matrix of eigenvectors of S N Eigenvectors of some large sample covariance matrix ensembles – p. 15/23
Estimating the Covariance Matrix (1) � Tr ( AA ∗ ) Frobenius norm: � A � = U N : matrix of eigenvectors of S N Find matrix closest to Σ N among those that have eigenvectors U N Eigenvectors of some large sample covariance matrix ensembles – p. 15/23
Estimating the Covariance Matrix (1) � Tr ( AA ∗ ) Frobenius norm: � A � = U N : matrix of eigenvectors of S N Find matrix closest to Σ N among those that have eigenvectors U N � U N D N U ∗ min N − Σ N � D N diagonal Eigenvectors of some large sample covariance matrix ensembles – p. 15/23
Estimating the Covariance Matrix (1) � Tr ( AA ∗ ) Frobenius norm: � A � = U N : matrix of eigenvectors of S N Find matrix closest to Σ N among those that have eigenvectors U N � U N D N U ∗ min N − Σ N � D N diagonal Solution: D N = Diag ( � � d 1 , . . . , � � d i = u ∗ d N ) where i Σ N u i Eigenvectors of some large sample covariance matrix ensembles – p. 15/23
Estimating the Covariance Matrix (2) Eigenvectors of some large sample covariance matrix ensembles – p. 16/23
Estimating the Covariance Matrix (2) Take g ( τ ) = τ Eigenvectors of some large sample covariance matrix ensembles – p. 16/23
Estimating the Covariance Matrix (2) Take g ( τ ) = τ N � N ( λ ) = 1 Ω g u ∗ i Σ N u i 1 [ λ i , + ∞ ) ( λ ) N i =1 � λ l → | 1 − c − clm F ( l ) | 2 dF ( l ) −∞ Eigenvectors of some large sample covariance matrix ensembles – p. 16/23
Estimating the Covariance Matrix (2) Take g ( τ ) = τ N � N ( λ ) = 1 Ω g u ∗ i Σ N u i 1 [ λ i , + ∞ ) ( λ ) N i =1 � λ l → | 1 − c − clm F ( l ) | 2 dF ( l ) −∞ λ i u ∗ i Σ N u i ≈ m F ( λ i ) | 2 | 1 − c − cλ i ˘ Eigenvectors of some large sample covariance matrix ensembles – p. 16/23
Oracle Estimator Eigenvectors of some large sample covariance matrix ensembles – p. 17/23
Oracle Estimator Keep same eigenvectors as those of S n , Eigenvectors of some large sample covariance matrix ensembles – p. 17/23
Oracle Estimator Keep same eigenvectors as those of S n , divide i th sample eigenvalue by | 1 − c − cλ i ˘ m F ( λ i ) | 2 Eigenvectors of some large sample covariance matrix ensembles – p. 17/23
Oracle Estimator Keep same eigenvectors as those of S n , divide i th sample eigenvalue by | 1 − c − cλ i ˘ m F ( λ i ) | 2 → oracle estimator � − S N Eigenvectors of some large sample covariance matrix ensembles – p. 17/23
Oracle Estimator Keep same eigenvectors as those of S n , divide i th sample eigenvalue by | 1 − c − cλ i ˘ m F ( λ i ) | 2 → oracle estimator � − S N Percentage Relative Improvement in Average Loss: � � 2 � � �� S N − U N � D N U ∗ E � N PRIAL = 100 × 1 − � � 2 � � � S N − U N � D N U ∗ E � N Eigenvectors of some large sample covariance matrix ensembles – p. 17/23
Monte-Carlo Simulations Eigenvectors of some large sample covariance matrix ensembles – p. 18/23
Monte-Carlo Simulations 10,000 simulations Eigenvectors of some large sample covariance matrix ensembles – p. 18/23
Monte-Carlo Simulations 10,000 simulations c=1/2 Eigenvectors of some large sample covariance matrix ensembles – p. 18/23
Monte-Carlo Simulations 10,000 simulations c=1/2 Population eigenvalues: 20% equal to 1 40% equal to 3 40% equal to 10 Eigenvectors of some large sample covariance matrix ensembles – p. 18/23
Monte-Carlo Simulations 10,000 simulations c=1/2 Population eigenvalues: 20% equal to 1 40% equal to 3 40% equal to 10 Compare with Ledoit-Wolf (2004) linear shrinkage estimator Eigenvectors of some large sample covariance matrix ensembles – p. 18/23
Simulation Results Eigenvectors of some large sample covariance matrix ensembles – p. 19/23
Simulation Results γ=2 100% Relative Improvement in Average Loss 90% 80% 70% 60% Optimal Nonlinear Shrinkage Optimal Linear Shrinkage 50% 10 20 40 60 80 100 120 140 160 180 200 Sample Size Eigenvectors of some large sample covariance matrix ensembles – p. 19/23
Inverse of the Covariance Matrix (1) Eigenvectors of some large sample covariance matrix ensembles – p. 20/23
Inverse of the Covariance Matrix (1) Find matrix closest to Σ − 1 N among those that have eigenvectors U N Eigenvectors of some large sample covariance matrix ensembles – p. 20/23
Inverse of the Covariance Matrix (1) Find matrix closest to Σ − 1 N among those that have eigenvectors U N � U N ∆ N U ∗ N − Σ − 1 min N � ∆ N diagonal Eigenvectors of some large sample covariance matrix ensembles – p. 20/23
Inverse of the Covariance Matrix (1) Find matrix closest to Σ − 1 N among those that have eigenvectors U N � U N ∆ N U ∗ N − Σ − 1 min N � ∆ N diagonal Solution: ∆ N = Diag ( � � δ 1 , . . . , � � δ i = u ∗ i Σ − 1 δ N ) where N u i Eigenvectors of some large sample covariance matrix ensembles – p. 20/23
Inverse of the Covariance Matrix (1) Find matrix closest to Σ − 1 N among those that have eigenvectors U N � U N ∆ N U ∗ N − Σ − 1 min N � ∆ N diagonal Solution: ∆ N = Diag ( � � δ 1 , . . . , � � δ i = u ∗ i Σ − 1 δ N ) where N u i i Σ N u i ) − 1 u ∗ i Σ − 1 N u i ≥ ( u ∗ Eigenvectors of some large sample covariance matrix ensembles – p. 20/23
Inverse of the Covariance Matrix (2) Eigenvectors of some large sample covariance matrix ensembles – p. 21/23
Inverse of the Covariance Matrix (2) Take g ( τ ) = 1 τ Eigenvectors of some large sample covariance matrix ensembles – p. 21/23
Inverse of the Covariance Matrix (2) Take g ( τ ) = 1 τ N � N ( λ ) = 1 Ω g u ∗ i Σ − 1 N u i 1 [ λ i , + ∞ ) ( λ ) N i =1 � λ 1 − c − 2 cl Re [ m F ( l )] → dF ( l ) l −∞ Eigenvectors of some large sample covariance matrix ensembles – p. 21/23
Inverse of the Covariance Matrix (2) Take g ( τ ) = 1 τ N � N ( λ ) = 1 Ω g u ∗ i Σ − 1 N u i 1 [ λ i , + ∞ ) ( λ ) N i =1 � λ 1 − c − 2 cl Re [ m F ( l )] → dF ( l ) l −∞ N u i ≈ 1 − c − 2 cλ i Re [ ˘ m F ( λ i )] u ∗ i Σ − 1 λ i Eigenvectors of some large sample covariance matrix ensembles – p. 21/23
Conclusion Eigenvectors of some large sample covariance matrix ensembles – p. 22/23
Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Eigenvectors of some large sample covariance matrix ensembles – p. 22/23
Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Gives location of sample eigenvectors relative to: Eigenvectors of some large sample covariance matrix ensembles – p. 22/23
Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Gives location of sample eigenvectors relative to: Population eigenvectors Eigenvectors of some large sample covariance matrix ensembles – p. 22/23
Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Gives location of sample eigenvectors relative to: Population eigenvectors Population covariance matrix as a whole Eigenvectors of some large sample covariance matrix ensembles – p. 22/23
Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Gives location of sample eigenvectors relative to: Population eigenvectors Population covariance matrix as a whole Inverse of population covariance matrix Eigenvectors of some large sample covariance matrix ensembles – p. 22/23
Conclusion Generalization of the Marˇ cenko-Pastur (1967)/Silverstein (1995) Equation Gives location of sample eigenvectors relative to: Population eigenvectors Population covariance matrix as a whole Inverse of population covariance matrix We do for sample eigenvectors what MP67/S95 did for sample eigenvalues Eigenvectors of some large sample covariance matrix ensembles – p. 22/23
Directions for Future Research Eigenvectors of some large sample covariance matrix ensembles – p. 23/23
Directions for Future Research 1. Construct bona fide nonlinear shrinkage estimator of the covariance matrix Eigenvectors of some large sample covariance matrix ensembles – p. 23/23
Recommend
More recommend