Harmonic Means of Wishart Matrices Hi! Im Asad Lodhia, Im a postdoc - PowerPoint PPT Presentation

Harmonic Means of Wishart Matrices Hi! I’m Asad Lodhia, I’m a postdoc at the University of Michigan. Feel free to email me at alodhia@umich.edu Based on joint work with Keith Levin and Liza Levina. Thanks to the organizers for the invitation! I hope you enjoy it.

What’s a Harmonic Mean? ake two positive definite matrices P 1 and P 2 � − 1 P − 1 + P − 1 � 1 2 2 he AMHM inequality: � − 1 P − 1 + P − 1 � P 1 + P 2 1 2 � 2 2 o the operator norm is smaller...

What’s a Harmonic Mean? Take two positive definite matrices P 1 and P 2 � − 1 P − 1 + P − 1 � 1 2 2 he AMHM inequality: � − 1 P − 1 + P − 1 � P 1 + P 2 1 2 � 2 2 o the operator norm is smaller...

What’s a Harmonic Mean? Take two positive definite matrices P 1 and P 2 � − 1 P − 1 + P − 1 � 1 2 2 The AMHM inequality: � − 1 P − 1 + P − 1 � P 1 + P 2 1 2 � 2 2 o the operator norm is smaller...

What’s a Harmonic Mean? Take two positive definite matrices P 1 and P 2 � − 1 P − 1 + P − 1 � 1 2 2 The AMHM inequality: � − 1 P − 1 + P − 1 � P 1 + P 2 1 2 � 2 2 So the operator norm is smaller...

The Wishart Ensemble et X be P × N , i.i.d complex standard normals P < N : � P � ≤ K � � N − γ for some K > 0 and γ ∈ (0 , 1) . P 2 , � � he matrix W = XX ∗ has limiting spectral measure N ((1 + √ γ ) 2 − x )( x − (1 − √ γ ) 2 ) � ρ γ ( x ) := 2 γπx his is invertible with probability 1.

The Wishart Ensemble Let X be P × N , i.i.d complex standard normals P < N : � P � ≤ K � � N − γ for some K > 0 and γ ∈ (0 , 1) . P 2 , � � he matrix W = XX ∗ has limiting spectral measure N ((1 + √ γ ) 2 − x )( x − (1 − √ γ ) 2 ) � ρ γ ( x ) := 2 γπx his is invertible with probability 1.

The Wishart Ensemble Let X be P × N , i.i.d complex standard normals P < N : � P � ≤ K � � N − γ for some K > 0 and γ ∈ (0 , 1) . P 2 , � � The matrix W = XX ∗ has limiting spectral measure N ((1 + √ γ ) 2 − x )( x − (1 − √ γ ) 2 ) � ρ γ ( x ) := 2 γπx his is invertible with probability 1.

The Wishart Ensemble Let X be P × N , i.i.d complex standard normals P < N : � P � ≤ K � � N − γ for some K > 0 and γ ∈ (0 , 1) . P 2 , � � The matrix W = XX ∗ has limiting spectral measure N ((1 + √ γ ) 2 − x )( x − (1 − √ γ ) 2 ) � ρ γ ( x ) := 2 γπx This is invertible with probability 1.

Covariance Estimation he matrix W is an estimate of the Covariance Matrix (in this case I ). he MP-Law show’s W isn’t good in this high-d setting. uantitatively P,N →∞ � W − I � → γ + 2 √ γ lim a.s. s there something closer to I in operator norm ? ptimizing the Frobenius Norm has been done. (Ledoit, Pech´ e, Wolf)

Covariance Estimation The matrix W is an estimate of the Covariance Matrix (in this case I ). he MP-Law show’s W isn’t good in this high-d setting. uantitatively P,N →∞ � W − I � → γ + 2 √ γ lim a.s. s there something closer to I in operator norm ? ptimizing the Frobenius Norm has been done. (Ledoit, Pech´ e, Wolf)

Covariance Estimation The matrix W is an estimate of the Covariance Matrix (in this case I ). The MP-Law show’s W isn’t good in this high-d setting. uantitatively P,N →∞ � W − I � → γ + 2 √ γ lim a.s. s there something closer to I in operator norm ? ptimizing the Frobenius Norm has been done. (Ledoit, Pech´ e, Wolf)

Covariance Estimation The matrix W is an estimate of the Covariance Matrix (in this case I ). The MP-Law show’s W isn’t good in this high-d setting. Quantitatively P,N →∞ � W − I � → γ + 2 √ γ lim a.s. s there something closer to I in operator norm ? ptimizing the Frobenius Norm has been done. (Ledoit, Pech´ e, Wolf)

Covariance Estimation The matrix W is an estimate of the Covariance Matrix (in this case I ). The MP-Law show’s W isn’t good in this high-d setting. Quantitatively P,N →∞ � W − I � → γ + 2 √ γ lim a.s. Is there something closer to I in operator norm ? ptimizing the Frobenius Norm has been done. (Ledoit, Pech´ e, Wolf)

Covariance Estimation The matrix W is an estimate of the Covariance Matrix (in this case I ). The MP-Law show’s W isn’t good in this high-d setting. Quantitatively P,N →∞ � W − I � → γ + 2 √ γ lim a.s. Is there something closer to I in operator norm ? Optimizing the Frobenius Norm has been done. (Ledoit, Pech´ e, Wolf)

Bear with me here... et’s imagine I have multiple matrices { X i } n i =1 all P × N . uppose we form the average n A = 1 � W i , n i =1 X i X ∗ here W i = i N ust line up the X i , side by side � γ P,N →∞ � A − I � → γ lim n + 2 a.s. n loser now.

Bear with me here... Let’s imagine I have multiple matrices { X i } n i =1 all P × N . uppose we form the average n A = 1 � W i , n i =1 X i X ∗ here W i = i N ust line up the X i , side by side � γ P,N →∞ � A − I � → γ lim n + 2 a.s. n loser now.

Bear with me here... Let’s imagine I have multiple matrices { X i } n i =1 all P × N . Suppose we form the average n A = 1 � W i , n i =1 X i X ∗ here W i = i N ust line up the X i , side by side � γ P,N →∞ � A − I � → γ lim n + 2 a.s. n loser now.

Bear with me here... Let’s imagine I have multiple matrices { X i } n i =1 all P × N . Suppose we form the average n A = 1 � W i , n i =1 X i X ∗ here W i = i N Just line up the X i , side by side � γ P,N →∞ � A − I � → γ lim n + 2 a.s. n loser now.

Bear with me here... Let’s imagine I have multiple matrices { X i } n i =1 all P × N . Suppose we form the average n A = 1 � W i , n i =1 X i X ∗ here W i = i N Just line up the X i , side by side � γ P,N →∞ � A − I � → γ lim n + 2 a.s. n Closer now.

Alternative Means ake their Harmonic Mean: n ) − 1 , H = n ( W − 1 + · · · + W − 1 1 esult: the limiting ESD is n � ( e + − x )( x − e − ) 2 πγx where � γ � e ± = 1 − γ + 2 γ 1 − γ + γ n ± 2 n n lso: � γ � P,N →∞ � H − I � = 1 − e − = γ − 2 γ 1 − γ + γ lim n + 2 n n

Alternative Means Take their Harmonic Mean: n ) − 1 , H = n ( W − 1 + · · · + W − 1 1 esult: the limiting ESD is n � ( e + − x )( x − e − ) 2 πγx where � γ � e ± = 1 − γ + 2 γ 1 − γ + γ n ± 2 n n lso: � γ � P,N →∞ � H − I � = 1 − e − = γ − 2 γ 1 − γ + γ lim n + 2 n n

Alternative Means Take their Harmonic Mean: n ) − 1 , H = n ( W − 1 + · · · + W − 1 1 Result: the limiting ESD is n � ( e + − x )( x − e − ) 2 πγx where � γ � e ± = 1 − γ + 2 γ 1 − γ + γ n ± 2 n n lso: � γ � P,N →∞ � H − I � = 1 − e − = γ − 2 γ 1 − γ + γ lim n + 2 n n

Alternative Means Take their Harmonic Mean: n ) − 1 , H = n ( W − 1 + · · · + W − 1 1 Result: the limiting ESD is n � ( e + − x )( x − e − ) 2 πγx where � γ � e ± = 1 − γ + 2 γ 1 − γ + γ n ± 2 n n Also: � γ � P,N →∞ � H − I � = 1 − e − = γ − 2 γ 1 − γ + γ lim n + 2 n n

Figure of the ESD vs LSD P = 500 , N = 1000 and n = 2

Improved Operator Norm Estimate e have the a.s. result P,N →∞ � H − I � < lim P,N →∞ � A − I � lim for n ≤ n ∗ ( γ ) . ndeed this is equivalent to � γ � � γ γ − 2 γ 1 − γ + γ n < γ n + 2 n + 2 n n or n = 2 it’s always true for γ ∈ (0 , 1) � 1 − γ 2 < γ � � 2 γ 2 + 2 γ

Improved Operator Norm Estimate We have the a.s. result P,N →∞ � H − I � < lim P,N →∞ � A − I � lim for n ≤ n ∗ ( γ ) . ndeed this is equivalent to � γ � � γ γ − 2 γ 1 − γ + γ n < γ n + 2 n + 2 n n or n = 2 it’s always true for γ ∈ (0 , 1) � 1 − γ 2 < γ � � 2 γ 2 + 2 γ

Improved Operator Norm Estimate We have the a.s. result P,N →∞ � H − I � < lim P,N →∞ � A − I � lim for n ≤ n ∗ ( γ ) . Indeed this is equivalent to � γ � � γ γ − 2 γ 1 − γ + γ n < γ n + 2 n + 2 n n or n = 2 it’s always true for γ ∈ (0 , 1) � 1 − γ 2 < γ � � 2 γ 2 + 2 γ

Improved Operator Norm Estimate We have the a.s. result P,N →∞ � H − I � < lim P,N →∞ � A − I � lim for n ≤ n ∗ ( γ ) . Indeed this is equivalent to � γ � � γ γ − 2 γ 1 − γ + γ n < γ n + 2 n + 2 n n For n = 2 it’s always true for γ ∈ (0 , 1) � 1 − γ 2 < γ � � 2 γ 2 + 2 γ

Error Comparison for n = 2 as a function of γ

Is this Just an Identity Matrix Fact? nswer: No, but general Covar is tricky. ubmultiplicative bound √ √ √ √ � � Σ �� Σ − 1 �� H − I � � � � � Σ H Σ − Σ � ≤ Σ A Σ − Σ � � � � � A − I � � � o if we have � Σ �� Σ − 1 �� H − I � lim sup < 1 � A − I � P,N →∞ then √ √ � � Σ H Σ − Σ � � � � lim sup < 1 √ √ � � P,N →∞ Σ A Σ − Σ � � � �