Upper Bound for Intermediate Singular Values of Random Sub-Gaussian - PowerPoint PPT Presentation

Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices 1 Feng Wei 2 University of Michigan July 29, 2016 1 This presentation is based a project under the supervision of M. Rudelson. 2 Partially supported by M. Rudelson’s NSF Grant DMS-1464514, and USAF Grant FA9550-14-1-0009. Feng Wei (University of Michigan) Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices July 29, 2016 1 / 16

Motivation and Backgound Asymptotic Distribution of Singular Values Let A be an n × n random matrix with i.i.d. entries of mean 0 and variance 1. s 1 ( A ) ≥ s 2 ( A ) ≥ · · · ≥ s n ( A ) denote the singular values of A . Consider � A � � � µ ( J ) = 1 √ n n # i : s i ∈ J , J ⊂ R . √ By Quarter Circular Law, d µ ( x ) → 1 4 − x 2 1 [0 , 2] ( x ) dx as n → ∞ . π Feng Wei (University of Michigan) Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices July 29, 2016 2 / 16

Motivation and Backgound Asymptotic Distribution of Singular Values Let A be an n × n random matrix with i.i.d. entries of mean 0 and variance 1. s 1 ( A ) ≥ s 2 ( A ) ≥ · · · ≥ s n ( A ) denote the singular values of A . Consider � A � � � µ ( J ) = 1 √ n n # i : s i ∈ J , J ⊂ R . √ By Quarter Circular Law, d µ ( x ) → 1 4 − x 2 1 [0 , 2] ( x ) dx as n → ∞ . π A simple computation using the limiting distribution shows the ℓ th ℓ smallest singular value s n +1 − ℓ ( A ) is in the order of √ n for ℓ = 1 , 2 , · · · , n . Feng Wei (University of Michigan) Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices July 29, 2016 2 / 16

Motivation and Backgound Asymptotic Distribution of Singular Values Let A be an n × n random matrix with i.i.d. entries of mean 0 and variance 1. s 1 ( A ) ≥ s 2 ( A ) ≥ · · · ≥ s n ( A ) denote the singular values of A . Consider � A � � � µ ( J ) = 1 √ n n # i : s i ∈ J , J ⊂ R . √ By Quarter Circular Law, d µ ( x ) → 1 4 − x 2 1 [0 , 2] ( x ) dx as n → ∞ . π A simple computation using the limiting distribution shows the ℓ th ℓ smallest singular value s n +1 − ℓ ( A ) is in the order of √ n for ℓ = 1 , 2 , · · · , n . Question What is the distribution of the singular values for a fixed large n ? Feng Wei (University of Michigan) Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices July 29, 2016 2 / 16

Motivation and Backgound Asymptotic Distribution of Singular Values Let A be an n × n random matrix with i.i.d. entries of mean 0 and variance 1. s 1 ( A ) ≥ s 2 ( A ) ≥ · · · ≥ s n ( A ) denote the singular values of A . Consider � A � � � µ ( J ) = 1 √ n n # i : s i ∈ J , J ⊂ R . √ By Quarter Circular Law, d µ ( x ) → 1 4 − x 2 1 [0 , 2] ( x ) dx as n → ∞ . π A simple computation using the limiting distribution shows the ℓ th ℓ smallest singular value s n +1 − ℓ ( A ) is in the order of √ n for ℓ = 1 , 2 , · · · , n . Question What is the distribution of the singular values for a fixed large n ? Feng Wei (University of Michigan) Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices July 29, 2016 2 / 16

Motivation and Backgound Sub-gaussian Random Variables Definition Let θ > 0. Let Z be a random variable. Then the ψ θ -norm of Z is defined as � � � θ � | Z | � Z � ψ θ := inf λ > 0 : E exp ≤ 2 λ If � Z � ψ θ < ∞ , then Z is called a ψ θ random variable. This condition is satisfied for broad classes of random variables. In particular, a bounded random variable is ψ θ for any θ > 0, a normal random variable is ψ 2 , and a Poisson variable is ψ 1 . A ψ 2 random variable is also called sub-gaussian . Feng Wei (University of Michigan) Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices July 29, 2016 3 / 16

Motivation and Backgound Sub-gaussian Random Variables Definition Let θ > 0. Let Z be a random variable. Then the ψ θ -norm of Z is defined as � � � θ � | Z | � Z � ψ θ := inf λ > 0 : E exp ≤ 2 λ If � Z � ψ θ < ∞ , then Z is called a ψ θ random variable. This condition is satisfied for broad classes of random variables. In particular, a bounded random variable is ψ θ for any θ > 0, a normal random variable is ψ 2 , and a Poisson variable is ψ 1 . A ψ 2 random variable is also called sub-gaussian . ⇒ P ( | X | > t ) ≤ exp(1 − ct 2 Moreover, � X � ψ 2 = K ⇐ K 2 ). Feng Wei (University of Michigan) Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices July 29, 2016 3 / 16

Motivation and Backgound Sub-gaussian Random Variables Definition Let θ > 0. Let Z be a random variable. Then the ψ θ -norm of Z is defined as � � � θ � | Z | � Z � ψ θ := inf λ > 0 : E exp ≤ 2 λ If � Z � ψ θ < ∞ , then Z is called a ψ θ random variable. This condition is satisfied for broad classes of random variables. In particular, a bounded random variable is ψ θ for any θ > 0, a normal random variable is ψ 2 , and a Poisson variable is ψ 1 . A ψ 2 random variable is also called sub-gaussian . ⇒ P ( | X | > t ) ≤ exp(1 − ct 2 Moreover, � X � ψ 2 = K ⇐ K 2 ). Feng Wei (University of Michigan) Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices July 29, 2016 3 / 16

Motivation and Backgound Non-asymptotic Distribution of Singular Values The extreme singular values are better studied: It is easy to show that the operator norm of N × n i.i.d. sub-gaussian √ matrix is in the order of N with high probability. Feng Wei (University of Michigan) Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices July 29, 2016 4 / 16

Motivation and Backgound Non-asymptotic Distribution of Singular Values The extreme singular values are better studied: It is easy to show that the operator norm of N × n i.i.d. sub-gaussian √ matrix is in the order of N with high probability. In 2008, M. Rudelson proved that the smallest singular value of a square i.i.d. sub-gaussian matrix is lower bounded by n − 3 / 2 with high probability. Feng Wei (University of Michigan) Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices July 29, 2016 4 / 16

Motivation and Backgound Non-asymptotic Distribution of Singular Values The extreme singular values are better studied: It is easy to show that the operator norm of N × n i.i.d. sub-gaussian √ matrix is in the order of N with high probability. In 2008, M. Rudelson proved that the smallest singular value of a square i.i.d. sub-gaussian matrix is lower bounded by n − 3 / 2 with high probability. This result was later extended and improved by A. Basak and M. Rudelson, M. Rudelson and R. Vershynin, T. Tao and V. Vu in square matrices case. Feng Wei (University of Michigan) Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices July 29, 2016 4 / 16

Motivation and Backgound Non-asymptotic Distribution of Singular Values The extreme singular values are better studied: It is easy to show that the operator norm of N × n i.i.d. sub-gaussian √ matrix is in the order of N with high probability. In 2008, M. Rudelson proved that the smallest singular value of a square i.i.d. sub-gaussian matrix is lower bounded by n − 3 / 2 with high probability. This result was later extended and improved by A. Basak and M. Rudelson, M. Rudelson and R. Vershynin, T. Tao and V. Vu in square matrices case. In 2009, M. Rudelson and R. Vershynin proved a sharp bound for smallest singular value of all rectangular sub-gaussian matrices. Feng Wei (University of Michigan) Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices July 29, 2016 4 / 16

Motivation and Backgound Non-asymptotic Distribution of Singular Values The extreme singular values are better studied: It is easy to show that the operator norm of N × n i.i.d. sub-gaussian √ matrix is in the order of N with high probability. In 2008, M. Rudelson proved that the smallest singular value of a square i.i.d. sub-gaussian matrix is lower bounded by n − 3 / 2 with high probability. This result was later extended and improved by A. Basak and M. Rudelson, M. Rudelson and R. Vershynin, T. Tao and V. Vu in square matrices case. In 2009, M. Rudelson and R. Vershynin proved a sharp bound for smallest singular value of all rectangular sub-gaussian matrices. Feng Wei (University of Michigan) Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices July 29, 2016 4 / 16

Motivation and Backgound Lower Bound for Singular Values Theorem.(M. Rudelson and R. Vershynin, 2009) Let G be an N × n random matrix, N ≥ n , whose elements are independent copies of a centered sub-gaussian random variable with unit variance. Then for every ε > 0, we have � √ √ � �� ≤ ( C ε ) N − n +1 + e − C ′ N s n ( G ) ≤ ε N − n − 1 P where C , C ′ > 0 depend (polynomially) only on the sub-gaussian moment K . Consider an n × n i.i.d. sub-gaussian matrix A and let B be the first n + 1 − ℓ columns of A . Then with high probability, √ � √ n − ≥ c ℓ � √ n . s n +1 − ℓ ( A ) ≥ s n +1 − ℓ ( B ) ≥ c n − ℓ Feng Wei (University of Michigan) Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices July 29, 2016 5 / 16