Fourier and Circulant Matrices are Not Rigid Allen Liu (MIT) Joint - PowerPoint PPT Presentation

Preliminaries Results Proof Overview Conclusion Fourier and Circulant Matrices are Not Rigid Allen Liu (MIT) Joint work with Zeev Dvir (Princeton) July 18, 2019

Preliminaries Results Proof Overview Conclusion Matrix Rigidity

Preliminaries Results Proof Overview Conclusion Matrix Rigidity A matrix M is rigid if it cannot be written as A + E where A is low-rank and E is sparse

Preliminaries Results Proof Overview Conclusion Matrix Rigidity A matrix M is rigid if it cannot be written as A + E where A is low-rank and E is sparse Definition R M ( r ) is the smallest number s for which M = A + E rank( A ) ≤ r E is s -sparse

Preliminaries Results Proof Overview Conclusion Motivation

Preliminaries Results Proof Overview Conclusion Motivation Matrix rigidity was introduced as a method for proving circuit lower bounds

Preliminaries Results Proof Overview Conclusion Motivation Matrix rigidity was introduced as a method for proving circuit lower bounds Theorem [Valiant 77] Let M be an N × N matrix over a field F . If for some constant ǫ > 0 � � N � N 1+ ǫ � R F ≥ Ω M log log N then M cannot be computed by circuits of size O ( N ) and depth O (log N ).

Preliminaries Results Proof Overview Conclusion Motivation Matrix rigidity was introduced as a method for proving circuit lower bounds Theorem [Valiant 77] Let M be an N × N matrix over a field F . If for some constant ǫ > 0 � � N � N 1+ ǫ � R F ≥ Ω M log log N then M cannot be computed by circuits of size O ( N ) and depth O (log N ). � � � N 2 − ǫ � A random matrix has R F N ≥ Ω M log log N

Preliminaries Results Proof Overview Conclusion Motivation Matrix rigidity was introduced as a method for proving circuit lower bounds Theorem [Valiant 77] Let M be an N × N matrix over a field F . If for some constant ǫ > 0 � � N � N 1+ ǫ � R F ≥ Ω M log log N then M cannot be computed by circuits of size O ( N ) and depth O (log N ). � � � N 2 − ǫ � A random matrix has R F N ≥ Ω M log log N It is a long standing open problem to give an explicit construction of a rigid matrix

Preliminaries Results Proof Overview Conclusion Previous Work on Rigid Matrices

Preliminaries Results Proof Overview Conclusion Previous Work on Rigid Matrices Theorem [Shokrollahi et. al., 1997] For any N × N matrix M for which all minors are nonsingular � N 2 � r log N R M ( r ) ≥ Ω r

Preliminaries Results Proof Overview Conclusion Previous Work on Rigid Matrices Theorem [Shokrollahi et. al., 1997] For any N × N matrix M for which all minors are nonsingular � N 2 � r log N R M ( r ) ≥ Ω r Theorem [Goldreich, Tal 2016] Let A ∈ F N × N be a (uniformly) random circulant matrix. Then for √ 2 N , N every r ∈ [ 32 ], with 1 − o (1) probability � � N 3 R F 2 A ( r ) = Ω r 2 log N

Preliminaries Results Proof Overview Conclusion Previous Work on Rigid Matrices Theorem [Shokrollahi et. al., 1997] For any N × N matrix M for which all minors are nonsingular � N 2 � r log N R M ( r ) ≥ Ω r Theorem [Goldreich, Tal 2016] Let A ∈ F N × N be a (uniformly) random circulant matrix. Then for √ 2 N , N every r ∈ [ 32 ], with 1 − o (1) probability � � N 3 R F 2 A ( r ) = Ω r 2 log N Neither of these is strong enough for Valiant’s method

Preliminaries Results Proof Overview Conclusion Special Families of Matrices

Preliminaries Results Proof Overview Conclusion Special Families of Matrices (Generalized) Hadamard Matrix H d , n H xy = ω � x , y � where ω = e 2 π i and x , y ∈ Z n d d

Preliminaries Results Proof Overview Conclusion Special Families of Matrices (Generalized) Hadamard Matrix H d , n H xy = ω � x , y � where ω = e 2 π i and x , y ∈ Z n d d Fourier Matrix F N F ij = ζ x · y where ζ = e 2 π i N and x , y ∈ Z N

Preliminaries Results Proof Overview Conclusion Special Families of Matrices (Generalized) Hadamard Matrix H d , n H xy = ω � x , y � where ω = e 2 π i and x , y ∈ Z n d d Fourier Matrix F N F ij = ζ x · y where ζ = e 2 π i N and x , y ∈ Z N Circulant Matrix M N M ij = f ( i − j mod N ) for i , j ∈ Z N

Preliminaries Results Proof Overview Conclusion Special Families of Matrices (Generalized) Hadamard Matrix H d , n H xy = ω � x , y � where ω = e 2 π i and x , y ∈ Z n d d Fourier Matrix F N F ij = ζ x · y where ζ = e 2 π i N and x , y ∈ Z N Circulant Matrix M N M ij = f ( i − j mod N ) for i , j ∈ Z N Group Algebra Matrix M G M ab = f ( ab − 1 ) for a , b ∈ G

Preliminaries Results Proof Overview Conclusion Special Families of Matrices (Generalized) Hadamard Matrix H d , n H xy = ω � x , y � where ω = e 2 π i and x , y ∈ Z n d d Fourier Matrix F N F ij = ζ x · y where ζ = e 2 π i N and x , y ∈ Z N Circulant Matrix M N M ij = f ( i − j mod N ) for i , j ∈ Z N Group Algebra Matrix M G M ab = f ( ab − 1 ) for a , b ∈ G Do any of these families contain rigid matrices?

Preliminaries Results Proof Overview Conclusion Special Families of Matrices (Generalized) Hadamard Matrix H d , n H xy = ω � x , y � where ω = e 2 π i and x , y ∈ Z n d d Fourier Matrix F N F ij = ζ x · y where ζ = e 2 π i N and x , y ∈ Z N Circulant Matrix M N M ij = f ( i − j mod N ) for i , j ∈ Z N Group Algebra Matrix M G M ab = f ( ab − 1 ) for a , b ∈ G Do any of these families contain rigid matrices? Showing that any of these families contains some rigid matrix would still imply circuit lower bounds

Preliminaries Results Proof Overview Conclusion Previous Work on Non-rigid Matrices

Preliminaries Results Proof Overview Conclusion Previous Work on Non-rigid Matrices The Hadamard matrix is not rigid [Alman, Williams 2016] For every ǫ > 0, there exists ǫ ′ > 0 such that for all sufficiently large n , � 2 n (1 − ǫ ′ ) � R Q ≤ 2 n (1+ ǫ ) H 2 , n

Preliminaries Results Proof Overview Conclusion Previous Work on Non-rigid Matrices The Hadamard matrix is not rigid [Alman, Williams 2016] For every ǫ > 0, there exists ǫ ′ > 0 such that for all sufficiently large n , � 2 n (1 − ǫ ′ ) � R Q ≤ 2 n (1+ ǫ ) H 2 , n Group algebra matrices for Z n q are not rigid [Dvir, Edelman 2017] Fix ǫ > 0. There is ǫ ′ > 0 such that group algebra matrices for Z n q with entries over F q satisfy � q n (1 − ǫ ′ ) � R F q ≤ q n (1+ ǫ ) M for fixed q and n sufficiently large

Preliminaries Results Proof Overview Conclusion Previous Work on Non-rigid Matrices The Hadamard matrix is not rigid [Alman, Williams 2016] For every ǫ > 0, there exists ǫ ′ > 0 such that for all sufficiently large n , � 2 n (1 − ǫ ′ ) � R Q ≤ 2 n (1+ ǫ ) H 2 , n Group algebra matrices for Z n q are not rigid [Dvir, Edelman 2017] Fix ǫ > 0. There is ǫ ′ > 0 such that group algebra matrices for Z n q with entries over F q satisfy � q n (1 − ǫ ′ ) � R F q ≤ q n (1+ ǫ ) M for fixed q and n sufficiently large Neither of these matrices is rigid enough to carry out Valiant’s method for proving circuit lower bounds.

Preliminaries Results Proof Overview Conclusion Our results

Preliminaries Results Proof Overview Conclusion Our results Theorem (informal) Generalized Hadamard, Fourier, circulant, and group algebra matrices for abelian groups are all not Valiant-rigid.

Preliminaries Results Proof Overview Conclusion Our results Theorem (informal) Generalized Hadamard, Fourier, circulant, and group algebra matrices for abelian groups are all not Valiant-rigid. Theorem For all sufficiently large N , if M is an N × N circulant matrix over C or some fixed finite field F q , � � N ≤ N 1+15 ǫ R M 2 ǫ 6 (log N ) 0 . 35

Preliminaries Results Proof Overview Conclusion Our results Theorem (informal) Generalized Hadamard, Fourier, circulant, and group algebra matrices for abelian groups are all not Valiant-rigid. Theorem For all sufficiently large N , if M is an N × N circulant matrix over C or some fixed finite field F q , � � N ≤ N 1+15 ǫ R M 2 ǫ 6 (log N ) 0 . 35 Previous results [AW16, DE17] only work with matrices whose symmetry group has small characteristic

Preliminaries Results Proof Overview Conclusion Our results Theorem (informal) Generalized Hadamard, Fourier, circulant, and group algebra matrices for abelian groups are all not Valiant-rigid. Theorem For all sufficiently large N , if M is an N × N circulant matrix over C or some fixed finite field F q , � � N ≤ N 1+15 ǫ R M 2 ǫ 6 (log N ) 0 . 35 Previous results [AW16, DE17] only work with matrices whose symmetry group has small characteristic In this presentation treat everything over C

Preliminaries Results Proof Overview Conclusion Proof Overview

Preliminaries Results Proof Overview Conclusion Diagonalization Trick

Preliminaries Results Proof Overview Conclusion Diagonalization Trick Observation: F N diagonalizes any N × N circulant matrix M

Preliminaries Results Proof Overview Conclusion Diagonalization Trick Observation: F N diagonalizes any N × N circulant matrix M M = F ∗ N DF N = ( F N − E ) ∗ DF N + E ∗ D ( F N − E ) + E ∗ DE