Algebraic Methods for Tensor Data Neriman Tokcan (Broad Institute, - PowerPoint PPT Presentation

Algebraic Methods for Tensor Data Neriman Tokcan (Broad Institute, MIT/Harvard) Harm Derksen (Dept. of Math., Northeastern University) Jonathan Gryak (BCIL lab, Univ. of Michigan) Kayvan Najarian (BCIL lab, Univ. of Michigan) August 25, 2020 Algebraic Methods for Tensor Data

Tensors V i = R p i V = V 1 ⊗ V 2 ⊗ · · · ⊗ V d = R p 1 × p 2 ×···× p d T ∈ V tensor Algebraic Methods for Tensor Data

Tensors V i = R p i V = V 1 ⊗ V 2 ⊗ · · · ⊗ V d = R p 1 × p 2 ×···× p d T ∈ V tensor tensors appear in many data applications e.g., chemometrics, psychometrics, imaging, algebraic complexity theory, signal processing, neural networks, . . . Algebraic Methods for Tensor Data

Tensors V i = R p i V = V 1 ⊗ V 2 ⊗ · · · ⊗ V d = R p 1 × p 2 ×···× p d T ∈ V tensor tensors appear in many data applications e.g., chemometrics, psychometrics, imaging, algebraic complexity theory, signal processing, neural networks, . . . T · S inner product of tensors √ �T � = |T � F = T · T Euclidean (Frobenius) norm Algebraic Methods for Tensor Data

Tensor Rank Definition rank( T ) is the smallest integer r for which we have a decomposition r � ( ⋆ ) T = v 1 j ⊗ v 2 j ⊗ · · · ⊗ v dj j =1 Algebraic Methods for Tensor Data

Tensor Rank Definition rank( T ) is the smallest integer r for which we have a decomposition r � ( ⋆ ) T = v 1 j ⊗ v 2 j ⊗ · · · ⊗ v dj j =1 if r is minimal then ( ⋆ ) is called the CP decomposition Algebraic Methods for Tensor Data

Tensor Rank Definition rank( T ) is the smallest integer r for which we have a decomposition r � ( ⋆ ) T = v 1 j ⊗ v 2 j ⊗ · · · ⊗ v dj j =1 if r is minimal then ( ⋆ ) is called the CP decomposition tensor rank is not continuous or even semi-continuous, i.e., {T ∈ V | rank( T ) ≤ r } not always a closed set Algebraic Methods for Tensor Data

Nuclear Norm use convex relaxation of tensor rank: Definition the nuclear norm �T � ⋆ of T is the minimal value of r � (#) � v i 1 ⊗ v i 2 ⊗ · · · ⊗ v id � j =1 where r is arbitrary and r � ( ⋆ ) T = v 1 j ⊗ v 2 j ⊗ · · · ⊗ v dj j =1 (a minimum is attained) Algebraic Methods for Tensor Data

Nuclear Norm use convex relaxation of tensor rank: Definition the nuclear norm �T � ⋆ of T is the minimal value of r � (#) � v i 1 ⊗ v i 2 ⊗ · · · ⊗ v id � j =1 where r is arbitrary and r � ( ⋆ ) T = v 1 j ⊗ v 2 j ⊗ · · · ⊗ v dj j =1 (a minimum is attained) if �T � ⋆ is equal to (#), then ( ⋆ ) is called the nuclear decomposition or convex decomposition Algebraic Methods for Tensor Data

Spectral Norm Definition the spectral norm of T is �T � σ = max {T · ( v 1 ⊗ v 2 ⊗· · ·⊗ v d ) | � v 1 � = � v 2 � = · · · = � v d � = 1 } (a maximum is attained) Algebraic Methods for Tensor Data

Spectral Norm Definition the spectral norm of T is �T � σ = max {T · ( v 1 ⊗ v 2 ⊗· · ·⊗ v d ) | � v 1 � = � v 2 � = · · · = � v d � = 1 } (a maximum is attained) � · � σ and � · � ⋆ are dual norms |T · S| ≤ �T � ⋆ �S� σ Algebraic Methods for Tensor Data

Matrices ( d = 2) A ∈ R p × q has a singular value decomposition A = UDV t with U ∈ O p , V ∈ O q , D diagonal with the nonzero diagonal entries λ 1 ≥ λ 2 ≥ · · · ≥ λ r > 0 Algebraic Methods for Tensor Data

Matrices ( d = 2) A ∈ R p × q has a singular value decomposition A = UDV t with U ∈ O p , V ∈ O q , D diagonal with the nonzero diagonal entries λ 1 ≥ λ 2 ≥ · · · ≥ λ r > 0 r = rank( A ) is the matrix rank and the tensor rank of A Algebraic Methods for Tensor Data

Matrices ( d = 2) A ∈ R p × q has a singular value decomposition A = UDV t with U ∈ O p , V ∈ O q , D diagonal with the nonzero diagonal entries λ 1 ≥ λ 2 ≥ · · · ≥ λ r > 0 r = rank( A ) is the matrix rank and the tensor rank of A √ AA t ) = λ 1 + λ 2 + · · · + λ r � A � ⋆ = Tr( Algebraic Methods for Tensor Data

Matrices ( d = 2) A ∈ R p × q has a singular value decomposition A = UDV t with U ∈ O p , V ∈ O q , D diagonal with the nonzero diagonal entries λ 1 ≥ λ 2 ≥ · · · ≥ λ r > 0 r = rank( A ) is the matrix rank and the tensor rank of A √ AA t ) = λ 1 + λ 2 + · · · + λ r � A � ⋆ = Tr( � A � σ = λ 1 is operator norm Algebraic Methods for Tensor Data

Matrices ( d = 2) A ∈ R p × q has a singular value decomposition A = UDV t with U ∈ O p , V ∈ O q , D diagonal with the nonzero diagonal entries λ 1 ≥ λ 2 ≥ · · · ≥ λ r > 0 r = rank( A ) is the matrix rank and the tensor rank of A √ AA t ) = λ 1 + λ 2 + · · · + λ r � A � ⋆ = Tr( � A � σ = λ 1 is operator norm the norms and the rank of A are easy to compute Algebraic Methods for Tensor Data

negative results ( d = 3) Theorem (H˚ astad) computing tensor rank is NP-complete Algebraic Methods for Tensor Data

negative results ( d = 3) Theorem (H˚ astad) computing tensor rank is NP-complete Theorem (Friedland–Lim) computing the nuclear norm is NP-complete Algebraic Methods for Tensor Data

negative results ( d = 3) Theorem (H˚ astad) computing tensor rank is NP-complete Theorem (Friedland–Lim) computing the nuclear norm is NP-complete Theorem (Hillar–Lim) computing the spectral norm is NP-complete Algebraic Methods for Tensor Data

Approximating Spectral Norm (case d = 3 for simplicity) � 1 �� d S p − 1 × S q − 1 × S r − 1 |T · ( x ⊗ y ⊗ z ) | d �T � σ, d = . normalize �T � σ, d �T � σ, d = � e 1 ⊗ e 1 ⊗ e 1 � σ, d Algebraic Methods for Tensor Data

Approximating Spectral Norm (case d = 3 for simplicity) � 1 �� d S p − 1 × S q − 1 × S r − 1 |T · ( x ⊗ y ⊗ z ) | d �T � σ, d = . normalize �T � σ, d �T � σ, d = � e 1 ⊗ e 1 ⊗ e 1 � σ, d �T � σ = lim d →∞ �T � σ, d . Algebraic Methods for Tensor Data

Approximating Spectral Norm (case d = 3 for simplicity) � 1 �� d S p − 1 × S q − 1 × S r − 1 |T · ( x ⊗ y ⊗ z ) | d �T � σ, d = . normalize �T � σ, d �T � σ, d = � e 1 ⊗ e 1 ⊗ e 1 � σ, d �T � σ = lim d →∞ �T � σ, d . when d is even there is an algebraic method for computing �T � σ, d ! Algebraic Methods for Tensor Data

Invariant Tensors V = R n , V ⊗ d = V ⊗ V ⊗ · · · ⊗ V � �� � d Algebraic Methods for Tensor Data

Invariant Tensors V = R n , V ⊗ d = V ⊗ V ⊗ · · · ⊗ V � �� � d O n acts on V and V ⊗ d Algebraic Methods for Tensor Data

Invariant Tensors V = R n , V ⊗ d = V ⊗ V ⊗ · · · ⊗ V � �� � d O n acts on V and V ⊗ d there is an O n -equivariant linear isomorphism between V ⊗ d and the space of multilinear maps V d → R Algebraic Methods for Tensor Data

Invariant Tensors V = R n , V ⊗ d = V ⊗ V ⊗ · · · ⊗ V � �� � d O n acts on V and V ⊗ d there is an O n -equivariant linear isomorphism between V ⊗ d and the space of multilinear maps V d → R so there is a linear isomorphism between the space ( V ⊗ d ) O n of O n -invariant tensors and the space of O n -invariant multilinear maps V d → R Algebraic Methods for Tensor Data

Brauer Diagrams a Brauer diagram is a perfect matching, for example 1 3 5 D = 2 4 6 Algebraic Methods for Tensor Data

Brauer Diagrams a Brauer diagram is a perfect matching, for example 1 3 5 D = 2 4 6 to a diagram E on d vertices we can associate an O n -invariant multilinear map M E : V d → R , for example M D ( v 1 , v 2 , . . . , v 6 ) = ( v 1 · v 3 )( v 2 · v 6 )( v 4 · v 5 ) Algebraic Methods for Tensor Data

Brauer Diagrams a Brauer diagram is a perfect matching, for example 1 3 5 D = 2 4 6 to a diagram E on d vertices we can associate an O n -invariant multilinear map M E : V d → R , for example M D ( v 1 , v 2 , . . . , v 6 ) = ( v 1 · v 3 )( v 2 · v 6 )( v 4 · v 5 ) the invariant multi-linear map M E corresponds to an invariant tensor T E using the linear isomorphism of before, for example n n n � � � T D = e i ⊗ e j ⊗ e i ⊗ e k ⊗ e k ⊗ e j i =1 j =1 k =1 Algebraic Methods for Tensor Data

Brauer Diagrams Theorem (FFT of Invariant Theory for O n ) the space ( V ⊗ d ) O n is spanned by all T D where D runs over all Brauer diagrams on d vertices Algebraic Methods for Tensor Data

Brauer Diagrams Theorem (FFT of Invariant Theory for O n ) the space ( V ⊗ d ) O n is spanned by all T D where D runs over all Brauer diagrams on d vertices the tensors T D are independent if n ≥ d Algebraic Methods for Tensor Data

Brauer Diagrams Theorem (FFT of Invariant Theory for O n ) the space ( V ⊗ d ) O n is spanned by all T D where D runs over all Brauer diagrams on d vertices the tensors T D are independent if n ≥ d the number of Brauer diagrams on d vertices is 1 · 3 · 5 · · · d − 1 (when d even) Algebraic Methods for Tensor Data

Partial Brauer Diagrams to a partial Brauer diagram with d vertices and e edges we associate an O( V )-equivariant multi-linear map M D : V d → V ⊗ ( d − 2 e ) and a linear map L D : V ⊗ d → V ⊗ d − 2 e Algebraic Methods for Tensor Data