Optimal Rank-1 Hankel Approximation of Matrices Gerlind Plonka - PowerPoint PPT Presentation

Optimal Rank-1 Hankel Approximation of Matrices Gerlind Plonka University of Göttingen CodEx Seminar November 3, 2020 Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 1 / 31

Outline Introduction: Low Rank Hankel Approximation ◮ Known Solution Strategies ◮ Applications of Low Rank Hankel Matrices ◮ Low Rank Hankel Operators Rank-1 Hankel Matrices Optimal Rank-1 Hankel Approximation in the Frobenius Norm Optimal Rank-1 Hankel Approximation in the Spectral Norm Comparison to Cadzow’s Algorithm Collaborations Hanna Knirsch Markus Petz Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 2 / 31

Before We Start: Theorem (Eckhart-Young-Mirsky Theorem) Let A ∈ C M × N with M ≥ N and let A = U D V ∗ be the singular value decomposition (SVD) of A , where U = ( u 0 , u 1 , . . . , u M − 1 ) ∈ C M × M and V = ( v 0 , v 1 , . . . , v N − 1 ) ∈ C N × N are unitary matrices and D ∈ C M × N is a diagonal matrix with entries σ 0 ≥ σ 1 ≥ . . . ≥ σ N − 1 ≥ 0 . Then for r ≤ rank A , r − 1 � σ i u i v T A r = i i =0 is the optimal rank-r approximation of A , and we have N − 1 � A − A r � 2 � σ 2 � A − A r � 2 = σ r , F = i . i = r Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 3 / 31

Optimal low-rank Hankel approximation Let A ∈ C M × N . Hankel matrices are of the form   h 0 h 1 h 2 · · · h N − 1 h 1 h 2 h N     h 2 H := ( h ℓ + k ) M − 1 , N − 1 ∈ C M × N . =   k ,ℓ =0 . .   . .   . .   h M − 1 h M h M +1 · · · h M + N − 2 We want to solve H r ∈ C M × N � A − H r � 2 H r ∈ C M × N � A − H r � 2 min or min 2 , F under the restriction that H r is a Hankel matrix of rank r . Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 4 / 31

Low-rank Hankel Approximation: Solution Strategies 1 Heuristic approaches: Cadzow Algorithm Cadzow ’88, Chu et al. ’03, . . . 2 Consider the problem as a nonlinear structured least squares problem (NSLSP) de Moor ’93, Lemmerling et al. ’00, ’01, Gillard et al. ’11, Ishteva et al. ’14, Markovsky ’05, ’08, ’18, Usevich et al ’14, . . . 3 Consider the problem as a nonlinear eigenvalue problem Bresler et al. ’86, Osborne et al. 95, Zhang et al. ’19 4 Relaxation of the optimization problem Fazel et al. ’13, Grussler et al. ’18, Andersson et al. ’19 5 Application of subspace methods Van Overschee et al. ’96, Liu et al. ’09 6 Application of AAK-Theory from complex analysis, transfer from infinite Hankel matrices to finite Hankel matrices Beylkin et al. ’05, ’10, Andersson et al. ’11, Plonka & Pototskaia ’16, ’19 7 Algebraic methods Ottaviani ’14 Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 5 / 31

Low Rank Hankel Approximation Non-convex optimization problem: For a given matrix A ∈ C M × N and r < min { M , N } , find H r := argmin � A − H � , H Hankel rank H ≤ r Numerous applications: linear system theory (minimal partial realizations) system identification problems approximation with finite rate of innovation signals Prony’s method for parameter estimation and approximation Singular Spectrum Analysis (SSA) Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 6 / 31

Application 1: Parameter identification M c j e α j x Assume, we have the signal structure f ( x ) = � j =1 We have f ( ℓ ), ℓ = 0 , . . . , L , L ≥ 2 M − 1. M , c j ∈ C \ { 0 } , e α j ∈ C , j = 1 , . . . , M . We want Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 7 / 31

Application 1: Parameter identification M c j e α j x Assume, we have the signal structure f ( x ) = � j =1 We have f ( ℓ ), ℓ = 0 , . . . , L , L ≥ 2 M − 1. M , c j ∈ C \ { 0 } , e α j ∈ C , j = 1 , . . . , M . We want Consider the Prony polynomial M M ( z − e α j ) = p ℓ z ℓ � � P ( z ) := j =1 ℓ =0 with unknown parameters α j and p M = 1. M M M M c j e α j m M c j e α j ( ℓ + m ) = � p ℓ e α j ℓ � � � � p ℓ f ( ℓ + m ) = p ℓ ℓ =0 j =1 j =1 ℓ =0 ℓ =0 M c j e α j m P ( e α j ) = 0 , = � m = 0 , . . . , L − M . j =1 Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 7 / 31

Reconstruction Algorithm Input : f ( ℓ ), ℓ = 0 , . . . , L Solve the Hankel system       f (0) f (1) . . . f ( M ) p 0 0 f (1) f (2) . . . f ( M + 1) p 1 0             = . . . . .  . . .   .   .  . . . . .             f ( L − M ) f ( M ) . . . f ( L ) p M 0 Compute the zeros z j = e α j , j = 1 , . . . , M of the Prony polynomial P ( z ) = � M ℓ =0 p ℓ z ℓ . Compute c j solving the linear system M c j e α j ℓ , � f ( ℓ ) = ℓ = 0 , . . . , L . j =1 Output : Parameters α j and c j , j = 1 , . . . , M . Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 8 / 31

Application 2: Low rank Hankel matrices for regularization Parallel MRI 1 Calibrationless parallel imaging reconstruction based on structured low-rank matrix completion (SAKE) (2013) Shin et al. 2 ESPIRiT- An eigenvalue approach to autocalibrating parallel MRI: Where SENSE meets GRAPPA (2014) Uecker et al. Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 9 / 31

Application 2: Low rank Hankel matrices for regularization Parallel MRI 1 Calibrationless parallel imaging reconstruction based on structured low-rank matrix completion (SAKE) (2013) Shin et al. 2 ESPIRiT- An eigenvalue approach to autocalibrating parallel MRI: Where SENSE meets GRAPPA (2014) Uecker et al. Signal denoising 1 Sparse and low-rank decomposition of a Hankel structured matrix for impulse noise removal (ALOHA) (2017) Jin et al. 2 Grid-free localization algorithm using low-rank Hankel matrix for super-resolution microscopy (2018) Min et al. 3 Low-rank seismic denoising with optimal rank selection for Hankel matrices (2019) Wang et al. Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 9 / 31

Is there hope to solve the problem exactly? j =0 ∈ ℓ 2 be a complex sequence and Let ( f j ) ∞   f 0 f 1 f 2 . . . f 1 f 2 f 3 . . .     Γ f := f 2 f 3 f 4 . . .     . . . ...   . . . . . . Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 10 / 31

Is there hope to solve the problem exactly? j =0 ∈ ℓ 2 be a complex sequence and Let ( f j ) ∞   f 0 f 1 f 2 . . . f 1 f 2 f 3 . . .     Γ f := f 2 f 3 f 4 . . .     . . . ...   . . . . . . Nehari : Γ f is bounded on ℓ 2 , if there exists a periodic function ψ ∈ L ∞ ([0 , 2 π )) with Fourier coefficients c j ( ψ ) = f j , j ≥ 0. Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 10 / 31

Is there hope to solve the problem exactly? j =0 ∈ ℓ 2 be a complex sequence and Let ( f j ) ∞   f 0 f 1 f 2 . . . f 1 f 2 f 3 . . .     Γ f := f 2 f 3 f 4 . . .     . . . ...   . . . . . . Nehari : Γ f is bounded on ℓ 2 , if there exists a periodic function ψ ∈ L ∞ ([0 , 2 π )) with Fourier coefficients c j ( ψ ) = f j , j ≥ 0. Peller : Γ f has finite rank N if and only if ∞ � f j z j f ( z ) := j =0 defines a rational function of type ( N − 1 , N ). For example, N N ∞ ∞ c k f j z j = c k z j � � � � k z j (1 − z k z ) = j =0 k =1 j =0 k =1 Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 10 / 31

Is there hope to solve the problem exactly? Aram, Arov and Krein: Let s 0 (Γ f ) ≥ s 1 (Γ f ) ≥ ... denote the decreasing sequence of singular values of the bounded Γ f . Then there exists a Hankel operator Γ N of rank at most N such that � Γ f − Γ N � ℓ 2 → ℓ 2 = s N (Γ f ) . Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 11 / 31

Is there hope to solve the problem exactly? Aram, Arov and Krein: Let s 0 (Γ f ) ≥ s 1 (Γ f ) ≥ ... denote the decreasing sequence of singular values of the bounded Γ f . Then there exists a Hankel operator Γ N of rank at most N such that � Γ f − Γ N � ℓ 2 → ℓ 2 = s N (Γ f ) . If Γ f , M has already finite rank M > N , then we have an algorithm to get the optimal low-rank matrix Γ N satisfying � Γ f , M − Γ N � ℓ 2 → ℓ 2 = s N (Γ f , M ) . Beylkin & Monzon (2005) , Plonka & Pototskaia (2016,19) . Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 11 / 31

Optimal Rank-1 Hankel Approximation We want to solve H 1 ∈ C M × N � A − H 1 � 2 H 1 ∈ C M × N � A − H 1 � 2 min or min 2 , F under the restriction that H 1 is a Hankel matrix of rank 1. Gerlind Plonka (University of Göttingen) Rank-1 Hankel Approximation CodEx Seminar 2020 12 / 31