Guided Signal Reconstruction with Application to Image Magnification - PowerPoint PPT Presentation

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Cambridge, Massachusetts Guided Signal Reconstruction with Application to Image Magnification Akshay Gadde (USC, MERL) Speaker: Andrew Knyazev (MERL) Hassan Mansour (MERL) Dong Tian (MERL) c � MERL December 16, 2015 1 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Outline 1. Introduction Problem Definition and Motivation Related Work 2. Reconstruction Set Geometric Interpretation Algorithm for Finding the Reconstruction Set Relation to Regularized Reconstruction 3. Experiments 4. Conclusion and Future Work c � MERL December 16, 2015 2 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Outline 1. Introduction Problem Definition and Motivation Related Work 2. Reconstruction Set Geometric Interpretation Algorithm for Finding the Reconstruction Set Relation to Regularized Reconstruction 3. Experiments 4. Conclusion and Future Work c � MERL December 16, 2015 3 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Problem Definition sampling subspace sample reconstruct • Lossy measurements • Prior information about the signal ⇒ Guiding subspace T ⊂ H � T ⊥ f � small f ∈ T or c � MERL December 16, 2015 4 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Problem Definition sampling subspace sample reconstruct • Lossy measurements • Prior information about the signal ⇒ Guiding subspace T ⊂ H � T ⊥ f � small f ∈ T or Questions Conditions on S and T for: • Uniqueness of reconstruction • Stability of reconstruction • Efficient algorithm for reconstruction • Effect of noise and model mismatch c � MERL December 16, 2015 4 / 21

τ τ MITSUBISHI ELECTRIC RESEARCH LABORATORIES Motivation • Image magnification • S : 2 × 2 averaging • T : low pass DCT c � MERL December 16, 2015 5 / 21

τ τ MITSUBISHI ELECTRIC RESEARCH LABORATORIES Motivation • Semi-supervised • Image magnification learning ? ? ? ? • S = { x | x U = 0 } • S : 2 × 2 averaging • T : low pass GFT • T : low pass DCT � K � � T = c i u i , i =1 { u i } e.v.’s of graph L c � MERL December 16, 2015 5 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Motivation • Semi-supervised • Bandwidth expansion of • Image magnification learning speech 8Khz (i) Khz ? 0Hz Khz ? (ii) Khz ? 0Hz ? Khz (iii) Khz 0Hz 50 100 150 200 250 300 350 • S = { x | x U = 0 } • S : 2 × 2 averaging • S : low pass DFT • T : low pass GFT • T : low pass DCT • T : learned from data τ � K τ � � T = c i u i , i =1 { u i } e.v.’s of graph L c � MERL December 16, 2015 5 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Related Work: Consistent Reconstruction • Consistent reconstruction ˆ f ⇔ S ˆ f = Sf (Unser and Aldroubi’94, Eldar’03) Existence and Uniqueness • Consistent reconstruction exists in T for any f ∈ H iff T + S ⊥ = H • Consistent reconstruction is unique iff T ∩ S ⊥ = { 0 } c � MERL December 16, 2015 6 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Related Work: Consistent Reconstruction • Consistent reconstruction ˆ f ⇔ S ˆ f = Sf (Unser and Aldroubi’94, Eldar’03) Existence and Uniqueness • Consistent reconstruction exists in T for any f ∈ H iff T + S ⊥ = H • Consistent reconstruction is unique iff T ∩ S ⊥ = { 0 } • Under the above assumptions ˆ f = P T ⊥S f (oblique projection) • If f ∈ T then ˆ f = f • If T ∩ S ⊥ � = { 0 } (non-unique consistent solutions), pick one by imposing additional constraints c � MERL December 16, 2015 6 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Related Work: Generalized Reconstruction • Existence of consistent reconstruction needs T + S ⊥ = H • Can lead to unstable reconstructions (if min. gap between T and S is large) • Oversampling for stability can cause T + S ⊥ ⊂ H Generalized reconstruction • Sample consistent plane Sf + S ⊥ f ∈ T closest to Sf + S ⊥ (relax Sf = S ˆ • ˆ f ) ˆ f = P T ⊥ S ( T ) f c � MERL December 16, 2015 7 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Related Work: Generalized Reconstruction • Existence of consistent reconstruction needs T + S ⊥ = H • Can lead to unstable reconstructions (if min. gap between T and S is large) • Oversampling for stability can cause T + S ⊥ ⊂ H Generalized reconstruction • Sample consistent plane Sf + S ⊥ f ∈ T closest to Sf + S ⊥ (relax Sf = S ˆ • ˆ f ) ˆ f = P T ⊥ S ( T ) f f ∈ Sf + S ⊥ (consistent) or ˆ Question: ˆ f ∈ T (generalized) or something else? c � MERL December 16, 2015 7 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Outline 1. Introduction Problem Definition and Motivation Related Work 2. Reconstruction Set Geometric Interpretation Algorithm for Finding the Reconstruction Set Relation to Regularized Reconstruction 3. Experiments 4. Conclusion and Future Work c � MERL December 16, 2015 8 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Reconstruction Set • sample consistent place Sf + S ⊥ • guiding subspace T Reconstruction set Shortest pathway between the consistent place and the guiding subspace t ∈T � ˆ � ˆ f ∈ Sf + S ⊥ � ˆ f ∈ Sf + S ⊥ min min f − t � = min f − t � = min min f − t � ˆ ˆ f ∈ Sf + S ⊥ t ∈T ˆ t ∈T • ˆ f : consistent reconstruction • t : generalized reconstruction c � MERL December 16, 2015 9 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Iterative Consistent Reconstruction Using Cojugate Gradient • Consistent reconstruction � T ⊥ f � S ˆ inf subject to f = Sf ˆ f c � MERL December 16, 2015 10 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Iterative Consistent Reconstruction Using Cojugate Gradient • Consistent reconstruction � T ⊥ f � S ˆ inf subject to f = Sf ˆ f Consistent reconstruction using CG x = (ˆ f − Sf ) ∈ S ⊥ . Then the above problem is equivalent to solving Define ˆ ( S ⊥ T ⊥ ) S ⊥ x = − S ⊥ T ⊥ Sf � ( Sf : measurement ) � • Restriction of S ⊥ T ⊥ to S ⊥ is self-adjoint • Use CG with initialization x 0 ∈ S ⊥ • CG: most efficient iterative method for solving linear systems • Frame-less algorithm: Needs only the (approximate) projector T c � MERL December 16, 2015 10 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Finding the Reconstruction Set t ∈T � ˆ � ˆ f ∈ Sf + S ⊥ � ˆ f − t � = f − t � = min f − t � f ∈ Sf + S ⊥ min min min min ˆ f ∈ Sf + S ⊥ ˆ t ∈T ˆ t ∈T • ˆ f : consistent reconstruction • t : generalized reconstruction • Relation between ˆ f and t t = T ˆ f c � MERL December 16, 2015 11 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Finding the Reconstruction Set t ∈T � ˆ � ˆ f ∈ Sf + S ⊥ � ˆ f − t � = f − t � = min f − t � f ∈ Sf + S ⊥ min min min min ˆ ˆ f ∈ Sf + S ⊥ t ∈T ˆ t ∈T • ˆ f : consistent reconstruction • t : generalized reconstruction • Relation between ˆ f and t t = T ˆ f Reconstruction Set = { α ˆ f + (1 − α ) T ˆ f , where α ∈ [0 , 1] } c � MERL December 16, 2015 11 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Connection with Regularization Reconstruction by regularization 2 2 � � � �� � � S ˆ f ρ − T ˆ ˆ inf f ρ − Sf + ρ , ρ > 0 f ρ � � � � � � � ˆ f ρ c � MERL December 16, 2015 12 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Connection with Regularization Reconstruction by regularization 2 2 � � � �� � � S ˆ f ρ − T ˆ ˆ inf f ρ − Sf + ρ , ρ > 0 f ρ � � � � � � � ˆ f ρ Theorem (Reconstruction set and Regularization) Let ˆ f be the consistent reconstruction given by S ˆ � T ⊥ f � inf f = Sf . subject to ˆ f The reconstruction set is given by { ˆ f α = α ˆ f + (1 − α ) T ˆ f , where 0 ≤ α ≤ 1 } . Then ˆ f α is a solution of the regularized reconstruction problem with ρ = (1 − α ) /α . • If a unique ˆ f ∈ T ∩ ( Sf + S ⊥ ) exists, then ˆ f ρ = ˆ f = T ˆ ∀ ρ > 0 f • No need to re-solve the regularization problem if ρ changes c � MERL December 16, 2015 12 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Reconstruction in the Presence of Noise • Noisy measurements: Sf ′ = Sf + e ⇒ Original signal f / ∈ ( Sf ′ + S ⊥ ) • Trust the guiding more than the samples f ∈ Sf ′ + S ⊥ be the consistent solution • Let ˆ ⇒ Good solution is ˆ f α = α ˆ f + (1 − α ) T ˆ f with α > 0 Good choice of α Noise energy � e � . Then pick α such that ˆ f − T ˆ � e � f ⇒ ˆ f α = ˆ 1 − α = f − � e � � ˆ f − T ˆ � ˆ f − T ˆ f � f � • Assumes that noise is orthogonal to T c � MERL December 16, 2015 13 / 21

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Outline 1. Introduction Problem Definition and Motivation Related Work 2. Reconstruction Set Geometric Interpretation Algorithm for Finding the Reconstruction Set Relation to Regularized Reconstruction 3. Experiments 4. Conclusion and Future Work c � MERL December 16, 2015 14 / 21