Global solution to non-convex optimization problems involving an - PowerPoint PPT Presentation

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Arthur Marmin In collaboration with: Jean-Christophe Pesquet and Marc Castella Center for Visual Computing, CentraleSup´ elec, INRIA, Universit´ e Paris-Saclay 9 th October 2020 Arthur Marmin GdR MIA 09/10/2020 1 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Motivation Need for accurate data acquisition of sparse data Arthur Marmin GdR MIA 09/10/2020 2 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Motivation Need for accurate data acquisition of sparse data Example in chromatography (joint work with IFPEN) Mixture of chemical components Find the components (peaks location) Find their concentrations (peaks amplitude) Arthur Marmin GdR MIA 09/10/2020 2 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Motivation Need for accurate data acquisition of sparse data Example in chromatography (joint work with IFPEN) Mixture of chemical components Find the components (peaks location) Find their concentrations (peaks amplitude) Sensor degradation Peak enlargement Nonlinear distortion (e.g. saturation) Noise Only sub-sampled signal available (e.g. performing long acquisition in a limited time) Arthur Marmin GdR MIA 09/10/2020 2 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Application to Chromatography 0.8 Original signal Altered signal Noisy ouput 0.6 0.4 0.2 0 -0.2 5 10 15 20 25 30 35 40 45 50 Arthur Marmin GdR MIA 09/10/2020 3 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Main Issues Common approach: Minimize a criterion Criterion = fit to observations + sparsity promoting regularizer Arthur Marmin GdR MIA 09/10/2020 4 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Main Issues Common approach: Minimize a criterion Criterion = fit to observations + sparsity promoting regularizer Issues: How to model nonlinearity? Arthur Marmin GdR MIA 09/10/2020 4 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Main Issues Common approach: Minimize a criterion Criterion = fit to observations + sparsity promoting regularizer Issues: How to model nonlinearity? − → Linearize the model − → Limited solution Arthur Marmin GdR MIA 09/10/2020 4 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Main Issues Common approach: Minimize a criterion Criterion = fit to observations + sparsity promoting regularizer Issues: How to model nonlinearity? − → Linearize the model − → Limited solution How to promote sparsity efficiently? (How to approximate ℓ 0 ?) Arthur Marmin GdR MIA 09/10/2020 4 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Main Issues Common approach: Minimize a criterion Criterion = fit to observations + sparsity promoting regularizer Issues: How to model nonlinearity? − → Linearize the model − → Limited solution How to promote sparsity efficiently? (How to approximate ℓ 0 ?) − → Use a convex approximation (like ℓ 1 ) − → Bias in solution − → Use nonconvex approximations − → Settle for suboptimal solutions (local minimizers) Arthur Marmin GdR MIA 09/10/2020 4 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Summary 1 Model and criterion 2 Rational formulation of the problem 3 Solving the rational optimization problem 4 Complexity of the relaxation Arthur Marmin GdR MIA 09/10/2020 5 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Modelling Decimation and Nonlinear Degradation Goal: Retrieve x from y Observation model y = x y = observed signal of size U x = initial sparse discrete signal of size T Arthur Marmin GdR MIA 09/10/2020 6 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Modelling Decimation and Nonlinear Degradation Goal: Retrieve x from y Observation model y = h ∗ x y = observed signal of size U x = initial sparse discrete signal of size T h = impulse response of convolution filter of length L Arthur Marmin GdR MIA 09/10/2020 6 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Modelling Decimation and Nonlinear Degradation Goal: Retrieve x from y Observation model y = Φ(h ∗ x) y = observed signal of size U x = initial sparse discrete signal of size T h = impulse response of convolution filter of length L Φ = nonlinear function (e.g. saturation) Arthur Marmin GdR MIA 09/10/2020 6 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Modelling Decimation and Nonlinear Degradation Goal: Retrieve x from y Observation model y = Φ(h ∗ x) + w y = observed signal of size U x = initial sparse discrete signal of size T h = impulse response of convolution filter of length L Φ = nonlinear function (e.g. saturation) w = white noise Arthur Marmin GdR MIA 09/10/2020 6 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Modelling Decimation and Nonlinear Degradation Goal: Retrieve x from y Observation model � � y = D Φ(h ∗ x) + w y = observed signal of size U x = initial sparse discrete signal of size T h = impulse response of convolution filter of length L Φ = nonlinear function (e.g. saturation) w = white noise D = decimation Arthur Marmin GdR MIA 09/10/2020 6 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Examples of Decimation Decimation D α − → Delete elements indexed by multiples of α � � D α ( v t ) 1 ≤ t ≤ T = ( v ∆( u ,α ) ) 1 ≤ u ≤ U ∆( u , α ) = Decimation index D ∞ = Identity operator Example:   v 1 v 2   D 2  =?   v 3  v 4 Arthur Marmin GdR MIA 09/10/2020 7 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Examples of Decimation Decimation D α − → Delete elements indexed by multiples of α � � D α ( v t ) 1 ≤ t ≤ T = ( v ∆( u ,α ) ) 1 ≤ u ≤ U ∆( u , α ) = Decimation index D ∞ = Identity operator Example:     v 1 v 1 v 2 v 2    ✚  ✚ D 2  =     v 3 v 3    v 4 v 4 ✚ ✚ Arthur Marmin GdR MIA 09/10/2020 7 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Examples of Decimation Decimation D α − → Delete elements indexed by multiples of α � � D α ( v t ) 1 ≤ t ≤ T = ( v ∆( u ,α ) ) 1 ≤ u ≤ U ∆( u , α ) = Decimation index D ∞ = Identity operator Example:   v 1 � v ∆(1 , 2) � v 2   D 2  =   v 3 v ∆(2 , 2)  v 4 Arthur Marmin GdR MIA 09/10/2020 7 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Criterion for Signal Reconstruction Criterion to minimize J ∗ = min J (x) x ∈ R T J (x) = f y (x) + R λ (x) � �� � ���� penalization data fidelity f y (x) = � y − D (Φ(h ∗ x)) � 2 2 = � y − D (Φ(Hx)) � 2 2 Arthur Marmin GdR MIA 09/10/2020 8 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Criterion for Signal Reconstruction Criterion to minimize J ∗ = min J (x) x ∈ R T J (x) = f y (x) + R λ (x) � �� � ���� penalization data fidelity f y (x) = � y − D (Φ(h ∗ x)) � 2 2 = � y − D (Φ(Hx)) � 2 2 t Φ rational element-wise − → Φ : t �→ 0 . 3+ | t | 1 0.5 (x) 0 -0.5 -1 -1 -0.5 0 0.5 1 x Saturation Function Φ Arthur Marmin GdR MIA 09/10/2020 8 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Criterion for Signal Reconstruction Criterion to minimize J ∗ = min J (x) x ∈ R T J (x) = f y (x) + R λ (x) � �� � ���� penalization data fidelity f y (x) = � y − D (Φ(h ∗ x)) � 2 2 = � y − D (Φ(Hx)) � 2 2 t Φ rational element-wise − → Φ : t �→ 0 . 3+ | t | � L �� 2 � U � � f y (x) = y u − Φ h l x ∆( u ,α ) − l +1 u =1 l =1 � �� � = g u ( x ∆( u ,α ) − L +1 ,..., x ∆( u ,α ) ) → rational functions Arthur Marmin GdR MIA 09/10/2020 8 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Choice of Sparsity Promoting Regularizer Ideal regularizer R λ (x) = λℓ 0 (x) . Lead to intricate optimization problem − → use surrogates Arthur Marmin GdR MIA 09/10/2020 9 / 22

Global solution to non-convex optimization problems involving an approximate ℓ 0 penalization Choice of Sparsity Promoting Regularizer Ideal regularizer R λ (x) = λℓ 0 (x) . Lead to intricate optimization problem − → use surrogates T � Separable approximation: R λ (x) = Ψ λ ( x t ) t =1 Arthur Marmin GdR MIA 09/10/2020 9 / 22