Convex regularization of discrete-valued inverse problems Christian - PowerPoint PPT Presentation

Convex regularization of discrete-valued inverse problems Christian Clason Faculty of Mathematics, Universität Duisburg-Essen joint work with Thi Bich Tram Do, Florian Kruse, Karl Kunisch New Trends in Parameter Identification for Mathematical Models IMPA, Rio de Janeiro, October 31, 2017 Overview Approach Multi-bang regularization Nonlinear problems 1 / 32

Motivation: hybrid discrete optimization u ∈ U F ( u ) + α 2 � u � 2 min F discrepancy term (involving PDEs) U discrete set � � u ∈ L p (Ω) : u ( x ) ∈ { u 1 , . . . , u d } a.e. U = u 1 , . . . , u d given voltages, velocities, materials, ... (assumed here: ranking by magnitude possible!) motivation: topology optimization, medical imaging Overview Approach Multi-bang regularization Nonlinear problems 2 / 32

Motivation: penalty convex relaxation: replace U by convex hull u ( x ) ∈ [ u 1 , u d ] works only for d = 2, cf. bang-bang control ( α = 0) � promote u ( x ) ∈ { u 1 , . . . , u d } by convex pointwise penalty � G ( u ) = g ( u ( x )) dx Ω generalize L 1 norm: polyhedral epigraph with vertices u 1 , . . . , u d not exact relaxation/penalization (in general)! Overview Approach Multi-bang regularization Nonlinear problems 3 / 32

Motivation: penalty generalize L 1 norm: polyhedral epigraph with vertices u 1 , . . . , u d 3 motivation: convex envelope 2 � u � 2 + δ U of 1 2 multi-bang (generalized bang-bang) control 1 � non-smooth optimization 0 in function spaces v u 1 u 2 u 3 Overview Approach Multi-bang regularization Nonlinear problems 3 / 32

Overview 1 Approach 2 Convex analysis Moreau–Yosida regularization Semismooth Newton method Multi-bang penalty 3 Multi-bang regularization Regularization properties Structure and numerical solution Nonlinear problems 4 Overview Approach Multi-bang regularization Nonlinear problems 4 / 32

Fenchel duality V Banach space, V ∗ dual space F : V → R := R ∪ {+ ∞ } convex, subdifferential � � v ∗ ∈ V ∗ : � v ∗ , v – ¯ ∂ F (¯ v ) = v � V ∗ , V � F ( v ) – F (¯ v ) for all v ∈ V Fenchel conjugate (always convex) F ∗ : V ∗ → R , F ∗ ( v ∗ ) = sup � v ∗ , v � V ∗ , V – F ( v ) v ∈ V “convex inverse function theorem”: v ∗ ∈ ∂ F ( v ) v ∈ ∂ F ∗ ( v ∗ ) ⇔ Overview Approach Multi-bang regularization Nonlinear problems 5 / 32

Fenchel duality: application F (¯ u ) + G (¯ u ) = min u F ( u ) + G ( u ) 1 Fermat principle: 0 ∈ ∂ ( F (¯ u ) + G (¯ u )) p ∈ V ∗ with 2 sum rule: 0 ∈ ∂ F (¯ u ) + ∂ G (¯ u ), i.e., there is ¯ � –¯ p ∈ ∂ F (¯ u ) p ∈ ∂ G (¯ ¯ u ) 3 Fenchel duality: � –¯ p ∈ ∂ F (¯ u ) u ∈ ∂ G ∗ (¯ ¯ p ) Overview Approach Multi-bang regularization Nonlinear problems 6 / 32

Regularization G non-smooth � subdifferential ∂ G ∗ set-valued � regularize u , p ∈ L 2 (Ω) Hilbert space � consider for γ > 0 Proximal mapping w G ∗ ( w ) + 1 2 γ � w – p � 2 prox γ G ∗ ( p ) = arg min single-valued, Lipschitz continuous coincides with resolvent (Id + γ ∂ G ∗ ) –1 ( p ) (also required for primal-dual first-order methods) Overview Approach Multi-bang regularization Nonlinear problems 7 / 32

Regularization Proximal mapping w G ∗ ( w ) + 1 2 γ � w – p � 2 prox γ G ∗ ( p ) = arg min Complementarity formulation of u ∈ ∂ G ∗ ( p ) u = 1 � � ( p + γu ) – prox γ G ∗ ( p + γu ) γ equivalent for every γ > 0 single-valued, Lipschitz continuous, implicit Overview Approach Multi-bang regularization Nonlinear problems 7 / 32

Regularization Proximal mapping w G ∗ ( w ) + 1 2 γ � w – p � 2 prox γ G ∗ ( p ) = arg min Moreau–Yosida regularization of u ∈ ∂ G ∗ ( p ) u = 1 =: ∂ G ∗ � � p – prox γ G ∗ ( p ) γ ( p ) γ 2 � · � 2 � ∗ → ∂ G ∗ as γ → 0 (no smoothing of G !) G + γ ∂ G ∗ � γ = ∂ single-valued, Lipschitz continuous, explicit � nonsmooth operator equation, Newton method Overview Approach Multi-bang regularization Nonlinear problems 7 / 32

Semismooth Newton method f locally Lipschitz, piecewise C 1 : f : R n → R f ( v ) = 0, Newton derivative D N f ( v ) δv ∈ ∂ C f ( v ) δv Clarke generalized gradient: convex hull of piecewise derivatives semismooth Newton method v k +1 = v k + δv D N f ( v k ) δv = – f ( v k ), converges locally superlinearly Overview Approach Multi-bang regularization Nonlinear problems 8 / 32

Semismooth Newton method f locally Lipschitz, piecewise C 1 : F : L r (Ω) → L s (Ω), F ( u ) = 0, [ F ( u )]( x ) = f ( u ( x )) Newton derivative [ D N F ( u ) δu ]( x ) ∈ ∂ C f ( δu ( x )) δu ( x ) any measurable selection of Clarke generalized gradient semismooth Newton method u k +1 = u k + δu D N F ( u k ) δu = – F ( u k ), converges locally superlinearly if r > s Overview Approach Multi-bang regularization Nonlinear problems 8 / 32

Numerical solution: summary � For (non)convex G : L 2 (Ω) → R , G ( u ) = Ω g ( u ( x )) dx , Approach: pointwise 1 compute subdifferential ∂ g (or Fenchel conjugate g ∗ ) 2 compute subdifferential ∂ g ∗ 3 compute proximal mapping prox γ g ∗ 4 compute Moreau–Yosida regularization ∂ g ∗ γ 5 compute Newton derivative D N ∂ g ∗ γ � semismooth Newton method, continuation in γ for [ ∂ G ∗ γ ( p )]( x ) = ∂ g ∗ superposition operator γ ( p ( x )) Overview Approach Multi-bang regularization Nonlinear problems 9 / 32

Multi-bang penalty � 1 2 (( u i + u i +1 ) v – u i u i +1 ) v ∈ [ u i , u i +1 ] g : R → R , v �→ else ∞ piecewise differentiable � subdifferential convex hull of derivatives  – ∞ , 1 � � 2 ( u 1 + u 2 ) v = u 1    � 1 �   2 ( u i + u i +1 ) v ∈ ( u i , u i +1 ) 1 � i < d ∂ g ( v ) = � 1 2 ( u i –1 + u i ), 1 � 2 ( u i + u i +1 ) v = u i 1 < i < d    � 1   � 2 ( u d –1 + u d ), ∞ v = u d Overview Approach Multi-bang regularization Nonlinear problems 10 / 32

Multi-bang penalty  – ∞ , 1 � � 2 ( u 1 + u 2 ) v = u 1    � 1 �   2 ( u i + u i +1 ) v ∈ ( u i , u i +1 ) 1 � i < d ∂ g ( v ) = � 1 2 ( u i –1 + u i ), 1 � 2 ( u i + u i +1 ) v = u i 1 < i < d    � 1   � 2 ( u d –1 + u d ), ∞ v = u d convex inverse function theorem:  – ∞ , 1 � � { u 1 } q ∈ 2 ( u 1 + u 2 )      q = 1 [ u i , u i +1 ] 2 ( u i + u i +1 ), 1 � i < d ∂ g ∗ ( q ) ∈ � 1 2 ( u i –1 + u i ), 1 � { u i } q ∈ 2 ( u i + u i +1 ) 1 < i < d ,    � 1   � { u d } q ∈ 2 ( u d –1 + u d ), ∞ Overview Approach Multi-bang regularization Nonlinear problems 10 / 32

Multi-bang penalty: sketch 3 3 2 2 1 1 0 0 v v u 1 u 2 u 3 u 1 u 2 u 3 (a) g ( u 1 = 0, u 2 = 1, u 3 = 2) (b) ∂ g ( u 1 = 0, u 2 = 1, u 3 = 2) Overview Approach Multi-bang regularization Nonlinear problems 11 / 32

Multi-bang penalty: sketch 3 2 2 1 1 0 0 v q u 1 u 2 u 3 0.5 1.5 (d) ∂ g ∗ ( u 1 = 0, u 2 = 1, u 3 = 2) (c) ∂ g ( u 1 = 0, u 2 = 1, u 3 = 2) Overview Approach Multi-bang regularization Nonlinear problems 11 / 32

Moreau–Yosida regularization prox γ g ∗ ( q ) = w iff q ∈ { w } + γ ∂ g ∗ ( w ) Proximal mapping case-wise inspection of subdifferential: � q ∈ Q γ u i γ ( q ) = 1 ∂ g ∗ i � � q – prox γ g ∗ ( q ) = 1 q – 1 q ∈ Q γ � � γ 2 ( u i + u i +1 ) i , i +1 γ � � Q γ 2 ( u i –1 + u i ) + γ u i , 1 1 i = 2 ( u i + u i +1 ) + γ u i � � Q γ 2 ( u i + u i +1 ) + γ u i , 1 1 i , i +1 = 2 ( u i + u i +1 ) + γ u i +1 Overview Approach Multi-bang regularization Nonlinear problems 12 / 32

Overview 1 Approach 2 Convex analysis Moreau–Yosida regularization Semismooth Newton method Multi-bang penalty 3 Multi-bang regularization Regularization properties Structure and numerical solution Nonlinear problems 4 Overview Approach Multi-bang regularization Nonlinear problems 13 / 32

Multi-bang regularization 1 2 � Ku – y δ � 2 min Y + α G ( u ) u ∈ L 2 (Ω) K : L 2 (Ω) → Y (linear) forward mapping, weakly closed y δ ∈ L 2 (Ω) noisy data with � y – y δ � Y � δ u 1 < · · · < u d given parameter values ( d > 2) G multi-bang penalty Overview Approach Multi-bang regularization Nonlinear problems 14 / 32

Multi-bang regularization 1 2 � Ku – y δ � 2 min Y + α G ( u ) u ∈ L 2 (Ω) G multi-bang penalty convex: existence of solution u δ α for every α > 0 1 δ → 0 implies u δ α ⇀ u α for every α > 0 2 δ → 0, α → 0, δα –2 → 0 implies u δ α ⇀ u † 3 (standard arguments, e.g. [Burger/Osher 04, Ito/Jin 14] ) Overview Approach Multi-bang regularization Nonlinear problems 14 / 32

Multi-bang regularization 1 2 � Ku – y δ � 2 min Y + α G ( u ) u ∈ L 2 (Ω) p † := K ∗ w ∈ ∂ G ( u † ) for w ∈ Y , standard source condition: a priori choice α ( δ ) = cδ 1 � Ku δ α ( δ ) – y δ � Y � τδ , τ > 1 a posteriori choice 2 � convergence rate d p † G ( u δ α , u † ) � C δ in Bregman distance d p 1 G ( u 2 , u 1 ) = G ( u 2 ) – G ( u 1 ) – � p 1 , u 2 – u 1 � X , p 1 ∈ ∂ G ( u 1 ) Overview Approach Multi-bang regularization Nonlinear problems 14 / 32