Evolution equations and Tikhonov regularization Juan PEYPOUQUET - PowerPoint PPT Presentation

Tikhonov regularization Evolution equations and Tikhonov regularization Juan PEYPOUQUET Universidad T´ ecnica Federico Santa Mar´ ıa joint work with R. Cominetti & S. Sorin J OURN ´ EES FRANCO - CHILIENNES D ’ OPTIMISATION Toulon, 20 mai 2008

Tikhonov regularization Outline 1 Generalized gradient method and Tikhonov regularization 2 Coupling evolution and regularization in convex minimization: strong and weak convergence 3 The case of differential inclusions governed by maximal monotone operators

Tikhonov regularization Generalized gradient method Let f be a proper lower-semicontinuous convex function on a Hilbert space H and let S = Argmin f . Functions u satisfying u ( t ) ∈ ∂ f ( u ( t )) − ˙ are minimizing. Moreover, if S � = ∅ they converge weakly as t → ∞ to some x ∞ ∈ S .

Tikhonov regularization Generalized gradient method Let f be a proper lower-semicontinuous convex function on a Hilbert space H and let S = Argmin f . Functions u satisfying u ( t ) ∈ ∂ f ( u ( t )) − ˙ are minimizing. Moreover, if S � = ∅ they converge weakly as t → ∞ to some x ∞ ∈ S .

Tikhonov regularization Tikhonov regularization For ε > 0 consider the strongly convex function f ε ( x ) = f ( x ) + ε 2 � x � 2 . The solutions of u ( t ) ∈ ∂ f ε ( u ( t )) − ˙ converge strongly as t → ∞ to x ε , the unique minimizer of f ε . If S � = ∅ then x ε converges strongly to the least-norm element x ∗ of S .

Tikhonov regularization Tikhonov regularization For ε > 0 consider the strongly convex function f ε ( x ) = f ( x ) + ε 2 � x � 2 . The solutions of u ( t ) ∈ ∂ f ε ( u ( t )) − ˙ converge strongly as t → ∞ to x ε , the unique minimizer of f ε . If S � = ∅ then x ε converges strongly to the least-norm element x ∗ of S .

Tikhonov regularization Tikhonov regularization For ε > 0 consider the strongly convex function f ε ( x ) = f ( x ) + ε 2 � x � 2 . The solutions of u ( t ) ∈ ∂ f ε ( u ( t )) − ˙ converge strongly as t → ∞ to x ε , the unique minimizer of f ε . If S � = ∅ then x ε converges strongly to the least-norm element x ∗ of S .

Tikhonov regularization Coupling Tikhonov regularization and the gradient method

Tikhonov regularization Coupling t →∞ ε ( t ) → 0 Let ε be a positive function on [ 0 , ∞ ) such that lim and let u : [ 0 , ∞ ) → H satisfy u ( t ) ∈ ∂ f ε ( t ) ( u ( t )) = ∂ f ( u ( t )) + ε ( t ) u ( t ) . − ˙ Theorem (Cominetti, P . & Sorin) � ∞ (i) If 0 ε ( t ) dt = ∞ then u ( t ) → x ∗ . � ∞ (ii) If 0 ε ( t ) dt < ∞ then u ( t ) ⇀ x ∞ for some x ∞ ∈ S.

Tikhonov regularization Coupling t →∞ ε ( t ) → 0 Let ε be a positive function on [ 0 , ∞ ) such that lim and let u : [ 0 , ∞ ) → H satisfy u ( t ) ∈ ∂ f ε ( t ) ( u ( t )) = ∂ f ( u ( t )) + ε ( t ) u ( t ) . − ˙ Theorem (Cominetti, P . & Sorin) � ∞ (i) If 0 ε ( t ) dt = ∞ then u ( t ) → x ∗ . � ∞ (ii) If 0 ε ( t ) dt < ∞ then u ( t ) ⇀ x ∞ for some x ∞ ∈ S.

Tikhonov regularization Proof (i) Let θ ( t ) = 1 2 | u ( t ) − x ∗ | 2 . One easily proves | x ∗ | 2 − | x ε ( t ) | 2 � θ ( t ) + ε ( t ) θ ( t ) ≤ 1 2 ε ( t ) � ˙ . A Gronwall-like inequality then gives 1 � | x ∗ | 2 − | x ε ( t ) | 2 � θ ( t ) ≤ lim sup 0 ≤ lim sup = 0 . 2 t →∞ t →∞ (ii) Will be done later. �

Tikhonov regularization Previous attempts Under additional assumptions on ε and x ε : • Attouch-Cominetti 1996. • Baillon-Cominetti 2001. (i) • Cabot 2004. • Reich 1976. • Cominetti-Alemany 1999. • Cabot 2004. (ii) • Furuya-Miyashiba-Kenmochi 1986. • Alvarez-P . 2007

Tikhonov regularization Tikhonov regularization and differential inclusions governed by monotone operators

Tikhonov regularization Monotone operators A (possibly multi-valued) map A : H → 2 H is monotone if � x ∗ − y ∗ , x − y � ≥ 0 y ∈ Ax , y ∗ ∈ Ax ∗ for all and maximal if its graph is not properly contained in the graph of any other monotone operator.

Tikhonov regularization Examples Examples: A = ∂ f ; with f closed, proper and convex. A = I − F ; with F nonexpansive. The solution set S = A − 1 0 coincides with the minimizers of f A = ∂ f if the fixed points of F A = I − F if

Tikhonov regularization Examples Examples: A = ∂ f ; with f closed, proper and convex. A = I − F ; with F nonexpansive. The solution set S = A − 1 0 coincides with the minimizers of f A = ∂ f if the fixed points of F A = I − F if

Tikhonov regularization Examples Examples: A = ∂ f ; with f closed, proper and convex. A = I − F ; with F nonexpansive. The solution set S = A − 1 0 coincides with the minimizers of f A = ∂ f if the fixed points of F A = I − F if

Tikhonov regularization Examples Examples: A = ∂ f ; with f closed, proper and convex. A = I − F ; with F nonexpansive. The solution set S = A − 1 0 coincides with the minimizers of f A = ∂ f if the fixed points of F A = I − F if

Tikhonov regularization u ∈ Au Asymptotic behavior of − ˙ In general, S � = ∅ does not imply that functions satisfying u ( t ) ∈ Au ( t ) converge (although it does imply they converge − ˙ in average ). Counterexample For A ( x , y ) = ( − y , x ) one gets u ( t ) = r 0 cos ( 2 t 0 − t ) , sin ( 2 t 0 − t ) � � , which does not converge unless r 0 = 0 .

Tikhonov regularization u ∈ Au Asymptotic behavior of − ˙ In general, S � = ∅ does not imply that functions satisfying u ( t ) ∈ Au ( t ) converge (although it does imply they converge − ˙ in average ). Counterexample For A ( x , y ) = ( − y , x ) one gets u ( t ) = r 0 cos ( 2 t 0 − t ) , sin ( 2 t 0 − t ) � � , which does not converge unless r 0 = 0 .

Tikhonov regularization Tikhonov regularization In what follows we assume ε is a positive function on [ 0 , ∞ ) t →∞ ε ( t ) → 0 and study the behavior as t → ∞ of such that lim functions u : [ 0 , ∞ ) → H satisfying u ( t ) ∈ Au ( t ) + ε ( t ) u ( t ) − ˙ according to whether the function ε is in L 1 or not.

Tikhonov regularization ε ∈ L 1 Theorem A perturbation ε ∈ L 1 ( 0 , ∞ ; R ) makes no difference in the asymptotic behavior of the system. There is no loss − and no gain! − in applying the Tikhonov regularization in this case.

Tikhonov regularization Proof I Let A ( · ) be a family of maximal monotone operators and let ε ∈ L 1 ( 0 , ∞ ; R ) . Lemma (Alvarez & P .) If every function u : [ 0 , ∞ ) → H satisfying u ( t ) ∈ A ( t ) u ( t ) − ˙ converges strongly (weakly) as t → ∞ , so does every function v : [ 0 , ∞ ) → H satisfying v ( t ) ∈ A ( t ) v ( t ) + ε ( t ) v ( t ) . − ˙

Tikhonov regularization Proof I Let A ( · ) be a family of maximal monotone operators and let ε ∈ L 1 ( 0 , ∞ ; R ) . Lemma (Alvarez & P .) If every function u : [ 0 , ∞ ) → H satisfying u ( t ) ∈ A ( t ) u ( t ) − ˙ converges strongly (weakly) as t → ∞ , so does every function v : [ 0 , ∞ ) → H satisfying v ( t ) ∈ A ( t ) v ( t ) + ε ( t ) v ( t ) . − ˙

Tikhonov regularization Proof II Let U ( t , s ) x = u ( t ) , where � − ˙ u ( t ) Au ( t ) ∈ u ( s ) x = and let y satisfy − y ( t ) ∈ Ay ( t ) + ε ( t ) y ( t ) . First on proves � � � y ( t + h ) − U ( t + h , t ) y ( t ) � lim sup = 0 . t →∞ h ≥ 0

Tikhonov regularization Proof II Let U ( t , s ) x = u ( t ) , where � − ˙ u ( t ) Au ( t ) ∈ u ( s ) x = and let y satisfy − y ( t ) ∈ Ay ( t ) + ε ( t ) y ( t ) . First on proves � � � y ( t + h ) − U ( t + h , t ) y ( t ) � lim sup = 0 . t →∞ h ≥ 0

Tikhonov regularization Proof III Next, � � h →∞ U ( t + h , t ) y ( t ) lim τ − lim = ζ. t →∞ Finally one writes y ( t + h ) − ζ = [ y ( t + h ) − U ( t + h , t ) y ( t )] + [ U ( t + h , t ) y ( t ) − ζ ] and concludes that t →∞ y ( t ) = ζ . τ − lim �

Tikhonov regularization Proof III Next, � � h →∞ U ( t + h , t ) y ( t ) lim τ − lim = ζ. t →∞ Finally one writes y ( t + h ) − ζ = [ y ( t + h ) − U ( t + h , t ) y ( t )] + [ U ( t + h , t ) y ( t ) − ζ ] and concludes that t →∞ y ( t ) = ζ . τ − lim �

Tikhonov regularization ∈ L 1 and bounded total variation ε / ∈ L 1 . Assume S � = ∅ and ε / Theorem (Cominetti, P . & Sorin) � ∞ If ε ( t ) | dt < ∞ then any function u : [ 0 , ∞ ) → H satisfying 0 | ˙ u ( t ) ∈ Au ( t ) + ε ( t ) u ( t ) − ˙ converges strongly as t → ∞ to the least-norm element of S .

Tikhonov regularization Idea of the proof One first proves that u ( t ) = 0. 1 t →∞ ˙ lim Next we verify that, as a consequence, all weak cluster points of u ( t ) for t → ∞ belong to S . Finally, the latter implies u ( t ) → x ∗ strongly. � 1 Actually we prove something weaker

Tikhonov regularization Idea of the proof One first proves that u ( t ) = 0. 1 t →∞ ˙ lim Next we verify that, as a consequence, all weak cluster points of u ( t ) for t → ∞ belong to S . Finally, the latter implies u ( t ) → x ∗ strongly. � 1 Actually we prove something weaker