GoBack Towards a general convergence theory for inexact Newton - PowerPoint PPT Presentation

GoBack

Towards a general convergence theory for inexact Newton regularizations Andreas Rieder Institut f¨ ur Angewandte und Numerische Mathematik Universit¨ at Karlsruhe Fakult¨ at f¨ ur Mathematik (jointly with Armin Lechleiter, Palaiseau) c � Andreas Rieder, Wien, AIP 09 – 1 / 20

Overview REGINN : An inexact Newton REGINN : An inexact Newton regularization regularization Level set based Level set based termination termination Local convergence Local convergence Bibliographical notes Bibliographical notes Conclusion Conclusion c � Andreas Rieder, Wien, AIP 09 – 2 / 20

REGINN : An inexact Newton ⊲ regularization Level set based termination Local convergence Bibliographical notes Conclusion REGINN : An inexact Newton regularization c � Andreas Rieder, Wien, AIP 09 – 3 / 20

Newton regularizations F : D ( F ) ⊂ X → Y, X, Y Hilbert spaces F ( x ) = y δ where � y − y δ � Y ≤ δ , y = F ( x + ) , and F ( x ) = y locally ill-posed in x + . Let x n be an approximation to x + : x n +1 = x n + s N n n = x + − x n satisfies ( A n := F ′ ( x n ) ) The exact Newton step s e A n s e n = y − F ( x n ) − E ( x + , x n ) c � Andreas Rieder, Wien, AIP 09 – 4 / 20

Newton regularizations F : D ( F ) ⊂ X → Y, X, Y Hilbert spaces F ( x ) = y δ where � y − y δ � Y ≤ δ , y = F ( x + ) , and F ( x ) = y locally ill-posed in x + . Let x n be an approximation to x + : x n +1 = x n + s N n n = x + − x n satisfies ( A n := F ′ ( x n ) ) The exact Newton step s e A n s e n = y − F ( x n ) − E ( x + , x n ) Determine s N = ⇒ n as regularized solution of n := y δ − F ( x n ) A n s = b δ b δ n , c � Andreas Rieder, Wien, AIP 09 – 4 / 20

Newton regularizations F : D ( F ) ⊂ X → Y, X, Y Hilbert spaces F ( x ) = y δ where � y − y δ � Y ≤ δ , y = F ( x + ) , and F ( x ) = y locally ill-posed in x + . Let x n be an approximation to x + : x n +1 = x n + s N n n = x + − x n satisfies ( A n := F ′ ( x n ) ) The exact Newton step s e A n s e n = y − F ( x n ) − E ( x + , x n ) Determine s N = ⇒ n as regularized solution of n := y δ − F ( x n ) A n s = b δ b δ n , Let { s n,m } m ∈ N a regularizing sequence. Then, s N n = s n,m n . For instance, s n,m = g m ( A ∗ n A n ) A ∗ n b δ n where g m : [0 , � A n � 2 ] → R is a so-called filter function. c � Andreas Rieder, Wien, AIP 09 – 4 / 20

Newton regularizations (continued) REGINN ( x N ( δ ) , R, { µ n } ) n := 0 ; x 0 := x N ( δ ) ; while � b δ n � Y > Rδ do { m := 0 , s n, 0 = 0 ; repeat m := m + 1 ; compute s n,m from A n s = b δ n ; until � A n s n,m − b δ n � Y < µ n � b δ n � Y x n +1 := x n + s n,m ; n := n + 1 ; } x N ( δ ) := x n ; m ∈ N : � A n s n,m − b δ n � Y < µ n � b δ � � m n = min n � Y c � Andreas Rieder, Wien, AIP 09 – 5 / 20

Assumptions on { s n,m } For the analysis of REGINN we require three properties of the regularizing sequence { s n,m } , namely 1. � A n s n,m , b δ n � Y > 0 ∀ m ≥ 1 whenever A ∗ n b δ n � = 0 , m →∞ A n s n,m = P R ( A n ) b δ 2. lim n , 3. ∃ Θ ≥ 1: � A n s n,m � Y ≤ Θ � b δ n � Y ∀ m, n. If s n,m = g m ( A ∗ n A n ) A ∗ n b δ n and 0 < λg m ( λ ) ≤ C g , λ > 0 , and m →∞ g m ( λ ) = 1 /λ, λ > 0 , lim then all three requirements are fulfilled where Θ ≤ C g . Examples: Landweber, implicit iteration, Tikhonov, Showalter, ν -methods, as well as non-linear methods: steepest decent and conjugate gradients c � Andreas Rieder, Wien, AIP 09 – 6 / 20

First results Lemma: Any direction s n,m is a descent direction in x n for the functional ϕ ( · ) = � y δ − F ( · ) � 2 Y , that is, n b δ �∇ ϕ ( x n ) , s n,m � X < 0 for m ≥ 1 whenever A ∗ n � = 0 . Lemma: Assume that � P R ( A n ) ⊥ b δ n � Y < � b δ n � Y . Then, for any tolerance � � P R ( A n ) ⊥ b δ n � Y � µ n ∈ , 1 � b δ n � Y the repeat -loop of REGINN terminates. Remark: Under � P R ( A n ) ⊥ b δ n � Y = � b δ n � Y , that is, � P R ( A n ) b δ n � Y = 0 we have s n,m = 0 for all m . c � Andreas Rieder, Wien, AIP 09 – 7 / 20

REGINN : An inexact Newton regularization Level set based ⊲ termination Local convergence Bibliographical notes Conclusion Level set based termination c � Andreas Rieder, Wien, AIP 09 – 8 / 20

Structural assumptions on non-linearity For x 0 ∈ D ( F ) such that � F ( x 0 ) − y δ � Y > δ define the level set x ∈ D ( F ): � F ( x ) − y δ � Y ≤ � F ( x 0 ) − y δ � � � L ( x 0 ) := . Note that x + ∈ L ( x 0 ) . Assume � F ( v ) − F ( w ) − F ′ ( w )( v − w ) � Y ≤ L � F ′ ( w )( v − w ) � Y � ⊥ F ′ ( w ) � for one L < 1 and for all v, w ∈ L ( x 0 ) with v − w ∈ N and � �� � � P R ( F ′ ( u )) ⊥ y − F ( u ) Y ≤ ̺ � y − F ( u ) � Y � for one ̺ < 1 and all u ∈ L ( x 0 ) . Remark: L < 1 L = ⇒ ̺ ≤ 1 − L < 1 2 c � Andreas Rieder, Wien, AIP 09 – 9 / 20

Example Let { v n } and { u n } be ONB in separable Hilbert spaces X and Y , resp. We define operator F : X → Y by ∞ 1 � � � F ( x ) = nf � x, v n � X u n n =1 where f : R → R is smooth with f ′ ( · ) ≥ f ′ min > 0 . Here, R ( F ′ ( x )) = Y for any x ∈ X . Thus, ̺ = 0 . min then L = f ′ max − f ′ If, further, f ′ ( · ) ≤ f ′ max with f ′ max < 2 f ′ < 1 . min f ′ min c � Andreas Rieder, Wien, AIP 09 – 10 / 20

Termination Theorem: Let Θ L + ̺ < Λ for one Λ < 1 . Further, choose 1 + ̺ R > Λ − Θ L − ̺. Finally, select all tolerances { µ n } such that with µ min ,n := (1 + ̺ ) δ � � µ n ∈ µ min ,n , Λ − Θ L , + ̺. � b δ n � Y Then, there exists an N ( δ ) such that { x 1 , . . . , x N ( δ ) } ⊂ L ( x 0 ) . Moreover, only the final iterate satisfies the discrepancy principle, that is, � y δ − F ( x N ( δ ) ) � Y ≤ Rδ, and � y δ − F ( x n +1 ) � Y < µ n + θ n L ≤ Λ , n = 0 , . . . , N ( δ ) − 1 , � y δ − F ( x n ) � Y where θ n = � A n s N n � Y / � b δ n � Y ≤ Θ . Remark: Although � y − F ( x N ( δ ) ) � Y < ( R + 1) δ we do not have convergence of { x N ( δ ) } as δ → 0 in general. c � Andreas Rieder, Wien, AIP 09 – 11 / 20

REGINN : An inexact Newton regularization Level set based termination Local ⊲ convergence Bibliographical notes Conclusion Local convergence c � Andreas Rieder, Wien, AIP 09 – 12 / 20

Additional assumptions on { s n,m } Monotonicity: Let there be a continuous and monotonically increasing function Ψ: R → R with t ≤ Ψ( t ) for t ∈ [0 , 1] such that if n − A n s e γ n = � b δ n � Y / � b δ n � Y < 1 and � b δ n − A n s n,m − 1 � Y / � b δ n � Y ≥ Ψ( γ n ) then � s n,m − s e n � X < � s n,m − 1 − s e n � X . δ → 0 s n,m ( y δ ) = s n,m ( y ) . Stability: lim Examples: Landweber iteration and steepest decent: Ψ( t ) = 2 t , Implicit iteration: Ψ( t ) = Ct where C > 2 , √ cg-method: Ψ( t ) = 2 t . c � Andreas Rieder, Wien, AIP 09 – 13 / 20

Modified structural assumption Assume � F ( v ) − F ( w ) − F ′ ( w )( v − w ) � Y ≤ L � F ′ ( w )( v − w ) � Y for one L < 1 and for all v, w ∈ B r ( x + ) ⊂ D ( F ) . c � Andreas Rieder, Wien, AIP 09 – 14 / 20

Monotonicity and Convergence Theorem: Let L � � Ψ + Θ L < Λ for one Λ < 1 1 − L and define � 1 � � 1 � µ min := Ψ R + L . 1 − L Choose R so large that µ min + Θ L < Λ . Restrict all tolerances { µ n } to ] µ min , Λ − Θ L ] and start with x 0 ∈ B r ( x + ) . Then, � x + − x n � X < � x + − x n − 1 � X , n = 1 , . . . , N ( δ ) , and, if x + is unique in B r ( x + ) , δ → 0 � x + − x N ( δ ) � X = 0 . lim c � Andreas Rieder, Wien, AIP 09 – 15 / 20

REGINN : An inexact Newton regularization Level set based termination Local convergence Bibliographical ⊲ notes Conclusion Bibliographical notes c � Andreas Rieder, Wien, AIP 09 – 16 / 20

Bibliographical notes B. Kaltenbacher, A. Neubauer, O. Scherzer Iterative Regularization Methods for Nonlinear Ill-posed Problems de Gruyter, Berlin, 2007 A. Lechleiter, A. Rieder Towards a general convergence theory for inexact Newton regularizations Numer. Math., to appear ( download from my webpage ), M. Hanke Regularizing properties of a truncated Newton-CG algorithm for nonlinear inverse problems Numer. Funct. Anal. Optim. 18 (1998), 971-993. Q. Jin and U. Tautenhahn On the discrepancy principle for some Newton type methods for solving nonlinear ill-posed problems Numer. Math., 111 (2009), 509-558. c � Andreas Rieder, Wien, AIP 09 – 17 / 20

REGINN : An inexact Newton regularization Level set based termination Local convergence Bibliographical notes ⊲ Conclusion Conclusion c � Andreas Rieder, Wien, AIP 09 – 18 / 20

What to remember from this talk We have presented a convergence theory for algorithm REGINN which is based on only 5 features of the underlying inner regularization scheme. These features are rather general and are shared by a variety of schemes being so different as Landweber, steepest decent, implicit iteration, and cg-method. c � Andreas Rieder, Wien, AIP 09 – 19 / 20