Dennis-Mor e Theorem Revisited Asen L. Dontchev Mathematical - PowerPoint PPT Presentation

Dennis-Mor´ e Theorem Revisited Asen L. Dontchev Mathematical Reviews and the University of Michigan Supported by NSF Grant DMS-1008341

Dennis-Mor´ e Theorem Quasi-Newton method for solving f ( x ) = 0: f ( x k ) + B k ( x k +1 − x k ) = 0 , R n → I R n and B k is a sequence of matrices. where f : I Let s k = x k +1 − x k , E k = B k − Df (¯ x ). Recall that { x k } converges superlinearly when � e k +1 � / � e k � → 0. Theorem [Dennis-Mor´ e, 1974] . Suppose that f is differentiable R n containing ¯ in an open convex set D in I x , a zero of f , the derivative Df is continuous at ¯ x and Df (¯ x ) is nonsingular. Let { B k } be a sequence of nonsingular matrices and let for some starting point x 0 in D the sequence { x k } be generated by the method, remain in D for all k and x k � = ¯ x for all k . Then x k → ¯ x superlinearly if and only if � E k s k � x k → ¯ x and lim = 0 . � s k � k →∞

Outline: 1. Strong metric subregularity 2. Dennis-Mor´ e theorem for Newton differentiable functions 3. Dennis-Mor´ e theorem for generalized equations

Strong metric subregularity → Y and a point Definition. Consider a mapping H : X → (¯ x , ¯ y ) ∈ X × Y . Then H is said to be strongly metrically subregular at ¯ x for ¯ y when ¯ y ∈ H (¯ x ) and there is a constant κ > 0 together with a neighborhood U of ¯ x such that � x − ¯ x � ≤ κ d (¯ y , H ( x )) for all x ∈ U . Obeys the paradigm of the implicit function theorem: f is s.m.s. if and only if the linearization x �→ f (¯ x ) + Df (¯ x )( x − ¯ x ) is s.m.s., which is the same as κ � h � ≤ � Df (¯ x ) h � for all h ∈ X . R n → → I R m , whose graph is the union of finitely Every mapping T : I many polyhedral convex sets, is s.m.s at ¯ x for ¯ y if and only if ¯ x is an isolated point in T − 1 (¯ y ).

Strong metric subregularity in optimization Convex optimization minimize g ( x ) − � p , x � over x ∈ C , R n → I R n , and C is a convex R is convex and C 2 , p ∈ I where g : I R n . polyhedral set in I First-order optimality condition ∇ g ( x ) + N C ( x ) ∋ p The mapping ∇ g + N C is strongly metrically subregular at ¯ x for ¯ p if and only if the standard second-order sufficient condition holds p : �∇ 2 g (¯ at ¯ x for ¯ x ) u , u � > 0 for all nonzero u in the critical cone K C (¯ x , ¯ p − ∇ g (¯ x )).

1฀3 1 Rockafellar Dontchev Asen L.Dontchev R.Tyrrell Rockafellar Asen L. Dontchev · R. Tyrrell Rockafellar springer smm Implicit Functions and Solution Mappings monographs in mathematics Implicit Functions The implicit function theorem is one of the most important theorems in analysis and its many variants are basic tools in partial differential equations and numerical analysis. This book treats the implicit function paradigm in the classical framework and beyond, focusing largely on properties of solution mappings of variational and Solution problems. Implicit Functions and Solution Mappings The purpose of this self-contained work is to provide a reference on the topic and to provide a unified collection of a number of results which are currently scattered throughout the literature. The first chapter of the book treats the classical implicit Mappings function theorem in a way that will be useful for students and teachers of undergraduate calculus. The remaining part becomes gradually more advanced, and considers implicit mappings defined by relations other than equations, e.g., variational problems. Applications to numerical analysis and optimization are also A View from provided. This valuable book is a major achievement and is sure to become a standard reference Variational Analysis on the topic. ISBN 978-0-387-87820-1 9 7 8 0 3 8 7 8 7 8 2 0 1

Convergence under strong subregularity Main Lemma. Let f : X → Y , for X and Y Banach spaces, be Lipschitz continuous in a neighborhood U of ¯ x and strongly metrically subregular at ¯ x with a neighborhood U . Consider any sequence { x k } the elements of which are in U for all k = 0 , 1 , . . . and x k � = ¯ x for all k . Then x k → ¯ x superlinearly if and only if � f ( x k +1 ) − f (¯ x ) � x k → ¯ and lim = 0 . x � s k � k →∞ Proof. Consider any { x k } with elements in U such that x k +1 � = ¯ x for all k and x k → ¯ x superlinearly. Then � e k � / � s k � → 1. Let ε > 0 and choose k 0 large enough so that � e k +1 � ≤ ε � s k � . Let L be a Lipschitz constant of f in U . We have � f ( x k +1 ) − f (¯ x ) � ≤ L � x k +1 − ¯ x � = L � e k +1 � ≤ L ε. � s k � � s k � � s k �

Proof continued. Let ε > 0 satisfy ε < 1 /κ and let k 0 be so large that for k ≥ k 0 , one has x k ∈ U and � f ( x k +1 ) − f (¯ x ) � ≤ ε � s k � for k ≥ k 0 . The assumed strong subregularity yields � x k +1 − ¯ x � ≤ κ � f ( x k +1 ) − f (¯ x ) � and hence, for all k ≥ k 0 , � e k +1 � ≤ κε � s k � . But then, for such k , � e k +1 � ≤ κε � s k � ≤ κε ( � e k � + � e k +1 � ) . Hence � e k +1 � κε ≤ � e k � 1 − κε for all sufficiently large k . Hence x k → ¯ x superlinearly.

Proof of Dennis-Mor´ e Theorem Let � 1 V k = Df (¯ x + τ e k ) d τ − Df (¯ x ) e k . 0 By elementary calculus � 1 f ( x k +1 ) = f (¯ x ) + Df (¯ x + τ e k +1 ) e k +1 d τ = Df (¯ x ) e k +1 + V k +1 0 = − f ( x k ) − B k s k + Df (¯ x ) s k + Df (¯ x ) e k + V k +1 = − f ( x k ) − E k s k + Df (¯ x ) e k + V k +1 = − E k s k − f ( x k ) + f (¯ x ) + Df (¯ x ) e k + V k +1 = − E k s k − V k + V k +1 . Apply Main Lemma.

Newton differentiable functions Definition. A function f is Newton differentiable at ¯ x ∈ int dom f when for each ε > 0 there exists a neighborhood U of ¯ x and such that for each x ∈ U there exists a mapping G ( x ) ∈ L ( X , Y ), called the N-derivative of f at x , such that � f ( x ) − f (¯ x ) − G ( x )( x − ¯ x ) � ≤ ε � x − ¯ x � . Reference: K. Ito and K. Kunisch, On the Lagrange multiplier approach to variational problems and applications, SIAM, Philadelphia, PA, 2008. f is strongly metrically subregular at ¯ x if and only if there is a constant κ > 0 such that for any h near 0 and any N-derivative G (¯ x + h ) one has � G (¯ x + h ) h � ≥ κ � h � .

Dennis-Mor´ e for Newton differentiable functions Theorem. Suppose that f has a zero ¯ x , is Lipschitz continuous in a neighborhood ¯ x , Newton differentiable at ¯ x and strongly subregular at ¯ x . Let { B k } be a sequence of linear and bounded mappings and suppose that there is a neighborhood of ¯ x such that for any starting point x 0 in U a sequence { x k } be generated by f ( x k ) + B k ( x k +1 − x k ) = 0 , remain in U and satisfy x k � = ¯ x for all k . Let G k ( x k ) be any Newton derivative of f for x k and denote E k = B k − G k ( x k ). Then x k → ¯ x superlinearly if and only if � E k s k � x k → ¯ x and lim = 0 . � s k � k →∞

Quasi-Newton method for generalized equations → Y f : X → Y , F : X → Generalized equation f ( x ) + F ( x ) ∋ 0 , Quasi-Newton method ( QN ) f ( x k ) + B k ( x k +1 − x k ) + F ( x k +1 ) ∋ 0 , → Y , B k ∈ L ( X , Y ). where f : X → Y , F : X →

Dennis-Mor´ e Theorem for generalized equations Let s k = x k +1 − x k , E k = B k − Df (¯ x ). Theorem [extended Dennis-Mor´ e] . Suppose that f is Fr´ echet differentiable in an open convex set D in X containing a solution ¯ x and Df is continuous at ¯ x . Let for some starting point x 0 in D the sequence { x k } be generated by (QN), remain in D and satisfy x k � = ¯ x for all k . If x k → ¯ x superlinearly, then d (0 , f (¯ x ) + E k s k + F ( x k +1 )) lim = 0 . � s k � k →∞ Conversely, suppose that x �→ G ( x ) = f (¯ x ) + Df (¯ x )( x − ¯ x ) + F ( x ) is strongly metrically subregular at ¯ x for 0 and consider a sequence { x k } generated by (QN) for some x 0 in D , which remain in D , satisfy x k � = ¯ x for all k , � E k s k � x k → ¯ x and lim = 0 . � s k � k →∞ Then x k → ¯ x superlinearly.