A Trust Funnel Algorithm for Nonconvex Equality Constrained - PowerPoint PPT Presentation

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization with O ( ǫ − 3 / 2 ) Complexity Mohammadreza Samadi , Lehigh University joint work with Frank E. Curtis , Lehigh University Daniel P. Robinson , Johns Hopkins University U.S.–Mexico Workshop OPTIMIZATION AND ITS APPLICATIONS Huatulco, Mexico January 8, 2018 A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 1 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Outline Motivation Proposed Algorithm Theoretical Results Numerical Results Summary A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 2 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Outline Motivation Proposed Algorithm Theoretical Results Numerical Results Summary A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 3 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Introduction Consider nonconvex equality constrained optimization problems of the form x ∈ R n f ( x ) min s . t . c ( x ) = 0 . where f : R n → R and c : R n → R m are twice continuously differentiable. ◮ We are interested in algorithm worst-case iteration / evaluation complexity. ◮ Constraints are not necessarily linear! A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 4 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Algorithms for equality constrained (nonconvex) optimization Sequential Quadratic Programming (SQP) / Newton’s method Trust Funnel; Gould & Toint (2010) Short-Step ARC; Cartis, Gould, & Toint (2013) A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 5 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Algorithms for equality constrained (nonconvex) optimization Sequential Quadratic Programming (SQP) / Newton’s method ◮ Global convergence: globally convergent (trust region/line search) Trust Funnel; Gould & Toint (2010) ◮ Global convergence: globally convergent Short-Step ARC; Cartis, Gould, & Toint (2013) ◮ Global convergence: globally convergent A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 5 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Algorithms for equality constrained (nonconvex) optimization Sequential Quadratic Programming (SQP) / Newton’s method ◮ Global convergence: globally convergent (trust region/line search) ◮ Worst-case complexity: No proved bound Trust Funnel; Gould & Toint (2010) ◮ Global convergence: globally convergent ◮ Worst-case complexity: No proved bound Short-Step ARC; Cartis, Gould, & Toint (2013) ◮ Global convergence: globally convergent ◮ Worst-case complexity: O ( ǫ − 3 / 2 ) A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 5 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Short-Step ARC f ( x ) � c ( x ) � 2 (0 , f ∗ ) A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 6 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Short-Step ARC f ( x ) � c ( x ) � 2 (0 , f ∗ ) A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 6 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Short-Step ARC f ( x ) � c ( x ) � 2 (0 , f ∗ ) A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 6 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Main Concerns ◮ Completely ignores the objective function during the first phase ◮ Question: Can we do better? A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 7 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Main Concerns ◮ Completely ignores the objective function during the first phase ◮ Question: Can we do better? ◮ Yes!(?) ◮ First, rather than two-phase approach that ignores objective in phase 1, wrap in a trust funnel framework that observes objective in both phases. ◮ Second, consider trace method for unconstrained nonconvex optimization ◮ F. E. Curtis, D. P. Robinson, MS, “A trust region algorithm with a worst-case iteration complexity of O ( ǫ − 3 / 2 ) for nonconvex optimization,” Mathematical Programming, 162, 2017. A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 7 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Outline Motivation Proposed Algorithm Theoretical Results Numerical Results Summary A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 8 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary SQP “core” ◮ Given x k , find s k as a solution of k s + 1 2 s T H k s s ∈ R n f k + g T min s . t . c k + J k s = 0 Issues: ◮ H k might not be positive definite over Null( J k ). ◮ Trust region!. . . but constraints might be incompatible. A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 9 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Step decomposition s 2 c k + J k s = 0 s 1 A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 10 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Step decomposition s 2 s 1 A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 10 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Step decomposition s 2 c k + J k s = c k + J k s n k s 1 A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 10 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Trust funnel basics Step decomposition approach: ◮ First, compute a normal step toward minimizing constraint violation � min s n ∈ R n m v k ( s n ) v ( x ) = 1 2 � c ( x ) � 2 ⇒ s . t . � s n � ≤ δ v k ◮ Second, compute multipliers y k (or take from previous iteration). ◮ Third, compute a tangential step toward optimality: s . t . J k s t = 0 , s t ∈ R n m f k + s t � ≤ δ f k ( s n k + s t ) � s n min k . A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 11 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Main idea Two-phase method combining trust funnel and trace . ◮ Trust funnel for globalization ◮ trace for good complexity bounds Phase 1 towards feasibility, two types of iterations: ◮ F-iteration s improve objective and reduce constraint violation. ◮ V-iteration s reduce constraint violation. Our algorithm vs. basic trust funnel ◮ modified F-iteration conditions and a different funnel updating procedure ◮ uses trace ideas (for radius updates) instead of tradition trust region ◮ after getting approximately feasible, switches to “phase 2”. A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 12 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Our algorithm-Illustration f ( x ) � c ( x ) � 2 (0 , f ∗ ) A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 13 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Our algorithm-Illustration f ( x ) � c ( x ) � 2 (0 , f ∗ ) A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 13 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Our algorithm-Illustration f ( x ) � c ( x ) � 2 (0 , f ∗ ) A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 13 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Our algorithm-Illustration f ( x ) � c ( x ) � 2 (0 , f ∗ ) A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 13 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Our algorithm-Illustration f ( x ) � c ( x ) � 2 (0 , f ∗ ) A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 13 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Our algorithm-Illustration f ( x ) � c ( x ) � 2 (0 , f ∗ ) A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 13 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Our algorithm-Illustration f ( x ) � c ( x ) � 2 (0 , f ∗ ) A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 13 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Our algorithm-Illustration f ( x ) � c ( x ) � 2 (0 , f ∗ ) A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 13 of 24

Motivation Proposed Algorithm Theoretical Results Numerical Results Summary Our algorithm-Illustration f ( x ) � c ( x ) � 2 (0 , f ∗ ) A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization 13 of 24