Formal Non-linear Optimization via Templates and Sum-of-Squares - PowerPoint PPT Presentation

Formal Non-linear Optimization via Templates and Sum-of-Squares Joint Work with B. Werner, S. Gaubert and X. Allamigeon Third year PhD Victor MAGRON LIX/INRIA, ´ Ecole Polytechnique TYPES 2013 Tuesday April 23 rd Third year PhD Victor MAGRON Templates SOS

Motivation: Flyspeck-Like Problems The Kepler Conjecture Kepler Conjecture (1611): The maximal density of sphere packings in 3D-space is π 18 It corresponds to the way people would intuitively stack oranges, as a pyramid shape The proof of T. Hales (1998) consists of thousands of non-linear inequalities Many recent efforts have been done to give a formal proof of these inequalities: Flyspeck Project Motivation: get positivity certificates and check them with Proof assistants like Coq Third year PhD Victor MAGRON Templates SOS

Contents Flyspeck-Like Problems 1 Certification Framework: who does what? 2 Polynomial Optimization via Sum-of-Squares 3 Non-Polynomial Optimization via Templates 4 Formal Non-Linear Optimization 5 Third year PhD Victor MAGRON Templates SOS

Flyspeck-Like Problems Lemma Example Inequalities issued from Flyspeck non-linear part involve: Multivariate Polynomials: 1 x 1 x 4 ( − x 1 + x 2 + x 3 − x 4 + x 5 + x 6 )+ x 2 x 5 ( x 1 − x 2 + x 3 + x 4 − x 5 + x 6 )+ x 3 x 6 ( x 1 + x 2 − x 3 + x 4 + x 5 − x 6 ) − x 2 ( x 3 x 4 + x 1 x 6 ) − x 5 ( x 1 x 3 + x 4 x 6 ) Semi-Algebraic functions algebra A : composition of 2 polynomials with | · | , √ , + , − , × , /, sup , inf , · · · Transcendental functions T : composition of semi-algebraic 3 functions with arctan , exp , + , − , × , · · · Lemma from Flyspeck (inequality ID 6096597438 ) ∀ x ∈ [3 , 64] , 2 π − 2( x arcsin(cos(0 . 797) sin( π/x )) − (0 . 591 − 0 . 0331 x + 1 . 506) ≥ 0 Third year PhD Victor MAGRON Templates SOS

Certification Framework: who does what? x ∈ R p ( x ) = 1 / 2 x 2 − bx + c Polynomial Optimization (POP): min y A program written in OCaml/C 1 x → p ( x ) provides the Sum-of-Squares decomposition: 1 / 2( x − b ) 2 c − b 2 / 2 A program written in Coq checks: 2 ∀ x ∈ R , p ( x ) = 1 / 2( x − b ) 2 + c − b 2 / 2 x b Sceptical approach: obtain certificates of positivity with efficient oracles and check them formally Questions: How to obtain the certificates? How to deal with non-polynomial case? Third year PhD Victor MAGRON Templates SOS

The Polynomial Case General POP min x ∈ K p ( x ) with K the compact set of constraints: K = { x ∈ R n : g 1 ( x ) ≥ 0 , · · · , g m ( x ) ≥ 0 } Let Σ[ x ] be the cone of Sum-of-Squares (SOS) and consider the restriction Σ d [ x ] to polynomials of degree at most 2 d : � � � q i ( x ) 2 , with q i ∈ R d [ x ] Σ d [ x ] = i Let g 0 := 1 and M ( g ) be the quadratic module generated by g 1 , · · · , g m : � m � � M ( g ) = σ j ( x ) g j ( x ) , with σ j ∈ Σ[ x ] j =0 Certificates for positive polynomials: Sum-of-Squares Third year PhD Victor MAGRON Templates SOS

The Polynomial Case: Putinar Theorem � m � � M ( g ) = σ j ( x ) g j ( x ) , with σ j ∈ Σ[ x ] j =0 Proposition (Putinar) Suppose x ∈ [ a , b ] . p ( x ) − p ∗ > 0 on K = ⇒ ( p ( x ) − p ∗ ) ∈ M ( g ) But the search space for σ 0 , · · · , σ m is infinite so consider the truncated module M d ( g ) : � m � � M d ( g ) = σ j ( x ) g j ( x ) , with σ j ∈ Σ[ x ] , ( σ j g j ) ∈ R 2 d [ x ] j =0 M 0 ( g ) ⊂ M 1 ( g ) ⊂ M 2 ( g ) ⊂ · · · ⊂ M ( g ) Hence, we consider the hierarchy of SOS relaxation � � programs: µ k := sup µ : ( p ( x ) − µ ) ∈ M k ( g ) µ,σ 0 , ··· ,σ m Third year PhD Victor MAGRON Templates SOS

The Polynomial Case: Examples x ∈ [4 , 6 . 3504] 6 ∆( x ) = x 1 x 4 ( − x 1 + x 2 + x 3 − x 4 + x 5 + x 6 ) + min x 2 x 5 ( x 1 − x 2 + x 3 + x 4 − x 5 + x 6 ) + x 3 x 6 ( x 1 + x 2 − x 3 + x 4 + x 5 − x 6 ) − x 2 ( x 3 x 4 + x 1 x 6 ) − x 5 ( x 1 x 3 + x 4 x 6 ) = µ 2 = 128 6 � ∆( x ) − µ 2 = σ 0 ( x ) + σ j ( x )(6 . 3504 − x j )( x j − 4) with j =1 σ 0 ∈ Σ 2 [ x ] , σ j ∈ Σ 1 [ x ] Also works for Semi-algebraic functions with lifting variables: � x 2 1 + x 2 f := ∆ x − 2 Define K = { ( x , z ) ∈ R n +1 : x ∈ [4 , 6 . 3504] 6 , z 2 ≥ 2 , z 2 ≤ x 2 x 2 1 + x 2 2 , x 2 1 + x 2 1 + x 2 2 , x 2 1 + x 2 2 , z ≥ 0 } x ∈ [4 , 6 . 3504] 6 f ( x ) = min ( x ,z ) ∈ K (∆( x ) − z ) (POP) min Third year PhD Victor MAGRON Templates SOS

Non-Polynomial Optimization: an Example n ( x i + x i +1 ) sin( √ x i ) � x ∈ [1 , 500] n f ( x ) = − min Example: i =1 Classical idea: approximate sin( √· ) by a degree- d Taylor n � Polynomial f d , solve x ∈ [1 , 500] n − min ( x i + x i +1 ) f d ( x i ) (POP) i =1 Lack of accuracy if d is not large enough No free lunch: the complexity to solve POP with Sum-of-Squares of degree 2 d is O ( n 2 d ) Alternative: deal with the complexity issue with low degree approximations: Templates method Third year PhD Victor MAGRON Templates SOS

Non-Polynomial Optimization via Templates √ Consider the univariate function ˆ f : b �→ sin( b ) on I = [1 , 500] y √ b �→ sin( b ) b b 1 b 2 b 3 = 500 1 Pick several points b j ∈ I ˆ f is semi-convex: there exists a constant c j > 0 s.t. f ( b ) + c j / 2( b − b j ) 2 is convex b �→ ˆ By convexity, ∀ b ∈ I, ˆ f ( b ) ≥ − c j / 2( b − b j ) 2 + ˆ f ′ ( b j )( b − b j )+ ˆ f ( b j ) = par − b j ( b ) Third year PhD Victor MAGRON Templates SOS

Non-Polynomial Optimization via Templates � � ∀ j, ˆ f ≥ par − ⇒ ˆ par − b j = f ≥ max : Max-Plus underestimator b j j � � ∀ j, ˆ f ≤ par + ⇒ ˆ par + b j = f ≤ min : Max-Plus overestimator b j j y par + √ b 3 par + b �→ sin( b ) b 1 b 1 b 1 b 2 b 3 = 500 par + par − b 2 b 3 par − b 2 par − b 1 Templates based on Max-plus Semi-algebraic Estimators for √ b �→ sin( b ) : b j ( x i ) } ≤ sin √ x i ≤ j ∈{ 1 , 2 , 3 } { par − j ∈{ 1 , 2 , 3 } { par + max min b j ( x i ) } Third year PhD Victor MAGRON Templates SOS

Non-Polynomial Optimization via Templates: Lifting Use a lifting variable z i to represent x i �→ sin( √ x i ) For each i , pick points b ji With 3 points b ji , we solve the POP:  n �  min − ( x i + x i +1 ) z i   x ∈ [1 , 500] n , z ∈ [ − 1 , 1] n i =1  z i ≤ par + b ji ( x i ) , j ∈ { 1 , 2 , 3 }  s.t.  POP with n + n variables ( n lifting = n variables), with Sum-of-Squares of degree 2 d : O ((2 n ) 2 d ) complexity Third year PhD Victor MAGRON Templates SOS

Full Lifting Templates / Lifting Free Templates Other choice: lifting variable y i to represent x i �→ √ x i and lifting variable z i to represent x i �→ sin( x i ) n  �  min − ( x i + x i +1 ) z i  √   x ∈ [1 , 500] n , y ∈ [1 , 500] n , z ∈ [ − 1 , 1] n  i =1  z i ≤ par + a ji ( y i ) , j ∈ { 1 , 2 , 3 } s.t.     y 2  i = x i  POP with n + 2 n variables ( n lifting = 2 n variables), with Sum-of-Squares of degree 2 d : O ((3 n ) 2 d ) complexity Taylor approximations: templates with n variables ( n lifting = 0 variables) Third year PhD Victor MAGRON Templates SOS

Templates and SOS: the Algorithm Algorithm template optim : Input: tree t , box K , number of lifting variables n lifting 1: if t is semi-algebraic then Define lifting variables and solve the resulting POP 2: 3: else if bop := root ( t ) is a binary operation with children c 1 and c 2 then Apply template optim recursively to c 1 , c 2 4: Compose the results 5: 6: else if r := root ( t ) is univariate transcendental function with child c then Apply template optim recursively to c 7: Build estimators for a sub-tree of t with up to n lifting variables 8: Solve the resulting POP 9: 10: end Third year PhD Victor MAGRON Templates SOS

Templates and SOS: Results for the Example n ( x i + ǫx i +1 ) sin( √ x i ) � x ∈ [1 , 500] n f ( x ) = − min i =1 n n lifting # boxes lower bound time 10( ǫ = 0) − 430 n 2 n 16 40 s 10( ǫ = 0) − 430 n 0 827 177 s 1000( ǫ = 1) − 967 n 2 n 1 543 s 1000( ǫ = 1) − 968 n n 1 272 s Third year PhD Victor MAGRON Templates SOS

Templates and SOS: Results for Flyspeck n = 6 variables, SOS of degree 2 k = 4 n T univariate transcendental functions # boxes sub-problems Inequality id n T n lifting # boxes time 9922699028 1 9 47 241 s 9922699028 1 3 39 190 s 3318775219 1 9 338 26 min 7726998381 3 15 70 43 min 7394240696 3 15 351 1 . 8 h 4652969746 1 6 15 81 1 . 3 h OXLZLEZ 6346351218 2 0 6 24 200 5 . 7 h Third year PhD Victor MAGRON Templates SOS

Towards Formal Non-Linear Optimization Use Sparsity/Symmetries for a positive domino effect: on the global optimization oracle to decrease the O ( n 2 d ) 1 complexity to check Sum-of-Squares with field tactic 2 Formal proofs for Max-Plus estimators: certify rigorous under/over estimators for univariate transcendental functions Third year PhD Victor MAGRON Templates SOS

End Thank you for your attention! Third year PhD Victor MAGRON Templates SOS