Motivation Preliminaries Sums-of-AM/GM-Exponentials Sparsity preservation A Hierarchy Extreme rays Conclusion Appendices Conditional Moment Relaxations and Sums-of-AM/GM-Exponentials Riley Murray California Institute of Technology MIT Virtual Seminar on Optimization and Related Areas 17 April 2020 Riley Murray 1
Motivation Preliminaries Sums-of-AM/GM-Exponentials Sparsity preservation A Hierarchy Extreme rays Conclusion Appendices The functions of interest polynomials signomials Parameters a i in N n , c i in R . Parameters a i in R n , c i in R . Using x a i = � n a ij j =1 x , In “exponential form”, j m m � � x �→ c i exp( a i · x ) . c i x a i . x �→ i =1 i =1 Care about number of terms: m . Care about degree: max i � a i � 1 . For historical and modeling reasons, signomials are often written in geometric form m � c i y a i y �→ i =1 where y ∈ R n ++ has the correspondence y i = exp( x i ) . We use the exponential form! Riley Murray 2
Motivation Preliminaries Sums-of-AM/GM-Exponentials Sparsity preservation A Hierarchy Extreme rays Conclusion Appendices Geometric Programming The signomial Optimization-based engineering design: electrical [2, 3, 4], structural [5, 6], environmental [7], and f ( x ) = � m i =1 c i exp( a i · x ) aeronautical [8, 9]. is called a posynomial when all c i ≥ 0 . Geometric programs (GPs): � � f ( x ) : g i ( x ) ≤ 1 ∀ i ∈ [ k ] inf x ∈ R n where f and { g i } k Epidemilogical process control [10, 11, 12], power control i =1 are posynomials. and storage [13, 14], self-driving cars [15], gas network operation [16]. Study of GPs initiated by Zener, Duffin, and Peterson (1967). Exponential-form GPs are convex & poly-time solvable via IPMs [1]. Additional applications in healthcare [17], biology [18], economics [19, 20, 21], and statistics [22, 23] Riley Murray 3
Motivation Preliminaries Sums-of-AM/GM-Exponentials Sparsity preservation A Hierarchy Extreme rays Conclusion Appendices Signomial programming A signomial program (SP) is an optimization problem stated with signomials, e.g. � � inf f ( x ) : g i ( x ) ≤ 0 for all i in [ k ] . x ∈ R n Major applications in aircraft design [24, 25, 26, 27, 28] and structrual engineering [29, 30, 31, 32]. Additional applications in EE [33], communications [34], and ML [35]. Riley Murray 4
Motivation Preliminaries Sums-of-AM/GM-Exponentials Sparsity preservation A Hierarchy Extreme rays Conclusion Appendices Motivation. Mathematical Preliminaries. Sums-of-AM/GM-Exponentials. Sparsity preservation. A hierarchy. Extreme rays. Conclusion. Riley Murray 5
Motivation Preliminaries Sums-of-AM/GM-Exponentials Sparsity preservation A Hierarchy Extreme rays Conclusion Appendices The AM/GM-inequality If u , λ ∈ R m are positive and 1 ⊺ λ = 1 , then u λ ≤ λ ⊺ u . Proof. If v = log u , then u λ = exp( λ ⊺ v ) ≤ � m i =1 λ i exp v i = λ ⊺ u . � A recent history of using the AM/GM inequality to certify function nonnegativity: 1978 and 1989: Reznick [36, 37]. 2009: P´ ebay, Rojas and Thompson [38]. 2012 and 2013: Ghasemi and Marshall [39], Ghasemi, Lasserre, and Marshall [40]. 2012: Paneta, Koeppl, and Craciun [41], and August, Craciun, and Koeppl [42]. 2016: Iliman and de Wolff [43]. When used for computation, exponents { a i } m i =1 were presumed to be highly structured . E.g. conv { a i } m i =1 has m − 1 extreme points, 1 point in its relative interior. Riley Murray 6
Motivation Preliminaries Sums-of-AM/GM-Exponentials Sparsity preservation A Hierarchy Extreme rays Conclusion Appendices Definitions from convex analysis A set convex set K is called a cone if x ∈ K ⇒ λ x ∈ K for all λ ≥ 0; the dual cone to K is K † = { y : y ⊺ x ≥ 0 for all x in K } . – and we have ( K † ) † = cl K A convex set X induces a support function σ X ( λ ) = sup { λ ⊺ x : x in X } . The relative entropy function continuously extends m � R m + × R m D ( u , v ) = u i log( u i /v i ) to + . i =1 Important: if you evaluate D ( · , · ) outside R m + × R m + , you get + ∞ . Riley Murray 7
Motivation Preliminaries Sums-of-AM/GM-Exponentials Sparsity preservation A Hierarchy Extreme rays Conclusion Appendices A trick with convex duality Start with a primal problem Val( c ) = inf x { c ⊺ x : Ax = b , x ≥ 0 } . Obtain a dual problem Val( c ) = sup µ {− b ⊺ µ : A ⊺ µ + c ≥ 0 } . We will encounter constraints like Val( c ) + L ≥ 0 . Write such a constraint as: there exists a µ where A ⊺ µ + c ≥ 0 and b ⊺ µ ≤ L. Riley Murray 8
Motivation Preliminaries Sums-of-AM/GM-Exponentials Sparsity preservation A Hierarchy Extreme rays Conclusion Appendices Nonnegativity and optimization We’ll work with sets X ⊂ R n . Speaking abstractly, for any f : R n → R f ⋆ X = inf { f ( x ) : x in X } = sup { γ : f − γ is nonnegative over X } . Make this more concrete. For signomials: 0 � � m a 2 C NNS ( A , X ) . � A . ∈ R m × n . = c : c i exp( a i · x ) ≥ 0 ∀ x ∈ X , = . . . i =1 a m So for f ( x ) = � m i =1 c i exp( a i · x ) , f ⋆ X = sup { γ : c − γ e 1 ∈ C NNS ( A , X ) } – where e 1 is the 1 st standard basis vector in R m . Riley Murray 9
Motivation Preliminaries Sums-of-AM/GM-Exponentials Sparsity preservation A Hierarchy Extreme rays Conclusion Appendices Duality and moment relaxations Abbreviate exp( Ax ) ∈ R m elementwise, and express C NNS ( A , X ) = { c : c ⊺ exp( Ax ) ≥ 0 ∀ x ∈ X } . The definition of “dual cone” requires C NNS ( A , X ) † = { v : c ⊺ v ≥ 0 ∀ c ∈ C NNS ( A , X ) } . So we end up getting C NNS ( A , X ) † = co { exp( Ax ) : x ∈ X } – a “ moment cone .” conv { exp( Ax ) : x ∈ X } = E x [exp( Ax )] : x ∼ F, supp F ⊂ X � �� � conditional probability Get moment relaxations from conic duality � � v ∈ C NNS ( A , X ) † sup γ { γ : c − γ e 1 ∈ C NNS ( A , X ) } = inf c ⊺ v : . satisfies v · e 1 = 1 � �� � f ⋆ X Riley Murray 10
Motivation Preliminaries Sums-of-AM/GM-Exponentials Sparsity preservation A Hierarchy Extreme rays Conclusion Appendices Motivation. Mathematical Preliminaries. Sums-of-AM/GM-Exponentials. Sparsity preservation. A hierarchy. Extreme rays. Conclusion. Riley Murray 11
Motivation Preliminaries Sums-of-AM/GM-Exponentials Sparsity preservation A Hierarchy Extreme rays Conclusion Appendices X -AGE functions Definition. An X - AGE function is an X -nonnegative signomial, which has at most one Generalizes X = R n from [44]; see [45]. negative coefficient. i x ) . If c ∈ R m has c \ k . Consider f ( x ) = � m i =1 c i exp( a ⊺ = ( c i ) i ∈ [ m ] \ k ≥ 0 , then m � � f ( x ) ≥ 0 ⇔ c i exp([ a i − a k ] · x ) ≥ 0 ⇔ c i exp([ a i − a k ] · x ) + c k ≥ 0 . i =1 i � = k � �� � convex ! Theorem ( M ., Chandrasekaran, & Wierman (2019)) If X is a convex set, then the conditions c \ k ≥ 0 and c ∈ C NNS ( A , X ) are equivalent to the existence of some ν ∈ R m satisfying 1 ⊺ ν = 0 and σ X ( − A ⊺ ν ) + D ( ν \ k , e c \ k ) ≤ c k . Riley Murray 12
Motivation Preliminaries Sums-of-AM/GM-Exponentials Sparsity preservation A Hierarchy Extreme rays Conclusion Appendices X -SAGE certificates & lower bounds A signomial is X -SAGE if it can be written as a sum of appropriate X -AGE functions. The cone of coefficients C SAGE ( A , X ) . � k =1 satisfy c = � m k =1 c ( k ) , 1 ⊺ ν ( k ) = 0 , c : { ( ν ( k ) , c ( k ) ) } m = � � − Aν ( k ) � � � ν ( k ) \ k , c ( k ) ≤ c ( k ) ∀ k ∈ [ m ] and σ X + D \ k k is contained within C NNS ( A , X ) . Consider f ( x ) = � m i =1 c i exp( a i · x ) with a 1 = 0 : f ⋆ X = sup { γ : c − γ e 1 in C NNS ( A , X ) } ≥ sup { γ : c − γ e 1 in C SAGE ( A , X ) } =: f SAGE . X MOSEK + sageopt = off-the-shelf software for computing f SAGE . X Riley Murray 13
Motivation Preliminaries Sums-of-AM/GM-Exponentials Sparsity preservation A Hierarchy Extreme rays Conclusion Appendices Conditional moment relaxations via SAGE Consider f ( x ) = � m i =1 c i exp( a i · x ) with a 1 = 0 . Applying conic duality ... � � c ⊺ v : v in C SAGE ( A , X ) † sup { γ : c − γ e 1 in C SAGE ( A , X ) } = f SAGE = inf X satisfies v · e 1 = 1 Conic duality reverses inclusions C NNS ( A , X ) † ⊂ C SAGE ( A , X ) † . The dual X -SAGE cone is C SAGE ( A , X ) † = cl { v : some z 1 , . . . , z m in R n satisfy v k log( v /v k ) ≥ [ A − 1 a k ] z k and z k /v k ∈ X for all k in [ m ] } . The dual helps with solution recovery. Useful even when f SAGE < f ⋆ X ! X Riley Murray 14
Motivation Preliminaries Sums-of-AM/GM-Exponentials Sparsity preservation A Hierarchy Extreme rays Conclusion Appendices An example in R 3 Minimize f ( x ) = exp( x 1 − x 2 ) / 2 − exp x 1 − 5 exp( − x 2 ) over � x : (70 , 1 , 0 . 5) ≤ exp x ≤ (150 , 30 , 21) X = � exp( x 2 − x 3 ) + exp x 2 + exp( x 1 + x 3 ) ≤ 1 . 100 100 2000 Compute f SAGE = − 147 . 85713 ≤ f ⋆ X , and recover feasible X x = (5 . 01063529 , 3 . 40119660 , − 0 . 48450710) ˜ satisfying f ( ˜ x ) = − 147 . 66666 . Riley Murray 15
Recommend
More recommend
