Solving Minimax Problems with Feasible Sequential Quadratic - PowerPoint PPT Presentation

Qianli Deng (Shally) E-mail: dqianli@umd.edu Advisor: Dr. Mark A. Austin Department of Civil and Environmental Engineering Institute for Systems Research University of Maryland, College Park, MD 20742 E-mail: austin@isr.umd.edu Solving Minimax Problems with Feasible Sequential Quadratic Programming 05/06/2014

Constrained minimax problem  ≤ ≤ bl x bu n B  ≤ = g ( ) x 0 j 1,..., n  j NI  ≡ − ≤ = g ( ) x c , x d 0 j 1,..., n  j j j LI  = = h ( ) x 0 j 1,..., n  j NE  ≡ − = = h ( ) x a , x b 0 j 1,..., n  j j j LE � ��������

Constrained minimax problem * Objectives  ≤ ≤ bl x bu n B  ≤ = g ( ) x 0 j 1,..., n  j NI  ≡ − ≤ = g ( ) x c , x d 0 j 1,..., n  j j j LI  = = h ( ) x 0 j 1,..., n  j NE  ≡ − = = h ( ) x a , x b 0 j 1,..., n  j j j LE � ��������

Constrained minimax problem * Objectives  ≤ ≤ bl x bu n * Bounds B  ≤ = g ( ) x 0 j 1,..., n  * Nonlinear inequality j NI  ≡ − ≤ = g ( ) x c , x d 0 j 1,..., n  * Linear inequality j j j LI  = = h ( ) x 0 j 1,..., n  * Nonlinear equality j NE  ≡ − = = h ( ) x a , x b 0 j 1,..., n  * Linear equality j j j LE � ��������

FSQP Algorithm � A random point � A point fitting all linear constraints � A point fitting all constraints � An optimal point fitting all constraints � ��������

FSQP Algorithm � A random point � A point fitting all linear constraints � A point fitting all constraints � An optimal point fitting all constraints � ��������

FSQP Algorithm � A random point � A point fitting all linear constraints • boundary + linear inequality + linear equality � A point fitting all constraints � An optimal point fitting all constraints � ��������

FSQP Algorithm � A random point � A point fitting all linear constraints • boundary + linear inequality + linear equality � A point fitting all constraints • boundary + linear inequality + linear equality + nonlinear inequality + nonlinear equality � An optimal point fitting all constraints � ��������

FSQP Algorithm � A random point � A point fitting all linear constraints • boundary + linear inequality + linear equality � A point fitting all constraints • boundary + linear inequality + linear equality + nonlinear inequality + nonlinear equality � An optimal point fitting all constraints � ��������

FSQP Algorithm Step 1. Initialization x H – a point; – Hessian matrix k k d Step 2. A search line – direction k t – distance Step 3. Line search k x + – an update point; k 1 Step 4. Updates H + – an update Hessian matrix k 1 �� ��������

Initialization x = an initial point fitting all constraints 0 H = an identity matrix 0 n ∑ NE p { } = a constant = − f ( , x p ) max f x ( ) p h x ( ) 0, j m i j j f ∈ i I = j 1 = j 1,..., n k = 0 NE �� ��������

Computation of a search line  η  1 0 1 0 1 − − + γ min d d d , d  0 0 0 + min , '( , , ) d H d f x d p  k k 1 ℝ n ℝ 2 ∈ γ ∈ d , k k k  0 2 d  1  ≤ + ≤ s t . . bl x d bu  0 ≤ + ≤ k bl x d bu   . . k st 1 ≤ γ f '( x d , , p )   k k 0  + ∇ ≤ g ( x ) g ( x ), d 0, = j 1,..., t  j k j k 1  + ∇ ≤ γ = g ( x ) g ( x ), d j 1,..., n i j k j k i   0 = + ∇ ≤ j 1,..., n h x ( ) h x ( ), d 0, e 1 j k j k  + ≤ = −  c x , d d j 1,..., t n j k j i i = − j 1,..., t n   0 + = a x , d b e e  1  + ∇ ≤ γ = ( ) ( ), 1,..., j k j h x h x d j n j k j k e  1  + = = − a x , d b j 1,..., t n  j k j e e 0 κ 0 κ ρ = + || d || /(|| d || v ) k k k k 1 τ 1 = v max(0.5,|| d || ) k k �� ��������

Line search Armijo line search  − 0 0  + = No nonlinear constraints d , H d n n 0 =  k k k i e δ k   + ≠ f '( x d , , p ) n n 0 k k k i e n ∑ NE = + ∇ − − ∇ f '( , , ) x d p max{ ( ) f x f x d ( ), } f ( ) x p h x d ( ), i i f j i I f ∈ i I = j 1 + ≤ + α δ f ( x t d , p ) f ( x , p ) t m k k k k m k k k n  − ∑ NE + ≤ = ( ) 0, 1,..., g x td j n = f ( , ) x p max{ ( )} f x p h x ( )  j k k NI  m i j j f ∈ i I = j 1  + ≤ = c , x td d , j 1,..., n j k k j LI   + ≤ = h x ( td ) 0, j 1,..., n  j k k NE �� ��������

Updates + = + x x t d k 1 k k k H + updated by BFGS formula with Powell’s modification k 1 η = − γ = ∇ − ∇ x x , L x ( , p ) L x ( , p ) + + + + k 1 k 1 k k 1 x k 1 k x k k  T T η γ ≥ η η 1, 0.2 H + 1 + 1 + 1 + 1  k k k k k =  θ T η η 0.8 H + k 1 k + 1 k k + 1  otherwise T T η η − η γ  H + + + + k 1 k k 1 k 1 k 1 ξ = θ ⋅ γ + − θ ⋅ γ (1 ) H k + 1 k + 1 k + 1 k + 1 k k + 1 T T η η ξ ξ H H + + + + = − k k 1 k 1 k + k 1 k 1 H H + k 1 k T T η η η ξ H + + + + k 1 k k 1 k 1 k 1  + µ ≥ ε p p =  k j , k j , j 1 p + k 1, j ε − µ δ max{ , p } otherwise  1 j k j , = + k k 1 �� ��������

Implementation JFSQP.java Initial.java MiniMax.java QP.java Direction_d0.java MiniMax.java QP.java Direction_d0.java KKT.java QP.java Direction_d1.java KKT.java QP.java Direction_d1.java Arcsearch.java QP.java Check.java Arcsearch.java BFGS_Powell.java Check.java BFGS_Powell.java �� ��������

Implementation JFSQP.java Initial.java Loop 1. MiniMax.java Find initial feasible point QP.java Direction_d0.java MiniMax.java QP.java Direction_d0.java KKT.java QP.java Direction_d1.java KKT.java QP.java Direction_d1.java Arcsearch.java QP.java Check.java Arcsearch.java BFGS_Powell.java Check.java Loop 2. BFGS_Powell.java Find optimal point �� ��������

Implementation JFSQP.java Initial.java Loop 1. MiniMax.java Find initial feasible point QP.java Direction_d0.java MiniMax.java QP.java Direction_d0.java KKT.java QP.java Direction_d1.java KKT.java QP.java Direction_d1.java Arcsearch.java QP.java Check.java Arcsearch.java BFGS_Powell.java Check.java Loop 2. BFGS_Powell.java Find optimal point �� ��������

Implementation JFSQP.java Initial.java MiniMax.java Find a point fitting all QP.java Direction_d0.java linear constraints MiniMax.java Find the QP.java optimal Direction_d0.java point also KKT.java Find point fitting all QP.java Direction_d1.java constraints fitting all KKT.java (linear + constraints QP.java nonlinear) (linear + Direction_d1.java nonlinear) Arcsearch.java QP.java Check.java Arcsearch.java BFGS_Powell.java Check.java BFGS_Powell.java �� ��������

Implementation Three open-source Java libraries were leveraged: 1) JAMA (Version 1.0.3; 11/2012) http://math.nist.gov/javanumerics/jama/ 2) Apache Commons.Lang (Version 3.3.2; 04/2014) http://commons.apache.org/ 3) Joptimizer (Version 3.3.0; 04/2014) http://www.joptimizer.com/ �� ��������

, Validation JFSQP.java Initial.java QP.java MiniMax.java Direction_d0.java QP.java KKT.java Direction_d1.java QP.java Arcsearch.java Check.java BFGS_Powell.java �� ��������

, Validation JFSQP.java testJFSQP.java Initial.java testInitial.java QP.java testQP.java MiniMax.java testMiniMax.java Direction_d0.java QP.java testQP.java KKT.java testKKT.java Direction_d1.java QP.java testQP.java Arcsearch.java Check.java BFGS_Powell.java testBFGS_Powell.java �� ��������

, Validation JFSQP.java testJFSQP.java Initial.java testInitial.java QP.java testQP.java MiniMax.java testMiniMax.java Direction_d0.java QP.java testQP.java KKT.java testKKT.java Direction_d1.java QP.java testQP.java Arcsearch.java Check.java BFGS_Powell.java testBFGS_Powell.java �� ��������

, Validation JFSQP.java testJFSQP.java Initial.java testInitial.java QP.java testQP.java MiniMax.java testMiniMax.java Direction_d0.java QP.java testQP.java KKT.java testKKT.java Direction_d1.java QP.java testQP.java Arcsearch.java Check.java BFGS_Powell.java testBFGS_Powell.java �� ��������