Nonlinear Programming Formulation of Chance-Constraints Andreas Wächter joint with Alejandra Peña-Ordieres and Jim Luedtke Department of Industrial Engineering and Management Sciences Northwestern University Evanston, IL, USA andreas.waechter@northwestern.edu US & Mexico Workshop on Optimization and Its Applications Huatulco, Mexico January 9, 2018 Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Problem Statement f ( x ) : R n → R min x ∈ X f ( x ) c ( x , ξ ) : R n × Ξ → R m s.t. P ξ [ c ( x , ξ ) ≤ 0] ≥ 1 − α ◮ Random variable ξ with support Ξ ◮ Assume f ( x ) and c ( x , ξ ) are sufficiently smooth for all ξ ∈ Ξ ◮ X ⊆ R n : Captures additional constraints Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Problem Statement f ( x ) : R n → R min x ∈ X f ( x ) c ( x , ξ ) : R n × Ξ → R m s.t. P ξ [ c ( x , ξ ) ≤ 0] ≥ 1 − α ◮ Random variable ξ with support Ξ ◮ Assume f ( x ) and c ( x , ξ ) are sufficiently smooth for all ξ ∈ Ξ ◮ X ⊆ R n : Captures additional constraints ◮ Goal: Formulate as continuous NLP ◮ Do not want to assume particular probability distribution ◮ Do not want to assume convexity (or convex approximations) ◮ Want to avoid combinatorial approach ◮ Use powerful NLP algorithms and techniques Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Constraint in Probability Space Impose p ( x ) = P [ c ( x , ξ ) ≤ 0] ≥ 1 − α 1.5 ( - 1) - p(x) y = 0 1 c ( x , ξ ) = x 2 − 2 + ξ 0.5 ξ ∼ N (0 , 1) 0 -0.5 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 Issues ◮ Linearization is poor approximation ◮ Always nonconvex ◮ Used in [Hu, Hong, Zhang 13], [Bremer, Henrion, Möller 15] Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Quantile Formulation Nomal CDF 1 Cumulative Distribution Function 0.8 0.6 0.4 0.2 0 -3 -2 -1 0 1 2 3 x ◮ Let Y be a real-valued random variable ◮ (1 − α )-quantile: Q Y 1 − α = inf { y ∈ R : P [ Y ≤ y ] ≥ 1 − α } Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Quantile Formulation Nomal CDF 1 Cumulative Distribution Function 0.8 0.6 0.4 0.2 0 -3 -2 -1 0 1 2 3 x ◮ Let Y be a real-valued random variable ( Y = c ( x , ξ )) ◮ (1 − α )-quantile: Q c ( x ,ξ ) = inf { y ∈ R : P [ c ( x , ξ ) ≤ y ] ≥ 1 − α } 1 − α Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Quantile Formulation Nomal CDF 1 Cumulative Distribution Function 0.8 0.6 0.4 0.2 0 -3 -2 -1 0 1 2 3 x ◮ Let Y be a real-valued random variable ( Y = c ( x , ξ )) ◮ (1 − α )-quantile: Q c ( x ,ξ ) = inf { y ∈ R : P [ c ( x , ξ ) ≤ y ] ≥ 1 − α } 1 − α q 1 − α ( x ) = Q c ( x ,ξ ) p ( x ) ≥ 1 − α ⇐ ⇒ ≤ 0 1 − α Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Choice of Formulation p ( x ) ≥ 1 − α ⇐ ⇒ q 1 − α ( x ) ≤ 0 1.5 ( - 1) - p(x) q 1 - (x) y = 0 1 0.5 0 -0.5 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 ◮ Quantile formulation more suitable for NLP solver Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Sample Average Approximation (SAA) f ( x ) : R n → R min x ∈ X f ( x ) c ( x , ξ ) : R n × Ξ → R s.t. P ξ [ c ( x , ξ ) ≤ 0] ≥ 1 − α ◮ Finite scenario set � Ξ N = { � ξ 1 , . . . , � ξ N } with � ξ i ∈ Ξ chosen i.i.d. ◮ For simplicity: c i ( x ) = c ( x , � ξ i ) Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Sample Average Approximation (SAA) f ( x ) : R n → R min x ∈ X f ( x ) c ( x , ξ ) : R n × Ξ → R s.t. P ξ [ c ( x , ξ ) ≤ 0] ≥ 1 − α ◮ Finite scenario set � Ξ N = { � ξ 1 , . . . , � ξ N } with � ξ i ∈ Ξ chosen i.i.d. ◮ For simplicity: c i ( x ) = c ( x , � ξ i ) SAA approximation of quantile ◮ Ensure that M = ⌈ (1 − α ) N ⌉ constraints are satisfied Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Sample Average Approximation (SAA) f ( x ) : R n → R min x ∈ X f ( x ) c ( x , ξ ) : R n × Ξ → R s.t. P ξ [ c ( x , ξ ) ≤ 0] ≥ 1 − α ◮ Finite scenario set � Ξ N = { � ξ 1 , . . . , � ξ N } with � ξ i ∈ Ξ chosen i.i.d. ◮ For simplicity: c i ( x ) = c ( x , � ξ i ) SAA approximation of quantile ◮ Ensure that M = ⌈ (1 − α ) N ⌉ constraints are satisfied ◮ Scenario vector: C ( x ) = ( c 1 ( x ) , . . . , c N ( x )) Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Sample Average Approximation (SAA) f ( x ) : R n → R min x ∈ X f ( x ) c ( x , ξ ) : R n × Ξ → R s.t. P ξ [ c ( x , ξ ) ≤ 0] ≥ 1 − α ◮ Finite scenario set � Ξ N = { � ξ 1 , . . . , � ξ N } with � ξ i ∈ Ξ chosen i.i.d. ◮ For simplicity: c i ( x ) = c ( x , � ξ i ) SAA approximation of quantile ◮ Ensure that M = ⌈ (1 − α ) N ⌉ constraints are satisfied ◮ Scenario vector: C ( x ) = ( c 1 ( x ) , . . . , c N ( x )) ◮ Order constraint values: c [1] ( x ) ≤ c [2] ( x ) ≤ . . . ≤ c [ N ] ( x ) Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Sample Average Approximation (SAA) f ( x ) : R n → R min x ∈ X f ( x ) c ( x , ξ ) : R n × Ξ → R s.t. P ξ [ c ( x , ξ ) ≤ 0] ≥ 1 − α ◮ Finite scenario set � Ξ N = { � ξ 1 , . . . , � ξ N } with � ξ i ∈ Ξ chosen i.i.d. ◮ For simplicity: c i ( x ) = c ( x , � ξ i ) SAA approximation of quantile ◮ Ensure that M = ⌈ (1 − α ) N ⌉ constraints are satisfied ◮ Scenario vector: C ( x ) = ( c 1 ( x ) , . . . , c N ( x )) ◮ Order constraint values: c [1] ( x ) ≤ c [2] ( x ) ≤ . . . ≤ c [ N ] ( x ) ◮ Empirical quantile: q 1 − α ( C ( x )) = c [ M ] ( x ) ≤ 0 � Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Empirical Quantile ◮ Feasible region for c ( x , ξ ) = ξ 1 x 1 + ξ 2 x 2 − 1 ξ 1 , ξ 1 ∼ N (0 , 1) 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.4 0.3 0.3 0.3 0.2 0.2 0.2 0.1 0.1 0.1 0 0 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 N = 2000 N = 200 N = 500 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.4 0.3 0.3 0.3 0.2 0.2 0.2 0.1 0.1 0.1 0 0 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Empirical Quantile ◮ Feasible region for c ( x , ξ ) = ξ 1 x 1 + ξ 2 x 2 − 1 ξ 1 , ξ 1 ∼ N (0 , 1) 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.4 0.3 0.3 0.3 0.2 0.2 0.2 0.1 0.1 0.1 0 0 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 N = 2000 N = 200 N = 500 Observations: ◮ Approximation improves as N increases ◮ Rough boundary of feasible region results in spurious local minima ◮ A lot of variance for small N as ξ i are resampled ◮ � q 1 − α ( C ( x )) is non-differentiable Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Variance Reduction using Kernels 2 1.5 1 0.5 0 0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1 ◮ Y [1] , . . . , Y [ N ] ordered realizations of a random variable ( c ( x , ξ i )) ◮ Empirical quantiles: For p ∈ [0 , 1] define p = Y [ j ] , where j ∈ { 1 , .., N } with j − 1 Q Y � N < p ≤ j N Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Variance Reduction using Kernels 2 1.5 1 0.5 0 0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1 ◮ Y [1] , . . . , Y [ N ] ordered realizations of a random variable ( c ( x , ξ i )) ◮ Empirical quantiles: For p ∈ [0 , 1] define p = Y [ j ] , where j ∈ { 1 , .., N } with j − 1 Q Y � N < p ≤ j N ◮ Kernel smoothing [Parson 79] � 1 � � Q Y , N , h � α − p Q Y 1 = h K dp 1 − α p h 0 Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Variance Reduction using Kernels 2 1.5 1 0.5 0 0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1 ◮ Y [1] , . . . , Y [ N ] ordered realizations of a random variable ( c ( x , ξ i )) ◮ Empirical quantiles: For p ∈ [0 , 1] define p = Y [ j ] , where j ∈ { 1 , .., N } with j − 1 Q Y � N < p ≤ j N ◮ Kernel smoothing [Parson 79] =: w N , h j � �� � � 1 � � � j � � N � N Q Y , N , h � α − p α − p Q Y 1 1 = h K dp = Y [ j ] h K dp 1 − α p h h j − 1 0 j =1 N Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
✶ Kernel Estimation of Quantiles ◮ Approximate q 1 − α ( C ( x )) = Q Y 1 − α with Y i = c ( x ; ξ i ) by kernel estimate N � q N , h w N , h c [ i ] ( x ) ≤ 0 1 − α ( C ( x )) := i i =1 ◮ The weights depend only on N and h and can be precomputed Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Kernel Estimation of Quantiles ◮ Approximate q 1 − α ( C ( x )) = Q Y 1 − α with Y i = c ( x ; ξ i ) by kernel estimate N � q N , h w N , h c [ i ] ( x ) ≤ 0 1 − α ( C ( x )) := i i =1 ◮ The weights depend only on N and h and can be precomputed ◮ Epanechnikov kernel: K ( u ) = 3 4 (1 − u 2 ) ✶ | u | < 1 Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Recommend
More recommend