Globalization strategies for Mesh Adaptive Direct Search Charles Audet, ´ Ecole Polytechnique de Montr´ eal John Dennis, Rice University ebastien Le Digabel, ´ S´ Ecole Polytechnique de Montr´ eal July 2009 Thanks to: AFOSR, ExxonMobil, Boeing, LANL, FQRNT, NSERC, IREQ.
Presentation outline 1 Handling constraints in real problems Three types of constraints Strategies to deal with constraints Three instantiations of mesh adaptive direct searches Hierarchical convergence analysis 2 Numerical results on engineering problems Three real test problems A feasible starting point An infeasible starting point Multiple runs 3 Discussion
Blackbox optimization problems My main research interest is nonsmooth optimization: minimize f ( x ) x ∈ Ω = { x ∈ X : c j ( x ) ≤ 0 , j ∈ J } ⊂ R n , subject to where f, c j : X → R ∪ {∞} for all j ∈ J = { 1 , 2 , . . . , m } , X is a subset of R n , evaluation of the functions are usually the result of a computer code (a black box) – costly to evaluate. Charles Audet (ISMP 2009) Handling constraints in real problems 3 / 22
Presentation outline 1 Handling constraints in real problems Three types of constraints Strategies to deal with constraints Three instantiations of mesh adaptive direct searches Hierarchical convergence analysis 2 Numerical results on engineering problems Three real test problems A feasible starting point An infeasible starting point Multiple runs 3 Discussion
Three types of constraints The domain: Ω = { x ∈ X : c j ( x ) ≤ 0 , j ∈ J } ⊂ R n Unrelaxable constraints define X Cannot be violated by any trial point. For example, logical conditions on the variables indicating if the simulation may be launched. Charles Audet (ISMP 2009) Handling constraints in real problems 5 / 22
Three types of constraints The domain: Ω = { x ∈ X : c j ( x ) ≤ 0 , j ∈ J } ⊂ R n Unrelaxable constraints define X Relaxable constraints c j ( x ) ≤ 0 Can be violated, and c j ( x ) provides a measure of how much the constraint is violated. A budget for example. Charles Audet (ISMP 2009) Handling constraints in real problems 5 / 22
Three types of constraints The domain: Ω = { x ∈ X : c j ( x ) ≤ 0 , j ∈ J } ⊂ R n Unrelaxable constraints define X Relaxable constraints c j ( x ) ≤ 0 Hidden constraints Is a convenient term to exclude the set of points in the feasible region for the relaxable or unrelaxable constraints at which the black box fails to return a value for one of the problem functions. A typical example is when the simulation crashes unexpectedly. Charles Audet (ISMP 2009) Handling constraints in real problems 5 / 22
Three strategies to deal with constraints Extreme barrier (EB) Treats the problem as being unconstrained, by replacing the objective function f ( x ) by � f ( x ) if x ∈ Ω , f Ω ( x ) := ∞ otherwise. The problem x ∈ R n f Ω ( x ) min is then solved. Remark : If x �∈ X (the non-relaxable constraints), then the costly evaluation of f ( x ) is not performed. Charles Audet (ISMP 2009) Handling constraints in real problems 6 / 22
Three strategies to deal with constraints Extreme barrier (EB) Progressive barrier (PB) Defined for the relaxable constraints. As in the filter methods of Fletcher and Leyffer, it uses the non-negative constraint violation function h : R n → R ∪ {∞} � (max( c j ( x ) , 0)) 2 if x ∈ X, h ( x ) := j ∈ J ∞ , otherwise. At iteration k , points with h ( x ) > h max are rejected by the k algorithm, and h max → 0 as k → ∞ . k Charles Audet (ISMP 2009) Handling constraints in real problems 6 / 22
Three strategies to deal with constraints Extreme barrier (EB) Progressive barrier (PB) h max 0 ✻ f s ✲ h Charles Audet (ISMP 2009) Handling constraints in real problems 6 / 22
Three strategies to deal with constraints Extreme barrier (EB) Progressive barrier (PB) h max 0 ✻ f Image of trial points s s s s ✲ h Charles Audet (ISMP 2009) Handling constraints in real problems 6 / 22
Three strategies to deal with constraints Extreme barrier (EB) Progressive barrier (PB) h max 0 ✻ f Image of trial points This trial point is dominated by the incumbent ց s s s s ✲ h Charles Audet (ISMP 2009) Handling constraints in real problems 6 / 22
Three strategies to deal with constraints Extreme barrier (EB) Progressive barrier (PB) h max 0 ✻ f Image of trial points This trial point improves h but worsens f ց s s s s ✲ h Charles Audet (ISMP 2009) Handling constraints in real problems 6 / 22
Three strategies to deal with constraints Extreme barrier (EB) Progressive barrier (PB) h max h max 1 0 ✻ f Image of trial points s s New incumbent solution ց s s ✲ h Charles Audet (ISMP 2009) Handling constraints in real problems 6 / 22
Three strategies to deal with constraints Extreme barrier (EB) Progressive barrier (PB) Progressive-to-Extreme Barrier (PEB) Initially treats a relaxable constraint by the progressive barrier. Then, if polling around the infeasible poll center generates a new infeasible incumbent that satisfies a constraint violated by the poll center, then that constraint moves from being treated by the progressive barrier to the extreme barrier. Charles Audet (ISMP 2009) Handling constraints in real problems 6 / 22
Infeasible starting point The progressive and progressive-to-extreme barrier approaches allow initial points that violate the relaxable constraints c j ( x ) ≤ 0 . A two-phase method can be ran on the relaxable constraints that we want to treat by the extreme barrier approach. The first phase minimizes the constraint violation function subject to x ∈ X , the unrelaxable constraints. Avoids expensive computations of f . The first phase terminates as soon as a h = 0 , providing an initial point for the second phase. Charles Audet (ISMP 2009) Handling constraints in real problems 7 / 22
Three instantiations of mesh adaptive direct searches Gps with coordinate search. t x 0 Charles Audet (ISMP 2009) Handling constraints in real problems 8 / 22
Three instantiations of mesh adaptive direct searches Gps with coordinate search. t p 3 t t t x 0 p 2 p 4 t p 1 f ( p 4 ) < f ( x 0 ) Charles Audet (ISMP 2009) Handling constraints in real problems 8 / 22
Three instantiations of mesh adaptive direct searches Gps with coordinate search. t t x 0 x 1 Charles Audet (ISMP 2009) Handling constraints in real problems 8 / 22
Three instantiations of mesh adaptive direct searches Gps with coordinate search. t x 1 Charles Audet (ISMP 2009) Handling constraints in real problems 8 / 22
Three instantiations of mesh adaptive direct searches Gps with coordinate search. t p 3 t t t x 1 p 2 p 4 t p 1 f ( p i ) ≥ f ( x 1 ) Charles Audet (ISMP 2009) Handling constraints in real problems 8 / 22
Three instantiations of mesh adaptive direct searches Gps with coordinate search. t x 2 Charles Audet (ISMP 2009) Handling constraints in real problems 8 / 22
Three instantiations of mesh adaptive direct searches Gps with coordinate search. LTMads a non-deterministic implementation of Mads . Union of normalized polling directions grows dense in the unit sphere with probability one. t x 1 Charles Audet (ISMP 2009) Handling constraints in real problems 8 / 22
Three instantiations of mesh adaptive direct searches Gps with coordinate search. LTMads a non-deterministic implementation of Mads . Union of normalized polling directions grows dense in the unit sphere with probability one. t ✄ p 3 ✄ ✄ ✟ t ✟✟✟✟✟✟✟✟✟✟ ✄ p 4 ✄ ✄ t x 1 ✄ ✄ t ✄ p 2 ✄ ✄ t p 1 Charles Audet (ISMP 2009) Handling constraints in real problems 8 / 22
Three instantiations of mesh adaptive direct searches Gps with coordinate search. LTMads a non-deterministic implementation of Mads . Union of normalized polling directions grows dense in the unit sphere with probability one. OrthoMads a deterministic implementation of Mads with orthogonal polling directions. Union of normalized polling directions grows dense in the unit sphere. t x 1 Charles Audet (ISMP 2009) Handling constraints in real problems 8 / 22
Three instantiations of mesh adaptive direct searches Gps with coordinate search. LTMads a non-deterministic implementation of Mads . Union of normalized polling directions grows dense in the unit sphere with probability one. OrthoMads a deterministic implementation of Mads with orthogonal polling directions. Union of normalized polling directions grows dense in the unit sphere. t ✄ p 3 ✄ ✄ ✄ ❳❳❳❳❳❳❳❳❳❳ t ✄ p 2 ✄ t x 1 ❳ ✄ t ✄ p 4 ✄ ✄ ✄ t p 1 Charles Audet (ISMP 2009) Handling constraints in real problems 8 / 22
Convergence analysis of Mads Assumptions At least one initial point in X is provided – but not required to be in Ω . All iterates belong to some compact set – it is sufficient to assume that level sets of f in X are bounded. Key to the analysis These assumptions ensure that there is a convergent subsequence of poll centers on meshes that get infinitely fine. The analysis is divided in two: the limit of feasible poll centers, and the limit of infeasible poll centers. Charles Audet (ISMP 2009) Handling constraints in real problems 9 / 22
Recommend
More recommend
