Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Black-box expensive optimization: Learn to optimize S´ ebastien Verel Laboratoire d’Informatique, Signal et Image de la Cˆ ote d’opale (LISIC) Universit´ e du Littoral Cˆ ote d’Opale, Calais, France http://www-lisic.univ-littoral.fr/~verel/ August, 29th/30th, 2019
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions AI : Machine Learning, Optimization, perception, etc. Learning : Minimize an error function { M θ } : models to learn on data Search θ ⋆ = arg min θ Error ( M θ , data ) According to the model dimension, variables, error function, etc., huge number of optimization algorithms Optimization : Learn a design algorithm { A θ } : search algorithms for problems ( X , f ) Learn A θ ⋆ such that x = A θ ⋆ ( X , f ) is a good solution According to the class of algorithms, search spaces, functions, etc., huge number of learning algorithms
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Black-box (expensive) optimization x − → − → f ( x ) No direct analytic definition of the objective function f Objective fonction : can be irregular, non continuous, non differentiable, etc. given by a computation or an (expensive) simulation Few examples from the OSMOSE team at LISIC : • Nuclear power plant (Valentin Drouet), • Logistic simulation (Brahim Aboutaib), • Mobility simulation (Florian Leprˆ etre), • Plant’s biology, plant growth (Amaury Dubois), • Cellular automata, grammatical inference (Bilal El Ghadyry),
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Real-world black-box expensive optimization PhD of Mathieu Muniglia, 2014-2017, Valentin Drouet, 2017-2020, Jean-Michel Do, Jean-Charles Le Pallec, CEA-Saclay, Paris x − → − → f ( x ) (73 , .., 8) → → (∆ z P , V e ) Multi-physic simulator Expensive multiobjective optimization : parallel computing, and surrogate model.
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Search algorithms Principle Enumeration of the search space A lot of ways to enumerate the search space Main topics : local search, evolutionary algorithms, metaheuristics, etc. Main challenges : Understand search dynamics, Parameters, design components
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Multiobjective optimization Multiobjective optimization problem X : set of feasible solutions in the decision space M � 2 objective functions f = ( f 1 , f 2 , . . . , f M ) (to maximize) R M : set of feasible outcome vectors in the Z = f ( X ) ⊆ I objective space x 2 f 2 x 1 f 1 Decision space Objective space
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Definition Pareto dominance relation A solution x ∈ X dominates a solution x ′ ∈ X ( x ′ ≺ x ) iff ∀ i ∈ { 1 , 2 , . . . , M } , f i ( x ′ ) � f i ( x ) ∃ j ∈ { 1 , 2 , . . . , M } such that f j ( x ′ ) < f j ( x ) x 2 f 2 non- dominated dominated vector non- dominated solution vector x 1 f 1 Decision space Objective space
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Pareto set, Pareto front x 2 f 2 Pareto front Pareto optimal solution Pareto optimal set x 1 f 1 Decision space Objective space Goal Find the Pareto Optimal Set, or a good approximation of the Pareto Optimal Set NPP problem discrete optimization problem, ≈ 30min of computation per value, 24h total, 10 3 to 3 . 10 3 cores.
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Decomposition based approaches, MOEA/D f 2 λ λ 1 2 λ i-1 Minimization problem z 1 z λ i 2 One solution x i for each sub pb. i λ z i-1 i+1 Representation of solutions in objective space : z i = g ( x i | λ i , z ⋆ i ) z i z i+1 Same reference point for all sub-pb. z ⋆ = z ⋆ 1 = . . . = z ⋆ λ z µ μ-1 μ-1 Scalar function g : z μ λ μ Weighted Tchebycheff z* Neighborhood size ♯ B ( i ) = T = 3 f 1 Population at iteration t
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Decomposition based approaches, MOEA/D f 2 λ λ 1 2 λ i-1 z 1 Minimization problem z λ i 2 One solution x i for each sub pb. i z λ i+1 i-1 B(i) Representation of solutions in z objective space : z i = g ( x i | λ i , z ⋆ i i ) z i+1 Same reference point for all λ sub-pb. z ⋆ = z ⋆ z 1 = . . . = z ⋆ μ-1 μ-1 µ z μ Scalar function g : λ μ z* Weighted Tchebycheff f 1 Neighborhood size ♯ B ( i ) = T = 3 From the neigh. B ( i ) of sub-pb. i , x i +1 is selected
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Decomposition based approaches, MOEA/D f 2 λ λ 1 2 λ i-1 Minimization problem z 1 z λ i 2 One solution x i for each sub pb. i λ z i-1 i+1 Representation of solutions in objective space : z i = g ( x i | λ i , z ⋆ i ) z i z i+1 Same reference point for all sub-pb. z ⋆ = z ⋆ 1 = . . . = z ⋆ λ z µ μ-1 μ-1 Scalar function g : z μ λ μ Weighted Tchebycheff z* Neighborhood size ♯ B ( i ) = T = 3 f 1 The mutated solution y is created
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Decomposition based approaches, MOEA/D f 2 λ λ 1 2 λ i-1 z 1 z λ i 2 Minimization problem z λ i-1 i+1 One solution x i for each sub pb. i Representation of solutions in z objective space : z i = g ( x i | λ i , z ⋆ i ) z i+1 i Same reference point for all λ z μ-1 μ-1 sub-pb. z ⋆ = z ⋆ 1 = . . . = z ⋆ µ z μ λ μ Scalar function g : z* Weighted Tchebycheff f 1 Neighborhood size ♯ B ( i ) = T = 3 According to scalar fonction, y is worst than x i − 1 , y is better than x i and replaces it.
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Decomposition based approaches, MOEA/D f 2 λ λ 1 2 λ i-1 z 1 z λ i 2 Minimization problem z λ i-1 i+1 One solution x i for each sub pb. i Representation of solutions in z i+1 objective space : z i = g ( x i | λ i , z ⋆ i ) z i Same reference point for all λ z μ-1 μ-1 sub-pb. z ⋆ = z ⋆ 1 = . . . = z ⋆ µ z μ λ μ Scalar function g : z* Weighted Tchebycheff f 1 Neighborhood size ♯ B ( i ) = T = 3 According to scalar fonction, y is also better than x i +1 and replaces it for the next iteration.
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Asynchronous distributed (1 + λ )-Evolution Strategy Master-slaves architecture Algorithm on Master { x 1 , . . . , x λ } ← Initialization () Slave for i = 1 ..λ do Send (Non-blocking) x i to slave S i Slave i end for 1-Receive fi x best ← ∅ , and f best ← ∞ Master repeat Slave if there is a pending mess. from S i then 3- Send xi 2- update xbest Receive fitness f i of x i from S i xi = mutation(xbest) if f i � f best then x best ← x i , and f best ← f i Slave end if x i ← mutation ( x best ) Send (Non-blocking) x i to slave S i end if until time limit Parameters of mutation : fitness landscape analysis (random walk, and var. importance)
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions R´ esultat
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Adaptive distributed optimization algorithms B. Aboutaib (2018-), C. Jankee (2014-2018), C. Fonlupt (LISIC), B. Derbel (univ. Lille)
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Adaptive distributed optimization algorithms B. Aboutaib (2018-), C. Jankee (2014-2018), C. Fonlupt (LISIC), B. Derbel (univ. Lille)
Introduction Nuclear power plant control design Learning for optimization Surrogate-assisted optimization Conclusions Main challenges How to select an algorithm ? Design reinforcement learning methods for distributed computing ( ǫ -greedy, adapt. pursuit, UCB, ...) How to compute a reward ? Aggregation function of local rewards (mean, max, etc.) for a global selection 8 5 4 3 2 4 Methodology : Use designed benchmark functions with designed properties and experimental analysis
Recommend
More recommend
