Dynamique(s) de descente pour loptimisation multi-objectif - PowerPoint PPT Presentation

Dynamique(s) de descente pour l’optimisation multi-objectif Guillaume Garrigos Istituto Italiano di Tecnologia & Massachusetts Institute of Technology Genova, Italie Journées SMAI-MODE 24 Mars, 2016 Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 1/20

Introduction/Motivation Multi-objective problem In engineering, decision sciences, it happens that various objective → R functions shall be minimized simultaneously: f 1 , ..., f m : H − Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 2/20

Introduction/Motivation Multi-objective problem In engineering, decision sciences, it happens that various objective → R functions shall be minimized simultaneously: f 1 , ..., f m : H − − → Needs appropriate tools: multi-objective optimization. Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 2/20

The multi-objective optimization problem Let F = ( f 1 , ..., f m ) : H → R m locally Lipschitz, H Hilbert. Solve MIN ( f 1 ( x ) , ..., f m ( x )) : x ∈ C ⊂ H convex. Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 3/20

The multi-objective optimization problem Let F = ( f 1 , ..., f m ) : H → R m locally Lipschitz, H Hilbert. Solve MIN ( f 1 ( x ) , ..., f m ( x )) : x ∈ C ⊂ H convex. We consider the usual order(s) on R m : a ĺ b ⇔ a i ≤ b i for all i = 1 , ..., m , a ă b ⇔ a i < b i for all i = 1 , ..., m . Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 3/20

The multi-objective optimization problem Let F = ( f 1 , ..., f m ) : H → R m locally Lipschitz, H Hilbert. Solve MIN ( f 1 ( x ) , ..., f m ( x )) : x ∈ C ⊂ H convex. We consider the usual order(s) on R m : a ĺ b ⇔ a i ≤ b i for all i = 1 , ..., m , a ă b ⇔ a i < b i for all i = 1 , ..., m . x is a Pareto point if ∄ y ∈ C such that F ( y ) ň F ( x ) x is a weak Pareto point if ∄ y ∈ C such that F ( y ) ă F ( x ) Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 3/20

The multi-objective optimization problem Let F = ( f 1 , ..., f m ) : H → R m locally Lipschitz. Solve MIN f 1 ( x ) , ..., f m ( x ) : x ∈ C ⊂ H convex. How to solve it? Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 4/20

The multi-objective optimization problem Let F = ( f 1 , ..., f m ) : H → R m locally Lipschitz. Solve MIN f 1 ( x ) , ..., f m ( x ) : x ∈ C ⊂ H convex. How to solve it? genetic algorithm − → no theoretical guarantees. Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 4/20

The multi-objective optimization problem Let F = ( f 1 , ..., f m ) : H → R m locally Lipschitz. Solve MIN f 1 ( x ) , ..., f m ( x ) : x ∈ C ⊂ H convex. How to solve it? genetic algorithm − → no theoretical guarantees. scalarization method: � argmin f θ ( x ) ⊂ { weak Paretos } ⊂ { Paretos } , x ∈ H θ ∈ ∆ m m where ∆ m is the simplex unit and f θ ( x ) := � θ i f i ( x ) . i = 1 Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 4/20

The multi-objective optimization problem Let F = ( f 1 , ..., f m ) : H → R m locally Lipschitz. Solve MIN f 1 ( x ) , ..., f m ( x ) : x ∈ C ⊂ H convex. We are going to present a method which: Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 4/20

The multi-objective optimization problem Let F = ( f 1 , ..., f m ) : H → R m locally Lipschitz. Solve MIN f 1 ( x ) , ..., f m ( x ) : x ∈ C ⊂ H convex. We are going to present a method which: generalizes the gradient descent dynamic ˙ x ( t ) + ∇ f ( x ( t )) = 0, Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 4/20

The multi-objective optimization problem Let F = ( f 1 , ..., f m ) : H → R m locally Lipschitz. Solve MIN f 1 ( x ) , ..., f m ( x ) : x ∈ C ⊂ H convex. We are going to present a method which: generalizes the gradient descent dynamic ˙ x ( t ) + ∇ f ( x ( t )) = 0, is cooperative , i.e. all objective functions decrease simultaneously, Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 4/20

The multi-objective optimization problem Let F = ( f 1 , ..., f m ) : H → R m locally Lipschitz. Solve MIN f 1 ( x ) , ..., f m ( x ) : x ∈ C ⊂ H convex. We are going to present a method which: generalizes the gradient descent dynamic ˙ x ( t ) + ∇ f ( x ( t )) = 0, is cooperative , i.e. all objective functions decrease simultaneously, is independent of any choice of parameters. Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 4/20

Towards a descent dynamic for multi-objective optimization Single objective optimization: x n + 1 = x n + λ n d n , where d n satisfies df ( x n ; d n ) < 0 (e.g. d n = −∇ f ( x n ) ). Multi-objective optimization: Can we find d n such that df i ( x n ; d n ) < 0 for all i ∈ { 1 , ..., m } ? Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 5/20

Towards a descent dynamic for multi-objective optimization Historical review Cornet (1981) ∇ f 2 ( x ) ∇ f 1 ( x ) � s ( x ) , ∇ f i ( x ) � < 0 s ( x ) := − [ ∇ f 1 ( x ) , ∇ f 2 ( x )] 0 PhD defense - Guillaume Garrigos 28/30

Multi-objective steepest descent → R m locally Lipschitz, C = H Hilbert. Let F = ( f 1 , ..., f m ) : H − Definition For all x ∈ H , s ( x ) := − ( co { ∂ C f i ( x ) } i = 1 ,..., m ) 0 is the (common) steepest descent direction at x . Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 6/20

Multi-objective steepest descent → R m locally Lipschitz, C = H Hilbert. Let F = ( f 1 , ..., f m ) : H − Definition For all x ∈ H , s ( x ) := − ( co { ∂ C f i ( x ) } i = 1 ,..., m ) 0 is the (common) steepest descent direction at x . Remarks in the smooth case If m = 1 then s ( x ) = −∇ f 1 ( x ) . Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 6/20

Multi-objective steepest descent → R m locally Lipschitz, C = H Hilbert. Let F = ( f 1 , ..., f m ) : H − Definition For all x ∈ H , s ( x ) := − ( co { ∂ C f i ( x ) } i = 1 ,..., m ) 0 is the (common) steepest descent direction at x . Remarks in the smooth case If m = 1 then s ( x ) = −∇ f 1 ( x ) . At each x , s ( x ) selects a convex combination: m m � � s ( x ) = − θ i ( x ) ∇ f i ( x ) = −∇ f θ ( x ) ( x ) where f θ ( x ) = θ i ( x ) f i . i = 1 i = 1 Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 6/20

Multi-objective steepest descent → R m locally Lipschitz, C = H Hilbert. Let F = ( f 1 , ..., f m ) : H − Definition For all x ∈ H , s ( x ) := − ( co { ∂ C f i ( x ) } i = 1 ,..., m ) 0 is the (common) steepest descent direction at x . Remarks in the smooth case If m = 1 then s ( x ) = −∇ f 1 ( x ) . At each x , s ( x ) selects a convex combination: m m � � s ( x ) = − θ i ( x ) ∇ f i ( x ) = −∇ f θ ( x ) ( x ) where f θ ( x ) = θ i ( x ) f i . i = 1 i = 1 s ( x ) is the steepest descent: � � s ( x ) � s ( x ) � = argmin i = 1 ,..., m �∇ f i ( x ) , d � max . d ∈ B H Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 6/20

The (multi-objective) Steepest Descent dynamic Algorithm: x n + 1 = x n + λ n s ( x n ) . Studied in the 2000’s by Svaiter, Fliege, Iusem, ... Continuous dynamic: (SD) x ( t ) = s ( x ( t )) , ˙ x ( t ) + ( co { ∂ C f i ( x ( t )) } i ) 0 = 0 i.e. (SD) ˙ Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 7/20

The (multi-objective) Steepest Descent dynamic Example x ( t ) = s ( x ( t )) with f 1 ( x ) = � x � 2 and f 2 ( x ) = x 1 . (SD) ˙ Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 8/20

The (multi-objective) Steepest Descent dynamic Example x ( t ) = s ( x ( t )) with f 1 ( x ) = � x � 2 and f 2 ( x ) = x 1 . (SD) ˙ Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 8/20

The (multi-objective) Steepest Descent dynamic Example f 1 ( x ) = � x � 2 and f 2 ( x ) = x 1 . (SD) x ( t ) = s ( x ( t )) with ˙ Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 8/20

The (multi-objective) Steepest Descent dynamic Example f 1 ( x ) = � x � 2 and f 2 ( x ) = x 1 . (SD) x ( t ) = s ( x ( t )) with ˙ Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 8/20

The (multi-objective) Steepest Descent dynamic Example f 1 ( x ) = � x � 2 and f 2 ( x ) = x 1 . (SD) x ( t ) = s ( x ( t )) with ˙ Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 8/20

The (multi-objective) Steepest Descent dynamic Example f 1 ( x ) = � x � 2 and f 2 ( x ) = x 1 . (SD) x ( t ) = s ( x ( t )) with ˙ Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 8/20

The (multi-objective) Steepest Descent dynamic Example x ( t ) = s ( x ( t )) with f 1 ( x ) = x 2 1 and f 2 ( x ) = x 2 (SD) ˙ 2 . Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 9/20

The (multi-objective) Steepest Descent dynamic Example x ( t ) = s ( x ( t )) with f 1 ( x ) = x 2 1 and f 2 ( x ) = x 2 (SD) ˙ 2 . Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 9/20

The (multi-objective) Steepest Descent dynamic Example x ( t ) = s ( x ( t )) with f 1 ( x ) = x 2 1 and f 2 ( x ) = x 2 (SD) ˙ 2 . Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 9/20

The (multi-objective) Steepest Descent dynamic Main results (Attouch, G., Goudou, 2014) A cooperative dynamic Let x : R + − → H be a solution of (SD) ˙ x ( t ) = s ( x ( t )) . For all i = 1 , ..., m , the function t �→ f i ( x ( · )) is decreasing. Journées SMAI-MODE 2016 - Toulouse - Guillaume Garrigos 10/20