nonlinear optimization introduction
play

Nonlinear Optimization: Introduction INSEAD, Spring 2006 - PowerPoint PPT Presentation

Jul 24, 2023 •187 likes •349 views

Nonlinear Optimization: Introduction INSEAD, Spring 2006 Jean-Philippe Vert Ecole des Mines de Paris Jean-Philippe.Vert@mines.org 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) p.1/15 Nonlinear optimization c

  1. Nonlinear Optimization: Introduction INSEAD, Spring 2006 Jean-Philippe Vert Ecole des Mines de Paris Jean-Philippe.Vert@mines.org � 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) – p.1/15 Nonlinear optimization c

  2. Optimization problem minimize f ( x ) subject to x ∈ X x : a decision variable X : the constraint set , i.e., available decisions f : X → R : the cost or objective function We want to find an optimal decision , i.e., an x ∗ ∈ X such that: f ( x ∗ ) ≤ f ( x ) , ∀ x ∈ X . � 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) – p.2/15 Nonlinear optimization c

  3. Where do optimization problems arise? Economics : Consumer theory / supplier theory Finance : Optimal hedging / pricing Statistics : data fitting, regression, pattern recognition Science / Engineering : Aerospace, product design, data mining Other Business decisions : scheduling, production, organizational decisions Government : Military applications, fund allocation, etc Other Personal decisions : Sports, on-field decisions, player acquisition, marketing � 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) – p.3/15 Nonlinear optimization c

  4. Example Portfolio optimization variables: amount invested in different assets constraint: budget, max/min investment per asset, minimum return objective: overall risk, or return variance Device sizing in electronic circuits variables: device width and lengths constraints: manufacturing limits, max area power consumption Data fitting variable: model parameters contraints: prior information, parameter limits objective: measure of misfit, prediction error � 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) – p.4/15 Nonlinear optimization c

  5. Classification of optimization problems X is discrete/finite , e.g. integer programming (scheduling, route planning) X ⊂ R d is continuous Linear programming (LP) : f is linear and X is a polyhedron specified by linear equalities and inequalities. Quadratic programming (QP) : f is convex quadratic and X is a polyhedron specified by linear equalities and inequalities. Convex programming : f is a convex function and X is a convex set. Nonlinear programming : f is nonlinear, and/or X is specified by nonlinear equalities and inequalities. � 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) – p.5/15 Nonlinear optimization c

  6. Solving optimization problems General optimization problem very difficult to solve methods involve some compromise , e.g., very long computation time, or not always finding the solution Exceptions : certain problem classes can be solved efficiently and reliably least-square problems linear programming problems convex optimization problems � 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) – p.6/15 Nonlinear optimization c

  7. Least squares � Ax − b � 2 minimize 2 � − 1 A ⊤ b analytical solution x ∗ = � A ⊤ A reliable and efficient algorithms and softwares a mature technology least-square problems are easy to recognize � 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) – p.7/15 Nonlinear optimization c

  8. Linear programming c ⊤ x minimize a ⊤ subject to i x ≤ b i , i = 1 , . . . , m no analytical formula for solution reliable and efficient algorithms and software (simplex algorithm, interior-point methods) a mature technology not as easy to recognize as least-square problems a few standard tricks to convert problems into linear programs � 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) – p.8/15 Nonlinear optimization c

  9. Convex optimization minimize f ( x ) subject to g i ( x ) ≤ b i , i = 1 , . . . , m f and g i are convex: f ( αx + βy ) ≤ αf ( x ) + βf ( y ) if α, β ≥ 0 , α + β = 1 . include least-squares and linear programming no analytical solution reliable and efficient algorithms almost a technology often difficult to recognize � 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) – p.9/15 Nonlinear optimization c

  10. Brief history 1947: simplex algorithm for linear programming (Dantzig) 1960s: early interior-point methods (Fiacco & McCormick, Dikin, ...) 1970s: ellipsoid method and other subgradient methods 1980s: polynomial-time interior-point methods for linear programming (Karmarkar 1984) late 1980s-now: polynomial-time interior-point methods for nonlinear convex optimization (Nesterov & Nemirovski 1994) � 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) – p.10/15 Nonlinear optimization c

  11. The optimization process (1/2) Formulate real life problems into mathematical models Study the environment and clearly understand the problem Formulate the problem using verbal description Define notations for parameters and decision variables Construct a model using mathematical expressions Collect necessary data; Transform the raw data to parameter values Implement the model and solution algorithms using a computer : analyze the models and develop efficient procedures to obtain best solutions � 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) – p.11/15 Nonlinear optimization c

  12. The optimization process (2/2) Interpret computer solutions and perform sensitivity analysis Implementation : put the knowledge gained from the solution to work Monitor the validity and effectiveness of the model and update it when necessary � 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) – p.12/15 Nonlinear optimization c

  13. What you will learn Models - the art: How we choose to represent real problems Theory - the science: What we know about different classes of models; e.g. necessary and sufficient conditions for optimality Algorithms - the tools: How we apply the theory to robustly and efficiently solve powerful models � 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) – p.13/15 Nonlinear optimization c

  14. Course outline Modeling and reformulating Optimality conditions Duality theory and sensitivity analysis Algorithms for unconstrained problems Algorithms for linearly constrained problems Algorithms for convex problems Applications � 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) – p.14/15 Nonlinear optimization c

  15. Practical informations Course web page: http://cbio.ensmp.fr/˜vert/teaching/2006insead schedule slides references, links.. 10 × 2 h Grading: weekly homework (50%), article presentation (10%), final exam (40%) � 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org) – p.15/15 Nonlinear optimization c

Recommend

Halo Orbit Stationkeeping using Nonlinear Outline Introduction MPC and Polynomial Optimization

Halo Orbit Stationkeeping using Nonlinear Outline Introduction MPC and Polynomial Optimization

Rutgers, The State University of New Jersey tugraz Halo Orbit Stationkeeping using Nonlinear Outline Introduction MPC and Polynomial Optimization Circular Restricted Three Body Problem Gaurav Misra, Hao Peng, and Xiaoli Bai Global

985 views • 54 slides
Introduction: Mathematical optimization Motivating Example Applications Least-squares(LS) and

Introduction: Mathematical optimization Motivating Example Applications Least-squares(LS) and

Introduction: Mathematical optimization Motivating Example Applications Least-squares(LS) and linear programming(LP) - Very common place Course goals and topics Nonlinear optimization Brief history of convex optimization Prof. G a n e s h

489 views • 31 slides
Nonlinear Equations and Optimization Motivation: Nonlinear Equations So far we have mostly

Nonlinear Equations and Optimization Motivation: Nonlinear Equations So far we have mostly

Nonlinear Equations and Optimization Motivation: Nonlinear Equations So far we have mostly focused on linear phenomena Interpolation leads to a linear system Vb = y (monomials) or I b = y (Lagrange polynomials) Linear least-squares leads

1.77k views • 149 slides
A Nonlinear Trust Region Framework for PDE-Constrained Optimization Using

A Nonlinear Trust Region Framework for PDE-Constrained Optimization Using

Motivation PDE-Constrained Optimization ROM-Constrained Optimization Numerical Experiments Conclusion References A Nonlinear Trust Region Framework for PDE-Constrained Optimization Using Progressively-Constructed Reduced-Order Models

458 views • 34 slides
Computational Optimization Advance Topics NonSmooth Optimization Reference: Nonlinear

Computational Optimization Advance Topics NonSmooth Optimization Reference: Nonlinear

Computational Optimization Advance Topics NonSmooth Optimization Reference: Nonlinear Optimization, Ruszynski,2006 Best Linear Separator: Supporting Plane Method Maximize distance Between two para supporting planes Distance = x

1.24k views • 26 slides
A Nonlinear Trust-Region Framework for PDE-Constrained Optimization Using Adaptive Model

A Nonlinear Trust-Region Framework for PDE-Constrained Optimization Using Adaptive Model

Introduction Optimization via Adaptive Model Reduction Large-Scale, Constrained Optimization Conclusion A Nonlinear Trust-Region Framework for PDE-Constrained Optimization Using Adaptive Model Reduction Matthew J. Zahr Institute for

336 views • 33 slides
Rapid Nonlinear Topology Optimization using Precomputed Reduced-Order Models Matthew J. Zahr and

Rapid Nonlinear Topology Optimization using Precomputed Reduced-Order Models Matthew J. Zahr and

Introduction Topology Optimization Model Order Reduction Applications Conclusion Rapid Nonlinear Topology Optimization using Precomputed Reduced-Order Models Matthew J. Zahr and Charbel Farhat Farhat Research Group Stanford University 17th

688 views • 23 slides
Introduction Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Mathematical

Introduction Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Mathematical

Introduction Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Mathematical Optimization Least-squares Linear Programming Convex Optimization Nonlinear Optimization Summary Outline Mathematical Optimization

549 views • 40 slides
Introduction Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Mathematical

Introduction Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Mathematical

Introduction Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Mathematical Optimization Least-squares Linear Programming Convex Optimization Nonlinear Optimization Summary Outline Mathematical Optimization

749 views • 38 slides
Nonlinear Optimization for Optimal Control Part 2 Pieter Abbeel UC Berkeley EECS Outline n

Nonlinear Optimization for Optimal Control Part 2 Pieter Abbeel UC Berkeley EECS Outline n

Nonlinear Optimization for Optimal Control Part 2 Pieter Abbeel UC Berkeley EECS Outline n From linear to nonlinear n Model-predictive control (MPC) From Linear to Nonlinear We know how to solve (assuming g t , U t , X t convex): n (1)

750 views • 13 slides
Bellmans curse of dimensionality n n-dimensional state space n Number of states grows

Bellmans curse of dimensionality n n-dimensional state space n Number of states grows

Nonlinear Optimization for Optimal Control Pieter Abbeel UC Berkeley EECS [optional] Boyd and Vandenberghe, Convex Optimization, Chapters 9 11 [optional] Betts, Practical Methods for Optimal Control Using Nonlinear Programming Bellmans

174 views • 16 slides
Deterministic Global Optimization Algorithm and Nonlinear Dynamics C. S. Adjiman and I.

Deterministic Global Optimization Algorithm and Nonlinear Dynamics C. S. Adjiman and I.

Deterministic Global Optimization Algorithm and Nonlinear Dynamics C. S. Adjiman and I. Papamichail Centre for Process Systems Engineering Department of Chemical Engineering and Chemical Technology Imperial College London Global Optimization

693 views • 41 slides
Response Surface Methods in Optimization A nonlinear Response Surface R(x) is a continuous

Response Surface Methods in Optimization A nonlinear Response Surface R(x) is a continuous

Response Surface Methods in Optimization A nonlinear Response Surface R(x) is a continuous nonlinear multivariate approximation to f(x). R(x) has also been called a response surface or a surrogate model. Response surfaces

331 views • 30 slides
Notes Solving Nonlinear Systems Most thoroughly explored in the context of optimization

Notes Solving Nonlinear Systems Most thoroughly explored in the context of optimization

Notes Solving Nonlinear Systems Most thoroughly explored in the context of optimization For systems arising in implicit time integration of stiff problems: Must be more efficient than taking k substeps of an explicit method ruling

73 views • 4 slides
Nonlinear Classifiers II 2 Nonlinear Classifiers: Introduction Classifiers Supervised

Nonlinear Classifiers II 2 Nonlinear Classifiers: Introduction Classifiers Supervised

1 Nonlinear Classifiers II 2 Nonlinear Classifiers: Introduction Classifiers Supervised Classifiers XOR problem Linear Classifiers Perceptron Least Squares Methods Linear Support Vector Machine Nonlinear

355 views • 20 slides
Optimization based on Mixed Integer Dealing with nonconvexities Global Nonlinear Programming

Optimization based on Mixed Integer Dealing with nonconvexities Global Nonlinear Programming

Claudia DAmbrosio What is MP? What is a MINLP? Subclasses of MINLP Optimization based on Mixed Integer Dealing with nonconvexities Global Nonlinear Programming methods Optimization methods Spatial Branch-and-Bound Expression trees

1.71k views • 144 slides
Robust nonlinear Optimization Maren Mahsereci Workshop on Uncertainty Quantification 09 / 15 /

Robust nonlinear Optimization Maren Mahsereci Workshop on Uncertainty Quantification 09 / 15 /

Robust nonlinear Optimization Maren Mahsereci Workshop on Uncertainty Quantification 09 / 15 / 2016, Sheffield Emmy Noether Group on Probabilistic Numerics Department of Empirical Inference Max Planck Institute for Intelligent Systems

889 views • 55 slides
Pros and Cons IIT Bombay In each case, we get highly nonlinear optimization problem

Pros and Cons IIT Bombay In each case, we get highly nonlinear optimization problem

Automation Lab Pros and Cons IIT Bombay In each case, we get highly nonlinear optimization problem Important to give good initial guess to obtain reliable model parameters: not an easy task It is assumed that structure of the

1.11k views • 87 slides
A FEASIBLE POINT ALGORITHM FOR NONLINEAR CONSTRAINED OPTIMIZATION with Applications to Parameter

A FEASIBLE POINT ALGORITHM FOR NONLINEAR CONSTRAINED OPTIMIZATION with Applications to Parameter

A FEASIBLE POINT ALGORITHM FOR NONLINEAR CONSTRAINED OPTIMIZATION with Applications to Parameter Identification in Structural Mechanics Jos Herskovits 1 1 COPPE - Federal University of Rio de Janeiro, Mechanical Engineering Department Rio de

691 views • 38 slides
Discontinuous Feedback in Nonlinear Control: Stabilization Under Disturbances and Optimization

Discontinuous Feedback in Nonlinear Control: Stabilization Under Disturbances and Optimization

Discontinuous Feedback in Nonlinear Control: Stabilization Under Disturbances and Optimization Yuri S. Ledyaev Western Michigan University ledyaev@wmich.edu +1-(269)-387-4557 DIMACS Workshop on Perspectives and Future Directions in

1.22k views • 119 slides
Metti 5 Optimization for nonlinear parameter estimation and function estimation Lecture 7

Metti 5 Optimization for nonlinear parameter estimation and function estimation Lecture 7

Metti 5 Optimization for nonlinear parameter estimation and function estimation Lecture 7 Roscoff, June 13-18, 2011 Objectives Direct problem input BC Model State solution IC Parameters

869 views • 69 slides
Nonlinear Optimization I Dr. Thomas M. Surowiec Humboldt University of Berlin Department of

Nonlinear Optimization I Dr. Thomas M. Surowiec Humboldt University of Berlin Department of

Introduction Optimality Conditions Nonlinear Optimization I Dr. Thomas M. Surowiec Humboldt University of Berlin Department of Mathematics Summer 2013 Dr. Thomas M. Surowiec BMS Couse NLO, Summer 2013 Introduction Optimality Conditions

537 views • 19 slides
Updating & Improving optim(): Unifying optimization algorithms in R for smooth, nonlinear

Updating & Improving optim(): Unifying optimization algorithms in R for smooth, nonlinear

Updating & Improving optim(): Unifying optimization algorithms in R for smooth, nonlinear problems John C. Nash University of Ottawa, Telfer School of Management Ottawa, Canada Ravi Varadhan The Center on Aging and Health, Johns Hopkins

881 views • 31 slides
NONLINEAR BUCKLING OPTIMIZATION OF LAMINATED COMPOSITES INCLUDING WORST SHAPE IMPERFECTIONS

NONLINEAR BUCKLING OPTIMIZATION OF LAMINATED COMPOSITES INCLUDING WORST SHAPE IMPERFECTIONS

18 TH INTERNATIONAL CONFERENCE ON COMPOSITE MATERIALS NONLINEAR BUCKLING OPTIMIZATION OF LAMINATED COMPOSITES INCLUDING WORST SHAPE IMPERFECTIONS E. Lund*, E. Lindgaard Department of Mechanical and Manufacturing Engineering, Aalborg

327 views • 6 slides

More recommend