Line Search 2 Lecture 4 ME EN 575 Andrew Ning aning@byu.edu - PDF document

Line Search 2 Lecture 4 ME EN 575 Andrew Ning aning@byu.edu Outline Root Finding Methods 1D Optimization Methods

Root Finding Methods Root Finding Methods How do we know when we have reached a local minimum?

Bisection Example: Refrigeration Tank Minimize the cost of a cylindrical refrigeration tank with a volume of 50 m 3 . • Circular ends cost $10 per m 2 • Cylindrical walls cost $6 per m 2 • Refrigerator costs $80 per m 2 over its life

45 πd 2 + 17200 minimize d d with respect to d ≥ 0 subject to Newton’s Method We can do better by using gradient information

1 x f ′′ ( x k ) x k +1 = x k − f ′ ( x k ) 2 3 f ( x )

f ( x ) f ( x ) 1 2 3 1 x x 2 Brent’s Method

1D Optimization Methods Golden Section Search

Extreme 1: bisection-like Extreme 2: small improvement

I1 I2 I2 I 2 = τI 1 I1 I2 I2 I3 I3 I 3 = τI 2

I 1 = I 2 + I 3 Polynomial Methods Approximate function locally as a polynomial (in this case quadratic): f = 1 2 ax 2 + bx + c ˜ If a > 0 , the minimum of this function is x ∗ = − b/a .

Brent’s Method Combines quadratic polynomial method with golden section search. Where are we going? Optimization • Nelder-Mead Simplex Smooth Non-smooth • Genetic Algorithms 1-D Optimization N-D Optimization • Bisection Search Unconstrained Constrained • Fibonnacci Search • Golden Section Search • Newton ’ s Method • Lagrange Multipliers Line-search Methods Trust-region Methods • Polynomial Interpolation • Steepest Descent • Polynomial Fits • Exterior Penalty Methods • Brent ’ s Method • Conjugate Gradient • Nonparametric Fits • Interior Point Methods • Newton ’ s Method • SQP • Quasi-Newton Methods

Intuition in Higher Dimensions Consider a hypersphere inscribed inside a hypercube volume of sphere ? volume of cube 1.0 0.8 V sphere /V cube 0.6 0.4 0.2 0.0 1 2 3 4 5 6 7 8 9 dimension