Warning CS 61A/CS 98-52 FYI: This lecture might get a little... - PDF document

0.8 1.0 0.6 0.4 0.2 0.0 1.0 0.8 0.6 0.4 0.2 0.0 Warning CS 61A/CS 98-52 FYI: This lecture might get a little... intense... and math-y Mehrdad Niknami If it’s hard, don’t panic! It’s okpy! They won’t all be like this! Just try to enjoy it, ask questions, & learn as much as you can. :) University of California, Berkeley Ready?! Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 1 / 25 Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 2 / 25 Preliminaries Brachistochrone Problem Let’s solve a realistic problem. It’s the brachistochrone (“shortest time”) problem: 1 Drop a ball on a ramp Last lecture was on equation-solving : 2 Let it roll down “Given f and initial guess x 0 , solve f ( x ) = 0” 3 What shape minimizes the travel time? This lecture is on optimization : arg min x F ( x ) 0.0 0.5 1.0 1.5 “Given F and initial guess x 0 , find x that minimizes F ( x )” - 0.5 - 1.0 - 1.5 = ⇒ How would you solve this? Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 3 / 25 Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 4 / 25 Brachistochrone Problem Brachistochrone Problem Ideally: Learn fancy math, derive the answer, plug in the formula. Joking aside... Oh, sorry... did you say you’re a programmer ? Most problems don’t have a nice formula, so you’ll need algorithms. 1 Math is hard Let’s get our hands dirty! 2 Physics is hard Remember Riemann sums? 3 We’re lazy This is similar: 1 Chop up the ramp into line 4 Why learn something new when you can burn electricity instead? segments (but hold ends fixed) OK but honestly the math is a little complicated... 2 Move around the anchors to Calculus of variations... Euler-Lagrange differential eqn... maybe? minimize travel time Take Physics 105... have fun! Q: How do you do this? Don’t get wrecked Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 5 / 25 Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 6 / 25 Algorithm Algorithm Newton-Raphson method for optimization : Use Newton-Raphson! 1 Assume F is approximately quadratic 1 (so f = F ′ approx. linear) ...but wasn’t that for finding roots ? Not optimizing? 2 Guess some x 0 intelligently Actually, it’s used for both: 3 Repeatedly solve linear approximation 2 of f = F ′ : If F is differentiable, minimizing F reduces to root-finding : f ( x k ) − f ( x k +1 ) = f ′ ( x k ) ( x k − x k +1 ) F ′ ( x ) = f ( x ) = 0 f ( x k +1 ) = 0 x k +1 = x k − f ′ ( x k ) − 1 f ( x k ) = ⇒ Caveat: must avoid maxima and inflection points We ignored F ! Avoid maxima and inflection points! (How?) Easy in 1-D: only ± directions to check for increase/decrease 4 ...Profit? Good luck in N -D... infinitely many directions 1 Why are quadratics common? Energy/cost are quadratic ( K = 1 2 mv 2 , P = I 2 R ...) 2 You’ll see linearization ALL the time in engineering Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 7 / 25 Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 8 / 25

Algorithm Algorithm Coordinate descent: Wait, but we have a function of many variables. What do? 1 Take x 1 , use it to minimize F , holding others fixed A couple options: 2 Take y 1 , use it to minimize F , holding others fixed 1 Fully multivariate Newton-Raphson: 3 Take x 2 , use it to minimize F , holding others fixed x k − � ∇ � x k ) − 1 � � x k +1 = � f ( � f ( � x k ) 4 Take y 2 , use it to minimize F , holding others fixed 5 . . . Taught in EE 219A, 227C, 144/244, etc... (need Math 53 and 54) 6 Cycle through again 2 Newton coordinate-descent Doesn’t work as often, but it works very well here. Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 9 / 25 Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 10 / 25 Algorithm Algorithm Computing derivatives numerically: Newton step for minimization : def derivative(f, coords, i, j, h): def newton_minimizer_step(F, coords, h): x = coords[i][j] delta = 0.0 coords[i][j] = x + h; f2 = f(coords) for i in range(1, len(coords) - 1): coords[i][j] = x - h; f1 = f(coords) for j in range(len(coords[i])): coords[i][j] = x def f(c): return derivative(F, c, i, j, h) return (f2 - f1) / (2 * h) def df(c): return derivative(f, c, i, j, h) step = -f(coords) / df(coords) Why not (f(x + h) - f(x)) / h ? delta += abs(step) Breaking the intrinsic asymmetry reduces accuracy coords[i][j] += step return delta ∼ Words of Wisdom ∼ Side note: Notice a potential bug? What’s the fix? If your problem has { fundamental feature } that your solution doesn’t, Notice a 33% inefficiency? What’s the fix? you’ve created more problems. Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 11 / 25 Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 12 / 25 Algorithm Algorithm What is our objective function F to minimize? Let’s define quadratic roots ... def falling_time(coords): # coords = [[x1,y1], [x2,y2], ...] t, speed = 0.0, 0.0 def quadratic_roots(two_a, b, c): prev = None D = b * b - 2 * two_a * c for coord in coords: if D >= 0: if prev != None: if D > 0: dy = coord[1] - prev[1] r = D ** 0.5 d = ((coord[0] - prev[0]) ** 2 + dy ** 2) ** 0.5 roots = [(-b + r) / two_a, (-b - r) / two_a] accel = -9.80665 * dy / d else: for dt in quadratic_roots(accel, speed, -d): roots = [-b / two_a] if dt > 0: else: speed += accel * dt roots = [] t += dt return roots prev = coord return t Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 13 / 25 Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 14 / 25 Algorithm Algorithm Aaaaaand put it all together def main(n=6): (y1, y2) = (1.0, 0.0) (x1, x2) = (0.0, 1.0) coords = [ # initial guess: straight line [x1 + (x2 - x1) * i / n, (Demo) y1 + (y2 - y1) * i / n] for i in range(n + 1) ] f = falling_time h = 0.00001 while newton_minimizer_step(f, coords, h) > 0.01: print(coords) if __name__ == '__main__': main() Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 15 / 25 Mehrdad Niknami (UC Berkeley) CS 61A/CS 98-52 16 / 25