Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Hack the Derivative! Erik Taubeneck GameChanger Media - Software Engineer and Data Maven We’re hiring! https://gc.com/about/careers July 25th, 2015
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Outline Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Try It Yourself The functions used through out this presentations can be installed with pip! $ mkdir somewhere_new $ cd somewhere_new $ virtualenv venv $ source venv/bin/activate $ pip install hackthederivative $ python >>> from hackthederivative import *
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Taking the Derivative Let f ( x ) be a function from R → X ⊆ R . e.g. • f ( x ) = x 2 • g ( x ) = sin ( x ) • h ( x ) = 5 x 3 − 6 x + 1 1 x 1 Note that h ( x ) is not defined at x = 0.
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Taking the Derivative The derivative of f ( x ), f ′ ( x ), is a new function which tells us the slope of the tangent line that intersects f at any given x .
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Taking the Derivative Formally, the derivative of f ( x ) at x is f ( x + h ) − f ( x ) f ′ ( x ) = lim h h → 0
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Taking the Derivative E x : Let f ( x ) = x 2 .
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Taking the Derivative E x : Let f ( x ) = x 2 . Then ( x + h ) 2 − x 2 f ′ ( x ) = lim h h → 0
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Taking the Derivative E x : Let f ( x ) = x 2 . Then ( x + h ) 2 − x 2 f ′ ( x ) = lim h h → 0 x 2 + 2 xh + h 2 − x 2 = lim h h → 0
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Taking the Derivative E x : Let f ( x ) = x 2 . Then ( x + h ) 2 − x 2 f ′ ( x ) = lim h h → 0 x 2 + 2 xh + h 2 − x 2 = lim h h → 0 2 xh + h 2 = lim h h → 0
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Taking the Derivative E x : Let f ( x ) = x 2 . Then ( x + h ) 2 − x 2 f ′ ( x ) = lim h h → 0 x 2 + 2 xh + h 2 − x 2 = lim h h → 0 2 xh + h 2 = lim h h → 0 = lim h → 0 2 x + h
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Taking the Derivative E x : Let f ( x ) = x 2 . Then ( x + h ) 2 − x 2 f ′ ( x ) = lim h h → 0 x 2 + 2 xh + h 2 − x 2 = lim h h → 0 2 xh + h 2 = lim h h → 0 = lim h → 0 2 x + h = 2 x �
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Finite Difference Suppose we wish to find the derivative for some function f ( x ) a x 0 .
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Finite Difference Suppose we wish to find the derivative for some function f ( x ) a x 0 . One approximation is the finite difference method.
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Finite Difference Suppose we wish to find the derivative for some function f ( x ) a x 0 . One approximation is the finite difference method. Recall f ( x + h ) − f ( x ) f ′ ( x ) = lim h h → 0
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Finite Difference Suppose we wish to find the derivative for some function f ( x ) a x 0 . One approximation is the finite difference method. Recall f ( x + h ) − f ( x ) f ′ ( x ) = lim h h → 0 We can estimate f ′ ( x o ) ≈ f ( x 0 + h ) − f ( x 0 ) h for some small h .
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Finite Difference Implemented in Python In [1]: def finite_difference(f, x, h): return (f(x+h) - f(x))/h
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Finite Difference Implemented in Python In [1]: def finite_difference(f, x, h): return (f(x+h) - f(x))/h In [2]: f = lambda x: x**2 In [2]: finite_difference(f, 1.0, 0.00001) Out [3]: 2.00001000001393
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Finite Difference But how do we choose h ?
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Finite Difference But how do we choose h ? The limit as h → 0 approaches the derivative, so the smaller the better! In [4]: import sys In [5]: h = sys.float_info.min
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Finite Difference But how do we choose h ? The limit as h → 0 approaches the derivative, so the smaller the better! In [4]: import sys In [5]: h = sys.float_info.min In [6]: print finite_difference(f, 1.0, h) Out [6]: 0.0 Uh oh.
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Finite Difference Hmm, 0 . 00001 seems pretty small, and worked well earlier. Maybe that will just work.
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Finite Difference Hmm, 0 . 00001 seems pretty small, and worked well earlier. Maybe that will just work. In [7]: finite_difference(f, 1.0e20, 0.00001) Out [7]: 0.0 Nope, that can break too. To figure out why, let’s look closer at Floating Point numbers.
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Floating Point Numbers A Float64, or a double-precision floating-point number, consists of and represents the real number x ∈ R with sign , e , and b i such that ( − 1) sign (1 . b 51 b 50 ... b 0 ) 2 2 e − 1023 or (possibly more readable) � 52 � � ( − 1) sign b 52 − i 2 − i × 2 e − 1023 1 + i =1
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Floating Point Numbers This means that the floating point numbers have gaps!
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Floating Point Gaps In [1]: import sys In [2]: plus_epsilon_identity(x, eps): return x + eps == x In [3]: eps = sys.float_info.min
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Floating Point Gaps In [1]: import sys In [2]: plus_epsilon_identity(x, eps): return x + eps == x In [3]: eps = sys.float_info.min In [4]: plus_epsilon_identity(0.0, eps) Out [4]: False In [5]: plus_epsilon_identity(1.0, eps) Out [5]: True
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Floating Point Gaps In [1]: import sys In [2]: plus_epsilon_identity(x, eps): return x + eps == x In [3]: eps = sys.float_info.min In [4]: plus_epsilon_identity(0.0, eps) Out [4]: False In [5]: plus_epsilon_identity(1.0, eps) Out [5]: True In [6]: plus_epsilon_identity(1.0e20, 1.0) Out [6]: True
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Problems with the Derivative If we choose h that is so small such that float ( x ) + float ( h ) = float ( x )
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Problems with the Derivative If we choose h that is so small such that float ( x ) + float ( h ) = float ( x ) then f ( x + h ) − f ( x ) = f ( x ) − f ( x ) = 0 h = 0 h h
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Choosing Epsilon In [1]: def eps(x): e = float(max(sys.float_info.min, abs(x))) while not plus_epsilon_identity(x, e): last = e e = e / 2. return last
Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative Choosing Epsilon In [1]: def eps(x): e = float(max(sys.float_info.min, abs(x))) while not plus_epsilon_identity(x, e): last = e e = e / 2. return last In [2]: eps(1.0) Out[2]: 2.220446049250313e-16 In [3]: eps(1.0e10) Out[3]: 1.1102230246251565e-06
Recommend
More recommend
