Nonlinear Equations Nonlinear Equations The scientist described what is: the engineer creates what never was. Theodor von Karman The father of supersonic flight 1 Fall 2010
Problem Description Problem Description � Given a non-linear equation f ( x )=0, find a x * q f ( ) such that f ( x *) = 0. Thus, x * is a root of f ( x )=0. � Galois theory in math tells us that only polynomials of degree ≤ 4 can be solved with ≤ 4 l i l f d b l d ith close forms using + , − , × , ÷ and taking roots. � General non-linear equations can be solved with � General non-linear equations can be solved with iterative methods. � Basically, we try to guess the location of a root, Basically, we try to guess the location of a root, and approximate it iteratively. � Unfortunately, this process can go wrong, leading to another root or even diverge. 2
Methods Will Be Discussed Methods Will Be Discussed � There are two types of methods, bracketing and There are two types of methods, bracketing and open . The bracketing methods require an interval that is known to contain a root, while interval that is known to contain a root, while the open method does not. � Commonly seen bracketing methods include � Commonly seen bracketing methods include the bisection method and the regula falsi method and the open methods are Newton’s method, and the open methods are Newton s method, the secant method, and the fixed-point iteration . iteration 3
Bisection Method: 1/6 Bisection Method: 1/6 � Idea : The key is the intermediate value theorem . Idea : The key is the intermediate value theorem . � If y = f ( x ) is a continuous function on [ a , b ] and r is between f ( a ) and f ( b ) then there is a x * such is between f ( a ) and f ( b ), then there is a x such that r = f ( x *). � Therefore, if f ( a ) × f ( b ) < 0 and r = 0, there is a if f ( ) × f ( b ) < 0 � Th f d 0 th i root x * of f ( x ) = 0 in [ a , b ]. y=f ( x ) f ( a ) a b X x* f ( b ) f ( b ) 4
Bisection Method: 2/6 Bisection Method: 2/6 � Note that f ( a ) × f ( b ) < 0 guarantees a root in [ a , b ]. f ( ) f ( ) g � Compute c = ( a + b )/2 and f ( c ). � If f ( c ) = 0, we have found a root. f � Otherwise, either f ( a ) × f ( c ) < 0 or f ( b ) × f ( c ) < 0 but not both. Use [ a , c ] for the former and [ c , b ] for the latter until | f ( c )| < ε . th l tt til | f ( )| < 1 2 3 y=f ( x ) f ( a ) f ( a ) 4 4 a b X x* x* f ( b ) 5
Bisection Method: 3/6 Bisection Method: 3/6 � Conver Convergence g ence : Since the first iteration reduces the interval length to | b - a |/2, the second to | b - a |/2 2 , the third to | b - a |/2 3 , etc, after the k -th iteration, the interval length is | b - a |/2 k . � If ε > 0 is a given tolerance value, we have | b - a |/2 k < ε or | b - a |/ ε < 2 k for some k . k � Taking base 2 logarithm, we have the value of k , the expected number of iterations with accuracy ε : − k = ⎡ ⎡ ⎤ ⎤ b a l log 2 k ⎢ ⎥ ε ⎢ ⎥ � Convergence is not fast but guaranteed and steady . g g y 6
Bisection Method: 4/6 Bisection Method: 4/6 � Al Algorithm: g orithm: Fa = f(a) Fa = f(a) � If ABS(b-a) < ε , Fb = f(b) ! F ! Fa* Fb * Fb must must be < 0 be < 0 DO then stop. p IF (ABS(b-a) < ε ) EXIT IF (ABS(b-a) < ε ) EXIT � Note the red framed c = (a+b)/2 IF (c ==a .OR. c == b) EXIT IF statement. Fc = f(c) Fc f(c) Without this, an IF (Fc == 0) EXIT IF (Fa*Fc < 0) THEN infinite loop can b = c occur if | b - a | is very Fb = Fc ELSE small and c can be a a = c or b due to b d t Fa = Fc rounding. END IF END DO 7
Bisection Method: 5/6 Bisection Method: 5/6 � However, there is a catch. ABS(b-a)< ε is an , absolute error and the accuracy of the root is approximately k accurate digits if ε = 10 - k . � What if the actual root is smaller than ε = 10 - k ? In this case, the bisection method returns no accurate digit at all! � Recall that –log 10 (| x - x *|/| x *|) is approximately the number of accurate digits of x . Prove P P rove It! It! I It! � Thus, if the anticipated root is very close to 0, one may change the absolute error to relative error. 8
Bisection Method: 6/6 Bisection Method: 6/6 � The ABS(b-a) < ε may be changed to S( ) y g ABS(b-a)/MIN(ABS(a),ABS(b)) < ε � This test has a potential problem: if the initial � Thi t t h t ti l bl if th i iti l bracket contains 0, MIN(ABS(a),ABS(b)) can approach 0 which would cause a division by h 0 hi h ld di i i b zero! � Nothing is perfect! 9
Bisection Method Example: 1/3 Bisection Method Example: 1/3 � Suppose we wish to find the root closest to 6 of pp the following equation: + + sin( i ( cos( )) ( )) x x = ( ) f x + + 2 1 1 x x � Plotting the equation f ( x ) = 0 around x = 6 is very helpful. very helpful. � You may use gnuplot for plotting. Gnuplot is available free on many platforms such as is available free on many platforms such as Windows, MacOS and Unix/Linux. 10
Bisection Method Example: 2/3 Bisection Method Example: 2/3 � Function f ( x ) has a root in [4,6], and f (4) < 0 and f (6) > 0. We may use [4,6] with the Bisection method. + sin( cos( )) x x = ( ) f x + + 2 1 1 x x 11
Bisection Method Example: 3/3 Bisection Method Example: 3/3 � 19 iterations, x * ≈ 5.5441017… f ( x * ) ≈ 0.863277 × 10 -7 . , f ( ) [5.5,5.5625] + sin( ( cos( )) ( )) x x [5.5,5.625] [5 5 5 625] = ( ) ( ) f f x + 2 1 x [5.5,5.75] [5.5,6] [5,6] [5,6] [4,6] 12
New ton’s Method: 1/13 New ton s Method: 1/13 � Idea Idea : If a is very close to a root of y = f ( x ) , the Idea Idea : If a is very close to a root of y f ( x ) , the tangent line though ( a , f ( a )) intersects the X -axis at a point b that is hopefully closer to the root. at a point b that is hopefully closer to the root. � The line through ( a , f ( a )) with slope f ’( a ) is: − f ( ) ( ) y f a = ' ( ) f a − x a ( a , f ( a )) y = f ( x ) y = f ( x ) a a x* x* b b y - f ( a ) = f ’ ( a ) × ( x - a ) 13
New ton s Method: 2/13 New ton’s Method: 2/13 � The line through ( a , f ( a )) with slope f ’( a ) is The line through ( a , f ( a )) with slope f ( a ) is − ( ) y f a = ' ( ) f a − x x a a � Its intersection point b with the X -axis can be found by setting y to zero and solving for x : ( ) f a = = − − b b a a ' ( ) f a ( a f ( a )) ( a , f ( a )) y = f ( x ) 14 a x* b
New ton’s Method: 3/13 New ton s Method: 3/13 � Starting with a x 0 that is close to a root x *, Starting with a x 0 that is close to a root x , Newton’s method uses the following to compute x 1 , x 2 , … until some x k converges to x *. x 1 , x 2 , … until some x k converges to x . ( ) f x + = − i x x + 1 1 i i i i ' ' ( ) f x i x* x 0 x 1 x 2 x 3 y = f ( x ) 15
New ton s Method: 4/13 New ton’s Method: 4/13 � Conver Convergence 1/2: g ence 1/2: � If there is a constant ρ , 0 < ρ < 1, such that + − * lim | lim | | | x x + = ρ ρ 1 1 k k − * →∞ | | x x k k the sequence x 0 , x 1 , …, is said to converge 0 1 linearly with ratio (or rate ) ρ . � If ρ is zero, the convergence is superlinear . � If x k +1 and x k are close to x *, the above expression means | x k +1 – x *| ≈ ρ | x k – x *| ρ | | *| *| � The error at x k +1 is proportional to ( i.e ., linear) and smaller than the error at x since linear) and smaller than the error at x k since ρ < 1. 16
New ton’s Method: 5/13 New ton s Method: 5/13 � Convergence Convergence 2/2: Convergence Convergence 2/2: 2/2: 2/2: � If there is a p > 1 and a C > 0 such that + − * lim | | | | x x = 1 k C − * p →∞ | | x x k k the sequence is said to converge with order p . � If x k +1 and x k are close to x *, the above If x k +1 and x k are close to x , the above expression yields | x k +1 - x * | ≈ C | x k - x *| p . � Since | x - x *| is close to 0 and p > 1 | x � Since | x k - x | is close to 0 and p > 1, | x k +1 - x | - x *| is even smaller, and has p more 0 digits. 17
New ton’s Method: 6/13 New ton s Method: 6/13 � The right shows a The right shows a x = initial value possible algorithm that Fx = f ( x ) implements Newton’s implements Newton s Dx = f ’( x ) f ( x ) DO method. IF (ABS(Fx) < ε ) EXIT � Newton s method � Newton’s method New_X = x – Fx/Dx Fx = f (New_X) converges with order 2. Dx = f’ (New_X) x = New_X � However, it may not � H it t END DO converge at all, and a maximum number of i b f iterations may be needed t to stop early. t l 18
New ton’s Method: 7/13 New ton s Method: 7/13 � Problem Problem 1/6: Problem Problem 1/6: 1/6: 1/6: � While Newton’s method has a fast convergence rate it may not converge at all no convergence rate, it may not converge at all no matter how close the initial guess is. + = 1/(2 1) n verify this fact yourself y y x x 3 x 1 x 2 x 4 19
Recommend
More recommend
