Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U First, suppose that Gaussian elimination can be performed on the system A x = b without row interchanges. With the notation used earlier, this is equivalent to having nonzero pivot elements a ( i ) ii , for each i = 1 , 2 , . . . , n . Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 8 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U First, suppose that Gaussian elimination can be performed on the system A x = b without row interchanges. With the notation used earlier, this is equivalent to having nonzero pivot elements a ( i ) ii , for each i = 1 , 2 , . . . , n . The first step in the Gaussian elimination process consists of performing, for each j = 2 , 3 , . . . , n , the operations a ( 1 ) j 1 ( E j − m j , 1 E 1 ) → ( E j ) , m j , 1 = where a ( 1 ) 11 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 8 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U First, suppose that Gaussian elimination can be performed on the system A x = b without row interchanges. With the notation used earlier, this is equivalent to having nonzero pivot elements a ( i ) ii , for each i = 1 , 2 , . . . , n . The first step in the Gaussian elimination process consists of performing, for each j = 2 , 3 , . . . , n , the operations a ( 1 ) j 1 ( E j − m j , 1 E 1 ) → ( E j ) , m j , 1 = where a ( 1 ) 11 These operations transform the system into one in which all the entries in the first column below the diagonal are zero. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 8 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: Constructing L & U (Cont’d) The system of operations in a ( 1 ) j 1 ( E j − m j , 1 E 1 ) → ( E j ) , m j , 1 = where a ( 1 ) 11 can be viewed in another way. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 9 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: Constructing L & U (Cont’d) The system of operations in a ( 1 ) j 1 ( E j − m j , 1 E 1 ) → ( E j ) , m j , 1 = where a ( 1 ) 11 can be viewed in another way. It is simultaneously accomplished by multiplying the original matrix A on the left by the matrix 1 0 0 · · · · · · . ... . − m 21 1 . M ( 1 ) = . . ... ... . . . 0 . . . ... ... . . . . 0 − m n 1 0 0 1 · · · Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 9 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: Constructing L & U (Cont’d) The system of operations in a ( 1 ) j 1 ( E j − m j , 1 E 1 ) → ( E j ) , m j , 1 = where a ( 1 ) 11 can be viewed in another way. It is simultaneously accomplished by multiplying the original matrix A on the left by the matrix 1 0 0 · · · · · · . ... . − m 21 1 . M ( 1 ) = . . ... ... . . . 0 . . . ... ... . . . . 0 − m n 1 0 0 1 · · · This is called the first Gaussian transformation matrix. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 9 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) We denote the product of this matrix with A ( 1 ) ≡ A by A ( 2 ) and with b by b ( 2 ) , Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 10 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) We denote the product of this matrix with A ( 1 ) ≡ A by A ( 2 ) and with b by b ( 2 ) , so A ( 2 ) x = M ( 1 ) A x = M ( 1 ) b = b ( 2 ) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 10 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) We denote the product of this matrix with A ( 1 ) ≡ A by A ( 2 ) and with b by b ( 2 ) , so A ( 2 ) x = M ( 1 ) A x = M ( 1 ) b = b ( 2 ) In a similar manner we construct M ( 2 ) , the identity matrix with the entries below the diagonal in the second column replaced by the negatives of the multipliers a ( 2 ) j 2 m j , 2 = . a ( 2 ) 22 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 10 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) The product of M ( 2 ) with A ( 2 ) has zeros below the diagonal in the first two columns, Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 11 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) The product of M ( 2 ) with A ( 2 ) has zeros below the diagonal in the first two columns, and we let A ( 3 ) x = M ( 2 ) A ( 2 ) x = M ( 2 ) M ( 1 ) A x = M ( 2 ) M ( 1 ) b = b ( 3 ) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 11 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) In general, with A ( k ) x = b ( k ) already formed, Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 12 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) In general, with A ( k ) x = b ( k ) already formed, multiply by the k th Gaussian transformation matrix 1 0 · · · · · · · · · · · · · · · 0 . ... . 0 1 . . . ... ... ... . . . . . . ... ... M ( k ) = . . . 0 . . . . ... ... . . . − m k + 1 , k . . . . . . ... ... ... . . . . . . 0 . . − m n , k . · · · 0 0 · · · 0 1 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 12 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) to obtain A ( k + 1 ) x M ( k ) A ( k ) x = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 13 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) to obtain A ( k + 1 ) x M ( k ) A ( k ) x = M ( k ) · · · M ( 1 ) A x = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 13 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) to obtain A ( k + 1 ) x M ( k ) A ( k ) x = M ( k ) · · · M ( 1 ) A x = M ( k ) b ( k ) = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 13 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) to obtain A ( k + 1 ) x M ( k ) A ( k ) x = M ( k ) · · · M ( 1 ) A x = M ( k ) b ( k ) = b ( k + 1 ) = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 13 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) to obtain A ( k + 1 ) x M ( k ) A ( k ) x = M ( k ) · · · M ( 1 ) A x = M ( k ) b ( k ) = b ( k + 1 ) = M ( k ) · · · M ( 1 ) b = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 13 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) The process ends with the formation of A ( n ) x = b ( n ) , where A ( n ) is the upper triangular matrix a ( 1 ) a ( 1 ) a ( 1 ) · · · · · · 1 n 11 12 . ... a ( 2 ) . 0 . 22 . . A ( n ) = ... ... ... . . . . . ... ... a ( n − 1 ) . . n − 1 , n a ( n ) 0 · · · · · · 0 n , n Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 14 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) The process ends with the formation of A ( n ) x = b ( n ) , where A ( n ) is the upper triangular matrix a ( 1 ) a ( 1 ) a ( 1 ) · · · · · · 1 n 11 12 . ... a ( 2 ) . 0 . 22 . . A ( n ) = ... ... ... . . . . . ... ... a ( n − 1 ) . . n − 1 , n a ( n ) 0 · · · · · · 0 n , n given by A ( n ) = M ( n − 1 ) M ( n − 2 ) · · · M ( 1 ) A Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 14 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) This process forms the U = A ( n ) portion of the matrix factorization A = LU . Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 15 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) This process forms the U = A ( n ) portion of the matrix factorization A = LU . To determine the complementary lower triangular matrix L , Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 15 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) This process forms the U = A ( n ) portion of the matrix factorization A = LU . To determine the complementary lower triangular matrix L , first recall the multiplication of A ( k ) x = b ( k ) by the Gaussian transformation of M ( k ) used to obtain: A ( k + 1 ) x = M ( k ) A ( k ) x = M ( k ) b ( k ) = b ( k + 1 ) , Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 15 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) This process forms the U = A ( n ) portion of the matrix factorization A = LU . To determine the complementary lower triangular matrix L , first recall the multiplication of A ( k ) x = b ( k ) by the Gaussian transformation of M ( k ) used to obtain: A ( k + 1 ) x = M ( k ) A ( k ) x = M ( k ) b ( k ) = b ( k + 1 ) , where M ( k ) generates the row operations ( E j − m j , k E k ) → ( E j ) , for j = k + 1 , . . . , n . Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 15 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) To reverse the effects of this transformation and return to A ( k ) requires that the operations ( E j + m j , k E k ) → ( E j ) be performed for each j = k + 1 , . . . , n . Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 16 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) To reverse the effects of this transformation and return to A ( k ) requires that the operations ( E j + m j , k E k ) → ( E j ) be performed for each M ( k ) � − 1 : j = k + 1 , . . . , n . This is equivalent to multiplying by � 1 0 · · · · · · · · · · · · · · · 0 . ... . 0 1 . . . ... ... ... . . . . M ( k ) � − 1 . . ... ... L ( k ) = � . . = . 0 . . . . ... ... . . m k + 1 , k . . . . . . . ... ... ... . . . . . . 0 . . m n , k . · · · 0 0 · · · 0 1 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 16 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) The lower-triangular matrix L in the factorization of A , then, is the product of the matrices L ( k ) : 1 0 · · · · · · 0 . ... . m 21 1 . L = L ( 1 ) L ( 2 ) · · · L ( n − 1 ) = . . ... ... ... . . . . . . ... ... . . . . 0 m n 1 m n , n − 1 1 · · · · · · Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 17 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) The lower-triangular matrix L in the factorization of A , then, is the product of the matrices L ( k ) : 1 0 · · · · · · 0 . ... . m 21 1 . L = L ( 1 ) L ( 2 ) · · · L ( n − 1 ) = . . ... ... ... . . . . . . ... ... . . . . 0 m n 1 m n , n − 1 1 · · · · · · since the product of L with the upper-triangular matrix U = M ( n − 1 ) · · · M ( 2 ) M ( 1 ) A gives Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 17 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) L ( 1 ) L ( 2 ) · · · L ( n − 3 ) L ( n − 2 ) L ( n − 1 ) LU = · M ( n − 1 ) M ( n − 2 ) M ( n − 3 ) · · · M ( 2 ) M ( 1 ) A Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 18 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) L ( 1 ) L ( 2 ) · · · L ( n − 3 ) L ( n − 2 ) L ( n − 1 ) LU = · M ( n − 1 ) M ( n − 2 ) M ( n − 3 ) · · · M ( 2 ) M ( 1 ) A [ M ( 1 ) ] − 1 [ M ( 2 ) ] − 1 · · · [ M ( n − 2 ) ] − 1 [ M ( n − 1 ) ] − 1 = · M ( n − 1 ) M ( n − 2 ) · · · M ( 2 ) M ( 1 ) A Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 18 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) L ( 1 ) L ( 2 ) · · · L ( n − 3 ) L ( n − 2 ) L ( n − 1 ) LU = · M ( n − 1 ) M ( n − 2 ) M ( n − 3 ) · · · M ( 2 ) M ( 1 ) A [ M ( 1 ) ] − 1 [ M ( 2 ) ] − 1 · · · [ M ( n − 2 ) ] − 1 [ M ( n − 1 ) ] − 1 = · M ( n − 1 ) M ( n − 2 ) · · · M ( 2 ) M ( 1 ) A A = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 18 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Constructing L & U (Cont’d) L ( 1 ) L ( 2 ) · · · L ( n − 3 ) L ( n − 2 ) L ( n − 1 ) LU = · M ( n − 1 ) M ( n − 2 ) M ( n − 3 ) · · · M ( 2 ) M ( 1 ) A [ M ( 1 ) ] − 1 [ M ( 2 ) ] − 1 · · · [ M ( n − 2 ) ] − 1 [ M ( n − 1 ) ] − 1 = · M ( n − 1 ) M ( n − 2 ) · · · M ( 2 ) M ( 1 ) A A = We now state a theorem which follows from these observations. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 18 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Theorem If Gaussian elimination can be performed on the linear system A x = b without row interchanges, Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 19 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Theorem If Gaussian elimination can be performed on the linear system A x = b without row interchanges, then the matrix A can be factored into the product of a lower-triangular matrix L and an upper-triangular matrix U , Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 19 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Theorem If Gaussian elimination can be performed on the linear system A x = b without row interchanges, then the matrix A can be factored into the product of a lower-triangular matrix L and an upper-triangular matrix U , that is, A = LU , where m ji = a ( i ) ji / a ( i ) ii , a ( 1 ) a ( 1 ) a ( 1 ) · · · · · · 1 0 · · · · · · 0 1 n 11 12 . . ... ... . . a ( 2 ) m 21 0 . 1 . 22 . . . . U = ... ... ... L = ... ... ... . . . . . . . . . ... ... . . ... ... a ( n − 1 ) . . . . . . 0 n − 1 , n m n 1 m n , n − 1 a ( n ) · · · · · · 1 0 · · · · · · 0 n , n Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 19 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Outline Computation Cost Rationale & Basic Solution Strategy 1 Constructing the Matrix Factorization 2 Example: LU Factorization of a 4 × 4 Matrix 3 The LU Factorization Algorithm 4 5 Permutation Matrices for Row Interchanges Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 20 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Example (a) Determine the LU factorization for matrix A in the linear system A x = b , where 1 1 0 3 1 2 1 − 1 1 1 A = b = and 3 − 1 − 1 2 − 3 − 1 2 3 − 1 4 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 21 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Example (a) Determine the LU factorization for matrix A in the linear system A x = b , where 1 1 0 3 1 2 1 − 1 1 1 A = b = and 3 − 1 − 1 2 − 3 − 1 2 3 − 1 4 (b) Then use the factorization to solve the system x 1 + x 2 + 3 x 4 = 8 2 x 1 + x 2 − x 3 + x 4 = 7 3 x 1 − x 2 − x 3 + 2 x 4 = 14 − x 1 + 2 x 2 + 3 x 3 − x 4 = − 7 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 21 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: 4 × 4 Example Part (a) Solution (1/2) The original system was considered under Gaussian Elimination Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 22 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: 4 × 4 Example Part (a) Solution (1/2) The original system was considered under Gaussian Elimination where we saw that the sequence of operations ( E 2 − 2 E 1 ) → ( E 2 ) ( E 3 − 3 E 1 ) → ( E 3 ) ( E 4 − ( − 1 ) E 1 ) → ( E 4 ) ( E 3 − 4 E 2 ) → ( E 3 ) ( E 4 − ( − 3 ) E 2 ) → ( E 4 ) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 22 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: 4 × 4 Example Part (a) Solution (1/2) The original system was considered under Gaussian Elimination where we saw that the sequence of operations ( E 2 − 2 E 1 ) → ( E 2 ) ( E 3 − 3 E 1 ) → ( E 3 ) ( E 4 − ( − 1 ) E 1 ) → ( E 4 ) ( E 3 − 4 E 2 ) → ( E 3 ) ( E 4 − ( − 3 ) E 2 ) → ( E 4 ) converts the system to the triangular system x 1 + x 2 + 3 x 4 = 4 − x 2 − x 3 − 5 x 4 = − 7 3 x 3 + 13 x 4 = 13 − 13 x 4 = − 13 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 22 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: 4 × 4 Example Part (a) Solution (2/2) The multipliers m ij and the upper triangular matrix produce the factorization 1 1 0 3 2 1 − 1 1 A = 3 − 1 − 1 2 − 1 2 3 − 1 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 23 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: 4 × 4 Example Part (a) Solution (2/2) The multipliers m ij and the upper triangular matrix produce the factorization 1 1 0 3 2 1 − 1 1 A = 3 − 1 − 1 2 − 1 2 3 − 1 1 0 0 0 1 1 0 3 2 1 0 0 0 − 1 − 1 − 5 = 3 4 1 0 0 0 3 13 − 1 − 3 0 1 0 0 0 − 13 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 23 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: 4 × 4 Example Part (a) Solution (2/2) The multipliers m ij and the upper triangular matrix produce the factorization 1 1 0 3 2 1 − 1 1 A = 3 − 1 − 1 2 − 1 2 3 − 1 1 0 0 0 1 1 0 3 2 1 0 0 0 − 1 − 1 − 5 = 3 4 1 0 0 0 3 13 − 1 − 3 0 1 0 0 0 − 13 LU = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 23 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: 4 × 4 Example Part (b) Solution (1/3) To solve x 1 1 0 0 0 1 1 0 3 x 2 2 1 0 0 0 − 1 − 1 − 5 A x = LU x = x 3 3 4 1 0 0 0 3 13 x 4 − 1 − 3 0 1 0 0 0 − 13 8 7 = 14 − 7 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 24 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: 4 × 4 Example Part (b) Solution (1/3) To solve x 1 1 0 0 0 1 1 0 3 x 2 2 1 0 0 0 − 1 − 1 − 5 A x = LU x = x 3 3 4 1 0 0 0 3 13 x 4 − 1 − 3 0 1 0 0 0 − 13 8 7 = 14 − 7 we first introduce the substitution y = U x . Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 24 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: 4 × 4 Example Part (b) Solution (1/3) To solve x 1 1 0 0 0 1 1 0 3 x 2 2 1 0 0 0 − 1 − 1 − 5 A x = LU x = x 3 3 4 1 0 0 0 3 13 x 4 − 1 − 3 0 1 0 0 0 − 13 8 7 = 14 − 7 we first introduce the substitution y = U x . Then b = L ( U x ) = L y . Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 24 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: 4 × 4 Example Part (b) Solution (2/3) First, solve L y = b (where y = U x : y 1 1 0 0 0 8 y 2 2 1 0 0 7 L y = = . y 3 3 4 1 0 14 y 4 − 1 − 3 0 1 − 7 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 25 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: 4 × 4 Example Part (b) Solution (2/3) First, solve L y = b (where y = U x : y 1 1 0 0 0 8 y 2 2 1 0 0 7 L y = = . y 3 3 4 1 0 14 y 4 − 1 − 3 0 1 − 7 This system is solved for y by a simple forward-substitution process: y 1 = 8 2 y 1 + y 2 = 7 ⇒ y 2 = 7 − 2 y 1 = − 9 3 y 1 + 4 y 2 + y 3 = 14 ⇒ y 3 = 14 − 3 y 1 − 4 y 2 = 26 − y 1 − 3 y 2 + y 4 = − 7 ⇒ y 4 = − 7 + y 1 + 3 y 2 = − 26 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 25 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: 4 × 4 Example Part (b) Solution (3/3) We then solve U x = y for x , the solution of the original system; Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 26 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: 4 × 4 Example Part (b) Solution (3/3) We then solve U x = y for x , the solution of the original system; that is, x 1 1 1 0 3 8 x 2 0 − 1 − 1 − 5 − 9 = x 3 0 0 3 13 26 x 4 0 0 0 − 13 − 26 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 26 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: 4 × 4 Example Part (b) Solution (3/3) We then solve U x = y for x , the solution of the original system; that is, x 1 1 1 0 3 8 x 2 0 − 1 − 1 − 5 − 9 = x 3 0 0 3 13 26 x 4 0 0 0 − 13 − 26 Using backward substitution we obtain x 4 = 2, x 3 = 0, x 2 = − 1, x 1 = 3. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 26 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Outline Computation Cost Rationale & Basic Solution Strategy 1 Constructing the Matrix Factorization 2 Example: LU Factorization of a 4 × 4 Matrix 3 The LU Factorization Algorithm 4 5 Permutation Matrices for Row Interchanges Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 27 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices LU Factorization Algorithm (1/3) To factor the n × n matrix A = [ a ij ] into the product of the lower-triangular matrix L = [ l ij ] and the upper-triangular matrix U = [ u ij ] ; that is, A = LU , where the main diagonal of either L or U consists of all ones: Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 28 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices LU Factorization Algorithm (1/3) To factor the n × n matrix A = [ a ij ] into the product of the lower-triangular matrix L = [ l ij ] and the upper-triangular matrix U = [ u ij ] ; that is, A = LU , where the main diagonal of either L or U consists of all ones: dimension n ; the entries a ij , 1 ≤ i , j ≤ n of A ; INPUT the diagonal l 11 = · · · = l nn = 1 of L or the diagonal u 11 = · · · = u nn = 1 of U . Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 28 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices LU Factorization Algorithm (1/3) To factor the n × n matrix A = [ a ij ] into the product of the lower-triangular matrix L = [ l ij ] and the upper-triangular matrix U = [ u ij ] ; that is, A = LU , where the main diagonal of either L or U consists of all ones: dimension n ; the entries a ij , 1 ≤ i , j ≤ n of A ; INPUT the diagonal l 11 = · · · = l nn = 1 of L or the diagonal u 11 = · · · = u nn = 1 of U . the entries l ij , 1 ≤ j ≤ i , 1 ≤ i ≤ n of L and the entries, OUTPUT u ij , i ≤ j ≤ n , 1 ≤ i ≤ n of U . Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 28 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices LU Factorization Algorithm (2/3) Select l 11 and u 11 satisfying l 11 u 11 = a 11 Step 1 If l 11 u 11 = 0 then OUTPUT (‘Factorization impossible’) STOP For j = 2 , . . . , n set u 1 j = a 1 j / l 11 ( First row of U ) Step 2 l j 1 = a j 1 / u 11 ( First column of L ) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 29 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices LU Factorization Algorithm (2/3) Select l 11 and u 11 satisfying l 11 u 11 = a 11 Step 1 If l 11 u 11 = 0 then OUTPUT (‘Factorization impossible’) STOP For j = 2 , . . . , n set u 1 j = a 1 j / l 11 ( First row of U ) Step 2 l j 1 = a j 1 / u 11 ( First column of L ) For i = 2 , . . . , n − 1 do Steps 4 and 5: Step 3 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 29 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices LU Factorization Algorithm (2/3) Select l 11 and u 11 satisfying l 11 u 11 = a 11 Step 1 If l 11 u 11 = 0 then OUTPUT (‘Factorization impossible’) STOP For j = 2 , . . . , n set u 1 j = a 1 j / l 11 ( First row of U ) Step 2 l j 1 = a j 1 / u 11 ( First column of L ) For i = 2 , . . . , n − 1 do Steps 4 and 5: Step 3 Select l ii and u ii satisfying l ii u ii = a ii − � i − 1 k = 1 l ik u ki Step 4 If l ii u ii = 0 then OUTPUT (‘Factorization impossible’) STOP Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 29 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices LU Factorization Algorithm (2/3) Select l 11 and u 11 satisfying l 11 u 11 = a 11 Step 1 If l 11 u 11 = 0 then OUTPUT (‘Factorization impossible’) STOP For j = 2 , . . . , n set u 1 j = a 1 j / l 11 ( First row of U ) Step 2 l j 1 = a j 1 / u 11 ( First column of L ) For i = 2 , . . . , n − 1 do Steps 4 and 5: Step 3 Select l ii and u ii satisfying l ii u ii = a ii − � i − 1 k = 1 l ik u ki Step 4 If l ii u ii = 0 then OUTPUT (‘Factorization impossible’) STOP For j = i + 1 , . . . , n Step 5 a ij − � i − 1 � � set u ij = 1 k = 1 l ik u kj ( ith row of U ) l ii a ji − � i − 1 � � l ji = 1 k = 1 l jk u ki ( ith column of L ) u ii Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 29 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices LU Factorization Algorithm (3/3) Select l nn and u nn satisfying l nn u nn = a nn − � n − 1 k = 1 l nk u kn Step 6 ( Note: If l nn u nn = 0 , then A = LU but A is singular ) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 30 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices LU Factorization Algorithm (3/3) Select l nn and u nn satisfying l nn u nn = a nn − � n − 1 k = 1 l nk u kn Step 6 ( Note: If l nn u nn = 0 , then A = LU but A is singular ) OUTPUT ( l ij for j = 1 , . . . , i and i = 1 , . . . , n ) Step 7 OUTPUT ( u ij for j = i , . . . , n and i = 1 , . . . , n ) STOP Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 30 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Using the LU Factorization to solve A x = b Once the matrix factorization is complete, Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 31 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Using the LU Factorization to solve A x = b Once the matrix factorization is complete, the solution to a linear system of the form A x = LU x = b Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 31 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Using the LU Factorization to solve A x = b Once the matrix factorization is complete, the solution to a linear system of the form A x = LU x = b is found by first letting y = U x Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 31 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Using the LU Factorization to solve A x = b Once the matrix factorization is complete, the solution to a linear system of the form A x = LU x = b is found by first letting y = U x and solving L y = b for y . Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 31 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Using the LU Factorization (Cont’d) Since L is lower triangular, we have y 1 = b 1 l 11 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 32 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Using the LU Factorization (Cont’d) Since L is lower triangular, we have y 1 = b 1 and, for each l 11 i = 2 , 3 , . . . , n , i − 1 y i = 1 b i − l ij y j � l ii j = 1 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 32 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization Using the LU Factorization (Cont’d) Since L is lower triangular, we have y 1 = b 1 and, for each l 11 i = 2 , 3 , . . . , n , i − 1 y i = 1 b i − l ij y j � l ii j = 1 After y is found by this forward-substitution process, the upper-triangular system U x = y is solved for x by backward substitution using the equations n x n = y n x i = 1 y i − u ij x j � and u nn u ii j = i + 1 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 32 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Outline Computation Cost Rationale & Basic Solution Strategy 1 Constructing the Matrix Factorization 2 Example: LU Factorization of a 4 × 4 Matrix 3 The LU Factorization Algorithm 4 5 Permutation Matrices for Row Interchanges Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 33 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: Permutation Matrices Limitations of the LU Factorization Algorithm We assumed that A x = b can be solved using Gaussian elimination without row interchanges. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 34 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: Permutation Matrices Limitations of the LU Factorization Algorithm We assumed that A x = b can be solved using Gaussian elimination without row interchanges. From a practical standpoint, this factorization is useful only when row interchanges are not required to control round-off error. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 34 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: Permutation Matrices Limitations of the LU Factorization Algorithm We assumed that A x = b can be solved using Gaussian elimination without row interchanges. From a practical standpoint, this factorization is useful only when row interchanges are not required to control round-off error. We will now consider the modifications that must be made when row interchanges are required. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 34 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: Permutation Matrices We begin with the introduction of a class of matrices that are used to rearrange, or permute, rows of a given matrix. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 35 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: Permutation Matrices We begin with the introduction of a class of matrices that are used to rearrange, or permute, rows of a given matrix. Permutation Matrix An n × n permutation matrix P = [ p ij ] is a matrix obtained by rearranging the rows of I n , the identity matrix. This gives a matrix with precisely one nonzero entry in each row and in each column, and each nonzero entry is a 1. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 35 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: Permutation Matrices Example The matrix 1 0 0 P = 0 0 1 0 1 0 is a 3 × 3 permutation matrix. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 36 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: Permutation Matrices Example The matrix 1 0 0 P = 0 0 1 0 1 0 is a 3 × 3 permutation matrix. For any 3 × 3 matrix A , multiplying on the left by P has the effect of interchanging the second and third rows of A : a 11 a 12 a 13 1 0 0 PA = a 21 a 22 a 23 0 0 1 a 31 a 32 a 33 0 1 0 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 36 / 46
Rationale Constructing LU Example Algorithm Permutation Matrices Matrix Factorization: Permutation Matrices Example The matrix 1 0 0 P = 0 0 1 0 1 0 is a 3 × 3 permutation matrix. For any 3 × 3 matrix A , multiplying on the left by P has the effect of interchanging the second and third rows of A : a 11 a 12 a 13 a 11 a 12 a 13 1 0 0 PA = a 21 a 22 a 23 a 31 a 32 a 33 = 0 0 1 a 31 a 32 a 33 a 21 a 22 a 23 0 1 0 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 36 / 46
Recommend
More recommend
