LU Decomposition INGE4035 Numerical Methods Applied to Engineering - PowerPoint PPT Presentation

LU Decomposition INGE4035 Numerical Methods Applied to Engineering Dr. Marco A Arocha October 2, 2014

Motivation • The motivation for an LU decomposition is based on the observation that systems of equations involving triangular coefficient matrices are easier to deal with. • Indeed, the whole point of Gaussian Elimination is to replace the coefficient matrix with one that is triangular. • The LU decomposition is another approach designed to exploit triangular systems.

Introduction • Matrices can be decomposed or factored into the product of two other matrices in many number of ways. • One way to decompose A is: A = LU • where L is a lower triangular and U an upper triangular matrices. • Specifying the diagonal elements of either L or U makes factoring unique. • There are three methods: Doolittle, Crout, Cholesky

Introduction • We assume that we can write A = LU • Our aim is to find L and U and once we have done so we have found an LU decomposition of A. • An LU decomposition of a matrix A is the product of a lower triangular matrix and an upper triangular matrix that is equal to A. Off course the multiplication process follows the rules of linear algebra.

There are three factorization methods: • Crout Method: • diag (U) = 1; u ii =1 • Doolittle Method: • diag (L) = 1; l ii =1 • Choleski Method: • diag (U) = diag (L); u ii = l ii

Doolittle 1 0 0 0 𝑦 1 0 0 x = any number; one’s in diagonal L = 𝑦 𝑦 1 0 𝑦 𝑦 𝑦 1

Crout 1 𝑦 𝑦 𝑦 0 1 𝑦 𝑦 x = any number; one’s in diagonal U = 0 0 1 𝑦 0 0 0 1

Cholesky • L ii = U ii diagonal elements are equal

Example (Crout) 𝑀 11 0 0 1 𝑉 12 𝑉 13 𝑀 21 𝑀 22 0 where 𝑀 = 𝑉 = 0 1 𝑉 23 𝑀 31 𝑀 32 𝑀 33 0 0 1

Example (Crout) Multiplying out LU and setting the answer equal to A gives 𝑀 11 𝑀 11 𝑉 12 𝑀 11 𝑉 13 1 2 4 𝑀 21 𝑀 21 𝑉 12 + 𝑀 22 𝑀 21 𝑉 13 + 𝑀 22 𝑉 23 = 3 8 14 𝑀 31 𝑀 31 𝑉 12 + 𝑀 32 𝑀 31 𝑉 13 + 𝑀 32 𝑉 23 + 𝑀 33 2 6 13 Matrix multiplication is reviewed at the end of the presentation

Example (Crout) 𝑀 11 𝑀 11 𝑉 12 𝑀 11 𝑉 13 1 2 4 𝑀 21 𝑀 21 𝑉 12 + 𝑀 22 𝑀 21 𝑉 13 + 𝑀 22 𝑉 23 = 3 8 14 𝑀 31 𝑀 31 𝑉 12 + 𝑀 32 𝑀 31 𝑉 13 + 𝑀 32 𝑉 23 + 𝑀 33 2 6 13 We begin by running along the first column: 𝑀 11 = 1; 𝑀 21 = 3; 𝑀 31 = 2 For the second column: 𝑀 11 𝑉 12 = 2, then 𝑉 12 = 2 𝑀 21 𝑉 12 + 𝑀 22 = 8 then 𝑀 22 = 2 𝑀 31 𝑉 12 + 𝑀 32 = 6 then 𝑀 32 = 2 For the third column: 𝑀 11 𝑉 13 = 4, then 𝑉 13 = 4 𝑀 21 𝑉 13 + 𝑀 22 𝑉 23 = 14 then 𝑉 23 = 1 𝑀 31 𝑉 13 + 𝑀 32 𝑉 23 + 𝑀 33 = 13 then 𝑀 33 = 3

Example (Crout) 1 0 0 1 2 4 where 𝑀 = 𝑉 = 3 2 0 0 1 1 2 2 3 0 0 1

Example (Doolittle)

Example (Doolittle) Matrix multiplication is reviewed at the end of the presentation

Example (Doolittle)

Example (Doolittle)

function [L, U] = CroutLU(A) matlab Crout LU [R, C] = size(A); for i = 1:R L(i, 1) = A(i, 1); % Crout LU Decomposition U(i, i) = 1; end clc, clear for j = 2:R A=[1,2,4; 3,8,14;2,6,13] U(1, j) = A(1, j) / L(1, 1); end [L, U] = CroutLU(A) for i = 2:R for j = 2:i L(i, j) = A(i, j) - L(i, 1:j - 1) * U(1:j - 1, j); end for j = i - 1:R U(i, j) = (A(i, j) - L(i, 1:i - 1) * U(1:i - 1, j)) / L(i, i); end end end https://rosettacode.org/wiki/LU_decomposition#Creating_a_MATLAB_function

matlab Crout LU: output

A = matlab lu function 1 2 4 3 8 14 2 6 13 % LU Decomposition % lu is a library matlab function L = clc, clear 0.3333 -1.0000 1.0000 A=[1,2,4; 3,8,14;2,6,13] not a triangular 1.0000 0 0 matrix 0.6667 1.0000 0 [L,U]=lu(A) U = 3.0000 8.0000 14.0000 0 0.6667 3.6667 0 0 3.0000

Using LU decomposition to solve systems of equations • Once a matrix A has been decomposed into lower and upper triangular parts it is possible to obtain the solution to AX = B in a direct way. • The procedure can be summarized as follows • Given A, find L and U so that • A = LU. • AX=B, hence LUX = B. • Let D = UX so that LD = B. Solve this triangular system for D . • Finally solve the triangular system UX = D for X. • The benefit of this approach is that we only ever need to solve triangular systems. The cost is that we have to solve two of them.

Consider the following system of equation:

The key components can be expressed as: coefficient matrix: right-hand side vector:

1 st Step: Decomposing A matrix in LU yields: Crout

The steps in finding the solution vector with LU Decomposition

2 nd Step: Find D vector

Solve by {D}: Expressing each equation and solving for {D}, i.e., d 1 , d 2 and d 3 : 1d 1 =15 2d 1 -2d 2 +0d 3 =22 3d 1 -5d 2 +3.5d 3 =39

1d 1 =15 2d 1 -2d 2 +0d 3 =22 3d 1 -5d 2 +3.5d 3 =39 which produces 𝑒 1 =15, 𝑒 2 =4, and 𝑒 3 =4 𝑒 1 15 𝑒 2 𝐸 = = 4 𝑒 3 4

3 rd Step: Find the solution vector x Third step: 𝑉 𝑌 = 𝐸 𝑒 1 15 𝑦 1 1 3 2 15 𝑒 2 𝐸 = = 4 𝑒 3 4 𝑦 2 = 0 1 0.5 4 𝑦 3 0 0 1 4

𝑉 𝑌 = 𝐸 𝑦 1 1 3 2 15 𝑦 2 = 0 1 0.5 4 𝑦 3 0 0 1 4 1𝑦 3 = 4 which produces 𝑦 3 = 4 , 𝑦 2 = 2 , 𝑦 1 = 1 1𝑦 2 + 0.5𝑦 3 = 4 1𝑦 1 + 3𝑦 2 + 2𝑦 3 = 15 𝑦 1 1 𝑦 2 𝑌 = = 2 𝑦 3 4

A = 1 3 2 2 4 3 Crout LU-matlab 3 4 7 B = 15 22 39 % Linear System Solution with LU % Decomposition with CroutLU function L = 1.0000 0 0 clc, clear 2.0000 -2.0000 0 3.0000 -5.0000 3.5000 A=[1,3,2; 2,4,3;3,4,7] B=[15;22;39] U = 1.0000 3.0000 2.0000 0 1.0000 0.5000 [L,U] = CroutLU(A) 0 0 1.0000 D = L\B X = U\D D = 15 4 4 X = 1 2 4

A = 1 3 2 2 4 3 lu function (matlab) 3 4 7 B = 15 22 39 % Linear System Solution with LU % Decomposition with lu matlab % function L = • 0.3333 1.0000 0 0.6667 0.8000 1.0000 clc, clear 1.0000 0 0 A=[1,3,2; 2,4,3;3,4,7] U = B=[15;22;39] 3.0000 4.0000 7.0000 0 1.6667 -0.3333 0 0 -1.4000 [L,U] = lu(A) D = L\B D = X = U\D 39.0000 2.0000 -5.6000 X = 1.0000 2.0000 4.0000

Matrix Multiplication Review

Matrix Multiplication • Compute the dot products of each row in A with each column in B • Each dot-prod result becomes an element with row equal to A and column equal to B in A*B A B the resulting matrix m    AB c a b ij ik kj  k 1  ( n x m ) * ( m x p ) n x p No commutative: AB ≠BA 34

Matrix Multiplication Math Syntax: AB MATLAB Syntax: A*B (NO DOT) >> B=[-2 4; 3 8; 12 -2] >> A=[1 3 5; 2 4 6] >> A*B B = A = ans = -2 4 1 3 5 3 8 67 18 2 4 6 12 -2 80 28 Sample calculation: The dot product of row-1 of A and column-1 of B: (1*-2)+(3*3)+(5*12)=67 35

Matrix Multiplication Math Syntax: AB MATLAB Syntax: A*B %(NO DOT) >> B=[-2 4; 3 8; 12 -2] >> A=[1 3 5; 2 4 6] >> matriX(A,B) B = A = ans = -2 4 1 3 5 3 8 67 18 2 4 6 12 -2 80 28 Sample calculation: The dot product of row-1 of A and column-1 of B: (1*-2)+(3*3)+(5*12)=67 36