LU Decomposition INGE4035 Numerical Methods Applied to Engineering - - PowerPoint PPT Presentation

LU Decomposition

INGE4035 Numerical Methods Applied to Engineering

• Dr. Marco A Arocha

October 2, 2014

Motivation

• The motivation for an LU decomposition is based on the observation

that systems of equations involving triangular coefficient matrices are easier to deal with.

• Indeed, the whole point of Gaussian Elimination is to replace the

coefficient matrix with one that is triangular.

• The LU decomposition is another approach designed to exploit

triangular systems.

Introduction

• Matrices can be decomposed or factored into the product of two
• ther matrices in many number of ways.
• One way to decompose A is:

A = LU

• where L is a lower triangular and U an upper triangular matrices.
• Specifying the diagonal elements of either L or U makes factoring

unique.

• There are three methods: Doolittle, Crout, Cholesky

Introduction

• We assume that we can write

A = LU

• Our aim is to find L and U and once we have done so we have found

an LU decomposition of A.

• An LU decomposition of a matrix A is the product of a lower triangular

matrix and an upper triangular matrix that is equal to A. Off course the multiplication process follows the rules of linear algebra.

There are three factorization methods:

• Crout Method:
• diag (U) = 1; uii=1
• Doolittle Method:
• diag (L) = 1; lii=1
• Choleski Method:
• diag (U) = diag (L); uii= lii

Doolittle

L = 1 𝑦 1 𝑦 𝑦 1 𝑦 𝑦 𝑦 1 x= any number; one’s in diagonal

Crout

U = 1 𝑦 𝑦 𝑦 1 𝑦 𝑦 1 𝑦 1 x= any number; one’s in diagonal

Cholesky

• Lii = Uii

diagonal elements are equal

Example (Crout)

where 𝑀 = 𝑀11 𝑀21 𝑀22 𝑀31 𝑀32 𝑀33 𝑉 = 1 𝑉12 𝑉13 1 𝑉23 1

Example (Crout)

Matrix multiplication is reviewed at the end of the presentation Multiplying out LU and setting the answer equal to A gives

𝑀11 𝑀11𝑉12 𝑀11𝑉13 𝑀21 𝑀21𝑉12 + 𝑀22 𝑀21𝑉13 + 𝑀22𝑉23 𝑀31 𝑀31𝑉12 + 𝑀32 𝑀31𝑉13 + 𝑀32𝑉23 + 𝑀33 = 1 2 4 3 8 14 2 6 13

Example (Crout)

𝑀11 𝑀11𝑉12 𝑀11𝑉13 𝑀21 𝑀21𝑉12 + 𝑀22 𝑀21𝑉13 + 𝑀22𝑉23 𝑀31 𝑀31𝑉12 + 𝑀32 𝑀31𝑉13 + 𝑀32𝑉23 + 𝑀33 = 1 2 4 3 8 14 2 6 13

We begin by running along the first column: 𝑀11 = 1; 𝑀21 = 3; 𝑀31 = 2 For the second column: 𝑀11𝑉12 = 2, then 𝑉12 = 2 𝑀21𝑉12 + 𝑀22 = 8 then 𝑀22 = 2 𝑀31𝑉12 + 𝑀32 = 6 then 𝑀32 = 2 For the third column: 𝑀11𝑉13 = 4, then 𝑉13 = 4 𝑀21𝑉13 + 𝑀22𝑉23 = 14 then 𝑉23 = 1 𝑀31𝑉13 + 𝑀32𝑉23 + 𝑀33 = 13 then 𝑀33 = 3

Example (Crout)

where 𝑀 = 1 3 2 2 2 3 𝑉 = 1 2 4 1 1 1

Example (Doolittle)

Example (Doolittle)

Matrix multiplication is reviewed at the end of the presentation

Example (Doolittle)

Example (Doolittle)

matlab Crout LU

% Crout LU Decomposition clc, clear A=[1,2,4; 3,8,14;2,6,13] [L, U] = CroutLU(A) function [L, U] = CroutLU(A) [R, C] = size(A); for i = 1:R L(i, 1) = A(i, 1); U(i, i) = 1; end for j = 2:R U(1, j) = A(1, j) / L(1, 1); end for i = 2:R

for j = 2:i L(i, j) = A(i, j) - L(i, 1:j - 1) * U(1:j - 1, j); end for j = i - 1:R U(i, j) = (A(i, j) - L(i, 1:i - 1) * U(1:i - 1, j)) / L(i, i); end

end end https://rosettacode.org/wiki/LU_decomposition#Creating_a_MATLAB_function

matlab Crout LU: output

matlab lu function

% LU Decomposition % lu is a library matlab function clc, clear A=[1,2,4; 3,8,14;2,6,13] [L,U]=lu(A) A = 1 2 4 3 8 14 2 6 13 L = 0.3333 -1.0000 1.0000 1.0000 0 0 0.6667 1.0000 0 U = 3.0000 8.0000 14.0000 0.6667 3.6667 0 0 3.0000

not a triangular matrix

Using LU decomposition to solve systems of equations

• Once a matrix A has been decomposed into lower and upper triangular

parts it is possible to obtain the solution to AX = B in a direct way.

• The procedure can be summarized as follows
• Given A, find L and U so that
• A = LU.
• AX=B, hence LUX = B.
• Let D = UX so that LD = B. Solve this triangular system for D .
• Finally solve the triangular system UX = D for X.
• The benefit of this approach is that we only ever need to solve triangular
• systems. The cost is that we have to solve two of them.

Consider the following system of equation:

The key components can be expressed as:

coefficient matrix: right-hand side vector:

1st Step: Decomposing A matrix in LU yields:

Crout

The steps in finding the solution vector with LU Decomposition

2nd Step: Find D vector

Solve by {D}:

Expressing each equation and solving for {D}, i.e., d1, d2 and d3: 1d1=15 2d1-2d2+0d3=22 3d1-5d2+3.5d3=39

1d1=15 2d1-2d2+0d3=22 3d1-5d2+3.5d3=39

which produces 𝑒1=15, 𝑒2=4, and 𝑒3=4 𝐸 = 𝑒1 𝑒2 𝑒3 = 15 4 4

3rd Step: Find the solution vector x

𝐸 = 𝑒1 𝑒2 𝑒3 = 15 4 4

Third step: 𝑉 𝑌 = 𝐸 1 3 2 1 0.5 1 𝑦1 𝑦2 𝑦3 = 15 4 4

𝑉 𝑌 = 𝐸 1 3 2 1 0.5 1 𝑦1 𝑦2 𝑦3 = 15 4 4

1𝑦3 = 4 1𝑦2 + 0.5𝑦3 = 4 1𝑦1 + 3𝑦2 + 2𝑦3 = 15 which produces 𝑦3 = 4, 𝑦2 = 2 , 𝑦1 = 1 𝑌 = 𝑦1 𝑦2 𝑦3 = 1 2 4

Crout LU-matlab

% Linear System Solution with LU % Decomposition with CroutLU function clc, clear A=[1,3,2; 2,4,3;3,4,7] B=[15;22;39] [L,U] = CroutLU(A) D = L\B X = U\D A = 1 3 2 2 4 3 3 4 7 B = 15 22 39 L = 1.0000 0 2.0000 -2.0000 0 3.0000 -5.0000 3.5000 U = 1.0000 3.0000 2.0000 1.0000 0.5000 0 0 1.0000 D = 15 4 4 X = 1 2 4

lu function (matlab)

% Linear System Solution with LU % Decomposition with lu matlab % function

• clc, clear

A=[1,3,2; 2,4,3;3,4,7] B=[15;22;39] [L,U] = lu(A) D = L\B X = U\D A = 1 3 2 2 4 3 3 4 7 B = 15 22 39 L = 0.3333 1.0000 0 0.6667 0.8000 1.0000 1.0000 0 U = 3.0000 4.0000 7.0000 1.6667 -0.3333 0 0

• 1.4000

D = 39.0000 2.0000

• 5.6000

X = 1.0000 2.0000 4.0000

Matrix Multiplication Review

Matrix Multiplication

• Compute the dot

products of each row in A with each column in B

• Each dot-prod result

becomes an element with row equal to A and column equal to B in the resulting matrix





 

m k kj ik ij

b a c AB

1

p n p m m n x ) x ( * ) x ( 

34

A B A*B

No commutative: AB≠BA

Matrix Multiplication

Math Syntax: AB MATLAB Syntax: A*B (NO DOT)

>> A=[1 3 5; 2 4 6] A = 1 3 5 2 4 6

35

Sample calculation: The dot product of row-1 of A and column-1 of B: (1*-2)+(3*3)+(5*12)=67

>> A*B ans = 67 18 80 28 >> B=[-2 4; 3 8; 12 -2] B =

• 2 4

3 8 12 -2

Matrix Multiplication

Math Syntax: AB MATLAB Syntax: A*B %(NO DOT)

>> A=[1 3 5; 2 4 6] A = 1 3 5 2 4 6

36

Sample calculation: The dot product of row-1 of A and column-1 of B: (1*-2)+(3*3)+(5*12)=67

>> matriX(A,B) ans = 67 18 80 28 >> B=[-2 4; 3 8; 12 -2] B =

• 2 4

3 8 12 -2