CS475/CS675 Lecture 2: May 3, 2016 Cholesky factorization, - PowerPoint PPT Presentation

CS475/CS675 Lecture 2: May 3, 2016 Cholesky factorization, tridiagonal, band matrices Reading: [TB] Chapt. 23 p. 172‐176 CS475/CS675 (c) 2016 P. Poupart & J. Wan 1

Special Linear Systems • Exploit special structures of linear systems • More efficient factorization • Symmetric systems – ��� � factorization (variant of �� ) • Symmetric positive definite systems – �� � factorization (a.k.a. Cholesky factorization) CS475/CS675 (c) 2016 P. Poupart & J. Wan 2

factorization • Theorem: If all the leading principal submatrices of are nonsingular, then there exist unique unit lower matrices and , and a unique diagonal matrix such � . that • Partial Proof: – Factor � � �� – Define � � ���� � � , … , � � , � � � � �� � � 1, … , � – Let � � � � �� � � unit upper ∆ ( � � unit lower ∆ ) – Thus � � �� � �� � �� � � ��� � � • Note: CS475/CS675 (c) 2016 P. Poupart & J. Wan 3

Symmetric systems � • Theorem: If is symmetric, then • Proof: � – By previous result, �� �� �� � �� �� �� is symmetric, so is �� �� – Since �� �� – Also, is lower is lower �� – So is both lower and symmetric �� �� is diag is diag �� – Since is also unit lower , �� then CS475/CS675 (c) 2016 P. Poupart & J. Wan 4

Symmetric systems • Notes We can save about half the work by computing � and � 1. only. One way is to compute the � factor only during the �� 2. factorization. CS475/CS675 (c) 2016 P. Poupart & J. Wan 5

Positive definite systems � • Definition: is positive definite iff for all . • Properties of positive definite matrices: CS475/CS675 (c) 2016 P. Poupart & J. Wan 6

Positive definite systems ��� is PD and ��� has rank • Theorem: If ��� is also PD � , then • Proof: – Let z ∈ � ��� . Then � � �� � � � � � ��� � �� � ����� – If �� � 0 , then � is not rank � . – Hence � � �� � 0 . • Corollary: If is PD, then all its principal submatrices are . In particular, all diag entries are positive. CS475/CS675 (c) 2016 P. Poupart & J. Wan 7

Positive definite systems � and • Corollary: If is PD, then has positive diag entries. • Proof: – Let � � � �� . Then � � �� � � �� ��� � � �� � �� � � �� is PD. – By previous corollary, ������� � � �� � has positive entries. – Note that � � and � �� are unit upper ∆ . ⟹ � � � �� is also unit upper ∆ ⟹ ���� �� � � �� � � CS475/CS675 (c) 2016 P. Poupart & J. Wan 8

Symmetric positive definite systems • Theorem: If is SPD, then there exists unique lower such that � • Proof: – � � ��� � and � � ������ � , … , � � � , � � � 0 . � � ≡ ���� – Define � � � , … , � � � – Let � � �� � . Then � is lower ∆ � � � � � ⟹ �� � � �� � � � � ��� � � � � �� � �� � � � CS475/CS675 (c) 2016 P. Poupart & J. Wan 9

Symmetric positive definite systems • Examples CS475/CS675 (c) 2016 P. Poupart & J. Wan 10

Cholesky factorization � is called the Cholesky factorization • of and the lower is called the Cholesky factor. CS475/CS675 (c) 2016 P. Poupart & J. Wan 11

Cholesky factorization • Algorithm big picture CS475/CS675 (c) 2016 P. Poupart & J. Wan 12

Cholesky factorization For � � 1,2, … , � � �� � � �� For � � � � 1, … , � � �� � � �� /� �� End For � � � � 1, … , � for � � �, … , � � �� � � �� � � �� � �� end End End � � ��������������� � � CS475/CS675 (c) 2016 P. Poupart & J. Wan 13

Banded systems • Definition: has upper bandwidth if �� and lower bandwidth if �� . • Picture CS475/CS675 (c) 2016 P. Poupart & J. Wan 14

Banded systems � � • If A is banded, so are • Theorem: Let . If has upper bandwidth and lower bandwidth , then has upper bandwidth and has lower bandwidth . • Picture CS475/CS675 (c) 2016 P. Poupart & J. Wan 15

Band Gaussian Elimination For � � 1,2, … , � � 1 For � � � � 1, … , min � � �, � � �� � � �� /� �� end for � � � � 1, … , min � � �, � for � � � � 1, … , min � � �, � � �� � � �� � � �� � �� end end End If � ≫ � and � ≫ � , then ���������� ��� � 2��� CS475/CS675 (c) 2016 P. Poupart & J. Wan 16

Tridiagonal systems • Assume is tridiagonal and symmetric • Then � � � � CS475/CS675 (c) 2016 P. Poupart & J. Wan 17

Tridiagonal System � implies • � �� � �� �,��� �� �� �,��� � � � � �� �� �� � � �� �� �,��� � � �� �� �� � �� ���,� � �� � �� � �� ���,��� ���,��� �,��� � � ���,��� �� �,��� �� ��� ��� � ��� ��� � CS475/CS675 (c) 2016 P. Poupart & J. Wan 18

Tridiagonal Factorization • Algorithm � � � � �� for � � 2, … , � � ��� � � �,��� /� ��� � � � � �� � � ��� � �,��� ( � ��� � ��� � � �,��� � end • ����� ������� � ���� CS475/CS675 (c) 2016 P. Poupart & J. Wan 19