CS475 / CM375 Lecture 11: Oct 18, 2011 QR Factorization and Gram - PDF document

19/10/2011 CS475 / CM375 Lecture 11: Oct 18, 2011 QR Factorization and Gram ‐ Schmidt Orthogonalization Reading: [TB] Chapters 7, 8 CS475/CM375 (c) 2011 P. Poupart & J. Wan 1 Gram ‐ Schmidt Orthogonalization • QR factorization algorithm – � � �� ( � orthogonal and � upper ∆ ) – Picture: • At the � �� step – � � is orthogonal to � � , … , � ��� � � � 1 – CS475/CM375 (c) 2011 P. Poupart & J. Wan 2 1

19/10/2011 Gram ‐ Schmidt Orthogonalization ��� • Consider � � � � � � ∑ � � � � ��� � � � • Since 0 � � � ��� � � � � ∑ � � � � � � � � � �� � � � 1, … , � � 1 ��� � � � � � � � � � � � � � � � � � � � � � 1 ∴ � � � �� � � � ��� � � � � � ⟹ � � � � � � ∑ � � ��� CS475/CM375 (c) 2011 P. Poupart & J. Wan 3 Gram ‐ Schmidt Orthogonalization � � • Normalize � �  � � � � � � � • Hence � � � � �� � � �� �� � � � � � � �� ⋮ ��� � � �∑ � �� � � � � � ��� � �� ��� � � � , � � � � ∑ • Where � �� � � � �� � � �� � � ��� � CS475/CM375 (c) 2011 P. Poupart & J. Wan 4 2

19/10/2011 Gram ‐ Schmidt Algorithm For � � 1,2, … , � � � � � � for � � 1, … , � � 1 � � � � �� � � � � � � � � � � �� � � end � �� � � � � � � � � � �� end CS475/CM375 (c) 2011 P. Poupart & J. Wan 5 Modified Gram ‐ Schmidt � � � ”  “ � � � � ” (more stable) • Change: “ � �� � � � �� � � � • In the � ‐ loop, � � changes for each � ��� � � � � � � � 1 : � � �� � � ��� � � � � � � �� � � � � � � � � � 2: � � �� � � � � �� � � ⋮ ��� � � � � ∑ ��� � � � � 1: � � � �� � � ��� • At � � �, � � � � �� � � � � � � � ∑ ��� � � � � �� � � �� � � �� � , … , � ��� �� ��� � � � ��� � � � CS475/CM375 (c) 2011 P. Poupart & J. Wan 6 3

19/10/2011 Complexity of Gram ‐ Schmidt • Consider the � ‐ loop: � � � or � � � � � � �� � � �  � mult, � � 1 adds � � � � � � � �� � �  � mult, � subs ∴ ����� ∽ 4� ��� � • Total flops � ∑ ∑ 4� ��� ��� � � � ∑ � � 1 4� ∼ 4� ∑ � ��� ��� �������� � � ∼ 2�� � • Note: when � � � , then ����� �� � 2� � � ��� � � � 3 � ��������� CS475/CM375 (c) 2011 P. Poupart & J. Wan 7 Example CS475/CM375 (c) 2011 P. Poupart & J. Wan 8 4

19/10/2011 Householder triangularization • More stable than Gram ‐ Schmidt • Idea: � � … � � � � � � � � � ∈ � ��� orthogonal matrices • Similar to GE, each � � will make the entries of col � become zero • Picture: CS475/CM375 (c) 2011 P. Poupart & J. Wan 9 Householder reflectors � � � 1 • Define � � � � 0 � � � �� � 1� 0 � • � is chosen to be a Householder reflector • Picture CS475/CM375 (c) 2011 P. Poupart & J. Wan 10 5

19/10/2011 Householder Reflector � | � | � 0 Suppose � � then �� � � � � � ⋮ • ⋮ � 0 � “reflects” � across hyperplane � orthogonal to � � � � � � � • The orthogonal projector of � onto � : • � � � � � � � � � � �� � � � � � � � � � Since � is a reflector, it should go twice as far: • �� � � � 2� � � � � � � CS475/CM375 (c) 2011 P. Poupart & J. Wan 11 Householder Reflectors • Two possibilities: • For stability reason, the further one is chosen – i.e., � � ����� � � � � � � � � � ���� � � � � � � � CS475/CM375 (c) 2011 P. Poupart & J. Wan 12 6

19/10/2011 Another Derivation �� � • Let � � � � 2 � � � . Find � s.t. �� ∈ ���� � � . � � � • �� � � � 2 � � � � ∈ ���� � � ⟺ � ∈ ������, � � � • Let � � � � �� � � � � � � � � �� � � � � � � � � � �� � � � � � � � �� � � � � �� � � � � � � 2�� � � � � CS475/CM375 (c) 2011 P. Poupart & J. Wan 13 Derivation Continued • ∴ �� � • Hence � � � � � � � and �� � ∓ � � � CS475/CM375 (c) 2011 P. Poupart & J. Wan 14 7

19/10/2011 Example CS475/CM375 (c) 2011 P. Poupart & J. Wan 15 8