CS475 / CS675 Lecture 20: July 7, 2016 Bidiagonalization SVD Image - PowerPoint PPT Presentation

CS475 / CS675 Lecture 20: July 7, 2016 Bidiagonalization SVD Image Compression Reading: [TB] Chapter 31 CS475/CS675 (c) 2016 P. Poupart & J. Wan 1

Alternative SVD Technique • Assume is square, i.e., • Consider the symmetric matrix: � � , � � • Since , � � � then � Λ CS475/CS675 (c) 2016 P. Poupart & J. Wan 2

Alternative SVD Technique • Hence, eigendeomposition of • Algorithm: – Compute eigendecomposition of � . – Set � � � |� � | – Extract �, � from � • Stable algorithm CS475/CS675 (c) 2016 P. Poupart & J. Wan 3

Two‐phase SVD • Idea: First reduce the matrix to bidiagonal form, then diagonalize it. • Picture: CS475/CS675 (c) 2016 P. Poupart & J. Wan 4

Golub‐Kahan Bidiagonalization • Apply Householder reflectors on the left and the right • reflectors on the left, on the right � � � • � CS475/CS675 (c) 2016 P. Poupart & J. Wan 5

Low‐Rank Approximation • Theorem: is the sum of rank‐one matrices: � � � � � ��� • Proof: CS475/CS675 (c) 2016 P. Poupart & J. Wan 6

Low‐Rank Approximation • Theorem: For any , , define � � � � � � ��� Then � ��� � � ���� � �� • Proof: first note that 0 � � � � ��� � � � � � � � � � � � � � � � � … � � ⋮ � ⋱ � � ����� � � � It is the SVD of � Hence: � ��� � CS475/CS675 (c) 2016 P. Poupart & J. Wan 7

Low‐Rank Approximation • Suppose with such that � ��� � � • Then ‐dim subspace such that • Note . Then � � � � ��� � CS475/CS675 (c) 2016 P. Poupart & J. Wan 8

Low‐Rank Approximation • But ‐dim subspace ��� such that ��� – E.g., � ��� � ������ � , � � , … , � ��� � – Note: �� � � � � � � , �� � � � � � � ��� � � • But ��� contradiction CS475/CS675 (c) 2016 P. Poupart & J. Wan 9

Low‐Rank Approximation • Notes � � � � � ⋱ � � � � � … � � ⋮ 1. � � � � � 0 � � � � � � � � … � � ⋮ ⋱ � � � � � � � � � Σ � � � � � is the best rank‐ � approximation of � . 2. The error of approximation is � ��� (in � � ‐norm) CS475/CS675 (c) 2016 P. Poupart & J. Wan 10

Application: Image Compression • An image can be represented by matrix where pixel value at �� • Compress the image by storing less than entries � � • Let , the best rank‐ approx of � � � ��� � • Keep the first singular values and use � to approximate ; i.e., compressed image � CS475/CS675 (c) 2016 P. Poupart & J. Wan 11

Application: Image Compression • Example: , • To store � , only need to store � � and � � � � – This requires only words • In contrast, to store one needs words ��� � � • Compression ratio: �� ��� CS475/CS675 (c) 2016 P. Poupart & J. Wan 12

Application: Image Compression � ��� k Compression rate Relative error � � 3 10 20 CS475/CS675 (c) 2016 P. Poupart & J. Wan 13