a 11 a 1 n . . ... . . A = . . a n 1 a nn - PowerPoint PPT Presentation

Lesson 16 L EAST S QUARES

• Let � � · · · a 11 a 1 n . . ... . . A = � . . � � � · · · a n 1 a nn be an n × n matrix with complex entries (i.e., A ∈ C n × n ) � It is helpful to view A as a row-vector whose columns are in C n : � a 1 | · · · | a n � A = • Recall that if is nonsingular, then we can always solve the linear system . . . . for . . • What if is singular? Can we find so that is "close" to ? . . • In other words, for , we want .

• Let � � · · · a 11 a 1 n . . ... . . A = � . . � � � · · · a n 1 a nn be an n × n matrix with complex entries (i.e., A ∈ C n × n ) � It is helpful to view A as a row-vector whose columns are in C n : � a 1 | · · · | a n � A = • Recall that if A is nonsingular, then we can always solve the linear system � � � � b 1 c 1 . . . . for A c = b , b = � , c = � ∈ C n � . � � . � � � b n c n • What if is singular? Can we find so that is "close" to ? . . • In other words, for , we want .

• Let � � · · · a 11 a 1 n . . ... . . A = � . . � � � · · · a n 1 a nn be an n × n matrix with complex entries (i.e., A ∈ C n × n ) � It is helpful to view A as a row-vector whose columns are in C n : � a 1 | · · · | a n � A = • Recall that if A is nonsingular, then we can always solve the linear system � � � � b 1 c 1 . . . . for A c = b , b = � , c = � ∈ C n � . � � . � � � b n c n • What if A is singular? Can we find c so that A c is "close" to b ? � � c 1 . . • In other words, for c = � , we want � � . � c n c 1 a 1 + · · · + c n a n ≈ b

• More generally, let A � C m × n : for a k � C m � a 1 | · · · | a n � A = • Can we numerically compute c 1 , . . . , c n so that c 1 a 1 + · · · c n a n � b • More precisely, we find such that takes its minimal value

• More generally, let A � C m × n : for a k � C m � a 1 | · · · | a n � A = • Can we numerically compute c 1 , . . . , c n so that c 1 a 1 + · · · c n a n � b • More precisely, we find c such that � A c � b � � 2 takes its minimal value

• Let's review the real A � R m � n , c � R n and b � R m • Minimizing � A c � b � is equivalent to minimizing � A c � b � 2 • We simplify • We can heuristically assume that the minimum is a stationary point of this equation; i.e., we want

• Let's review the real A � R m � n , c � R n and b � R m • Minimizing � A c � b � is equivalent to minimizing � A c � b � 2 • We simplify � A c � b � 2 = ( A c � b ) � ( A c � b ) • We can heuristically assume that the minimum is a stationary point of this equation; i.e., we want

• Let's review the real A � R m � n , c � R n and b � R m • Minimizing � A c � b � is equivalent to minimizing � A c � b � 2 • We simplify � A c � b � 2 = ( A c � b ) � ( A c � b ) = � A c � 2 � ( A c ) � b � b � A c + � b � 2 • We can heuristically assume that the minimum is a stationary point of this equation; i.e., we want

• Let's review the real A � R m � n , c � R n and b � R m • Minimizing � A c � b � is equivalent to minimizing � A c � b � 2 • We simplify � A c � b � 2 = ( A c � b ) � ( A c � b ) = � A c � 2 � ( A c ) � b � b � A c + � b � 2 = c � A � A c � 2 c � A � b + � b � 2 • We can heuristically assume that the minimum is a stationary point of this equation; i.e., we want

• Let's review the real A � R m � n , c � R n and b � R m • Minimizing � A c � b � is equivalent to minimizing � A c � b � 2 • We simplify � A c � b � 2 = ( A c � b ) � ( A c � b ) = � A c � 2 � ( A c ) � b � b � A c + � b � 2 = c � A � A c � 2 c � A � b + � b � 2 • We can heuristically assume that the minimum is a stationary point of this equation; i.e., we want 0 = � c � A c � b � 2

������� : Suppose A has linearly independent columns. The vector c = ( A � A ) � 1 A � b is the unique minimizer of � A c � b � ����� : • We first remark that is positive definite, i.e., for all (real) . (Why?) • Minimizing is equivalent to minimizing . • For all , we have

������� : Suppose A has linearly independent columns. The vector c = ( A � A ) � 1 A � b is the unique minimizer of � A c � b � ����� : • We first remark that A � A is positive definite, i.e., x � A � A x > 0 for all (real) x . (Why?) • Minimizing is equivalent to minimizing . • For all , we have

������� : Suppose A has linearly independent columns. The vector c = ( A � A ) � 1 A � b is the unique minimizer of � A c � b � ����� : • We first remark that A � A is positive definite, i.e., x � A � A x > 0 for all (real) x . (Why?) • Minimizing � A c � b � is equivalent to minimizing � A c � b � 2 . • For all , we have

������� : Suppose A has linearly independent columns. The vector c = ( A � A ) � 1 A � b is the unique minimizer of � A c � b � ����� : • We first remark that A � A is positive definite, i.e., x � A � A x > 0 for all (real) x . (Why?) • Minimizing � A c � b � is equivalent to minimizing � A c � b � 2 . • For all x , we have � A ( c + x ) � b � 2 = ( c + x ) � A � A ( c + x ) � 2( c + x ) � A � b + � b � 2

������� : Suppose A has linearly independent columns. The vector c = ( A � A ) � 1 A � b is the unique minimizer of � A c � b � ����� : • We first remark that A � A is positive definite, i.e., x � A � A x > 0 for all (real) x . (Why?) • Minimizing � A c � b � is equivalent to minimizing � A c � b � 2 . • For all x , we have � A ( c + x ) � b � 2 = ( c + x ) � A � A ( c + x ) � 2( c + x ) � A � b + � b � 2 = c � A � A ( c + x ) + x � A � A ( c + x ) � 2( c + x ) � b + � b � 2

������� : Suppose A has linearly independent columns. The vector c = ( A � A ) � 1 A � b is the unique minimizer of � A c � b � ����� : • We first remark that A � A is positive definite, i.e., x � A � A x > 0 for all (real) x . (Why?) • Minimizing � A c � b � is equivalent to minimizing � A c � b � 2 . • For all x , we have � A ( c + x ) � b � 2 = ( c + x ) � A � A ( c + x ) � 2( c + x ) � A � b + � b � 2 = c � A � A ( c + x ) + x � A � A ( c + x ) � 2( c + x ) � b + � b � 2 = x � A � A x + c � A � A c + 2 x � A � A c � 2( c + x ) � A � b + � b � 2

������� : Suppose A has linearly independent columns. The vector c = ( A � A ) � 1 A � b is the unique minimizer of � A c � b � ����� : • We first remark that A � A is positive definite, i.e., x � A � A x > 0 for all (real) x . (Why?) • Minimizing � A c � b � is equivalent to minimizing � A c � b � 2 . • For all x , we have � A ( c + x ) � b � 2 = ( c + x ) � A � A ( c + x ) � 2( c + x ) � A � b + � b � 2 = c � A � A ( c + x ) + x � A � A ( c + x ) � 2( c + x ) � b + � b � 2 = x � A � A x + c � A � A c + 2 x � A � A c � 2( c + x ) � A � b + � b � 2 = x � A � A x + c � A � A c + 2 x � A � A ( A � A ) � 1 A � b � 2( c + x ) � A � b + � b � 2

������� : Suppose A has linearly independent columns. The vector c = ( A � A ) � 1 A � b is the unique minimizer of � A c � b � ����� : • We first remark that A � A is positive definite, i.e., x � A � A x > 0 for all (real) x . (Why?) • Minimizing � A c � b � is equivalent to minimizing � A c � b � 2 . • For all x , we have � A ( c + x ) � b � 2 = ( c + x ) � A � A ( c + x ) � 2( c + x ) � A � b + � b � 2 = c � A � A ( c + x ) + x � A � A ( c + x ) � 2( c + x ) � b + � b � 2 = x � A � A x + c � A � A c + 2 x � A � A c � 2( c + x ) � A � b + � b � 2 = x � A � A x + c � A � A c + 2 x � A � A ( A � A ) � 1 A � b � 2( c + x ) � A � b + � b � 2 = x � A � A x + c � A � A c + 2 x � A � b � 2( c + x ) � A � b + � b � 2

������� : Suppose A has linearly independent columns. The vector c = ( A � A ) � 1 A � b is the unique minimizer of � A c � b � ����� : • We first remark that A � A is positive definite, i.e., x � A � A x > 0 for all (real) x . (Why?) • Minimizing � A c � b � is equivalent to minimizing � A c � b � 2 . • For all x , we have � A ( c + x ) � b � 2 = ( c + x ) � A � A ( c + x ) � 2( c + x ) � A � b + � b � 2 = c � A � A ( c + x ) + x � A � A ( c + x ) � 2( c + x ) � b + � b � 2 = x � A � A x + c � A � A c + 2 x � A � A c � 2( c + x ) � A � b + � b � 2 = x � A � A x + c � A � A c + 2 x � A � A ( A � A ) � 1 A � b � 2( c + x ) � A � b + � b � 2 = x � A � A x + c � A � A c + 2 x � A � b � 2( c + x ) � A � b + � b � 2 = x � A � A x + c � A � A c � 2 c � A � b + � b � 2