The separation of two matrices and its application in eigenvalue - PowerPoint PPT Presentation

The separation of two matrices and its application in eigenvalue perturbation theory Michael Karow Matheon, TU-Berlin

Outline. • The 3 definitions of separation • Inclusion theorems for pseudospectra of block triangular matrices • Perturbation bounds for invariant subspaces

The definitions of separation

10 5 0 −5 Pseudospectra −10 −10 −5 0 5 10 The pseudospectrum of A ∈ C n × n to the perturbation level ǫ > 0 is Λ ǫ ( A ) := set of all eigenvalues of all matrices of the form A + E, where E ∈ C n × n , � E � ≤ ǫ . union of the spectra Λ( A + E ) where E ∈ C n × n , � E � ≤ ǫ = � ( zI − A ) − 1 � − 1 ≤ ǫ } . = Λ( A ) ∪ { z ∈ C \ Λ( A ) | In this talk � · � denotes the spectral norm. Then Λ ǫ ( A ) := { z ∈ C | σ min ( zI − A ) ≤ ǫ } .

Separation of two matrices: Demmel’s definition Pseudospectra of L ∈ C ℓ × ℓ (blue) and M ∈ C m × m (red): 6 4 ǫ = 0 . 50 2 0 −2 −4 −6 −6 −4 −2 0 2 4 6

Separation of two matrices: Demmel’s definition Pseudospectra of L ∈ C ℓ × ℓ (blue) and M ∈ C m × m (red): 6 4 ǫ = 0 . 80 2 0 −2 −4 −6 −6 −4 −2 0 2 4 6

Separation of two matrices: Demmel’s definition Pseudospectra of L ∈ C ℓ × ℓ (blue) and M ∈ C m × m (red): 6 4 ǫ = 1 . 19 2 0 −2 −4 −6 −6 −4 −2 0 2 4 6

Separation of two matrices: Demmel’s definition Pseudospectra of L ∈ C ℓ × ℓ (blue) and M ∈ C m × m (red): 6 4 ǫ = 1 . 19 = sep D λ ( L, M ) 2 0 −2 −4 −6 −6 −4 −2 0 2 4 6 sep D λ ( L, M ) = min { ǫ | Λ ǫ ( L ) ∩ Λ ǫ ( M ) � = ∅ } = min z ∈ C max { σ min ( zI − L ) , σ min ( zI − M ) }

Separation of two matrices: Varah’s definition Pseudospectra of L ∈ C ℓ × ℓ (blue) and M ∈ C m × m (red): 6 4 ǫ 1 = 1 . 5 2 0 ǫ 2 = 0 . 85 −2 −4 −6 −6 −4 −2 0 2 4 6 sep V λ ( L, M ) = min { ǫ 1 + ǫ 2 | Λ ǫ 1 ( L ) ∩ Λ ǫ 2 ( M ) � = ∅ } = min z ∈ C [ σ min ( zI − L ) + σ min ( zI − M )]

Separation of two matrices: Stewart’s definition Definition uses Sylvester-operator Z �− → T ( Z ) = MZ − ZL : sep( L, M ) = min | =1 | | MZ − ZL | | | . | | | Z | | | Facts: • sep( L, M ) � = 0 iff T nonsingular iff Λ( L ) ∩ Λ( M ) � = ∅ • sep( L, M ) ≤ sep V λ ( L, M ) if | | · | | | is unitarily invariant. | Proof: Λ( L + E 1 ) ∩ Λ( M + E 2 ) � = ∅ ⇒ 0 = sep( L + E 1 , M + E 2 ) | =1 | | ( M + E 2 ) Z − Z ( L + E 1 ) | | | | = min | | Z | | | ≥ sep( L, M ) − � E 1 � − � E 2 � ⇒ � E 1 � + � E 2 � ≥ sep( L, M )

Comparison of the separations Stewart’s definition: sep( L, M ) = min | =1 | | MZ − ZL | | | | | | Z | | | Varah’s definition: sep V λ ( L, M ) = min { ǫ 1 + ǫ 2 | Λ ǫ 1 ( L ) ∩ Λ ǫ 2 ( M ) � = ∅} Demmel’s definition: sep D λ ( L, M ) = min { ǫ | Λ ǫ ( L ) ∩ Λ ǫ ( M ) � = ∅} Computation of sep D λ in [Gu,Overton, 2006] . We have sep( L, M ) ≤ sep V λ ( L, M ) ≤ 2 sep D λ ( L, M ) ≤ dist(Λ(L) , Λ(M)) Equality holds if L and M are both normal and | | ·| | | is the Frobenius norm. | Remark: For (scaled) Jordan blocks L , M : sep( L, M ) << sep D λ ( L, M ) << dist(Λ(L) , λ (M))

Application: Inclusion theorems for pseudospectra of block triangular matrices

The Problem Let A ∈ C n × n be given in block Schur form: � � L C U ∗ , A = U U unitary , Λ( L ) ∩ Λ( M ) = ∅ . 0 M We always have Λ ǫ ( L ) ∪ Λ ǫ ( M ) ⊆ Λ ǫ ( A ) . Problem: Find a tight function g of ǫ such that Λ ǫ ( A ) ⊆ Λ g ( ǫ ) ǫ ( L ) ∪ Λ g ( ǫ ) ǫ ( M ) . ( ∗ ) Relevance: If � E � = ǫ and the union in ( ∗ ) is disjoint then precisely dim L eigenvalues of A + E are contained in Λ g ( ǫ ) ǫ ( L ). The others are contained in Λ g ( ǫ ) ǫ ( M ).

Visualisation of the Problem Problem again: Find a tight function g of ǫ such that      L C Λ ǫ ⊆ Λ g ( ǫ ) ǫ ( L ) ∪ Λ g ( ǫ ) ǫ ( M ) .    0 M      L C grey region: Λ ǫ 6    0 M 4 blue region: Λ ǫ ( L ) 2 red region: Λ ǫ ( M ) 0 −2 blue curve: boundary of Λ g ( ǫ ) ǫ ( L ) −4 red curve: boundary of Λ g ( ǫ ) ǫ ( M ) −6 −6 −4 −2 0 2 4 6

Upper bounds in terms of C Let A ∈ C n × n be given in block Schur form: � � L C U ∗ , A = U U unitary , Λ( L ) ∩ Λ( M ) = ∅ . 0 M Then Λ ǫ ( A ) ⊆ Λ g ( ǫ ) ǫ ( L ) ∪ Λ g ( ǫ ) ǫ ( M ) for � 1 + � C � g ( ǫ ) = ( Grammont, Largillier, 2002) ǫ and for � g ( ǫ ) = 1 1 4 + � C � 2 + ( Bora, 2001) ǫ Good: Simple bounds which show that Λ ǫ ( A ) ≈ Λ ǫ ( L ) ∪ Λ ǫ ( M ) for large ǫ. Bad: g ( ǫ ) → ∞ as ǫ → 0.

Proof of the Grammont-Largillier-bound Let a z := max {� ( z I − L ) − 1 � , � ( z I − M ) − 1 �} . Then we have the following chain of inclusions and inequalities. ǫ − 1 � ( z I − A ) − 1 � z ∈ Λ ǫ ( A ) ⇒ ≤ � − ( z I − L ) − 1 C ( z I − M ) − 1 �� � ( z I − L ) − 1 � � = � � ( z I − M ) − 1 0 � � � a z 2 � C � �� � a z � � ≤ � � � 0 a z � 2 a z � C � + √ ( a z � C � ) 2 +4 = a z 2 � 2( ǫa z ) − 1 − a z � C � ( a z � C � ) 2 + 4 ⇒ ≤ � 1 + � C � /ǫ ) − 1 ⇒ ≤ ( ǫ a z ⇒ z ∈ Λ ǫ √ 1+ � C � /ǫ ( L ) ∪ Λ ǫ √ 1+ � C � /ǫ ( M ) .

Demmel’s bound (1983) Let T be such that � � � � L C L 0 T − 1 T = . 0 M 0 M Then the Bauer-Fike-Theorem yields �� �� L C Λ ǫ ⊆ Λ � T � � T − 1 � ǫ ( L ) ∪ Λ � T � � T − 1 � ǫ ( M ) 0 M Problem: Find such T with smallest condition number � T � � T − 1 � . Solution: Let R be such that RM − LR = C . Then � � � I R/p 1 + � R � 2 T = p = , 0 I/p has smallest possible condition number � p 2 − 1 ≤ 2 p. κ := � T � � T − 1 � = p + � R � = p + � � � � � � L C I R Note: has invariant subspaces range , range 0 0 M I and p is the norm of the associated spectral projector.

Illustration: invariant subspaces of � � � � L C L RM − LR A = = , Λ( L ) ∩ Λ( M ) = ∅ . 0 M 0 M invariant subspace R x I spectral projection I invariant subspace Px � � � � I R Invariant subspaces: range range , 0 I � � − R � I 1 + � R � 2 . Spectral projector: P = , p := � P � = 0 0

Demmel’s result and the separation. Let A ∈ C n × n be given in block Schur form: � � � � L C L RM − LR U ∗ = U U ∗ , A = U U unitary , Λ( L ) ∩ Λ( M ) = ∅ . 0 M 0 M Let � � � R � 2 + 1 = p 2 − 1 + p. κ = � R � + Then for all ǫ ≥ 0, Λ ǫ ( A ) ⊆ Λ κǫ ( L ) ∪ Λ κǫ ( M ) , Moreover, if ǫ < sep D λ ( L, M ) /κ then Λ κǫ ( L ) ∩ Λ κǫ ( M ) = ∅ .

Corollary to Demmel’s result. If L = λ I (i.e. λ is a semisimple eigenvalue of A ) then Λ ǫ ( A ) ⊆ Λ κ ǫ ( L ) ∪ Λ κ ǫ ( M ) 10 5 ∪ = D κ ǫ ( λ ) Λ κ ǫ ( M ) , 0 � �� � Disk of radius κǫ −5 � −10 p 2 − 1 + p where κ = � R � + p = ≈ 2 p −10 −5 0 5 10 � 1 + � R � 2 is the norm of the spectral projector. and p = Furthermore, if ǫ is small enough then D κ ǫ ( λ ) contains only one connected component C ǫ ( λ ) of Λ ǫ ( A ). But we know that for small ǫ C ǫ ( λ ) ≈ D p ǫ ( λ ) since p is the condition number of λ . Question: Is Demmel’s bound to large (factor ≈ 2)?

Inclusion bound for small ǫ : Demmel’s separation Let A ∈ C n × n be given in block Schur form: � � � � L C L RM − LR U ∗ = U U ∗ , A = U U unitary , Λ( L ) ∩ Λ( M ) = ∅ . 0 M 0 M � � � R � 2 + 1 = p 2 − 1 + p . Let s D = sep D λ ( L, M ), κ = � R � + Then for ǫ ≤ s D /κ , 2.6 p+||R||= κ 2.4 Λ ǫ ( A ) ⊆ Λ g D ( ǫ ) ǫ ( L ) ∪ Λ g D ( ǫ ) ǫ ( M ) , g D ( ε ) where p g D ( ǫ ) = p + � R � 2 ǫ s D / κ s D − p ǫ. 1 0 0.05 0.1 0.15 0.2

Inclusion bound for small ǫ : Varah’s separation Let A ∈ C n × n be given in block Schur form: � � � � RM − LR L C L U ∗ = U U ∗ , A = U U unitary , Λ( L ) ∩ Λ( M ) = ∅ . 0 0 M M � � � R � 2 + 1 = p 2 − 1 + p . Let s V = sep V λ ( L, M ), κ = � R � + Then for ǫ ≤ s V / (2 κ ), 2.6 p+||R||= κ 2.4 ⊆ Λ g V ( ǫ ) ǫ ( L ) ∪ Λ g V ( ǫ ) ǫ ( M ) , Λ ǫ ( A ) g V ( ε ) where p − ǫ/s V p g V ( ǫ ) = � . � � s V /(2 κ ) 1 4 − ǫ 1 p − ǫ 2 + s V s V 1 0 0.05 0.1 0.15 0.2