On two variants of incremental condition estimation Jurjen Duintjer Tebbens joint work with Miroslav T˚ uma Institute of Computer Science Academy of Sciences of the Czech Republic SNA11, Rožnov pod Radhoštˇ em, January 27, 2011.
1. Motivation: BIF ● This work is motivated by the recently introduced Balanced Incomplete Factorization (BIF) method for Cholesky (or LDU) decomposition [Bru, Marín, Mas, T˚ uma - 2008] ● The method is remarkable, among others, in that it computes the inverse triangular factors simultaneously during the factorization process. ● In BIF the presence of the inverse could be used to obtain robust dropping criteria ● In this talk we are interested in complete factorization (BIF without dropping any entries) 2 J. Duintjer Tebbens, M. T˚ uma
1. Motivation: BIF ● The main question is: How can the presence of the inverse factors be exploited in complete factorization? ● Perhaps the first thing that comes to mind, is to use the inverse triangular factors for improved condition estimation. Recall that κ ( L ) = σ max ( L ) σ min ( L ) = � L �� L − 1 � . ● In this talk we concentrate on estimation of the 2-norm condition number. ● We will see that exploiting the inverse factors for better condition estimation is possible, but not as straightforward as it may seem. 3 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation Traditionally, 2-norm condition estimators assume a triangular decomposition and compute estimates for the factors. E.g., if A is symmetric positive definite with Cholesky decomposition A = LL T , then the condition number of A satisfies κ ( A ) = κ ( L ) 2 = κ ( L T ) 2 . κ ( L T ) can be cheaply estimated with a technique called incremental condition number estimation. Main idea: Subsequent estimation of leading submatrices: L T = ⇒ all columns are accessed only once. k k+1 0 4 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation We will call the original incremental technique, introduced by Bischof [Bischof - 1990], simply incremental condition estimation (ICE): Let R be upper triangular with a given approximate maximal singular value σ maxICE ( R ) and corresponding approximate singular vector y, � y � = 1 , σ maxICE ( R ) = � y T R � ≈ σ max ( R ) = max � x � =1 � x T R � . ICE approximates the maximal singular value of the extended matrix � � R v R ′ = 0 γ by maximizing � �� � � R v � � c 2 + s 2 = 1 . � over all s, c satisfying sy, c � , � � 0 � γ � � 5 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation We have 2 � �� � � � � � � � � � R T 0 R v R v sy � � � � max = max sy, c sy, c � � v T 0 0 � γ � γ γ c s,c,c 2 + s 2 =1 s,c,c 2 + s 2 =1 � � σ maxICE ( R ) 2 + ( y T v ) 2 � � γ ( v T y ) � � � s � = max . s, c γ ( v T y ) γ 2 c s,c,c 2 + s 2 =1 The maximum is obtained with the normalized eigenvector corresponding to the maximum eigenvalue λ max ( B ICE ) of σ maxICE ( R ) 2 + ( y T v ) 2 γ ( v T y ) � � B ICE ≡ . γ ( v T y ) γ 2 � � � � ˆ ˆ s sy We denote the normalized eigenvector by and put ˆ y ≡ . ˆ ˆ c c 6 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation Then we define the approximate maximal singular value of the extended matrix as y T R ′ � ≈ σ max ( R ′ ) . σ maxICE ( R ′ ) ≡ � ˆ Similarly, if for some y with unit norm, σ minICE ( R ) = � y T R � ≈ σ min ( R ) = min � x � =1 � x T R � , then ICE uses the minimal eigenvalue λ min ( B ICE ) of σ minICE ( R ) 2 + ( y T v ) 2 � γ ( v T y ) � B ICE = γ ( v T y ) γ 2 y T and The corresponding eigenvector of B ICE yields the new vector ˆ y T R ′ � ≈ σ min ( R ′ ) . σ minICE ( R ′ ) ≡ � ˆ 7 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation Experiment: ● We generate 50 random matrices B of dimension 100 with the Matlab command B = randn ( 100 , 100 ) ● We compute the Cholesky decompositions LL T of the 50 symmetric positive definite matrices A = BB T ● We compute the estimations σ maxICE ( L T ) and σ minICE ( L T ) ● In the following graph we display the quality of the estimations through the number � 2 σ maxICE ( L T ) � σ minICE ( L T ) , κ ( A ) where κ ( A ) is the true condition number. Note that we always have � σ maxICE ( L T ) � 2 ≤ κ ( A ) . σ minICE ( L T ) 8 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 10 20 30 40 50 Quality of the estimator ICE for 50 random s.p.d. matrices of dimension 100. 9 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation With the decomposition method from [Bru, Marín, Mas, T˚ uma - 2008] we have to our disposal not only the Cholesky decomposition of A , A = LL T but also the inverse Cholesky factors as is the case in balanced factorization, i.e. we have A − 1 = L − T L − 1 . Then we can run ICE on L − T and use the additional estimations 1 1 σ maxICE ( L − T ) ≈ σ min ( L T ) , σ minICE ( L − T ) ≈ σ max ( L T ) . In the following graph we take the best of both estimations, we display � 2 max( σ maxICE ( L T ) ,σ minICE ( L − T ) − 1 ) � min( σ minICE ( L T ) ,σ maxICE ( L − T ) − 1 ) . κ ( A ) 10 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 10 20 30 40 50 Quality of ICE for 50 random s.p.d. matrices of dimension 100. 11 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 10 20 30 40 50 Quality of the estimator ICE without (black) and with exploiting (green) the inverse for 50 random s.p.d. matrices of dimension 100. 11 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation Plugging in the inverse gives no improvement! By comparing � � R v R ′ = σ maxICE ( R ′ ) , 0 γ with � − R − 1 v � R − 1 1 ( R ′ ) − 1 = γ σ minICE (( R ′ ) − 1 ) , , 1 0 γ it can be proven that 1 1 σ maxICE ( A ) = σ minICE ( A − 1 ) , σ minICE ( A ) = σ maxICE ( A − 1 ) , which we denote as ICE ( A ) = ICE ( A − 1 ) . 12 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation An alternative technique called incremental norm estimation (INE) was proposed in [Duff, Vömel - 2002]: Let R be upper triangular with given approximate maximal singular value σ maxINE ( R ) and corresponding approximate right singular vector z , � z � = 1 , σ maxINE ( R ) = � Rz � ≈ σ max ( R ) = max � x � =1 � Rx � . INE approximates the maximal singular value of the extended matrix � � R v R ′ = 0 γ by maximizing � �� � � � R v sz � � over all s, c satisfying c 2 + s 2 = 1 . � , � � � 0 γ c � � 13 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation We have 2 � �� � � � � � � � � � � � R T 0 R v sz sz R v sz � � max = max � � v T 0 0 � γ c � c γ γ c s,c,c 2 + s 2 =1 s,c,c 2 + s 2 =1 � � � � z T R T Rz z T R T v � � � s � = max . s c z T R T v v T v + γ 2 c s,c,c 2 + s 2 =1 The maximum is obtained for the normalized eigenvector corresponding to the maximum eigenvalue λ max ( B INE ) of � z T R T Rz z T R T v � B INE ≡ . z T R T v v T v + γ 2 � � � � ˆ ˆ s sz We denote the normalized eigenvector by and put ˆ z ≡ . c ˆ ˆ c 14 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation Then we define the approximate maximal singular value of the extended matrix as σ maxINE ( R ′ ) ≡ � R ′ ˆ z � ≈ σ max ( R ′ ) . Similarly, if for a unit vector z , � Rz � ≈ σ min ( R ) = min � x � =1 � Rx � , then INE uses the minimal eigenvalue λ min ( B INE ) of � z T R T Rz z T R T v � B INE = . z T R T v v T v + γ 2 The corresponding eigenvector of B INE yields the new vector ˆ z and σ minINE ( R ′ ) ≡ � R ′ ˆ z � ≈ σ min ( R ′ ) . 15 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation By comparing � � R v R ′ = σ maxINE ( R ′ ) , 0 γ with � − R − 1 v � R − 1 1 ( R ′ ) − 1 = γ σ minINE (( R ′ ) − 1 ) , , 1 0 γ it can be checked, that for INE we have in general INE ( A ) � = INE ( A − 1 ) ! Let us demonstrate this in the same experiment as before. 16 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 10 20 30 40 50 Quality of the estimator INE(A) 17 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 10 20 30 40 50 Quality of the estimator INE(A) (black) and of INE( A − 1 ) (green). 17 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 10 20 30 40 50 Quality of the estimators INE(A) (black), INE( A − 1 ) (green) and best of both INE(A) and INE( A − 1 ) (red). 17 J. Duintjer Tebbens, M. T˚ uma
2. Condition estimation 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 10 20 30 40 50 Quality of the estimators INE(A) (black), INE( A − 1 ) (green), best of both INE(A) and INE( A − 1 ) (red) and ICE(A) (blue). 17 J. Duintjer Tebbens, M. T˚ uma
Recommend
More recommend
