Parallel solution of large sparse eigenproblems using a - PowerPoint PPT Presentation

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Parallel solution of large sparse eigenproblems using a Block-Jacobi-Davidson method Melven Röhrig-Zöllner Simulation and Software Technology

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Background Aim: Calculate a set of outer eigenpairs ( λ i , v i ) of a large sparse matrix: Av i = λ i v i Project: Equipping Sparse Solvers for Exascale (ESSEX) of the DFG SPPEXA programme

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Outline Block-Jacobi-Davidson algorithm Performance analysis Implementation Results

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Outline Block-Jacobi-Davidson algorithm Jacobi-Davidson QR method Block JDQR method Performance analysis Implementation Results

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Jacobi-Davidson QR method Background ◮ Introduced 1998 by Fokkema et. al. ◮ Iteratively calculates a small set of eigenvalues (near a specific target)

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Jacobi-Davidson QR method Background ◮ Introduced 1998 by Fokkema et. al. ◮ Iteratively calculates a small set of eigenvalues (near a specific target) ◮ Based on a subspace iteration and a correction equation

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Jacobi-Davidson QR method Background ◮ Introduced 1998 by Fokkema et. al. ◮ Iteratively calculates a small set of eigenvalues (near a specific target) ◮ Based on a subspace iteration and a correction equation ◮ Motivated by the Rayleigh quotient iteration (RQI) � = ( A − λ k I ) − 1 v k , ¯ v k + 1 v k + 1 ¯ v k + 1 = � v k + 1 � RQI: = v T λ k + 1 k + 1 Av k + 1 ◮ Can also be derived from an inexact Newton process

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Jacobi-Davidson QR method Sketch of the algorithm 1: while not converged do ⊲ Outer iteration Project the problem to a small subspace 2: Solve the small eigenvalue problem 3: Calculate an approximation and its residual 4: Approximately solve the correction equation ⊲ Inner iteration 5: Orthogonalize the new direction 6: Enlarge the subspace 7: 8: end while

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Outer iteration Subspace iteration Deflation

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Outer iteration Subspace iteration ◮ Galerkin projection onto a small subspace W = span { w 1 , w 2 , . . . } : Av − λ v ⊥ W , v ∈ W ( W T AW ) s − ˜ ⇔ λ s = 0 , for orth. basis W Deflation

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Outer iteration Subspace iteration ◮ Galerkin projection onto a small subspace W = span { w 1 , w 2 , . . . } : Av − λ v ⊥ W , v ∈ W ( W T AW ) s − ˜ ⇔ λ s = 0 , for orth. basis W ◮ Approximation ˜ λ with ˜ v = Ws Deflation

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Outer iteration Subspace iteration ◮ Galerkin projection onto a small subspace W = span { w 1 , w 2 , . . . } : Av − λ v ⊥ W , v ∈ W ( W T AW ) s − ˜ ⇔ λ s = 0 , for orth. basis W ◮ Approximation ˜ λ with ˜ v = Ws Deflation ◮ Project out the already known eigenvector space Q = span { q 1 , q 2 , . . . } : ¯ A := ( I − QQ T ) A ( I − QQ T )

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Inner iteration Jacobi-Davidson correction equation v − ˜ ◮ Based on the current approximation A ˜ λ ˜ v = r

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Inner iteration Jacobi-Davidson correction equation v − ˜ ◮ Based on the current approximation A ˜ λ ˜ v = r λ I ) − 1 through a deflation of ◮ Avoid the (near) singularity of ( A − ˜ ˜ � Q v � Q = ˜

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Inner iteration Jacobi-Davidson correction equation v − ˜ ◮ Based on the current approximation A ˜ λ ˜ v = r λ I ) − 1 through a deflation of ◮ Avoid the (near) singularity of ( A − ˜ ˜ � Q v � Q = ˜ → Solve approximately: ( I − ˜ Q ˜ Q T )( A − ˜ λ I )( I − ˜ Q ˜ Q T ) w k + 1 = − r

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Inner iteration Jacobi-Davidson correction equation v − ˜ ◮ Based on the current approximation A ˜ λ ˜ v = r λ I ) − 1 through a deflation of ◮ Avoid the (near) singularity of ( A − ˜ ˜ � Q v � Q = ˜ → Solve approximately: ( I − ˜ Q ˜ Q T )( A − ˜ λ I )( I − ˜ Q ˜ Q T ) w k + 1 = − r ◮ Use some steps of an iterative solver (GMRES, BiCGStab, . . . )

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Inner iteration Jacobi-Davidson correction equation v − ˜ ◮ Based on the current approximation A ˜ λ ˜ v = r λ I ) − 1 through a deflation of ◮ Avoid the (near) singularity of ( A − ˜ ˜ � Q v � Q = ˜ → Solve approximately: ( I − ˜ Q ˜ Q T )( A − ˜ λ I )( I − ˜ Q ˜ Q T ) w k + 1 = − r ◮ Use some steps of an iterative solver (GMRES, BiCGStab, . . . ) ◮ Provides a new direction w k + 1 for the subspace iteration

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Block JDQR method Idea ◮ Calculate corrections for n b eigenvalues at once Numerical properties

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Block JDQR method Idea ◮ Calculate corrections for n b eigenvalues at once ◮ Block correction equation with ˜ � Q � Q = v 1 ˜ . . . v n b ˜ : ( I − ˜ Q ˜ Q T )( A − ˜ λ i I )( I − ˜ Q ˜ Q T ) w k + i = − r i i = 1 , . . . , n b Numerical properties

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Block JDQR method Idea ◮ Calculate corrections for n b eigenvalues at once ◮ Block correction equation with ˜ � Q � Q = v 1 ˜ . . . ˜ v n b : ( I − ˜ Q ˜ Q T )( A − ˜ λ i I )( I − ˜ Q ˜ Q T ) w k + i = − r i i = 1 , . . . , n b → Approximately solve n b linear systems at once Numerical properties

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Block JDQR method Idea ◮ Calculate corrections for n b eigenvalues at once ◮ Block correction equation with ˜ � Q � Q = v 1 ˜ . . . ˜ v n b : ( I − ˜ Q ˜ Q T )( A − ˜ λ i I )( I − ˜ Q ˜ Q T ) w k + i = − r i i = 1 , . . . , n b → Approximately solve n b linear systems at once ◮ Provides new directions w k + 1 , . . . , w k + n b for the subspace iteration Numerical properties

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Block JDQR method Idea ◮ Calculate corrections for n b eigenvalues at once ◮ Block correction equation with ˜ � Q � Q = v 1 ˜ . . . v n b ˜ : ( I − ˜ Q ˜ Q T )( A − ˜ λ i I )( I − ˜ Q ˜ Q T ) w k + i = − r i i = 1 , . . . , n b → Approximately solve n b linear systems at once ◮ Provides new directions w k + 1 , . . . , w k + n b for the subspace iteration Numerical properties ◮ More robust ◮ Usually needs more operations

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Block JDQR method Sketch of the complete algorithm 1: 2: while not converged do Project the problem to a small subspace 3: Solve the small eigenvalue problem 4: Calculate an n b approximations and their residual 5: 6: 7: Approximately solve the n b correction equations 8: Block-Orthogonalize the new directions 9: Enlarge the subspace 10: 11: end while

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Block JDQR method Sketch of the complete algorithm 1: Setup initial subspace 2: while not converged do Project the problem to a small subspace 3: Solve the small eigenvalue problem 4: Calculate an n b approximations and their residual 5: Lock converged eigenvalues 6: Shrink subspace if required (thick restart) 7: Approximately solve the n b correction equations 8: Block-Orthogonalize the new directions 9: Enlarge the subspace 10: 11: end while

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Outline Block-Jacobi-Davidson algorithm Performance analysis Required linear algebra operations spMMVM single node spMMVM inter-node Block vector operations Jacobi-Davidson Operator Implementation Results

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Required linear algebra operations Sparse matrix-multiple-vector multiplication (spMMVM) Block vector operations

Block-Jacobi-Davidson algorithm Performance analysis Implementation Results Required linear algebra operations Sparse matrix-multiple-vector multiplication (spMMVM) ◮ Large distributed sparse matrix A in CRS or SELL-C- σ format ◮ Distributed blocks of vectors X , Y ∈ R n × n b ◮ Shifted spMMVM: y i ← ( A − ˜ λ i I ) x i , i = 1 , . . . , n b Block vector operations