Nonlinear Preconditioning in PETSc Matthew Knepley PETSc Team - PowerPoint PPT Presentation

Nonlinear Preconditioning in PETSc Matthew Knepley ∈ PETSc Team Computation Institute University of Chicago Challenges in 21st Century Experimental Mathematical Computation ICERM, Providence, RI July 22, 2014 M. Knepley (UC) NPC ICERM ’14 1 / 53

The PETSc Team Bill Gropp Barry Smith Satish Balay Jed Brown Matt Knepley Lisandro Dalcin Hong Zhang Mark Adams Peter Brune M. Knepley (UC) NPC ICERM ’14 3 / 53

Why Experiment with Solvers? Asymptotic convergence rate Rarely get asymptotics in practice Really want the constant Manner of convergence Stationary iterative methods decrease high frequency error quickly Tuminaro, Walker, Shadid, JCP , 180, pp. 549-558 (2002). Robustness Line search Solver composition M. Knepley (UC) NPC ICERM ’14 4 / 53

Why Experiment with Solvers? Asymptotic convergence rate Rarely get asymptotics in practice Really want the constant Manner of convergence Stationary iterative methods decrease high frequency error quickly Tuminaro, Walker, Shadid, JCP , 180, pp. 549-558 (2002). Robustness Line search Solver composition M. Knepley (UC) NPC ICERM ’14 4 / 53

Why Experiment with Solvers? Asymptotic convergence rate Rarely get asymptotics in practice Really want the constant Manner of convergence Stationary iterative methods decrease high frequency error quickly Tuminaro, Walker, Shadid, JCP , 180, pp. 549-558 (2002). Robustness Line search Solver composition M. Knepley (UC) NPC ICERM ’14 4 / 53

Algorithmics Outline Algorithmics 1 Experiments 2 M. Knepley (UC) NPC ICERM ’14 5 / 53

Algorithmics Abstract System Out prototypical nonlinear equation is: x ) = � F ( � b (1) and we define the residual as x ) − � � r ( � x ) = F ( � b (2) M. Knepley (UC) NPC ICERM ’14 6 / 53

Algorithmics Abstract System Out prototypical nonlinear equation is: x ) = � F ( � b (1) and we define the (linear) residual as x − � � r ( � x ) = A � b (3) M. Knepley (UC) NPC ICERM ’14 6 / 53

Algorithmics Linear Left Preconditioning The modified equation becomes P − 1 � � x − � A � b = 0 (4) M. Knepley (UC) NPC ICERM ’14 7 / 53

Algorithmics Linear Left Preconditioning The modified defect correction equation becomes P − 1 � � x i − � A � = � x i + 1 − � b x i (5) M. Knepley (UC) NPC ICERM ’14 7 / 53

Algorithmics Additive Combination The linear iteration x i − ( α P − 1 + β Q − 1 )( A � x i − � � x i + 1 = � b ) (6) becomes the nonlinear iteration M. Knepley (UC) NPC ICERM ’14 8 / 53

Algorithmics Additive Combination The linear iteration x i − ( α P − 1 + β Q − 1 ) � � x i + 1 = � r i (7) becomes the nonlinear iteration M. Knepley (UC) NPC ICERM ’14 8 / 53

Algorithmics Additive Combination The linear iteration x i − ( α P − 1 + β Q − 1 ) � � x i + 1 = � r i (7) becomes the nonlinear iteration x i ,� x i ,� � x i + 1 = � x i + α ( N ( F ,� b ) − � x i )+ β ( M ( F ,� b ) − � x i ) (8) M. Knepley (UC) NPC ICERM ’14 8 / 53

Algorithmics Nonlinear Left Preconditioning From the additive combination, we have x i ,� P − 1 � ⇒ � x i − N ( F ,� r = b ) (9) so we define the preconditioning operation as x ,� � r L ≡ � x − N ( F ,� b ) (10) M. Knepley (UC) NPC ICERM ’14 9 / 53

Algorithmics Multiplicative Combination The linear iteration x i − ( P − 1 + Q − 1 − Q − 1 AP − 1 ) � � x i + 1 = � r i (11) becomes the nonlinear iteration M. Knepley (UC) NPC ICERM ’14 10 / 53

Algorithmics Multiplicative Combination The linear iteration x i − P − 1 � � x i + 1 / 2 = � r i (12) x i + 1 / 2 − Q − 1 � x i = � � r i + 1 / 2 (13) becomes the nonlinear iteration M. Knepley (UC) NPC ICERM ’14 10 / 53

Algorithmics Multiplicative Combination The linear iteration x i − P − 1 � � x i + 1 / 2 = � r i (12) x i + 1 / 2 − Q − 1 � � x i = � r i + 1 / 2 (13) becomes the nonlinear iteration x i ,� b ) , � � x i + 1 = M ( F , N ( F ,� b ) (14) M. Knepley (UC) NPC ICERM ’14 10 / 53

Algorithmics Nonlinear Right Preconditioning For the linear case, we have y = � AP − 1 � b (15) x = P − 1 � � y (16) so we define the preconditioning operation as y = M ( F ( N ( F , · ,� x i ,� � b )) , � b ) (17) y ,� � x = N ( F ,� b ) (18) M. Knepley (UC) NPC ICERM ’14 11 / 53

Algorithmics Nonlinear Preconditioning Type Sym Statement Abbreviation x ,� � x + α ( M ( F ,� b ) − � Additive + x ) M + N x ,� + β ( N ( F ,� b ) − � x ) x ,� b ) ,� M ( F , N ( F ,� Multiplicative ∗ b ) M ∗ N x ,� x ,� − L M ( � x − N ( F ,� b ) ,� M − L N Left Prec. b ) x ,� x ,� − R M ( F ( N ( F ,� b )) ,� M − R N Right Prec. b ) y = � x ) = K ( � y 0 ,� � J ( � x ) − 1 � r ( � J ( � x ) ,� \ NEWT \ K Inner Lin. Inv. b ) M. Knepley (UC) NPC ICERM ’14 12 / 53

Algorithmics Nonlinear Richardson 1: procedure NRICH( � x i ,� F ,� b ) � d = − � r ( � x i ) 2: x i + λ� � x i + 1 = � ⊲ λ determined by line search d 3: 4: end procedure 5: return � x i + 1 M. Knepley (UC) NPC ICERM ’14 13 / 53

Algorithmics Line Search Equivalent to NRICH − L N : NRICH − L N M. Knepley (UC) NPC ICERM ’14 14 / 53

Algorithmics Line Search Equivalent to NRICH − L N : NRICH − L N x ,� x ,� NRICH ( � x − N ( F ,� b ) ,� b ) M. Knepley (UC) NPC ICERM ’14 14 / 53

Algorithmics Line Search Equivalent to NRICH − L N : NRICH − L N x ,� x ,� NRICH ( � x − N ( F ,� b ) ,� b ) � x i + 1 = � x i − λ� r L M. Knepley (UC) NPC ICERM ’14 14 / 53

Algorithmics Line Search Equivalent to NRICH − L N : NRICH − L N x ,� x ,� NRICH ( � x − N ( F ,� b ) ,� b ) � x i + 1 = � x i − λ� r L x i ,� � x i + 1 = � x i + λ ( N ( F ,� b ) − � x i ) M. Knepley (UC) NPC ICERM ’14 14 / 53

Algorithmics Newton-Krylov 1: procedure NEWT \ K( � x i ,� F ,� b ) � d = � x i ,� J ( � x i ) − 1 � r ( � b ) ⊲ solve by Krylov method 2: x i + λ� � x i + 1 = � d ⊲ λ determined by line search 3: 4: end procedure 5: return � x i + 1 M. Knepley (UC) NPC ICERM ’14 15 / 53

Algorithmics Left Preconditioned Newton-Krylov x − � x ,� 1: procedure NEWT \ K( � M ( F ,� b ) ,� x i , 0) − 1 x i ,� d = ∂ ( � x i −M ( F ,� � b )) x i ,� ( � x i − M ( F ,� b )) 2: ∂� x i x i + λ� � x i + 1 = � d 3: 4: end procedure 5: return � x i + 1 M. Knepley (UC) NPC ICERM ’14 16 / 53

Algorithmics Jacobian Computation x i ,� x i ,� ∂ ( � x − M ( F ,� = I − ∂ M ( F ,� b )) b ) , � ∂� x i x i Direct differencing would require one inner nonlinear iteration per Krylov iteration. M. Knepley (UC) NPC ICERM ’14 17 / 53

Algorithmics Jacobian Computation x i ,� x i ,� ∂ ( � x − M ( F ,� = I − ∂ M ( F ,� b )) b ) , � ∂� x i x i Direct differencing would require one inner nonlinear iteration per Krylov iteration. M. Knepley (UC) NPC ICERM ’14 17 / 53

Algorithmics Jacobian Computation Impractical! x i ,� x i ,� ∂ ( � x − M ( F ,� = I − ∂ M ( F ,� b )) b ) , � ∂� x i x i Direct differencing would require one inner nonlinear iteration per Krylov iteration. M. Knepley (UC) NPC ICERM ’14 17 / 53

Algorithmics Jacobian Computation Approximation for NASM x − � x ,� ∂ ( � x − M ( F ,� = ∂ ( � x − ( � b J b ( � x b ) − 1 F b ( � x b ))) b )) ∂� ∂� x x � x b ∗ ) − 1 J ( � J b ( � ≈ x ) b This would require one inner nonlinear iteration small number of block solves per outer nonlinear iteration. X.-C. Cai and D. E. Keyes, SIAM J. Sci. Comput. , 24 (2002), pp. 183–200 M. Knepley (UC) NPC ICERM ’14 18 / 53

Algorithmics Right Preconditioned Newton-Krylov 1: procedure NK( � F ( � M ( � F , · ,� y i ,� b )) ,� b ) x i = � M ( � y i ,� � F ,� b ) 2: d = � � J ( � x ) − 1 � r ( � x i ) 3: x i + λ� � x i + 1 = � d ⊲ λ determined by line search 4: 5: end procedure 6: return � x i + 1 M. Knepley (UC) NPC ICERM ’14 19 / 53

Algorithmics Jacobian Computation First-Order Approximation Only the action of the original Jacobian is needed at first order: − 1 y i − λ∂ M ( F ,� y i ) y i )) − 1 F ( M ( F ,� � y i + 1 = � J ( M ( F ,� y i )) ∂� y i − 1 y i − λ∂ M ( F ,� y i ) y i )) − 1 F ( M ( F ,� M ( F ,� y i + 1 ) = M ( F ,� J ( M ( F ,� y i ))) ∂� y i ≈ M ( F ,� y i ) − 1 − λ∂ M ( F ,� ∂ M ( F ,� y i ) y i ) y i )) − 1 F ( M ( F ,� J ( M ( F ,� y i ))) ∂� ∂� y i y i y i )) − 1 F ( M ( F ,� = M ( F ,� y i ) − λ J ( M ( F ,� y i )) x i ) − 1 F ( � � x i + 1 = � x i − λ J ( � x i ) NEWT \ K − R � M is equivalent to NEWT \ K ∗ � M at first order M. Knepley (UC) NPC ICERM ’14 20 / 53