Scalability analysis of the distributed-memory implementation of the - PowerPoint PPT Presentation

Scalability analysis of the distributed-memory implementation of the Aggregated unfitted Finite Element Method (AgFEM) Alberto F. Martín ∗ , Santiago Badia, Francesc Verdugo, Eric Neiva MWNDEA 2020, Melbourne, Australia, 12/02/2020

Embedded Finite elements CutFEM, Finite Cell Method, AgFEM, X-FEM, ... body-fitted mesh unfitted mesh ✔ Simplified mesh generation Alberto F . Martín (Monash University) MWNDEA2020 2/35

Embedded Finite elements CutFEM, Finite Cell Method, AgFEM, X-FEM, ... body-fitted mesh unfitted mesh ✔ Simplified mesh generation ✘ Dirichlet BC? ✘ Numerical integration? ✘ ill-conditioning? (this talk) Alberto F . Martín (Monash University) MWNDEA2020 2/35

Parallel distributed-memory simulation pipeline 1. Unfitted (adaptive) Cartesian grids (p4est) Alberto F . Martín (Monash University) MWNDEA2020 3/35

Parallel distributed-memory simulation pipeline 1. Unfitted (adaptive) Cartesian grids (p4est) 2. Partition using space filling-curves (p4est) Alberto F . Martín (Monash University) MWNDEA2020 3/35

Parallel distributed-memory simulation pipeline 1. Unfitted (adaptive) Cartesian grids (p4est) 2. Partition using space filling-curves (p4est) 3. Unfitted FE discretization (AgFEM) 4. AMG linear solver (PETSc) Alberto F . Martín (Monash University) MWNDEA2020 3/35

Unfitted methods at large scales: pros and cons ✔ Highly scalable mesh generation based on octrees (e.g. p4est ) Alberto F . Martín (Monash University) MWNDEA2020 4/35

Unfitted methods at large scales: pros and cons ✔ Highly scalable mesh generation based on octrees (e.g. p4est ) ✔ Highly scalable mesh partition with space-filling curves (Parmetis not needed) Alberto F . Martín (Monash University) MWNDEA2020 4/35

Unfitted methods at large scales: pros and cons ✔ Highly scalable mesh generation based on octrees (e.g. p4est ) ✔ Highly scalable mesh partition with space-filling curves (Parmetis not needed) ✔ Highly scalable adaptive mesh refinement + load balancing Alberto F . Martín (Monash University) MWNDEA2020 4/35

Unfitted methods at large scales: pros and cons ✔ Highly scalable mesh generation based on octrees (e.g. p4est ) ✔ Highly scalable mesh partition with space-filling curves (Parmetis not needed) ✔ Highly scalable adaptive mesh refinement + load balancing ✘ Not guaranteed that highly scalable linear solvers keep their optimal properties for cut elements. Alberto F . Martín (Monash University) MWNDEA2020 4/35

PETSc CG + AMG preconditioner on unfitted meshes Poisson equation (weak scaling test with 5 meshes) AMG+AgFEM AMG + Naive un fi tted FEM* 16 16 14 14 GC iterations GC iterations 12 12 10 10 8 8 6 6 4 4 10 3 10 4 10 5 10 6 10 7 10 8 10 3 10 4 10 5 10 6 10 7 10 8 DOFs DOFs * Nitsche BCs + modified integration in cut cells Alberto F . Martín (Monash University) MWNDEA2020 5/35

Why linear solvers are affected by cut cells? Condition number estimates (Poisson Eq.) (a) Body-fitted case (b) Naive unfitted FEM k 2 ( A ) ∼ | η | − (2 p +1 − 2 k 2 ( A ) ∼ h − 2 d ) "small cut cell problem" Alberto F . Martín (Monash University) MWNDEA2020 6/35

Possible remedies Fix the linear solver Taylor your parallel solver to deal with k 2 ( A ) ∼ | η | − (2 p +1 − 2 /d ) Example: [S. Badia, F . Verdugo. Robust and scalable domain decomposition solvers for unfitted finite element methods. Journal of Computational and Applied Mathematics (2018) ]. Fix the linear system (this talk) Enhance the unfitted FE method so that k 2 ( A ) ∼ h − 2 Use a standard scalable solver Examples: CutFEM, AgFEM Alberto F . Martín (Monash University) MWNDEA2020 7/35

Agenda 1. The AgFEM method (serial case) 2. Parallel implementation 3. Performance of parallel AgFEM + AMG solvers Alberto F . Martín (Monash University) MWNDEA2020 8/35

Agenda 1. The AgFEM method (serial case) 2. Parallel implementation 3. Performance of parallel AgFEM + AMG solvers Alberto F . Martín (Monash University) MWNDEA2020 9/35

AgFEM method for the Poisson Eq. � − ∆ u = f in Ω u = u D on ∂ Ω Alberto F . Martín (Monash University) MWNDEA2020 10/35

Weak imposition of Dirichlet BCs Nitsche’s Method Find u h ∈ V h such that ∀ v h ∈ V h a h ( u h , v h ) = l h ( v h ) ( v h does not vanish on ∂ Ω !) where � � ∇ u h · ∇ v h − ( ∇ u h · n ) v h � � a h ( u h , v h ) := K ∩ Ω F K ∈T act F ∈T act ∩ ∂ Ω h h � � u h ( ∇ v h · n ) + u h v h � � βh − 1 − F F ∂ Ω F ∈ ( T act F ∈ ( T act ∩ ∂ Ω) ∩ ∂ Ω) h h � � � f v h + u D v h − ∇ v h · n � � � l h ( v h ) := βh − 1 u D � � F K ∩ Ω F F K ∈T act F ∈ ( T act F ∈ ( T act ∩ ∂ Ω) ∩ ∂ Ω) h h h The key feature of AgFEM is the definition of the discrete space V h Alberto F . Martín (Monash University) MWNDEA2020 11/35

Starting point: "naive" FE space V std := { u ∈ C 0 (Ω act ) : u | K ∈ Q p ( K ) ∀ K ∈ T act } h h T act , Ω act V std h h Alberto F . Martín (Monash University) MWNDEA2020 12/35

Aggregated FE space Basic idea: improve conditioning by removing problematic DOFs     V agg � :=  u ∈ V h : u × = C × • u • ∀ × ∈ P h  •∈ masters( × ) • well-posed dofs × problematic dofs ( P ) Alberto F . Martín (Monash University) MWNDEA2020 13/35

Definition of constraints via cell aggregates Alberto F . Martín (Monash University) MWNDEA2020 14/35

Definition of constraints via cell aggregates 1. Generate cell aggregates (1 interior cell + several cut cells) Alberto F . Martín (Monash University) MWNDEA2020 14/35

Definition of constraints via cell aggregates 1. Generate cell aggregates (1 interior cell + several cut cells) 2. Define dof to root cell map root( × ) via the aggregates Alberto F . Martín (Monash University) MWNDEA2020 14/35

Definition of constraints via cell aggregates 1. Generate cell aggregates (1 interior cell + several cut cells) 2. Define dof to root cell map root( × ) via the aggregates 3. Define constraints: φ root( × ) � u × = ( x × ) u • • •∈ dofs(root( × )) Alberto F . Martín (Monash University) MWNDEA2020 14/35

Results for the unfitted aggregated FEM (Poisson Eq.) 1 κ ( A ) ≤ c 1 h − 2 (Condition number bound) β ≤ c 2 h − 2 (Nitsche’s penalty coef.) � u − u h � H 1 (Ω) ≤ c 3 h p (Optimal convergence order) � u − u h � L 2 (Ω) ≤ c 4 h p +1 (Optimal convergence order) and others (inverse/trace inequalities, bound of aggregate size, bound of the extended solution, ...) 1 [Badia, Verdugo, Martín. The aggregated unfitted finite element method for elliptic problems. Comput. Methods Appl. Mech. Eng. (2018).] Alberto F . Martín (Monash University) MWNDEA2020 15/35

30 p=1, standard p=2, standard p=1, aggr. 25 p=2, aggr. log10(condest(A)) 20 15 10 5 0 0.3 0.4 0.5 0.6 0.7 ℓ Alberto F . Martín (Monash University) MWNDEA2020 16/35

30 p=1, standard 25 p=2, standard p=1, aggr. log10(condest(A)) p=2, aggr. 20 15 10 5 0 0.3 0.4 0.5 0.6 0.7 ℓ Alberto F . Martín (Monash University) MWNDEA2020 17/35

Convergence test 1 1 0 0 -1 -1 -2 -2 log10(Error energy norm) log10(Error energy norm) -3 -3 -4 -4 -5 -5 -6 -6 p=1, standard p=1, standard p=2, standard p=2, standard p=1, aggr. p=1, aggr. p=2, aggr. p=2, aggr. -7 -7 slope 1 slope 1 slope 2 slope 2 -8 -8 -2.5 -2 -1.5 -1 -2.5 -2 -1.5 -1 log10(h) log10(h) (a) 2D (b) 3D Alberto F . Martín (Monash University) MWNDEA2020 18/35

Extension to the Stokes problem 2 75 . 0 | u | 0 . 0  − ∆ u + ∇ p = f in Ω    ∇ · u = 0 in Ω   u = 0 on Γ D     ( ∇ u − pI ) · n = g on Γ N  2 [Badia, Martín, Verdugo. Mixed aggregated finite element methods for the unfitted discretization of the stokes problem. SIAM J. Sci. Comput., 40(6). 2018.] Alberto F . Martín (Monash University) MWNDEA2020 19/35

Alberto F . Martín (Monash University) MWNDEA2020 20/35

40 ❆❣❣r❡❣❛t❡❞ 35 ❙t❛♥❞❛r❞ log 10 ( ❝♦♥❞❡st ( A )) 30 25 20 15 10 5 0 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 ℓ 40 35 log 10 ( ❝♦♥❞❡st ( A )) 30 25 ❆❣❣r❡❣❛t❡❞ 20 ❙t❛♥❞❛r❞ 15 10 5 0 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 ℓ Alberto F . Martín (Monash University) MWNDEA2020 21/35

− 2 − 2 − 2 � � � − 3 − 3 − 3 � u h − u � H 1 � u h − u � L 2 � p h − p � L 2 � u � H 1 � u � L 2 � p � L 2 − 4 − 4 − 4 � � � log 10 log 10 log 10 − 5 − 5 − 5 ❆❣❣r❡❣❛t❡❞ ❆❣❣r❡❣❛t❡❞ ❆❣❣r❡❣❛t❡❞ ❙t❛♥❞❛r❞ ❙t❛♥❞❛r❞ ❙t❛♥❞❛r❞ s❧♦♣❡ ✲✷ s❧♦♣❡ ✲✸ s❧♦♣❡ ✲✷ − 6 − 6 − 6 1 . 2 1 . 4 1 . 6 1 . 8 2 . 0 1 . 2 1 . 4 1 . 6 1 . 8 2 . 0 1 . 2 1 . 4 1 . 6 1 . 8 2 . 0 � (DOF) 1 /d � � (DOF) 1 /d � � (DOF) 1 /d � log 10 log 10 log 10 Alberto F . Martín (Monash University) MWNDEA2020 22/35

Agenda 1. The AgFEM method (serial case) 2. Parallel implementation 3. Performance of parallel AgFEM + AMG solvers Alberto F . Martín (Monash University) MWNDEA2020 23/35