Westf¨ alische Wilhelms-Universit¨ at M¨ unster Stencil-like operations on unstructured meshes wissen leben Christian Engwer 13.04.2015, WWU M¨ unster joint work with P. Bastian, Schloss Dagstuhl J. Fahlke, S. M¨ uthing
Westf¨ alische Wilhelms-Universit¨ at Stencil-like operations on unstructured meshes 1 /20 M¨ unster What we aim at... Solving Partiell Differential Equations ◮ Wide range of applications ◮ In general requires unstructured meshes ◮ Not accessible to classical stencil approaches unster wissen leben WWU M¨ , , WWU M¨ unster Christian Engwer 13.04.2015, Schloss Dagstuhl
Westf¨ alische Wilhelms-Universit¨ at Stencil-like operations on unstructured meshes 1 /20 M¨ unster What we aim at... Solving Partiell Differential Equations ◮ Wide range of applications ◮ In general requires unstructured meshes ◮ Not accessible to classical stencil approaches → unster wissen leben WWU M¨ , , WWU M¨ unster Christian Engwer 13.04.2015, Schloss Dagstuhl
Westf¨ alische Wilhelms-Universit¨ at Stencil-like operations on unstructured meshes 1 /20 M¨ unster What we aim at... Solving Partiell Differential Equations ◮ Wide range of applications ◮ In general requires unstructured meshes ◮ Not accessible to classical stencil approaches unster wissen leben WWU M¨ [Unat et.al. – 2012] , , WWU M¨ unster Christian Engwer 13.04.2015, Schloss Dagstuhl
Westf¨ alische Wilhelms-Universit¨ at Stencil-like operations on unstructured meshes 2 /20 M¨ unster Outline 1 The EXA-DUNE Project 2 Stencils vs. FEM? 3 Introducing Local Structure unster 4 Exploiting Local Structure for Vectorization wissen leben WWU M¨ 5 Matrix-free vs. matrix-based solvers 6 Discussion , , WWU M¨ unster Christian Engwer 13.04.2015, Schloss Dagstuhl
Westf¨ alische Wilhelms-Universit¨ at Stencil-like operations on unstructured meshes 3 /20 M¨ unster The EXA-DUNE Project Framework approach to software development applications ◮ Open-Source C++ framework ◮ Integrated toolbox of simulation discretization modules extra grids external modules pdelab fem components ◮ Existing body of complex applications core modules grid istl localfunctions ◮ Scalability for traditional MPI unster [Bastian, Blatt, Dedner, E, Kl¨ ofkorn, wissen leben Kornhuber, Ohlberger, Sander 2008] http://www.dune-project.org/ WWU M¨ , , WWU M¨ unster Christian Engwer 13.04.2015, Schloss Dagstuhl
Westf¨ alische Wilhelms-Universit¨ at Stencil-like operations on unstructured meshes 3 /20 M¨ unster The EXA-DUNE Project Framework approach to software development applications ◮ Open-Source C++ framework ◮ Integrated toolbox of simulation discretization modules extra grids external modules pdelab fem components ◮ Existing body of complex applications core modules grid istl localfunctions ◮ Scalability for traditional MPI unster [Bastian, Blatt, Dedner, E, Kl¨ ofkorn, wissen leben Kornhuber, Ohlberger, Sander 2008] Challenges http://www.dune-project.org/ WWU M¨ ◮ Incorporate new algorithms, hardware paradigms ◮ Integrate changes across simulation stages (Ahmdahl’s Law) ◮ Provide “reasonable” upgrade path for existing applications , , WWU M¨ unster Christian Engwer 13.04.2015, Schloss Dagstuhl
Westf¨ alische Wilhelms-Universit¨ at Stencil-like operations on unstructured meshes 4 /20 M¨ unster The EXA-DUNE Project (2) Framework approach to software development ◮ DUNE + FEAST = Flexibility + MPI M M Performance M M CPU CPU CPU Accl UMA ◮ General Software Frameworks ... L2 L2 L2 IC IC IC IC L1 L1 L1 M M M M ... P P P SIMD ... ... → co-designed to specific hardware platforms is not sufficient EXA-DUNE hardware model ◮ Hardware-Oriented Numerics Coarse-grained : MPI between unster wissen leben heterogeneous nodes → standard low order algorithms Medium-grained : multicore-CPUs, do not scale any more GPUs, MICs, APUs, ... WWU M¨ Fine-grained : vectorization, ⇒ Much more than a pure implementational issue GPU ‘threads’, ... , , WWU M¨ unster Christian Engwer 13.04.2015, Schloss Dagstuhl
Westf¨ alische Wilhelms-Universit¨ at Stencil-like operations on unstructured meshes 4 /20 M¨ unster The EXA-DUNE Project (2) Framework approach to software development ◮ DUNE + FEAST = Flexibility + MPI M M Performance M M CPU CPU CPU Accl UMA ◮ General Software Frameworks ... L2 L2 L2 IC IC IC IC L1 L1 L1 M M M M ... P P P SIMD ... ... → co-designed to specific hardware platforms is not sufficient EXA-DUNE hardware model ◮ Hardware-Oriented Numerics Coarse-grained : MPI between unster wissen leben heterogeneous nodes → standard low order algorithms Medium-grained : multicore-CPUs, do not scale any more GPUs, MICs, APUs, ... WWU M¨ ⇒ Much more than a pure Fine-grained : vectorization, GPU ‘threads’, ... implementational issue , , WWU M¨ unster Christian Engwer 13.04.2015, Schloss Dagstuhl
Westf¨ alische Wilhelms-Universit¨ at Stencil-like operations on unstructured meshes 4 /20 M¨ unster The EXA-DUNE Project (2) Framework approach to software development ◮ DUNE + FEAST = Flexibility + MPI M M Performance M M CPU CPU CPU Accl UMA ◮ General Software Frameworks ... L2 L2 L2 IC IC IC IC L1 L1 L1 M M M M ... P P P SIMD ... ... → co-designed to specific hardware platforms is not sufficient EXA-DUNE hardware model ◮ Hardware-Oriented Numerics Coarse-grained : MPI between unster wissen leben heterogeneous nodes → standard low order algorithms Medium-grained : multicore-CPUs, do not scale any more GPUs, MICs, APUs, ... WWU M¨ ⇒ Much more than a pure implementational issue PS: CPUs are catching up: E5-2699v3: 0.9TF@145W, K80: 2.9TF@300W , , WWU M¨ unster Christian Engwer 13.04.2015, Schloss Dagstuhl
Westf¨ alische Wilhelms-Universit¨ at Stencil-like operations on unstructured meshes 5 /20 M¨ unster Stencil vs. FEM? Stencil approach ◮ Structured data layout ◮ Define update operation, based on local neighbourhood as y i = f ( x i , N ( x i )) for each i ∈ [0 , # DOFs − 1] , for some neighbourhood N unster wissen leben ◮ Data parallel pattern ◮ Easily vectorized WWU M¨ , , WWU M¨ unster Christian Engwer 13.04.2015, Schloss Dagstuhl
Westf¨ alische Wilhelms-Universit¨ at Stencil-like operations on unstructured meshes 6 /20 M¨ unster Stencil vs. FEM? Local Stiffness Matrix approach ◮ Based on a weak formulation ◮ Loop over cells ◮ Compute local contributions to global residual/stiffness-matrix ◮ Similar to Map/Reduce pattern � R E A E R T A = E E unster A E = ( a ( φ i , φ j )) i , j with supp φ ∩ E � = ∅ wissen leben WWU M¨ , , WWU M¨ unster Christian Engwer 13.04.2015, Schloss Dagstuhl
Westf¨ alische Wilhelms-Universit¨ at Stencil-like operations on unstructured meshes 6 /20 M¨ unster Stencil vs. FEM? Local Stiffness Matrix approach ◮ Based on a weak formulation ◮ Loop over cells ◮ Compute local contributions to global residual/stiffness-matrix ◮ Similar to Map/Reduce pattern � R E A E R T A = E E unster wissen leben A E = ( a ( φ i , φ j )) i , j with supp φ ∩ E � = ∅ Challenges: WWU M¨ ◮ size of A E varies ◮ indirect memory access (gather/scatter) ◮ read/write conflicts , , WWU M¨ unster Christian Engwer 13.04.2015, Schloss Dagstuhl
Westf¨ alische Wilhelms-Universit¨ at Stencil-like operations on unstructured meshes 7 /20 M¨ unster Possible Patterns ◮ Neighbor data ◮ CCFV and DG method ◮ simililar to classical stencil semantics ◮ explicit read access to neighbor data ◮ Vertex, Face or Edge data ◮ e.g. Conforming FEM, Raviar-Thomas, Nedelec unster ◮ several cells contributing to the same wissen leben DOF ◮ implicitly shared: read/write conflicts WWU M¨ ◮ Element local data ◮ DG methods ◮ No coupling to other cells ◮ simple, no conflicts , , WWU M¨ unster Christian Engwer 13.04.2015, Schloss Dagstuhl
Westf¨ alische Wilhelms-Universit¨ at Stencil-like operations on unstructured meshes 7 /20 M¨ unster Possible Patterns ◮ Neighbor data ◮ CCFV and DG method ◮ simililar to classical stencil semantics ◮ explicit read access to neighbor data ◮ Vertex, Face or Edge data ◮ e.g. Conforming FEM, Raviar-Thomas, Nedelec unster ◮ several cells contributing to the same wissen leben DOF ◮ implicitly shared: read/write conflicts WWU M¨ ◮ Element local data ◮ DG methods ◮ No coupling to other cells ◮ simple, no conflicts , , WWU M¨ unster Christian Engwer 13.04.2015, Schloss Dagstuhl
Westf¨ alische Wilhelms-Universit¨ at Stencil-like operations on unstructured meshes 7 /20 M¨ unster Possible Patterns ◮ Neighbor data ◮ CCFV and DG method ◮ simililar to classical stencil semantics ◮ explicit read access to neighbor data ◮ Vertex, Face or Edge data ◮ e.g. Conforming FEM, Raviar-Thomas, Nedelec unster ◮ several cells contributing to the same wissen leben DOF ◮ implicitly shared: read/write conflicts WWU M¨ ◮ Element local data ◮ DG methods ◮ No coupling to other cells ◮ simple, no conflicts , , WWU M¨ unster Christian Engwer 13.04.2015, Schloss Dagstuhl
Recommend
More recommend
