XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion eXtended Discrete Element Method (XDEM) Research on UL HPC platform A. Rousset X. Besseron W. Mainassara LuXDEM Research Team, RUES Prof. Bernhard Peters http://luxdem.uni.lu UL HPC School 2017 June 12, 2017 eXtended Discrete Element Method (XDEM) Research on UL HPC platform 1 / 16
XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion XDEM = Extended Discrete Element Method XDEM software is multi-physics simulation toolbox modeling granular materials and processes Particle Motion Chemical Reactions Snow, Sand, ... Coke, Iron ore, Biomass, ... + coupling external libraries: CFD with OpenFoam, FEM with Diffpack eXtended Discrete Element Method (XDEM) Research on UL HPC platform 2 / 16
XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion Domains of Applications • Storage and transport of granular material • Mining and its machinery • Agriculture and its machinery • Processing industry: Fluidised beds, fixed and moving bed reactors for • Drying • Thermal conversion (combustion, gasification) • Processing of raw materials • Pharmaceutical industry e.g. coating, drug production • Food industry (transport, coating, processing) • Material science • Additive manufacturing eXtended Discrete Element Method (XDEM) Research on UL HPC platform 3 / 16
XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion XDEM Research Examples Blast furnace Rolling tire on snow eXtended Discrete Element Method (XDEM) Research on UL HPC platform 4 / 16
XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion XDEM Workflow (Dynamic phase) Distance/Overlap Particle A Particle B at time t at time t Interaction/Collision/Contact Interaction step − Impact − Attraction Force, Torque Species fractions − Bond − Radiation Timestep t Heat flux, etc. − Chemical reaction Resultants on Resultants on particle B particle A Integration Integration New acceleration, velocity, position New acceleration, velocity, position Integration step temperature, species concentration temperature, species concentration − Dynamics solvers volume, mass, etc. for particle A volume, mass, etc. for particle B − Conversion solvers Update Update Timestep t+Δt Particle A Particle B at time t+Δt at time t+Δt eXtended Discrete Element Method (XDEM) Research on UL HPC platform 5 / 16
XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion Why using UL HPC facilities? Granular dynamics applications • Computation intensive = Time consumming • Require the use of UL HPC facilities eXtended Discrete Element Method (XDEM) Research on UL HPC platform 6 / 16
XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion Usage of UL HPC platform XDEM model studies • model validation (using parameter studies) • execution of full simulations Parallel/distributed code development • Debugging, validation • Scalability studies eXtended Discrete Element Method (XDEM) Research on UL HPC platform 7 / 16
XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion XDEM - Parallel code design Domain Decomposition • Domain = whole simulation space • Cell = regular subdivision with a fixed size • Partition = subset of cells assiged to a given process Partitioner and Load-Balancer • Responsible for assigning a cell to a process • Balance the workload • Based on Orthogonal Recursive Bisection (ORB), Zoltan, PHG, etc... MPI Parallel Simulation Driver • Schedule the modules on all the processes • Exchange the data between processes (Ghost particles in ghost cells) • Independent of the module implementation • Based on Message Passing Interface (MPI) eXtended Discrete Element Method (XDEM) Research on UL HPC platform 8 / 16
XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion Testcase - Tire Sinkage of a Tire • Simulate the vertical motion of the tire tread • The tire is a rigid surface of 18,707 triangles • Soil layer is made of 43,624 sand grains • Sand grains are described as spherical particles of different radii eXtended Discrete Element Method (XDEM) Research on UL HPC platform 9 / 16
XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion Testcase - Tire: Scalability Parallel execution from 1 process to 64 processes (8 processes/node) 80 T 1 /N DPM execution 70 60 Iteration Time (s) 50 40 30 20 10 0 0 10 20 30 40 50 60 70 Number of processes 2 processes 1.3X speedup → 8 processes 4.6X speedup → 64 processes 17.6X speedup → eXtended Discrete Element Method (XDEM) Research on UL HPC platform 10 / 16
XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion Testcase - Hopper Hopper discharge • Simulate hopper discharge • The hopper is a rigid surface of 31,500 triangles • Spherical particles of different radii eXtended Discrete Element Method (XDEM) Research on UL HPC platform 11 / 16
XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion eXtended Discrete Element Method (XDEM) Research on UL HPC platform 12 / 16
XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion Testcase - Hopper: Scalability Parallel execution from 1 process to 48 processes (24 processes/node) Xdem parallel results ● ● ● 1000 Simulation time[s] ORB ● ● ● RIB RCB 500 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0 0 10 20 30 40 50 Number of processes eXtended Discrete Element Method (XDEM) Research on UL HPC platform 13 / 16
XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion Testcase - Hopper discharge: Scalability Xdem parallel results 50 ● ● ● 40 ● ● 30 Speed Up ● ORB RIB RCB 20 Ideal Speed Up ● ● ● 10 ● ● ● ● ● ● ● ● ● ● ● ● 0 0 10 20 30 40 50 Number of processes 48 processes 21X speedup → eXtended Discrete Element Method (XDEM) Research on UL HPC platform 14 / 16
XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion Conclusion Discrete Particle Method (DPM) sofware • Multi-physics simulation toolbox • Flexible and extensible design • Parallel execution using MPI: 21x speedup with 48 processes Future work • Hydrid approach: MPI + (OpenMP or GPU) • OpenMP or GPU: collision detection eXtended Discrete Element Method (XDEM) Research on UL HPC platform 15 / 16
XDEM HPC usages Enhanced Design Parallel Experiments/Results Conclusion Thank you for your attention eXtended Discrete Element Method (XDEM) Research on UL HPC platform 16 / 16
Recommend
More recommend
