Semi-Lagrangian Simulations for Solving 2d2v Vlasov-Poisson Systems - PowerPoint PPT Presentation

Semi-Lagrangian Simulations for Solving 2d2v Vlasov-Poisson Systems (one and two species) Yann Barsamian 1,3 , Joackim Bernier 4 , Sever Hirstoaga 1,2 , Michel Mehrenberger 1,2 , ´ Eric Violard 1,3 1. 2. CNRS IRMA (MoCo), INRIA (TONUS) 3. CNRS ICube (ICPS), INRIA (CAMUS) 4. Universit´ e de Rennes June 2017 Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 1 / 28

Outline Motivation Comparison of two standard parallel paradigms Optimization of the domain decomposition paradigm Fluorescent Light Sun Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 2 / 28

Kinetic Modeling (one species, dimensionless quantities)  ∂ f x f − − → ∂ t + − → v f = 0 Vlasov v · ∇ − E · ∇ −  → →  − →  E = ρ Poisson ∇ −  → x f ( − → x , − → v , t ) : distribution function of the electrons − → E ( − → x , t ) : the self-induced electric field t : time − → x : position (2d with periodic boundaries: flat torus) → − v : velocity (2d) � ρ ( − → f ( − → x , − → v , t ) d − → x , t ) = 1 − v : volume charge density Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 3 / 28

Semi-Lagrangian Methods: Splitting splitting of the system  ∂ f x f − − → ∂ t + − → v f = 0 Vlasov v · ∇ − E · ∇ −  → →  − →  E = ρ Poisson ∇ −  → x Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 4 / 28

Semi-Lagrangian Methods: Splitting splitting of the system into two simpler systems 1 :  ∂ f x f − − → ∂ t + − → v f = 0 Vlasov v · ∇ − E · ∇ −  → →  − →  E = ρ Poisson ∇ −  → x ∂ f ∂ t − − →   ∂ f v f = 0 E · ∇ − ∂ t + − → →  v · ∇ − x f = 0   →   ∂ − → − → E ∂ t = − →  E = ρ  ∇ − 0   → x  1 Cheng & Knorr, 1976 Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 4 / 28

Semi-Lagrangian Methods: Splitting splitting of the system into two simpler systems 1 :  ∂ f x f − − → ∂ t + − → v f = 0 Vlasov v · ∇ − E · ∇ −  → →  − →  E = ρ Poisson ∇ −  → x ∂ f ∂ t − − →   ∂ f v f = 0 E · ∇ − ∂ t + − → →  v · ∇ − x f = 0   →   ∂ − → → − E ∂ t = − →  E = ρ  ∇ − 0   → x  advection: ∂ g ∂ t + a · ∇ x g = 0 is a translation: g ( x , T ) = g ( x − aT , 0 ) g ( x , 0 ) aT x x 1 Cheng & Knorr, 1976 Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 4 / 28

Semi-Lagrangian Methods: Advection on − → x y y x x Values after k time steps. Values after k + 1 time steps. Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 5 / 28

Semi-Lagrangian Methods: Advection on − → x f ∗ ( x , y , − → v , ( k + 1 )∆ t ) y y x x Values after k time steps. Values after k + 1 time steps. Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 5 / 28

Semi-Lagrangian Methods: Advection on − → x Advection f ∗ ( x , y , − → v , ( k + 1 )∆ t ) f ( x − ∆ x , y − ∆ y , − → v , k ∆ t ) y y x x Values after k time steps. Values after k + 1 time steps. Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 5 / 28

Semi-Lagrangian Methods: Advection on − → x Interpolation f ( x − ∆ x , y − ∆ y , − → v , k ∆ t ) y y x x Values after k time steps. Values after k + 1 time steps. Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 5 / 28

Semi-Lagrangian Methods: Pseudo-Code Parameters: ∆ t , the time step. Algorithm: 1 Initialize f 2 Foreach time iteration, do 3 Output diagnostics Advection of f on − → 4 x for 0 . 5 ∆ t 5 Compute ρ from f 6 Compute E from ρ Advection of f on − → 7 v for ∆ t Advection of f on − → 8 x for 0 . 5 ∆ t 9 Compute ρ from f 10 Compute E from ρ 11 End Foreach Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 6 / 28

Semi-Lagrangian Methods: Pseudo-Code Parameters: ∆ t , the time step. Algorithm: 1 Initialize f 2 Foreach time iteration, do 3 Output diagnostics Advection of f on − → 4 x for 0 . 5 ∆ t 5 Compute ρ from f 6 Compute E from ρ Advection of f on − → 7 v for ∆ t Advection of f on − → 8 x for 0 . 5 ∆ t 9 Compute ρ from f 10 Compute E from ρ 11 End Foreach Advections (Lagrange interpolations): 90-99% of execution time Poisson solver (FFT): 1-10% of execution time Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 6 / 28

Supercomputers Marconi : 1,512 nodes (each composed of 2 sockets of 18 Intel Broadwell cores and 4 memory channels) = 54,432 cores 2 . 2 Compare to 10,649,600 (Sunway TaihuLight ), 361,760 (Piz Daint ), 560,640 (Titan ), 556,104 (Oakforest-PACS ): https://www.top500.org Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 7 / 28

Parallelization Scheme 1: Remap v v x f split in v . x f split in x . Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 8 / 28

Parallelization Scheme 1: Remap v v x f split in v . x f split in x . Example with a 64 2 × 64 2 grid and 4 processors for f split in v : processor 0 will have f [ 00 . . . 63 ][ 00 . . . 63 ][ 00 . . . 31 ][ 00 . . . 31 ] processor 1 will have f [ 00 . . . 63 ][ 00 . . . 63 ][ 00 . . . 31 ][ 32 . . . 63 ] processor 2 will have f [ 00 . . . 63 ][ 00 . . . 63 ][ 32 . . . 63 ][ 00 . . . 31 ] processor 3 will have f [ 00 . . . 63 ][ 00 . . . 63 ][ 32 . . . 63 ][ 32 . . . 63 ] Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 8 / 28

Parallelization Scheme 1: Remap v v x f split in v . x f split in x . Example with a 64 2 × 64 2 grid and 4 processors for f split in x : processor 0 will have f [ 00 . . . 31 ][ 00 . . . 31 ][ 00 . . . 63 ][ 00 . . . 63 ] processor 1 will have f [ 00 . . . 31 ][ 32 . . . 63 ][ 00 . . . 63 ][ 00 . . . 63 ] processor 2 will have f [ 32 . . . 63 ][ 00 . . . 31 ][ 00 . . . 63 ][ 00 . . . 63 ] processor 3 will have f [ 32 . . . 63 ][ 32 . . . 63 ][ 00 . . . 63 ][ 00 . . . 63 ] Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 8 / 28

Parallelization Scheme 2: Domain Decomposition v x Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 9 / 28

Parallelization Scheme 2: Domain Decomposition v x Example with a 64 2 × 64 2 grid and 4 processors: processor 0 will have f [ 00 . . . 31 ][ 00 . . . 63 ][ 00 . . . 31 ][ 00 . . . 63 ] processor 1 will have f [ 00 . . . 31 ][ 00 . . . 63 ][ 32 . . . 63 ][ 00 . . . 63 ] processor 2 will have f [ 32 . . . 63 ][ 00 . . . 63 ][ 00 . . . 31 ][ 00 . . . 63 ] processor 3 will have f [ 32 . . . 63 ][ 00 . . . 63 ][ 32 . . . 63 ][ 00 . . . 63 ] Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 9 / 28

Advection with Remap: Pseudo-Code Local variable: buffer[max(ncx, ncy)]. Algorithm: 1 Remap 2 Foreach vx, do 3 Foreach vy, do 4 Compute the displacement on the x-axis 5 Foreach y, do 6 buffer ← f[:][y][vx][vy] 7 Foreach x, do 8 Interpolate on the x-axis from buffer 9 Compute the displacement on the y-axis 10 Foreach x, do 11 buffer ← f[x][:][vx][vy] 12 Foreach y, do 13 Interpolate on the y-axis from buffer Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 10 / 28

Advection with Domain Decomposition: Pseudo-Code 1 Local variable: buffer[max(ncx, ncy) + Lagrange degree]. Algorithm: 1 Foreach vx, do 2 Foreach vy, do 3 Compute the displacement on the x-axis 4 Foreach y, do 5 Communicate the needed points 6 Foreach x, do 7 Interpolate on the x-axis from buffer 8 Compute the displacement on the y-axis 9 Foreach x, do 10 Communicate the needed points 11 Foreach y, do 12 Interpolate on the y-axis from buffer Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 11 / 28

Advection with Domain Decomposition: Pseudo-Code 2 Local variable: buffer[max(ncx * (ncy + Lag. degree), ncy * (ncx + Lag. degree))]. Algorithm: 1 Foreach vx, do 2 Foreach vy, do 3 Compute the displacement on the x-axis 4 Communicate the needed points 5 Foreach y, do 6 Foreach x, do 7 Interpolate on the x-axis from buffer 8 Compute the displacement on the y-axis 9 Communicate the needed points 10 Foreach x, do 11 Foreach y, do 12 Interpolate on the y-axis from buffer Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 12 / 28

Advection with Domain Decomposition: Drawing v x x Y. Barsamian (Strasbourg, France) Parallelization of 2d2v semi-Lagrangian Simulations 27/06/2017 13 / 28