Sparse Grid Regression for Performance Prediction Using - PowerPoint PPT Presentation

Chair for High-Performance Computing Philipp Neumann Sparse Grid Regression for Performance Prediction Using High-Dimensional Run Time Data Slide 1 Euro-Par 2019: P. Neumann

Outline • Performance Analysis and Higher Dimensions • Sparse Grids in a Nutshell • Regression on Sparse Grids • Results: Molecular Dynamics, Climate, Weather • Summary Slide 2 Euro-Par 2019: P. Neumann

Performance Analysis and Higher Dimensions: Parameters Affecting Performance Outline • Performance Analysis and Higher • Algorithmic parameters Dimensions • Sparse Grids in a  convergence criteria, mesh size, time step, … Nutshell • Hardware-aware optimization • Regression on  Sparse Grids params for cache blocking, data alignment, • Results: vector widths, … Molecular Dynamics Climate • Parallelization settings Weather  number of MPI processes, OMP threads, … • Summary • Scenario-dependent parameters  domain size/shape, number of cells/particles, …  High-Dimensional Parameter Space Slide 3 Euro-Par 2019: P. Neumann

Performance Analysis and Higher Dimensions: Exploring High-Dimensional Spaces Outline • Performance Analysis and Higher • (Semi-)Analytical models Dimensions • Sparse Grids in a  Only available for small subset of params Nutshell • Neural networks/ deep learning • Regression on Sparse Grids  Effective approach • Results:  Interesting for hard (e.g., combinatorial) problems Molecular Dynamics  Decisions/results not necessarily transparent Climate Weather • Regression and related methods • Summary  Effective approach  Application in higher dimensions? Slide 4 Euro-Par 2019: P. Neumann

Sparse Grids in a Nutshell Outline • Performance Analysis and Higher Dimensions J. Garcke. • Sparse Grids in a Sparse grids in a nutshell Nutshell Full Cart. grid: O(N d ) points • Regression on Sparse Grids SG: O(N(log N) d-1 ) points • Results: Molecular Dynamics Climate  hierarchical Weather representation • Summary  prerequisite for “good” approximations: sufficiently smooth settings/params Slide 5 Euro-Par 2019: P. Neumann

Sparse Grids: Local Mesh Refinement Outline • Performance Analysis and Higher Dimensions • Sparse Grids in a Nutshell • Regression on Sparse Grids • Results: Molecular Dynamics • No. refinement iterations: 3 Climate Weather • No. adaptable grid points: 3 • Summary • Example: 2 refinement iterations, 3 adaptable grid points, start from level-2 grid • Software in use: SG++ Slide 6 Euro-Par 2019: P. Neumann

Sparse Grids: Local Mesh Refinement Outline • Performance Analysis and Higher Dimensions • Sparse Grids in a Nutshell • Regression on Sparse Grids • Results: Molecular Dynamics • No. refinement iterations: 3 Climate Weather • No. adaptable grid points: 3 • Summary • Example: 2 refinement iterations, 3 adaptable grid points, start from level-2 grid • Software in use: SG++ Slide 7 Euro-Par 2019: P. Neumann

Sparse Grids: Local Mesh Refinement Outline • Performance Analysis and Higher Dimensions • Sparse Grids in a Nutshell • Regression on Sparse Grids • Results: Molecular Dynamics • No. refinement iterations: 3 Climate Weather • No. adaptable grid points: 3 • Summary • Example: 2 refinement iterations, 3 adaptable grid points, start from level-2 grid • Software in use: SG++ Slide 8 Euro-Par 2019: P. Neumann

Sparse Grids: Local Mesh Refinement Outline • Performance Analysis and Higher Dimensions • Sparse Grids in a Nutshell • Regression on Sparse Grids • Results: Molecular Dynamics • No. refinement iterations: 3 Climate Weather • No. adaptable grid points: 3 • Summary • Example: 2 refinement iterations, 3 adaptable grid points, start from level-2 grid • Software in use: SG++ Slide 9 Euro-Par 2019: P. Neumann

Sparse Grids: Local Mesh Refinement Outline • Performance Analysis and Higher Dimensions • Sparse Grids in a Nutshell • Regression on Sparse Grids • Results: Molecular Dynamics • No. refinement iterations: 3 Climate Weather • No. adaptable grid points: 3 • Summary • Example: 2 refinement iterations, 3 adaptable grid points, start from level-2 grid • Software in use: SG++ Slide 10 Euro-Par 2019: P. Neumann

Regression on Sparse Grids Outline • Performance Analysis and Higher • Define linear hat function φ i per sparse grid point Dimensions  defines function space V n • Sparse Grids in a Nutshell • Solve regression problem on run time data y j , • Regression on given parameter combinations x j : Sparse Grids • Results: Molecular Dynamics Climate Weather • Summary with • Results in linear system: Slide 11 Euro-Par 2019: P. Neumann

Results: Evaluation Procedure Outline • Performance Analysis and Higher • Dimensions Data splitting: Use s % of data for learning and 1-s % for validation • Sparse Grids in a Nutshell • Mean relative error: • Regression on Sparse Grids – Start from one data split • Results: – Compute and average relative errors for this data split Molecular Dynamics Climate – Repeat this procedure for 10 data splits and average Weather errors • Summary • Consider different initial sparse grid level refinements (level-2 and level-3 grids) Slide 12 Euro-Par 2019: P. Neumann

Results: Molecular Dynamics (1) Outline • Performance Analysis and Higher Dimensions • Sparse Grids in a Nutshell • Regression on Sparse Grids • Results: Molecular Dynamics Climate Weather • Summary Slide 13 Euro-Par 2019: P. Neumann

Results: Molecular Dynamics (2) Outline • Performance Max/min run time ratio: Analysis and Higher Dimensions 1557  • Sparse Grids in a Nutshell • Regression on Sparse Grids • Results: Molecular Dynamics Climate Weather • Summary Slide 14 Euro-Par 2019: P. Neumann

Results: Molecular Dynamics (2) Outline • Performance Max/min run time ratio: Analysis and Higher Dimensions 7  • Sparse Grids in a Nutshell • Regression on Sparse Grids • Results: Molecular Dynamics Climate Weather • Summary Slide 15 Euro-Par 2019: P. Neumann

Results: Molecular Dynamics (3) Outline • Performance Analysis and Higher Dimensions • Sparse Grids in a Nutshell • Regression on Sparse Grids • Results: Molecular Dynamics Climate Weather • Upper left: SG • Summary • Upper right: 1 st order reg. • Lower right: 2 nd order reg. Slide 16 Euro-Par 2019: P. Neumann

Results: Weather and Climate – ICON Model Outline • Performance Analysis and Higher Dimensions • Sparse Grids in a Nutshell • Regression on Sparse Grids • Results: Molecular Dynamics Climate Weather • Summary • ICON=ICOsahedral Non-hydrostatic model • Developed by Deutscher Wetterdienst/ Max-Planck-Institut für Meteorologie • Triangular grids on the sphere + vertical columns • Multiscale, multiphysics: dynamical core, climate/weather physics, radiation, land surface interaction, … Slide 17 Euro-Par 2019: P. Neumann

Results: Climate – ICON V16.0 Benchmark 1 Outline • Performance Max/min run time ratio: Analysis and Higher Dimensions 1.14 • Sparse Grids in a Nutshell • Regression on Sparse Grids • Results: Molecular Dynamics Climate Weather • Summary • Params: # OpenMP threads (1,2,4,6,8,12,18,36), nproma (col. blocking; 2,8,16,24,32) 1 https://redmine.dkrz.de/projects/icon-benchmark/wiki/ Instructions_on_download_execution_and_analysis_ICON_Benchmark_v160 Slide 18 Euro-Par 2019: P. Neumann

Results: Weather Outline • Performance Analysis and Higher Dimensions • Sparse Grids in a Nutshell • Regression on Sparse Grids • Results: Molecular Dynamics Climate Weather • Summary • Params: # OpenMP threads (2,4,6,12,18), # nodes (100,200,300,400), nproma (col. blocking; 2,4,8,16,32), # vert. levels (60,70,80,90) Slide 19 Euro-Par 2019: P. Neumann

Results: Weather Outline • Performance Analysis and Higher Dimensions • Sparse Grids in a Nutshell • Regression on Max/min run time ratio: 2 Sparse Grids • Results: Molecular Dynamics Climate Weather • Summary • Params: # OpenMP threads (2,4,6,12,18), # nodes (100,200,300,400), nproma (col. blocking; 2,4,8,16,32), # vert. levels (60,70,80,90) Slide 20 Euro-Par 2019: P. Neumann

Summary Outline • Performance • Application of the sparse grid regression Analysis and Higher  Training of SG with performance data Dimensions  Prediction of run times via SG basis functions • Sparse Grids in a • Nutshell Molecular dynamics: Accurate prediction (ca 15% dev.) using >= 180 samples to describe nonlinear 5D parameter space • Regression on • Sparse Grids Climate: ca 2.5% deviation for small-deviation case • Results: (max/min run time ratio: 1.14) • Molecular Dynamics Future work: Climate • Comparison with other methods Weather  Neural networks, Gaussian process regression • Summary • On-the-fly data collection and prediction P. Neumann acknowledges ESiWACE. ESiWACE has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 675191. This material reflects only the author’s view and the European Commission is not responsible for any use that may be made of the information it contains. P. Neumann acknowledges funding by the Federal Ministry of Education and Research, grant No 01IH16008B, project TaLPas. Slide 21 Euro-Par 2019: P. Neumann