NVIDIA TO ACCELERATE THE HPC-AI CONVERGENCE Workshop on the Convergence of ML & HPC Gunter Roeth gunterr@nvidia.com March 2020
GRAND CHALLENGES REQUIRE MASSIVE COMPUTING REINVENTING THE LI-ION BATTERY UNDERSTANDING HIV’S STRUCTURE CLOUD RESOLVING CLIMATE SIMULATIONS 3M Node Hours | 7 Days on Titan 100M Node Hours | 840 Days on Piz Daint 10M node Hours |16 Days on BlueWaters 2
TOP500 EFFECTS 100 PFLOPS 10 PFLOPS 1 PFLOPS 100 TFLOPS 10 TFLOPS 1 TFLOPS 100 GFLOPS All #1 #500 3
SOMETHING NEW: AI + HPC = REVOLUTION 4
INGREDIENTS: BIG DATA 5
BIG DATA IN SCIENCE Big Science ingests/outputs Big Data Johns Hopkins Square Kilometer Array Large Hadron Collider Turbulence Database NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 6
AI WORKFLOW FOR HPC DATA NEW DATA TRAINING SET REGRESSION SET SIMULATION TRAINING REGRESSION TESTING INFERENCE (FP64/FP32) (FP32/FP16) (FP16/INT8) (FP16/INT8) ERRORS 7
THE CONVERGENCE OF HPC * AI Integrating the Third and Fourth Pillars of Scientific Discovery HPC AI 40+ years of algorithms New algorithms and models based on first principles with potential to increase theory model size and accuracy Dramatically Improves Accuracy and /or Time-to-Solution at Large Scale Commercially Improve or validate the Understanding Climate/weather Clinically viable viable fusion Standard Model of cosmological dark forecasts with ultra- precision medicine energy Physics energy and matter high fidelity
AI FOR HPC Transformative Tool To Accelerate The Pace of Scientific Innovation 5,000X Faster 300,000X Faster 90% accuracy 33% Faster Process LIGO Signal Predict Molecular Energetics Fusion Sustainment Track Neutrinos Understanding Universe Drug Discovery Clean Energy Particle Physics 70% accuracy 11% higher accuracy Weeks to 10 milliseconds 14X Faster Score Protein Ligand Monitor Earth’s Vital Analyze Gravitational Lensing Generate Bose-Einstein Drug Discovery Climate Astrophysics Condensate (Physics) Improves Accuracy Accelerates Time to Solution Enabling realization of full scientific potential Unlocking the use of science in exciting new ways 9
INTELLIGENT HPC DL Driving Future HPC Breakthroughs From Trained networks as solvers • calendar • Super-resolution of coarse simulations time to real Low- and mixed-precision • time? Simulation for training, network in production • Pre- Post- Simulation processing processing • Select/classify/augment/ • Analyze/reduce/augment distribute input data output data • Control job parameters • Act on output data 10
THE SHAPE OF AI SUPERCOMPUTING 11
VOLTA TENSOR CORE GPU FUELS WORLD'S FASTEST SUPERCOMPUTER Fused HPC and AI Computing In a Unified Platform Genomics (CoMet) 150X World’s First Exascale Run Finding Genes-to-disease Connection Over Titan Same accuracy as FP64 w/ Tensor Core 50X Quantum Chemistry (QMCPack) Simulate New Materials High-Temperature Semiconductors Over Titan Summit Supercomputer AI: 3 Exaflops Oakridge National Labs HPC: 122 Petaflops Measured performance: Summit node vs Titan node 12
AI: A NEW COMPUTING PARADIGM 1000X GPU-Computing perf 10 7 by 1.5X per year 2025 10 6 1.1X per year 10 5 10 4 √ 10 3 1.5X per year 10 2 Single-threaded perf 1980 1990 2000 2010 2020 Original data up to the year 2010 collected and plotted by M. Horowitz, F. Labonte, O. Shacham, K. Olukotun, L. Hammond, and C. Batten New plot and data collected for 2010-2015 by K. Rupp 13
NVIDIA DL AND HPC – JOINTLY SOLVE NEW PROBLEMS, BETTER 14
AI SUPERCOMPUTING IS HERE Extending The Reach of HPC By Combining Computational & Data Science Clean Energy Drug Discovery Turbulent Flow Molecular Dynamics “ What’s happening?” “Is there cancer?” Monitoring Climate “Next move?” Understanding Universe Change Structural Analysis N-body Simulation “What does she mean?” COMPUTATIONAL SCIENCE DATA SCIENCE COMPUTATIONAL & DATA SCIENCE S8242 – DL for Computational Science, Jeff Adie & Yang Juntao Presented ~20 Success Stories of DL in Computational Science (GTC on-demand: http://on-demand-gtc.gputechconf.com)
Computational Chemistry 16
AI Quantum Breakthrough Background Developing a new drug costs $2.5B and takes 10-15 years. Quantum chemistry (QC) simulations are important to accurately screen millions of potential drugs to a few most promising drug candidates. Challenge QC simulation is computationally expensive so researchers use approximations, compromising on accuracy. To screen 10M drug candidates, it takes 5 years to compute on CPUs. Solution Researchers at the University of Florida and the University of North Carolina leveraged GPU deep learning to develop ANAKIN-ME, to reproduce molecular energy surfaces with super speed (microseconds versus several minutes), extremely high (DFT) accuracy, and at 1-10/millionths of the cost of current computational methods. Essentially the DL model is trained to learn Hamiltonian of the Schrodinger equation. Impact Faster, more accurate screening at far lower cost 55 17
NEURAL NETWORK MODEL APPROACH Training set: ~20M DFT data points. Molecules with 1 to 8 atoms from GDB database 18
Computational Mechanics 20
Deep Learning for Solid Mechanics FEA UPDATED WITH NEURAL NETWORK FEA trained deep neural network for surrogate modelling of estimated stress distribution. Deepvirtuality, a spinoff from Volkswagen Data:Lab under Nvidia Inception Program has demonstrate with their software aimed for a quicker prediction of structural data. An demonstration of Structure Born Noise of a V12 Engine with Deepvirtuality Torsional Frequencies of a Car Body by Deepvirtuallity 21
Eulerian Fluid Simulation Approximating PDE solutions NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 22 “Accelerating Eulerian Fluid Simulation With Convolutional Networks” , Thompson et al., 2016
SimRes at SC19 in Denver Physics Informed NN 23
EXAMPLES OF PINN Vortex induced vibrations problem of flow past a circular cylinder. (eta) 24 Incompressible Navier-Stokes equations
Train first Data Driven Networks (DDNN) 25
AUTOMOTIVE AERODYNAMICS Inference Training 26
DATA DRIVEN METHODS Pros & Cons - Need to generate a lot of Simulations (accuracy dependent on the simulation code) - No Physics Awareness; Generalizability may be limited - Not very efficient for Complex 3D Geometries/Curved Surfaces - Interpolation/Extrapolation Errors + Not dependent on Physics 27 27
Use Physics Informed NN (PINNs) 28
PHYSICS INFORMED NEURAL NETS: ARCHITECTURE A Neural Network Architecture for Computational Mechanics/Physics problems ❑ Point Cloud for 3D Geometries & Meshes (Fixed/Moving, Deforming, Structured & Unstructured) ❑ Physics Driven & Physics Aware Networks (respects the governing PDEs, Multi-disciplinary) ❑ Performance optimized for GPU tensor cores PINN - Physics Informed Neural Point Cloud representation of 29 29 Networks Computational Domain & Data on 3D Geometries
PHYSICS DRIVEN METHODS Special Considerations Problem Modeling: ❑ o Complex Geometries Sampling Insensitivity ❑ Network Architecture: ❑ Faithfully represent the Physics with initial & boundary conditions o Architectural Requirements for n th order derivatives o Loss Convergence Acceleration o Activation Functions o Gradients & Discontinuities o Global vs. Local o 30 30
Results of Physics Informed NN (PINNs) 32
STEADY STATE: 2D LID DRIVEN CAVITY SIMNET VS. OPENFOAM U velocity difference = 0.2% V velocity difference = 0.4% 33
HEAT SINK: A MULTI-PHYSICS PROBLEM Heat Sink – * Temperatures to not exceed the design criteria Objectives – * Similar accuracy as the Solver * Geometry representation with Point Clouds * Multiple simultaneous parametrized & unparametrized geometries Physics involved – CFD & Heat Transfer Ansys IcePack used for Simulation (** we kindly acknowledge Ansys’s support **) 34
NETWORK ARCHITECTURE Multi-Physics Neural Networks Multi-Physics PDEs CFD (with turbulence) – 2 nd Order PDE CFD Heat Transfer in Solids & Fluid (turbulent) Fluid-Solid Interface Conditions Heat Transfer Temperature in Fluid Heat Flux PINN Network Architecture Heat Transfer 10 layers for non-Physics Informed Network 10 x 2n layers for n th order PDEs in Solid 50-500 neurons per layer Swish Activation Function 35
HEATSINK DESIGN OPTIMIZATION Physics Informed Neural Net for Coupled CFD-Heat Transfer Problems 37
Earth Science 38
Deep Learning for Climate Modeling ANOMALY DETECTION IN CLIMATE DATA Identifying “extreme” weather events in multi -decadal datasets with 5-layered Convolutional Neural Network. Reaching 99.98% of detection accuracy. (Kim et al, 2017) Dataset: Visualization of historic cyclones from JWTC hurricane report from 1979 to 2016 Systemic framework for detection and localization of extreme climate event 39
Recommend
More recommend
