S6253 VMD: Petascale Molecular Visualization and Analysis with - PowerPoint PPT Presentation

S6253 — VMD: Petascale Molecular Visualization and Analysis with Remote Video Streaming John E. Stone Theoretical and Computational Biophysics Group Beckman Institute for Advanced Science and Technology University of Illinois at Urbana-Champaign http://www.ks.uiuc.edu/Research/gpu/ S6253, GPU Technology Conference 1:00pm-1:50pm, Room LL21D, San Jose Convention Center, San Jose, CA, Tuesday April 5 th , 2016 Biomedical Technology Research Center for Macromolecular Modeling and Bioinformatics Beckman Institute, University of Illinois at Urbana-Champaign - www.ks.uiuc.edu

VMD – “Visual Molecular Dynamics” • Visualization and analysis of: – molecular dynamics simulations – particle systems and whole cells – cryoEM densities, volumetric data – quantum chemistry calculations – sequence information • User extensible w/ scripting and plugins Whole Cell Simulation MD Simulations • http://www.ks.uiuc.edu/Research/vmd/ CryoEM, Cellular Biomedical Technology Research Center for Macromolecular Modeling and Bioinformatics Sequence Data Quantum Chemistry Tomography Beckman Institute, University of Illinois at Urbana-Champaign - www.ks.uiuc.edu

Goal: A Computational Microscope Study the molecular machines in living cells Ribosome: target for antibiotics Poliovirus Biomedical Technology Research Center for Macromolecular Modeling and Bioinformatics Beckman Institute, University of Illinois at Urbana-Champaign - www.ks.uiuc.edu

VMD Interoperability Serves Many Communities • Uniquely interoperable with a broad range of tools: – AMBER, CHARMM, CPMD, DL_POLY, GAMESS, GROMACS, HOOMD, LAMMPS, NAMD, and many more … • Supports key data types, file formats, and databases • Incorporates tools for simulation preparation, visualization, and analysis Biomedical Technology Research Center for Macromolecular Modeling and Bioinformatics Beckman Institute, University of Illinois at Urbana-Champaign - www.ks.uiuc.edu

NAMD and VMD Use GPUs and Petascale Computing to Meet Computational Biology’s Insatiable Demand for Processing Power 10 8 HIV capsid 10 7 Number of atoms Ribosome 10 6 STMV ATP Synthase 10 5 ApoA1 Lysozyme 10 4 1986 1990 1994 1998 2002 2006 2010 2014 Biomedical Technology Research Center for Macromolecular Modeling and Bioinformatics Beckman Institute, University of Illinois at Urbana-Champaign - www.ks.uiuc.edu

NAMD Titan XK7 Performance August 2013 NAMD XK7 vs. XE6 GPU Speedup: 2x-4x HIV-1 Trajectory: ~1.2 TB/day @ 4096 XK7 nodes Biomedical Technology Research Center for Macromolecular Modeling and Bioinformatics Beckman Institute, University of Illinois at Urbana-Champaign - www.ks.uiuc.edu

VMD Petascale Visualization and Analysis • Analyze/visualize large trajectories too large to transfer off-site: – User-defined parallel analysis operations, data types – Parallel rendering, movie making • Supports GPU-accelerated Cray XK7 nodes for both visualization and analysis: GPU accelerated trajectory analysis w/ CUDA – OpenGL and GPU ray tracing for visualization and – movie rendering Parallel I/O rates up to 275 GB/sec on 8192 Cray • NCSA Blue Waters Hybrid Cray XE6 / XK7 XE6 nodes – can read in 231 TB in 15 minutes! 22,640 XE6 dual-Opteron CPU nodes 4,224 XK7 nodes w/ Telsa K20X GPUs Parallel VMD currently available on: ORNL Titan, NCSA Blue Waters, Indiana Big Red II, CSCS Piz Daint, and similar systems Biomedical Technology Research Center for Macromolecular Modeling and Bioinformatics Beckman Institute, University of Illinois at Urbana-Champaign - www.ks.uiuc.edu

Interactive Remote Visualization and Analysis • Enabled by hardware H.264/H.265 video encode/decode • Enable visualization and analyses not possible with conventional workstations • Access data located anywhere in the world – Same VMD session available to any device Biomedical Technology Research Center for Macromolecular Modeling and Bioinformatics Beckman Institute, University of Illinois at Urbana-Champaign - www.ks.uiuc.edu

Interactive Collaboration • Enable interactive VMD sessions Experimentalist with multiple-endpoints Collaborators Pittsburgh, PA • Enable collaboration features that Supercomputer, MD Simulation were previously impractical: – Remote viz. overcomes local computing and visualization limitations for interactive display Urbana, IL Biomedical Technology Research Center for Macromolecular Modeling and Bioinformatics Beckman Institute, University of Illinois at Urbana-Champaign - www.ks.uiuc.edu

Adaptation of VMD to EGL for in-situ and parallel rendering on clouds, clusters, and supercomputers • Eliminate dependency on windowing systems • Simplified deployment of parallel VMD builds supporting off-screen rendering • Maintains 100% of VMD OpenGL shaders and rendering features • Support high-quality vendor- supported commercial OpenGL implementations in HPC systems that were previously limited to Mesa Poliovirus Biomedical Technology Research Center for Macromolecular Modeling and Bioinformatics Beckman Institute, University of Illinois at Urbana-Champaign - www.ks.uiuc.edu

OpenGL: GLX vs. EGL Viz Application Viz Application (user) (user) GLX OpenG OpenGL OpenGL X server L (root) EGL Driver Driver GPU GPU NIH BTRC for Macromolecular Modeling and Bioinformatics Beckman Institute, U. Illinois at Urbana-Champaign http://www.ks.uiuc.edu/

Molecular Structure Data and Global VMD State Scene Graph Graphical User Interface Representations Subsystem Tcl/Python Scripting DrawMolecule Mouse + Windows Non-Molecular 6DoF Input “Tools” Geometry Display GLX+X11+Drv VMDDisplayList OpenGL Pbuffer/FBO Windowed OpenGL Subsystem DisplayDevice EGL+Drv OpenGLRenderer OpenGL Pbuffer/FBO NIH BTRC for Macromolecular Modeling and Bioinformatics Beckman Institute, U. Illinois at Urbana-Champaign http://www.ks.uiuc.edu/

Swine Flu A/H1N1 neuraminidase bound to Tamiflu: VMD EGL rendering demonstrating full support for all VMD shaders and OpenGL features, multisample antialiasing, ray cast spheres, 3-D texture mapping, ... NIH BTRC for Macromolecular Modeling and Bioinformatics Beckman Institute, U. Illinois at Urbana-Champaign http://www.ks.uiuc.edu/

Benefits of EGL Platform Interfaces • Minor similarity to OpenCL’s platform interfaces • Enumerate and select among available implementations, potentially supporting multiple vendors in the same node • Allows specific target implementation to be bound, e.g. GPU, CPU- integrated GPU, software rasterizer EGL interfaces make it EASY to bind a GPU to a thread with optimal • CPU affinity with respect to NUMA topology – High-perf. multi-GPU image compositing, video – NVIDIA EGL implementation supports multiple indexing schemes, e.g. PCIe ordering – EGL plays nicely with MPI, CUDA/OpenCL, OptiX, NVENC, etc NIH BTRC for Macromolecular Modeling and Bioinformatics Beckman Institute, U. Illinois at Urbana-Champaign http://www.ks.uiuc.edu/

Example Node NUMA Topology DRAM GPU 1 QPI/HT IOH 1 PCIe 3.0 x16 CPU 1 GPU 2 PCIe 3.0 x16 CPU Bus 25GB/sec NET QPI/ PCIe 3.0 x4/x8/x16 QuickPath (QPI) HT HyperTransport (HT) PCIe 3.0 x16 12GB/sec GPU 3 CPU 2 PCIe 3.0 x16 IOH 2 GPU 4 QPI/HT DRAM PCIe 3.0 x16 NIH BTRC for Macromolecular Modeling and Bioinformatics Beckman Institute, U. Illinois at Urbana-Champaign http://www.ks.uiuc.edu/

Example Cloud Node NUMA Topology DRAM vGPU 1 QPI/HT IOH 1 PCIe 3.0 x16 CPU 1 vGPU 2 Board 1 CPU Bus 25GB/sec QuickPath (QPI) PCIe 3.0 x8/x16 QPI/ NET HT HyperTransport (HT) PCIe 3.0 x16 vGPU 3 12GB/sec CPU 2 IOH 2 vGPU 4 QPI/HT DRAM PCIe 3.0 x16 Board 2 NIH BTRC for Macromolecular Modeling and Bioinformatics Beckman Institute, U. Illinois at Urbana-Champaign http://www.ks.uiuc.edu/

VMD EGL Performance on Amazon EC2 Cloud EC2 “G2.8xlarge” HIV-1 movie rendering time MPI (sec), (I/O %) Ranks GPU Instances 3840x2160 resolution 1 1 626s (10% I/O) 2 1 347s (19% I/O) 4 1 221s (31% I/O) 8 2 141s (46% I/O) 16 4 107s (64% I/O) 32 8 90s (76% I/O) Performance at 32 nodes reaches ~48 frames per second High Performance Molecular Visualization: In-Situ and Parallel Rendering with EGL. 64M atom HIV-1 capsid J. E. Stone, P. Messmer, R. Sisneros, and K. Schulten. High Performance Data Analysis and simulation rendered via EGL Visualization Workshop, IEEE IPDPSW, 2016. NIH BTRC for Macromolecular Modeling and Bioinformatics Beckman Institute, U. Illinois at Urbana-Champaign http://www.ks.uiuc.edu/

Close-up view of HIV-1 hexamer rendered via EGL 64M atom HIV-1 capsid simulation rendered via EGL NIH BTRC for Macromolecular Modeling and Bioinformatics Beckman Institute, U. Illinois at Urbana-Champaign http://www.ks.uiuc.edu/

Molecular Dynamics Flexible Fitting (MDFF) X-ray crystallography MDFF Electron microscopy APS at Argonne FEI microscope ORNL Titan Flexible fitting of atomic structures into electron microscopy maps using molecular dynamics. L. Trabuco, E. Villa, K. Mitra, J. Frank, and K. Schulten. Structure, 16:673-683, 2008. Biomedical Technology Research Center for Macromolecular Modeling and Bioinformatics Beckman Institute, University of Illinois at Urbana-Champaign - www.ks.uiuc.edu