Deep Neural Networks for Non-Equilibrium Molecular Dynamics GPU - PowerPoint PPT Presentation

Deep Neural Networks for Non-Equilibrium Molecular Dynamics GPU Tech Conf (GTC), San Jose Session S7373, R210C Monday, May 8 th , 2017 Jon Belof, Will Lowe † , Adam Hogan ‡ †Seven Deep Cognition, LLC ‡University of South Florida LLNL-PRES-XXXXXX This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344. Lawrence Livermore National Security, LLC

Equilibrium vs. Non-equilibrium Life Li ordinary Death De rock extraordinary creature “The inner life of a cell”, (Cellular Visions/Harvard) kinesin walker on microtubule silica crystal structure There are non-equilibrium states of scientific interest of intermediate complexity: phase transformations of matter Lawrence Livermore National Laboratory 2 LLNL-PRES-XXXXXX

Phase transformation = whole-sale atomic rearrangement • Phase transformation of Fe under shock- wave loading • Transition occurs from a highly metastable state due to large driving force • Microstructure of the system evolves far-from- equilibrium K. Kadau, T.C. Germann, P.S. Lomdahl, and B.L. Holian, “Microscopic view of structural phase transitions induced by shock waves”, Science, 296:1681 (2002) Lawrence Livermore National Laboratory 3 LLNL-PRES-XXXXXX

Phase transitions and pattern formation Snowflakes dendrites dissipative structures chemical instabilities hydrodynamic instabilities Lawrence Livermore National Laboratory 4 LLNL-PRES-XXXXXX

Phase transitions and pattern formation Phase field crystal Phase field method ? Snowflakes dendrites dissipative structures Reaction-diffusion eqn Navier-Stokes chemical instabilities hydrodynamic instabilities In principle, coarse-graining over atomistic dynamics provides a path to derive non-equilibrium constitutive relations Lawrence Livermore National Laboratory 5 LLNL-PRES-XXXXXX

Phase transitions have played a fundamental role in the history of statistical physics Internal consistency of statmech Onsager, Ginzburg, Landau ~ 1950 Renormalization group methods, Wilson, Kadanoff, Fisher, Widom ~ 1970 universality and scaling Mandelbrot, Feigenbaum, ~1980 Complexity theory, self- Langer, many others... organization and emergence Non-equilibrium work, Prigogine, ~1970 entropy production theory, Evans, Jarzynski, Crooks ~1990 systems far from equilibrium Tsallis ~2000 “Where is the frontier of physics? Some would say 10 -33 cm, some 10 -15 cm and some 10 28 cm. My vote is for 10 -6 cm. Two of the greatest puzzles of our age have their origins at this interface between the macroscopic and microscopic worlds.” -L.S. Schulman, Time’s Arrows and Quantum Measurement (1997) Far-from-equilibrium phase transitions open a new avenue to the discovery of a universal theory of non-equilibrium statistical mechanics Lawrence Livermore National Laboratory 6 LLNL-PRES-XXXXXX

What is molecular dynamics (MD)? apply periodic boundaries retrieve positions, velocities calculate forces evaluate observables from the potential and average advance positions, apply thermostat velocities by dt and/or barostat finite propagator from Liouville expansion, e.g. velocity verlet To numerical precision, the dynamics are nearly exact with one important exception: the MD potential Lawrence Livermore National Laboratory 7 LLNL-PRES-XXXXXX

Example: solidification of Cu L. Zepeda-Ruiz (LLNL) • 64 million atoms • Cu EAM potential (Mishin et al., PRB, 2001) • Temperature quench at constant pressure • Several ns dynamics • Common neighbor analysis for phase detection: • liquid = transparent • fcc = green • hcp = red • bcc = blue The accuracy of result is only as good as the potential! Lawrence Livermore National Laboratory 8 LLNL-PRES-XXXXXX

Commonly used MD potentials are empirical and fit to DFT so as to reproduce equilibrium properties Lennard-Jones Tersoff Lennard-Jones, Proc. R. Soc. Lond. A, 106:463 (1924) J. Tersoff, Phys Rev B, 37, 6991 (1988) 2-body potentials 3-body potentials Embedded Atom Method Stillinger-Weber Daw, Baskes, Phys Rev Lett, 50, 1285 (1983) Stillinger and Weber, Phys Rev B, 31, 5262 (1985) Pair potentials sometimes present reasonable accuracy for the lowest computational cost Lawrence Livermore National Laboratory 9 LLNL-PRES-XXXXXX

Many-body effects can be captured through inclusion of high-order terms Moriarty, Physical Review B, 38, 3199 (1988) Modified Generalized Pseudo-potential Theory (MGPT) T is the many-body dipole-dipole tensor Thole, Chem Phys, 59, 341 (1981); Applequist, JCP, 83, 809 (1985) Thole-Applequist Polarization All of these potentials have coarse-grained over the electronic states: where do these methods fail? Lawrence Livermore National Laboratory 10 LLNL-PRES-XXXXXX

In solidification, nucleation from pre-existing clusters can be influenced by electronic states nucleation growth coarsening • Fluctuations in the (metastable, undercooled) liquid result in an atomic configuration that resembles the solid, biased by electronic states • Forming this small solid in the liquid creates an interface which has an entropic penalty (surface free energy), opposing the thermodynamic (bulk) driving force Current MD potentials are incapable of describing these effects Lawrence Livermore National Laboratory 11 LLNL-PRES-XXXXXX

In solidification, thermal transport during interface growth of the crystal thermal diffusion ç latent heat Interface velocity solid liquid temperature distribution hot electrons flow this way See also: Broughton, Gilmer, Jackson, PRL 49:1496 (1982) Mikheev and Chernov, J. Crystal Growth , 112:591 (1991) L liquid diffusion è interface attachment What is needed for non-equilibrium is an “intelligent” potential that understands that the underlying electronic states have changed Lawrence Livermore National Laboratory 12 LLNL-PRES-XXXXXX

Can a Deep Neural Network be trained to provide an MD potential? Challenges: Could a DNN learn how to impose a thermostat, or even propagate o DNN is “fuzzy”, get noise equations of motion themselves? o Conservation constraints Behler, Parrinello, PRL 98:146401 (2007) o Performance See recent result on DNN trained to o Need large training set solve Schrodinger equation: o Non-analytic, not easily modified Mills, Spanner, Tamblyn, “Deep learning and the Schrodinger equation”, arXiv 1702.01361v1 (2017) This is the kind of many-body physics ideally matched for DNNs Goal is to train from non-equilibrium QMD but our first step is to validate methodology on pairwise and many-body potentials Lawrence Livermore National Laboratory 13 LLNL-PRES-XXXXXX

Our approach: flowtential – coupling MD and DNN Keras / TensorFlow backend pairwise matrix translation LAMMPS MD • Rectified linear unit atomic coordinates • Dropout = 0.2 input vector • Neurons / layer: • 2048 timestep iterator … • 1024 • 512 6 layers • 256 • 512 atomic forces • 2048 output vector The flowtential code propagates dynamics via LAMMPS library routines and calls Keras/TF for force “prediction” via callback Lawrence Livermore National Laboratory 14 LLNL-PRES-XXXXXX

flowtential training methodology Trained over “exact” solution via LAMMPS MD GTX 1080 (cuDNN) • 3 MD test systems <= 100 epochs w/ applied: RMSProp LJ Cu • Lennard-Jones Initial learning rate = 0.001 Cu potential Reduction factor 0.1 • SPC H 2 O patience of 10 • Many-body H 2 O • Trained over 100k SPC H 2 O MB H 2 O MD configurations • System size: 108 Cu atoms / 64 H 2 O molecules RMS error in forces: 3 % for Cu LJ, ~2 % for H 2 O How does prediction-step ( i.e., propagating MD dynamics) look? Lawrence Livermore National Laboratory 15 LLNL-PRES-XXXXXX

Cu Lennard-Jones dynamics with flowtential • Cu DNN potential linked with LAMMPS MD • Melting from an fcc lattice (NVT, Nose-Hoover thermostat) • 300 timesteps temperature of the system After performance improvements, we will compare standard metrics (pair correlation functions, energy conservation, etc. ) Lawrence Livermore National Laboratory 16 LLNL-PRES-XXXXXX

Conclusions and Path Forward v The development of a universal theory for non-equilibrium phenomena would be a tremendous break-through worthy of effort v Phase transformations present a wide array of complex behavior, for which we would like to have formally derived constitutive relations v MD should formally be able to provide coarse-grained constitutive relations for non-equilibrium phenomena, but current potentials are inadequate v We’ve explored the concept that a DNN can serve to provide the MD potential: ideally trained on QMD but, as a first step, trained on potentials of increasing complexity v We’ve developed a new code called flowtential that couples Keras/TF and the LAMMPS MD code and demonstrated it on several test systems v Preliminary results show stable dynamics, future work will focus on comparison of pair correlation functions at multiple temperature, etc. after performance improvements Lawrence Livermore National Laboratory 17 LLNL-PRES-XXXXXX