FLASH and Its Research Communities D. Q. Lamb Flash Center for - PowerPoint PPT Presentation

FLASH and It’s Research Communities D. Q. Lamb Flash Center for Computational Science University of Chicago NSF Community Code Workshop Chicago, IL 6 September 2012 Flash Center for Computational Science The University of Chicago

Outline of Talk  Role of community codes  Properties community codes need to have  Related needs  FLASH basics  FLASH capabilities  Code contribution policy and major contributions  FLASH research communities:  Astrophysics and cosmology (examples: thermo- nuclear-powered supernovae and galaxy cluster mergers)  High energy density physics (example: shock- generated magnetic fields)  Computational fluid dynamics Flash Center for Computational Science The University of Chicago

Role of community codes Community codes serve their scientific communities by providing:  Extensive physics capabilities that would otherwise not be available and that can be used to address cutting edge research problems in science and engineering.  A framework with existing capabilities into which new open or proprietary solvers can be imported.  A means of maintaining solvers over time and migrating them to new platforms, which is particularly important given the radical new architectures expected in the next generation of platforms.  An opportunity to support the education and training of the applied mathematicians, computer scientists, and computational scientists needed to develop the algorithms and codes needed to run at scale on these architectures. Flash Center for Computational Science The University of Chicago

Properties community codes need to have The extensive experience the Flash Center has had with designing, implementing, disseminating, and maintaining FLASH has shown the importance of a community code having certain properties in order to successfully serve a large research community:  It must be fully modular, extensible, and open — only if this is the case can it support creative approaches to algorithms and solvers, facilitate prototyping, and enable individual researchers to use its capabilities while adding their own solvers, whether the researchers make them open or retain proprietary control of them for shorter or longer periods of time.  It must be professionally written and documented – this is essential to maintaining the code over time, and making it possible for members of the user community to change it.  It must be subject to continuous, rigorous verification, including unit and regression testing – this is essential if it is to be robust. Flash Center for Computational Science The University of Chicago

Related needs  Having more than one community code for a research community is usually desirable – one size/ shape usually does not fit all.  Development of algorithms, solvers, and codes by individuals and small groups also needs to be supported – this encourages the development of innovative new algorithms and solvers, many of which will end up enhancing the capabilities of the code. Flash Center for Computational Science The University of Chicago

FLASH code contributors  Current Flash Center Contributors :  John Bachan, Sean Couch, Chris Daley, Milad Fatenejad, Norbert Flocke, Carlo Graziani, Cal Jordan, Dongwook Lee, Min Long, Prateeti Mohapatra, Anthony Scopatz, Petros Tzeferacos, Klaus Weide  Current External Contributors:  Paul Ricker, John Zuhone, Marcos Vanella, Mats Holmstrom, Seyit Hocuk, Chris Orban, Igor Golovkin, Tommaso Vinci, William Gray, Cristoph Federrath, Richard Wunsch  Past Major Contributors:  Katie Antypas, Alan Calder, Jonathan Dursi, Robert Fisher, Bruce Fryxell, Murali Ganapathy, Shravan Gopal, Nathan Hearn, Timur Linde, Bronson Messer, Kevin Olson, Tomek Plewa, Lynn Reid, Katherine Riley, Andrew Siegel, Dan Sheeler, Frank Timmes, , Dean Townsley, Natalia Vladimirova, Greg Weirs, Mike Zingale Flash Center for Computational Science The University of Chicago

FLASH basics  FLASH is a multi-physics finite-volume Eulerian code and framework implemented on a block-structured mesh with adaptive mesh refinement (AMR)  FLASH scales to well over a hundred thousand processors; it uses a variety of parallelization techniques including domain decomposition, mesh replication, and threading to best utilize hardware resources  FLASH is extremely portable and can run on a variety of platforms from laptops to supercomputing systems such as the IBM BG/P and BG/Q Flash Center for Computational Science The University of Chicago

FLASH basics (continued)  FLASH is composed of interoperable units/modules; particular modules are combined to run individual simulations, so only the code relevant to a particular problem is included when FLASH is compiled, allowing important compile-time optimizations.  FLASH is professionally managed software with daily, automated regression testing on a variety of platforms, version control, coding standards, extensive documentation, user support, and integration of code contributions from external users.  FLASH has been downloaded more than 3,000 times; more than 1,000 scientists around the world have now used FLASH, and more than 700 papers have been published that directly use it. Flash Center for Computational Science The University of Chicago

A Snapshot of FLASH Simulations Shortly: Relativistic accretion onto NS Turbulent Nuclear Burning Gravitationally confined detonation Shock-generated B-field experiment Wave breaking on white dwarfs Intracluster interactions Laser-driven shock instabilities Nova outbursts on white dwarfs Rayleigh-Taylor instability Orzag/Tang MHD Helium burning on neutron stars vortex Cellular detonation Magnetic Flash Center for Computational Science Rayleigh-Taylor The University of Chicago Richtmyer-Meshkov instability

A Snapshot of FLASH Simulations FLASH capabilities include: • Adaptive mesh refinement (AMR) on a block-structured mesh Shortly: Relativistic accretion onto NS • Multiple state-of-the-art hydrodynamic solvers (1T and 3T) • State-of-the-art magnetohydrodynamics (currently 1T) • Implicit solvers for diffusion using the HYPRE library (currently Turbulent Nuclear Burning Gravitationally confined being used to model thermal conduction, radiation diffusion, detonation and viscosity) Shock-generated B-field experiment Wave breaking on white dwarfs • Many physics modules relevant to astrophysics and cosmology, including gravity and nuclear burning • Many physics modules relevant to high energy density physics, Intracluster interactions including laser drive, 3T, electron conduction, multi-group Laser-driven shock instabilities Nova outbursts on white dwarfs Rayleigh-Taylor instability radiation diffusion, tabular EOS and opacities • Incompressible Navier-Stokes solver • Generic, highly scalable parallel particles framework (currently used for PIC simulations, laser ray tracing, dark matter, Lagrangian tracer particles, fluid-structure interactions) Orzag/Tang MHD Helium burning on neutron stars vortex Cellular detonation Magnetic Flash Center for Computational Science Rayleigh-Taylor The University of Chicago Richtmyer-Meshkov instability

Flash Center code contribution policy  The code contribution policy is designed with both the intellectual property rights of the contributors and the benefits to other researchers in mind.  Often researchers want to add some functionality to the code that they may not wish to share in a public release.  The Flash Center's policy is to allow individual contributors a pre- negotiated time when their code can exist in the main FLASH source tree without being included in the public release.  During this period the testing and maintenance of the code is carried out as an integral part of the code without any cost to the contributor, as long as the contributor provides supporting documentation and test problems for verification.  At the end of this period, the contributor agrees to release the code with the public distribution.  In return, the Flash Center assumes the responsibility for maintaining the contributed code and migrating it to new platforms. Flash Center for Computational Science The University of Chicago

Major code contributions so far  Already in the code:  2D staggered mesh unsplit MHD solver – contributed by Dongwook Lee before he joined the Center  Huang Greengard method-based multigrid solver – contributed by Paul Ricker after he left the Center  Ionization – Salvatore Orlando  Radiation transfer using hybrid characteristics – Eric Jan Rijkhorst  Several direct solvers for uniform grid – Marcos Vanella  Hybrid particle-in-cell capability – Mats Holmstrom  In the next release (FLASH 4.0 in mid-September):  Sink particles – Christoph Federrath, Robi Banerjee, Martin Schron,  Barnes-Hut tree – Richard Wunsch  Primordial chemistry – William Gray Flash Center for Computational Science The University of Chicago

FLASH research communities  Astrophysics and cosmology (examples: thermonuclear-powered supernovae and mergers of galaxy clusters)  High energy density physics (example: shock-generated magnetic fields)  Computational fluid dynamics Flash Center for Computational Science The University of Chicago

“Pulsationally assisted” gravitationally confined detonation model of thermonuclear-powered supernova Flash Center for Computational Science The University of Chicago

Flash Center for Computational Science The University of Chicago