Realistic modelling of complex Realistic modelling of complex Supercomputing, Visualization & e-Science problems on Grids problems on Grids Manchester Computing John Brooke (University of Manchester) Peter Coveney PI RealityGrid (University College London) Stephen Pickles (University of Manchester) Thanks also to the other RealityGrid co-Investigators John Darlington (Imperial College) Steve Kenny and Roy Kalawsky (Loughborough University) John Gurd (University of Manchester) Mike Cates (University of Edinburgh) Adrian Sutton (University of Oxford)
The RealityGrid project Mission: “Using Grid technology to closely couple high performance computing, high throughput experiment and visualization, RealityGrid will move the bottleneck out of the hardware and back into the human mind.” Scientific aims: � to predict the realistic behavior of matter using diverse simulation methods (Lattice Boltzmann, Molecular Dynamics and Monte Carlo) spanning many time and length scales � to discover new materials through integrated experiments. 2 http://www.realitygrid.org
Partners Academic Industrial � University College London � Schlumberger � Queen Mary, University of London � Edward Jenner Institute for � Imperial College Vaccine Research � University of Manchester � Silicon Graphics Inc � University of Edinburgh � Computation for Science � University of Oxford Consortium � University of Loughborough � Advanced Visual Systems Fujitsu � 3 http://www.realitygrid.org
Access Grid meetings for problem solving � 4 http://www.realitygrid.org
RealityGrid “Instruments”: XMT devices, LUSI,… Storage devices Grid infrastructure (Globus, Unicore,…) HPC resources User with laptop/PDA Steering Scalable MD, MC, (web based portal) ReG steering API mesoscale modelling Performance control/monitoring Visualization engines VR and/or AG nodes Moving the bottleneck out of the hardware and into the human mind… d… Moving the bottleneck out of the hardware and into the human min 5 http://www.realitygrid.org
RealityGrid Characteristics � Grid-enabled (Globus, UNICORE) � Component-based, service-oriented � Steering is central – Computational steering – On-line visualisation of large, complex datasets – Feedback-based performance control – Remote control of novel, grid-enabled, instruments (LUSI) � Advanced Human-Computer Interfaces (Loughborough) � Everything is (or should be) distributed and collaborative � High performance computing, visualization and networks � All in a materials science domain – multiple length scales, many "legacy" codes (Fortran90, C, C++, mostly parallel) 6 http://www.realitygrid.org
Three dimensional Lattice-Boltzmann simulations Code (LB3D) written in Fortran90 and � parallelized using MPI. � Scales linearly on all available resources. Fully steerable. � � Future plans include move to parallel data format PHDF5. Data produced during a single large scale � simulation can exceed hundreds of gigabytes or even terabytes. � Simulations require supercomputers � High end visualization hardware and parallel rendering software (e.g. VTK) needed for data analysis. 3D datasets showing snapshots from a simulation of spinodal decomposition: A binary mixture of water and oil phase separates. ‘Blue’ areas denote high water densities and ‘red’ visualizes the interface between both fluids. 7 http://www.realitygrid.org
Exploring parameter space through computational steering Cubic micellar phase, high surfactant density gradient. Cubic micellar phase, low surfactant density gradient. Initial condition: Self-assembly Random water/ starts. Lamellar phase: surfactant mixture. surfactant bilayers between water layers. Rewind and restart from checkpoint. 8 http://www.realitygrid.org
Computational Steering - Why? � Terascale simulations can generate in days data that takes months to understand � Problem: to efficiently explore and understand the parameter spaces of materials science simulations � Computational steering aims to short circuit post facto analysis – Brute force parameter sweeps create a huge data-mining problem – Instead, we use computational steering to navigate to interesting regions of parameter space – Simultaneous on-line visualization develops and engages scientist's intuition – thus avoiding wasted cycles exploring barren regions, or even doing the wrong calculation 9 http://www.realitygrid.org
Computational steering – how? � We instrument (add "knobs" and "dials" to) simulation codes through a steering library � Library provides: – Pause/resume – Checkpoint and windback – Set values of steerable parameters – Report values of monitored (read-only) parameters – Emit "samples" to remote systems for e.g . on-line visualization – Consume "samples" from remote systems for e.g. resetting boundary conditions � Images can be displayed at sites remote from visualization system, using e.g. SGI OpenGL VizServer, or Chromium � Implemented in 5+ independent parallel simulation codes, F90, C, C++ 10 http://www.realitygrid.org
Philosophy � Provide right level of steering functionality to application developer � Instrumentation of existing code for steering – should be easy – should not bifurcate development tree � Hide details of implementation and supporting infrastructure – eg. application should not be aware of whether communication with visualisation system is through filesystem, sockets or something else – permits multiple implementations – application source code is proof against evolution of implementation and infrastructure 11 http://www.realitygrid.org
Steering and Visualization Simulation Steering library Client data Steering library transfer Display Steering library Display Visualization Visualization Display 12 http://www.realitygrid.org
Architecture Communication modes: • Shared file system Simulation • Files moved by UNICORE daemon • GLOBUS-IO Steering library • SOAP over http/https Client data Steering library transfer Steering library Data mostly flows from simulation to visualization. Reverse direction is being exploited to Visualization integrate NAMD&VMD into RealityGrid Visualization framework. 13 http://www.realitygrid.org
Steering in the OGSA Steering Simulation GS bind Steering library Steering library publish Client connect data Steering Registry Steering library transfer client find publish Steering library bind Steering Visualization Visualization GS 14 http://www.realitygrid.org
Steering in OGSA continued… � Each application has an associated OGSI-compliant “Steering Grid Service” (SGS) � SGS provides public interface to application – Use standard grid service technology to do steering – Easy to publish our protocol – Good for interoperability with other steering clients/portals – Future-proofed next step to move away from file-based steering or Modular Visualisation Environments with steering capabilities � SGSs used to bootstrap direct inter-component connections for large data transfers � Early working prototype of OGSA Steering Grid Service exists – Based on light-weight Perl hosting environment OGSI::Lite – Lets us use OGSI on a GT2 Grid such as UK e-Science Grid today 15 http://www.realitygrid.org
Steering client � Built using C++ and Qt library – currently have execs. for Linux and IRIX Attaches to any steerable RealityGrid � application � Discovers what commands are supported � Discovers steerable & monitored parameters � Constructs appropriate widgets on the fly � Web client (portal) under development 16 http://www.realitygrid.org
RealityGrid-L2: LB3D on the L2G Visualization SGI Onyx Vtk + VizServer SGI OpenGL VizSe rver Laptop Vizserver Client hared L) Simulation Steering GUI Steering (XM File based communication via s filesystem: Steering GUI X output is Data GLOBUS used to GLOBUS-IO launch jobs tunnelled back using ssh. program lbe use lbe_init_module use lbe_steer_module use lbe_invasion_module Simulation LB3D with RealityGrid ReG steering GUI Steering API 17 http://www.realitygrid.org
Performance Control application application performance steerer component component component performance performance performance steerer steerer steerer component component component 1 3 2 18 http://www.realitygrid.org
Advance Reservation and Co-allocation: Summary of Requirements � Computational steering + remote, on-line visualization demand: – co-allocation of HPC (processors) and visualization (graphics pipes and processors) resources – at times to suit the humans in the loop • advanced reservation � For medium to large datasets, Network QoS is important – between simulation and visualization, – visualisation and display � Integration with Access Grid – want to book rooms and operators too � Cannot assume that all resources are owned by same VO � Want programmable interfaces that we can rely on – must be ubiquitous, standard, and robust � Reservations (agreements) should be re-negotiable Hard to change attitudes of sysadmins and (some) vendors � 19 http://www.realitygrid.org
Recommend
More recommend
