Working Group Outbrief Defense Programs g Visualization and Data Analysis James Ahrens, David Rogers, Becky Springmeyer Eric Brugger, Cyrus Harrison, Laura Monroe, Dino Pavlakos Scott Klasky, Kwan-Liu Ma, Hank Childs March 23-24, 2011 Workshop on R&D Challenges for HPC Simulation Environments LLNL-PRES-481881 1
Working Group Description Defense Programs g • The scope of our working group is scientific visualization and data analysis. – Scientific visualization • refers to the process of transforming scientific simulation and experimental data into images to facilitate visual understanding – Data analysis • refers to the process of transforming data into an information-rich form via mathematical or computational algorithms to promote better understanding understanding – Data management - shared with IONS • refers to the process of tracking, organizing and enhancing the use of scientific data • The purpose of our work is to enable scientific discovery and understanding. – Our scope includes an exascale software and hardware infrastructure Ou scope c udes a e asca e so t a e a d a d a e ast uctu e that effectively supports visualization and data analysis. March 23-24, 2011 Workshop on R&D Challenges for HPC Simulation Environments 2
Identify current state-of-the-art Defense Programs g • Production visualization tools scalably process large data – Suite of data-parallel visualization, analysis and rendering algorithms Suite of data parallel visualization, analysis and rendering algorithms • Open-source tools – ParaView, Visit • Commercial tool – Ensight • ASC success story – NNSA/ASC created the large-data scientific visualization tools in use by other agencies (NSF, DOD) and around the world y g ( , ) March 23-24, 2011 3
Identify Exascale Visualization and Data Analysis Needs Defense Programs g • Visualization and Data Analysis (VDA) • Broad range of scope for VDA – VDA as an application – VDA as a service – VDA as a systems infrastructure • Note: like apps, VDA capabilities will require development to exploit opportunities in evolving platforms March 23-24, 2011 4
1. Exascale Challenges – storing a full-range of results for later analysis becomes impossible due to technology trends p gy Defense Programs g • The rate of performance improvement of rotating storage is not keeping pace with compute. • Provisioning additional disks is a possible mitigation strategy, however, power, cost and reliability issues will become a significant issue. • A new in-situ exascale visualization and data analysis approach is needed: needed: – Slow output to data hierarchy / Data movement = power/cost – Where will we process data? • Different customer-driven approaches require integration at different HW ‘levels’ March 23-24, 2011 Workshop on R&D Challenges for HPC Simulation Environments 5
2. Exascale Challenges - Exascale simulation results must be distilled with quantifiable data reduction techniques q Defense Programs g • Exascale as massive data – Defacto data reduction technique Defacto data reduction technique ① Visualization algorithms ② Rendering massive numbers of polygons – This puts lots of data into a single pixel, combined by the renderer • This is a workable method but is it what the user wants? – This approach provides the foundation for our current successes • brute force approach that requires significant computing resources • difficult to quantify the bias of this approach • Approaches that quantifiably reduce data as it is generated need to be explored March 23-24, 2011 Workshop on R&D Challenges for HPC Simulation Environments 6
3. Exascale Challenges - New exascale-enabled physics approaches require corresponding new visualization and data analysis approaches Defense Programs g • Implication of exascale as massive compute – Statistical physics approaches Statistical physics approaches • Statistical modeling of a physical process – Parametric studies • record how a simulation responds in a parameter space of possibilities – Multi-physics approaches • simulate a linked model of different related phenomena such as a linked physics and chemistry simulation – Multi-scale approaches M lti l h • simulate phenomena at different spatial and temporal scales • Understanding and presenting both summarized and highlighted results from multiple sources is an important technical challenge that needs to be addressed. March 23-24, 2011 Workshop on R&D Challenges for HPC Simulation Environments 7
4. Exascale Challenges - Visualization and data analysis approaches will need to run efficiently on exascale platform architectures y p Defense Programs g – Need to take advantage of a very high degree of parallelism – Technical challenges include achieving portability, efficiency and integration flexibility with simulation codes March 23-24, 2011 Workshop on R&D Challenges for HPC Simulation Environments 8
1. Path Forward - New visualization and data analysis software infrastructure Defense Programs g • Required Partnership – When?: Run-time, Postprocessing – IONS, tools, systems – How?: Interactive, Batch – Apps for co-design • In-situ analysis within the simulation • Metric code – Our success will be measured Our success will be measured – Run-time, (batch or interactive) by our readiness for • Post-processing --- advanced query- applications as machine based approach delivery milestones are met • Revolutionary approach – Phase 1 • Risks • Prototype approaches in P t t h i – How to do discovery science applications in an in-situ world? – Phase 2&3 – Won’t find an effective • Develop and deploy p p y analysis approach for • Continue R&D exascale applications Workshop on R&D Challenges for HPC Simulation Environments 9
2. Advanced quantifiable data reduction algorithms Defense Programs g • Data triage • Required Partnership – How do we significantly How do we significantly – Applied math Applied math reduce the data as it is – Apps for co-design generated? • Metric • Statistical sampling – Measure of amount of data M f t f d t • Compression reduced and quality of result, • Multi-resolution time • Science-based feature extraction extraction • Revolutionary approach – Phase 1 • Risks • Prototype approaches P t t h – Won’t find an effective independently and with analysis approach for applications exascale applications – Phase 2&3 • Develop and deploy • Continue R&D Workshop on R&D Challenges for HPC Simulation Environments 10
3. Visualization and data analysis techniques to help understand advanced exascale physics p y Defense Programs g • Visualization and Data Analysis for: • Required Partnership – Statistical physics approaches – Applications for co-design Applications for co design – Parametric studies • Metric – Multi-physics approaches – Multi-scale approaches – Ties to appropriate application milestones application milestones • h • how results from different aspects of a lt f diff t t f simulation suite relate to each other • Evolutionary/Revolutionary • Evolutionary/Revolutionary • Risks approach – Won’t understand output of – Phase 1 exascale applications exascale applications • Prototype approaches P t t h independently and with applications – Phase 2&3 • Develop and deploy • Continue R&D Workshop on R&D Challenges for HPC Simulation Environments 11
4. Implement core visualization and data- analysis capability using a scalable parallel infrastructure Defense Programs g • – Our visualization and data • Required Partnership analysis solutions need to – Programming models, tools Programming models, tools work on the exascale and applications groups supercomputers on both swim • Metric lanes. – Our success will be measured Our success will be measured by our readiness for applications as machine delivery milestones are met • Evolutionary approach • Risks – Phase 1 – Not running on the machines • Prototype approaches P t t h independently and with applications – Phase 2&3 • Develop and deploy • Continue R&D Workshop on R&D Challenges for HPC Simulation Environments 12
5. Exascale visualization and data analysis hardware infrastructure Defense Programs g • Data-intensive hardware • Required Partnership infrastructure for the exascale – HW, Systems, I/O HW, Systems, I/O platform f • Metric – Memory buffers for staged – Ties to appropriate machine analysis and storage milestones milestones – Analysis-enabled storage – Large node memory portion of the supercomputer • Evolutionary/Revolutionary • Risks approach – HW platform that makes data – Phase 1 – Phase 1 analysis difficult analysis difficult • Prototype approaches independently and with applications – Phase 2&3 • Develop and deploy • Continue R&D Workshop on R&D Challenges for HPC Simulation Environments 13
Recommend
More recommend
