The HPC Challenge Benchmark The HPC Challenge Benchmark http://icl.cs.utk.edu/hpcc/ Jack Dongarra Innovative Computing Laboratory University of Tennessee and Computer Science and Mathematics Division Oak Ridge National Laboratory 1
Phases I - - III III Phases I Early Academia Early Metrics, Metrics and Products Software Research Pilot Benchmarks Benchmarks HPCS Tools Platforms Platforms Capability or Products Requirements Application and Metrics Analysis Performance Technology Research Assessment Assessments Prototypes System Concept & Pilot Systems Design Reviews PDR DDR Review Industry Phase II Phase III Readiness Review Readiness Reviews Fiscal Year 02 03 04 05 06 07 08 09 10 Phase III Reviews Phase I Full Scale Development Phase II Industry Procurements Industry commercially ready in the 2007 to 2010 timeframe. R&D Concept Study Critical Program 3 companies $100M ? Milestones 5 companies ~$50M each $10M each Productivity Team Industry: Mission partners: Productivity team (Lincoln Lab lead) MIT Lincoln Lab PI: Dongarra PIs: Benson, PI: Kepner PI: Lucas PI: Basili Snavely LCS Ohio State PIs: Gilbert, Edelman, PI: Koester PIs: Vetter, Lusk, Post, Bailey Ahalt, Mitchell 2
Motivation for Additional Benchmarks Motivation for Additional Benchmarks ♦ From Linpack Benchmark and Top500: “no single number can reflect overall performance” ♦ Without HPL Linpack only peak will be reported ♦ Clearly need something more than Linpack ♦ HPC Challenge Benchmark Goals HPC Challenge Benchmark Goals HPC Challenge Benchmark ♦ Stress CPU, memory system, interconnect ♦ Allow for optimizations � Record effort needed for tuning ♦ Provide verification of results ♦ Archive results ♦ Requires: MPI and BLAS 3
HPC Challenge Benchmark HPC Challenge Benchmark Initial Release 11/03 Initial Release 11/03 Consists of basically 5 benchmarks; Think of it as a framework or harness for adding benchmarks of interest. � HPL (LINPACK) ― MPI on whole system (Ax = b) 1. STREAM ― single CPU 2. *STREAM ― embarrassingly parallel whole system PTRANS (A A + B T ) ― MPI on whole system 3. RandomAccess ― single CPU Random integer 4. *RandomAccess ― embarrassingly parallel read; update; & write RandomAccess ― MPI on whole system BW and Latency – MPI 5. proc i proc k Coming soon: FFT and Matrix Multiply Memory Access Patterns low Traveling Digital Sales Signal Spatial locality Processing Person Applications Applications Computational Radar Fluid Cross Dynamics Section high low Temporal locality 4
Memory Access Patterns FFT low (coming soon) RandomAccess Traveling Digital Sales Signal Spatial locality Processing Person Applications Applications Signatures Signatures Computational Radar Fluid Cross Dynamics Section HPL Linpack STREAM / PTRANS high low low Temporal locality How Will The Benchmarking Work? How Will The Benchmarking Work? ♦ Single program to download and run � Simple input file similar to HPL input ♦ Base Run and Optimization Run � Base run must be made � User supplies MPI and the BLAS � Optimized run allowed to replace certain routines � User specifies what was done ♦ Results upload via website ♦ html table and Excel spreadsheet generated with performance results � Intentionally we are not providing a single figure of merit (no over all ranking) ♦ Goal: no more than 2 X the time to execute HPL. 5
http://icl.cs.utk.edu/hpcc/ http://icl.cs.utk.edu/hpcc/ Coming soon FFT and Matrix Multiply Go to… … Go to ♦ http://icl.cs.utk.edu/hpcc/ 6
Example of Output Example of Output http://icl.cs.utk.edu/hpcc/ 7
8
http://icl.cs.utk.edu/hpcc icl.cs.utk.edu/hpcc/ / http:// Expanded Set of Benchmarks Expanded Set of Benchmarks ♦ Constructing a framework for benchmarks ♦ Developing machine signatures ♦ Plans are to expand the benchmark collection ♦ Currently working on � DGEMM and *DGEMM � FFT (1d Complex) 9
Future Directions Future Directions ♦ Port to new systems ♦ Provide more implementations � Languages (Fortran, UPC, Co-Array) � Environments � Paradigms ♦ Other basic operations � Sparse matrix � I/O Collaborators Collaborators ♦ Piotr Ł uszczek, U of Tennessee ♦ David Bailey, NERSC/LBL ♦ Jeremy Kepner, MIT Lincoln Lab ♦ David Koester, MITRE ♦ Bob Lucas, ISI/USC ♦ John McCalpin, IBM, Austin ♦ Rolf Rabenseifner, HLRS Stuttgart http://icl.cs.utk.edu/hpcc/ 10
Recommend
More recommend
