Insightful Automatic Performance Modeling Software Alexandru Calotoiu - PowerPoint PPT Presentation

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Insightful Automatic Performance Modeling Software Alexandru Calotoiu 1 , Torsten Hoefler 2 , Martin Schulz 3 , Sergei Shudler 1 and Felix Wolf 1 1 TU Darmstadt , 2 ETH Zürich , 3 Lawrence Livermore National Laboratory

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Installing Extra-P

5 VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Installing Extra-P � Download and install QT4 � Download and install Python3+ and PYQT � Download and install Cube � http://www.scalasca.org/software/cube-4.x/download.html � Download Extra-P � http://www.scalasca.org/software/extra-p/download.html � Unpack & install Extra-P � ./configure --prefix=<extra-p-install-dir> --with-cube=<cube-dir> CPPFLAGS=-I<python.h path>; make; make install INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 3

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Using Extra-P

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Automatic performance modeling with Extra-P Performance measurements (profiles) Myapp.cpp Myapp.weak.p128.r1/profile.cubex main() { foo() Myapp.weak.p256.r1/profile.cubex Instrumentation & bar() Myapp.weak.p512.r1/profile.cubex Measurement compute() Myapp.weak.p1024.r1/profile.cubex Score-P } Myapp.weak.p2048.r1/profile.cubex Input Extra-P Output Results.cubex Results.xtrap Region 1: main Model: (3) + (3.14 * x^( 2 )) ... INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 5

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Extra-P input in text form � Useful to debug or when a small data set must be modeled � Example provided in input.txt POINTS 1000 2000 4000 8000 16000 EXPERIMENT ImportantData DATA 1 1 1 1 1 DATA 4 4 4 3.99 4.01 DATA 16 15.999 16.01 16.01 15.99 DATA 64 64 64 64.01 63.99 DATA 256.01 255.99 256 256 INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 6

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Extra-P input in text form � Useful to debug or when a small data set must be modeled � Example provided in input.txt Measurement points Use at least 4, preferably 5, POINTS 1000 2000 4000 8000 16000 but in general the more the better EXPERIMENT ImportantData DATA 1 1 1 1 1 DATA 4 4 4 3.99 4.01 DATA 16 15.999 16.01 16.01 15.99 DATA 64 64 64 64.01 63.99 DATA 256.01 255.99 256 256 INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 7

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Extra-P input in text form � Useful to debug or when a small data set must be modeled � Example provided in input.txt POINTS 1000 2000 4000 8000 16000 EXPERIMENT ImportantData Experiment identifier Acts as both identifier and separator DATA 1 1 1 1 1 for experiments and their data DATA 4 4 4 3.99 4.01 DATA 16 15.999 16.01 16.01 15.99 DATA 64 64 64 64.01 63.99 DATA 256.01 255.99 256 256 INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 8

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Extra-P input in text form � Useful to debug or when a small data set must be modeled � Example provided in input.txt POINTS 1000 2000 4000 8000 16000 EXPERIMENT ImportantData DATA 1 1 1 1 1 DATA 4 4 4 3.99 4.01 Data points DATA 16 15.999 16.01 16.01 15.99 Each row corresponds to a point; DATA 64 64 64 64.01 63.99 all values in a row are considered DATA 256.01 255.99 256 256 repeat measurements of the same experiment INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 9

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Extra-P input in text form � Useful to debug or when a small data set must be modeled � Example provided in input.txt Measurement points POINTS 1000 2000 4000 8000 16000 EXPERIMENT ImportantData Experiment identifier DATA 1 1 1 1 1 DATA 4 4 4 3.99 4.01 Data points DATA 16 15.999 16.01 16.01 15.99 DATA 64 64 64 64.01 63.99 DATA 256.01 255.99 256 256 INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 10

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Results in text form

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Extra-P output – Text form Measurements and model data for each experiment and metric: Metric name; either Score-P metrics (time, Callpath/Region: exp4 bytes, etc.) or custom metrics Metric: Test Data: Measurements for each input element ( 1000, 1e+06) 95% CI [1.00001e+06, 999989] (e.g., #processes) ( 2000, 4e+06) 95% CI [4.00003e+06, 3.99998e+06] ( 4000, 1.6e+07) 95% CI [1.6e+07, 1.6e+07] ( 8000, 6.4e+07) 95% CI [6.4e+07, 6.4e+07] ( 16000, 2.56e+08) 95% CI [2.56e+08, 2.56e+08] Best-fit model Model: RSS: Residual sum of squares 0+1*(p^2) RSS: 3.35017 Adjusted R 2 (explained previously) Adjusted R^2: 1 INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 12

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Modeling sets of Cube experiments

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Extra-P Cube input description � Modeling tool expects Cube files in the following format: <DIR>/<PREFIX><X><POSTFIX>.r<{1,..,REPS}>/<FILENAME> � DIR, PREFIX, X, POSTFIX, REPS and FILENAME must all be defined. � X – value of varied parameter e.g. number of processes � REPS – number of repeated experiments with same parameter value INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 14

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Extra-P Cube input description <DIR>/<PREFIX><X><POSTFIX>.r<{1,..,REPS}>/<FILENAME> Best effort approach to identify and populate the fields automatically based on selected directory contents. INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 15

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Visualization with Extra-P

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Extra-P user interface Selected kernel(s) Plot of the model Call tree exploration INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 17

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Extra-P call tree view Metric selection Call tree exploration Model Quality of fit metrics: Residual sum of squares and Adjusted R 2 Asymptotic view of model functions vs. value at given value Impact of each kernel on the metric at the selected process count compared to the other kernels INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 18

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Extra-P model view Models selected in the Call path view Measurement values X axis scale control for prediction of behavior at other process counts INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 19

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Extra-P model generation tool Number of terms. Recommended value 1, as it captures dominant behavior Search space definition Note: When a new experiment is loaded the values here are used for the initial modeling INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 20

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Feedback � What additional features would you like to see? � What additional capabilities would you like to see? INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 21

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING Thank You!