Towards Visualization Scalability through Time Intervals and Hierarchical Organization of Monitoring Data Lucas Mello Schnorr, Guillaume Huard, Philippe Olivier Alexandre Navaux Federal University of Rio Grande do Sul, Brazil Grenoble Institute of Technology, France – IEEE/ACM CCGrid – Shanghai, China – May 2009 – 1 / 23
Motivation Large-Scale Grids Grid’5000: limited heterogeneity, strong hierarchy Many geographically distributed computing resources Size of Parallel Applications: processes and threads KAAPI Applications executing on top of Grid’5000 2007 Grid@Work Contest: 3654 processes Behavior Analysis of Large Parallel Applications Enormous quantity of monitoring data Number of processes and threads How much detail is collected for each of them Often is a complex task How existing tools deal with these issues? 2 / 23
Motivation Most tools work on the monitoring data Reduction and Selection Mechanisms Paj´ e, ParaProf Hierarchical Representation with Gantt-Chart: Vampir Scalable file format: Jumpshot Traditional Space-Time Visualization Limited number of processes that can be represented Difficult to compare processes’s behavior 3 / 23
Motivation Most tools work on the monitoring data Reduction and Selection Mechanisms Paj´ e, ParaProf Hierarchical Representation with Gantt-Chart: Vampir Scalable file format: Jumpshot Traditional Space-Time Visualization Limited number of processes that can be represented Difficult to compare processes’s behavior Our Approach Definition of time intervals + Dynamic creation of a hierarchical structure + Use of the Treemap technique to visualization = Visualization Scalability 4 / 23
Outline 1 The Scalable Hierarchical Visualization Hierarchical Monitoring Data Time-Slice Algorithm Treemap Representation Triva Prototype 2 3 Results Conclusion 4 5 / 23
The Scalable Hierarchical Visualization Series of transformations on monitoring data Interactive Analysis Three steps Creating a Hierarchical Structure for Monitoring Data Applying the Time-Slice Algorithm Visualizing with the Treemap Technique Easy to merge with space-time visualizations 6 / 23
Step 1: Hierarchical Monitoring Data Monitoring systems register entities behavior Entities can be processes and threads They can be organized as a hierarchy Logical hierarchy Geographical Location hierarchy Other possibilities: libraries, components 7 / 23
Step 1: Hierarchical Monitoring Data Monitoring systems register entities behavior Entities can be processes and threads They can be organized as a hierarchy Logical hierarchy Geographical Location hierarchy Other possibilities: libraries, components 8 / 23
Step 1: Hierarchical Monitoring Data Monitoring systems register entities behavior Entities can be processes and threads They can be organized as a hierarchy Logical hierarchy Geographical Location hierarchy Other possibilities: libraries, components 9 / 23
Step 1: Hierarchical Monitoring Data Monitoring systems register entities behavior Entities can be processes and threads They can be organized as a hierarchy Logical hierarchy Geographical Location hierarchy Other possibilities: libraries, components 10 / 23
Step 2: The Time-Slice Algorithm To attribute a value for each leaf node Define these values based on two criterias Time Interval Fixed size (example: always 1 second) Dynamic size (from microseconds to days) Can be moved through the total time of analysis Summary of Events Define what we want to observe (e.g. a blocked state) More than one property of an entity can be observed Example 11 / 23
Step 2: The Time-Slice Algorithm - Example 12 / 23
Step 3: The Treemap Representation Created in 1991, by Johnson and Shneiderman Scalable hierarchical representation Squarified Treemaps Algorithm For a given node, split screen space among children Repeat the screen division on all of the children 13 / 23
The Treemap Representation - Example 14 / 23
Triva Prototype Implemented with Objective-C, C++ Component-based implementation DIMVisual Reader (defines location and hierarchy) Paj´ e Simulator / Storage Controller Time Slice Algorithm (annotates the hierarchy) Triva Display (implements Treemap) Input data: traces from KAAPI applications Library to define tasks and relations among them Runtime that executes the graph of tasks Work-stealing to achieve load-balancing 15 / 23
Results Traces from large-scale KAAPI application Analyze load balancing versus work stealing Processes are in one of two states: executing or steal Grid’5000 platform in France used for experiments Limited heterogeneity, strong hierarchy Grid composed of interconnected clusters Grid – Site – Cluster – Machine – Process – State Experiments specification #1: 200 machines, divided in 2 sites Site nancy with 100 machines from 2 clusters Site rennes with 100 machines from 3 clusters Application with 200 processes #2: 310 machines, divided in 4 sites Sites rennes , sophia and bordeaux , 3 clusters each Site lille , with 4 clusters Application with 2900 processes 16 / 23
Experiment #1 Start of the execution – → Stealing, → Executing 17 / 23
Experiment #1 End of the execution – → Stealing, → Executing 18 / 23
Experiment #1 Total execution time – → Stealing, → Executing 19 / 23
Experiment #1 20 / 23
Experiment #2 Application with 2900 processes 21 / 23
Conclusion Visualization Scalability problem Our approach Creation of an annotated hierarchical structure Use Treemaps to representation Main contributions Technique to create the hierarchical structure More powerful and scalable statistical tool Complements traditional space-time visualization Easy view of snapshots of grid application behavior Future Work Summary of other types of data: variables, events Sub-Level Hierarchical Aggregation 22 / 23
Thank You! Questions? Work supported by the Brazilian Government CAPES and CNPq Research Councils Contact information Lucas.Schnorr@inf.ufrgs.br http://www.inf.ufrgs.br/ ~ lmschnorr 23 / 23
Recommend
More recommend
