OCELOTL: LARGE TRACE OVERVIEWS BASED ON MULTIDIMENSIONAL DATA AGGREGATION OCELOTL: CLOSE ENCOUNTERS OF THE THIRD KIND 8th International Parallel Tools Workshop 1st October 2014 Damien Dosimont 1 2 , Youenn Corre 1 2 , Lucas M. Schnorr 3 , Guillaume Huard 2 1 , Jean-Marc Vincent 2 1 1 Inria, first.last@inria.fr, 2 Univ. Grenoble Alpes, LIG, CNRS, F-38000 Grenoble, France first.last@imag.fr 3 Informatics Institute, UFRGS, Porto Alegre schnorr@inf.ufrgs.br
INTRODUCTION
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . TRACE VISUALIZATION PROBLEMATIC ▶ Trace contents : • SPACE = application structure: ▶ hardware components: clusters, machines, cores, etc. ▶ software components: processes, threads, etc. • TIME = timestamped events: ▶ function calls, communications, CPU load, malloc, etc. ▶ Traces can be HUGE → scalability issues of space-time representations 3
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . PROBLEMATIC VISUALIZATION 4
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . OUR PROPOSAL MULTIDIMENSIONAL OVERVIEWS ▶ Several overviews generated thanks to data aggregation • Temporal • Spatiotemporal ▶ Showing meaningful information (phases, perturbations) ▶ Possibility to adjust dynamically the level of details 5
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . EXAMPLE : TEMPORAL OVERVIEW 6
S B S A S C Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . EXAMPLE : SPATIOTEMPORAL OVERVIEW MPI_Init MPI_Allreduce MPI_Recv Heterogeneous Spatiotemporal Behavior Temporal Perturbation 0s 20s 40s 60s 7 Grid'5000 Nancy, 700 Cores, NASBP LU, class C
THEORETICAL BACKGROUND : LAMARCHE-PERRIN METHODOLOGY
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . ADAPTING AN AGGREGATION METHODOLOGY 9
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . INFORMATION LOSS: KL DIVERGENCE ( ρ e ) ∑ loss E = ρ e log 2 ρ E e ∈ E 10
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . COMPLEXITY REDUCTION: SHANNON ENTROPY ∑ gain E = ρ E log 2 ρ E − ρ e log 2 ρ e e ∈ E 11
pIC E Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . TRADE-OFF: PIC ∑ pIC E = p gain E − (1 − p ) loss E pIC P = E ∈P ▶ For a given p: choose P with the highest pIC ▶ Aggregate in priority most homogeneous values 12
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . VIVA: SPATIAL AGGREGATION (SCHNORR & LP) 13
TEMPORAL OVERVIEW
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . TEMPORAL AGGREGATION AND VISUALIZATION IRQ Process 1 Process 2 0 1 2 3 Time Temporal Aggregation 0 12 3 Spatial Aggregation 15
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . MINIMUM INFORMATION LOSS: P=0 16
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . MAXIMUM COMPLEXITY REDUCTION: P=1 17
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . INTERESTING TRADE-OFF 18
SPATIOTEMPORAL OVERVIEW
S A S S B S C T Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . GENERATE A TRACE MICROSCOPIC MODEL s 1 s 2 s 3 s 4 s 5 s 6 s 7 s 8 s 9 s 10 s 11 s 12 t 1 t 6 t 9 t 14 t 19 | X | = 2 , ρ x ( s , t ) = d x ( s , t )/ d ( t ) ∈ [0 , 1] , ρ 1 ( s , t ) = 1 − ρ 2 ( s , t ) 20
S S C S B S A S S C S B S A Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . AGGREGATE THE MICROSCOPIC MODEL s 1 s 2 s 3 s 4 s 5 s 6 s 7 s 8 s 9 s 10 s 11 s 12 p = a , 0 < a < 1 p = 0 s 1 s 2 s 3 s 4 s 5 s 6 s 7 s 8 s 9 s 10 s 11 s 12 p = b , 0 < a < b < 1 p = 1 21
IMPLEMENTATION AND FEATURES
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . OCELOTL TOOL ▶ Implementation of the overview techniques ▶ Generic architecture. Add: • Your own aggregation operator (dimensions, metric) • Your own visualization ▶ Persistent caches to avoid long recomputations ▶ Integrated in Framesoc : • Trace and tools management • Fast trace reading (DB queries) • Interaction with other analysis tools • Also enable to add you own tools 23
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . FRAMESOC ▶ Trace format compatibility : Pajé (Akypuera: tool to convert from OTF2, Tau), LTTng, KPTrace 24
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . PERFORMANCE: TEMPORAL ANALYSIS Total analysis time as a function of trace size (100 time slices) 4 3,5 3 2,5 Time (min) 2 1,5 1 0,5 0 0 2 4 6 8 10 12 14 Size (GB) No cache With cache 25
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . PERFORMANCE: TEMPORAL ANALYSIS Total analysis time as a function of trace size (1000 time slices) 9 8 7 6 Time (min) 5 4 3 2 1 0 0 2 4 6 8 10 12 14 Size (GB) No Cache With Cache 26
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . PERFORMANCE: SPATIOTEMPORAL ANALYSIS Total analysis time as a function of trace size (30 time slices) 6 5 4 Time (min) 3 2 1 0 0 2 4 6 8 10 12 14 Size (GB) No cache With cache 27
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . DEMONSTRATION 28
CONCLUSION
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . CONCLUSION ▶ Visualizations based on spatiotemporal data aggregation • Solves screen, computing and analyst capability limitations • Gives meaningful information about homogeneity (phases, perturbations) ▶ Implementation : • Interaction (zoom, switch to other tools) • Helps to drastically reduce computation times (caches) • Generic architecture : add your own aggregation and visualization ▶ Future work : • Extend methodology and design new algorithms ( H ( S ) × H ( S ) × I ( T ) , surface, etc.) • Improve visualization and interaction to get more details • Framesoc: native compatibility with OTF2 (soon) 30
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . LINKS Ocelotl: http://github.com/dosimont/ocelotl Framesoc: http://github.com/generoso/framesoc Viva: http://github.com/schnorr/viva 31
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . THANK YOU FOR YOUR ATTENTION 32
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . OCELOTL: TEMPORAL AGGREGATION (1) 33
Introduction Theoretical Background Temporal Overview Spatiotemporal Overview Implementation and Features Conclusion . . . . . . . . . . . . . . . . . . . . . . OCELOTL: TEMPORAL AGGREGATION (2) 34
Recommend
More recommend
