Scalable and Live Trace Processing with Kieker Utilizing Cloud - PowerPoint PPT Presentation

Scalable and Live Trace Processing with Kieker Utilizing Cloud Computing Florian Fittkau, Jan Waller, Peer Brauer, and Wilhelm Hasselbring 2013-11-28 Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 1 / 18

1. Introduction 2. ExplorViz 3. Scalable Trace Processing Architecture 4. High-Throughput Tunings for Kieker 5. Preliminary Performance Evaluation 6. Related Work 7. Future Work and Conclusions 8. References Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 1 / 18

Introduction Introduction ◮ Knowledge of the internal behavior often gets lost ◮ Application-level monitoring ◮ Can cause large impact on the performance ◮ High-throughput trace processing reducing the overhead ◮ Cloud infrastructures Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 2 / 18

Landscape Level Perspective ExplorViz Figure 1 : Macro view on landscape level showing the communication between applications in the PubFlow ( http://pubflow.de ) software landscape [FWWH13] Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 3 / 18

System Level Perspective ExplorViz (a) Macro view visualizing four (b) Relationship view with opened service components of jPetStore component Figure 2 : Mockup of system level perspective on the example of jPetStore for demonstrating the exploration concept [FWWH13] Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 4 / 18

ExplorViz Dataflow ExplorViz Existing Preprocessed Traces Aggregated Traces Landscape Model Monitoring Data Application A1 A2 A3 132743373;CartBean;addItem;52.168 Existing 132416973;CartBean;addItem;58.163 132419877;CartBean;addItem;52.188 Application 132419877;CartBean;addItem;52.188 … A4 Visualization Landscape Level Perspective System Level Perspective Legend A1: Monitoring A2: Preprocessing A5 A3: Aggregation A4: Transformation A5: Navigation Figure 3 : Activities in our ExplorViz approach for live trace visualization of large software landscapes [FWWH13] Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 5 / 18

Basic Approach Scalable Trace Processing Architecture <<executionEnvironment>> ClientWorkstation Cloud <<component>> ExplorViz ApplicationNodes <<component>> Applications SLAsticNode <<component>> Kieker . Monitoring ExplorVizServerNode <<component>> SLAstic <<component>> <<component>> AnalysisWorkerLoadBalancer ExplorVizServer <<component>> LandscapeModelRepository AnalysisWorkerNodes <<component>> <<component>> <<component>> AnalysisWorker AnalysisMaster MQProvider <<component>> Kieker.Monitoring <<component>> ModelDatabase Figure 4 : Overview on our general trace processing architecture Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 6 / 18

Chaining of Analysis Workers Scalable Trace Processing Architecture <<component>> <<component>> Monitored Application1 AnalysisWorker1 <<component>> AnalysisWorker5 <<component>> <<component>> Monitored Application2 AnalysisWorker2 <<component>> AnalysisMaster <<component>> <<component>> Monitored Application3 AnalysisWorker3 <<component>> AnalysisWorker6 <<component>> <<component>> Monitored Application4 AnalysisWorker4 Figure 5 : Example for chaining of analysis workers Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 7 / 18

Chaining of Analysis Workers Scalable Trace Processing Architecture ◮ Levels of chaining are not restricted to one or two ◮ On each level, the number of analysis workers should be lower than before ◮ SLAstic can be used to scale each group of analysis workers ◮ SLAstic can be extended to decide whether a new analysis worker level should be opened Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 8 / 18

Kieker.Monitoring Tunings High-Throughput Tunings for Kieker Figure 6 : Our high-throughput tuned version of Kieker.Monitoring Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 9 / 18

Kieker.Analysis Tunings High-Throughput Tunings for Kieker Figure 7 : Our high-throughput tuned version of Kieker.Analysis Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 10 / 18

Experimental Setup Preliminary Performance Evaluation ◮ Extended version of the monitoring overhead benchmark MooBench [WH12] ◮ 2 virtual machines (VMs) in our OpenStack private cloud ◮ Each physical machine in our private cloud contains two 8-core Intel Xeon E5-2650 (2 GHz) processors, 128 GiB RAM, and a 10 Gbit network connection Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 11 / 18

Results for Kieker 1.8 Preliminary Performance Evaluation No inst. Deactiv. Collecting Writing Reconst. Reduction Mean 2 500.0k 1 176.5k 141.8k 39.6k 0.5k 0.5k 95% CI ± 371.4k ± 34.3k ± 2.0k ± 0.4k ± 0.001k ± 0.001k Q 1 2 655.4k 1 178.0k 140.3k 36.7k 0.4k 0.4k Median 2 682.5k 1 190.2k 143.9k 39.6k 0.5k 0.5k Q 3 2 700.4k 1 208.0k 145.8k 42.1k 0.5k 0.5k Table 1 : Throughput for Kieker 1.8 (traces per second) Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 12 / 18

Results for Our Tuned Kieker Version Preliminary Performance Evaluation No inst. Deactiv. Collecting Writing Reconst. Reduction Mean 2 688.2k 770.4k 136.5k 115.8k 116.9k 112.6k 95% CI ± 14.5k ± 8.4k ± 0.9k ± 0.7k ± 0.7k ± 0.8k Q 1 2 713.6k 682.8k 118.5k 102.5k 103.3k 98.4k Median 2 720.8k 718.1k 125.0k 116.4k 116.6k 114.4k Q 3 2 726.8k 841.0k 137.4k 131.9k 131.3k 132.4k Table 2 : Throughput for our high-throughput tuned Kieker version (traces per second) Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 13 / 18

Resulting Response Times Preliminary Performance Evaluation Figure 8 : Comparison of the resulting response times Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 14 / 18

Threats to Validity Preliminary Performance Evaluation ◮ Only on one type of virtual machine/hardware ◮ Virtualized cloud environment might resulted in unfortunate scheduling effects ◮ Minimized this threat by prohibiting over-provisioning Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 15 / 18

Related Work Related Work ◮ Dapper ◮ Magpie ◮ X-Trace Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 16 / 18

Future Work Future Work and Conclusions ◮ Evaluate the scalability and performance of our trace processing architecture in our private cloud environment ◮ Search for guidelines which number of levels of analysis workers is suitable in which situation ◮ Feedback our high-throughput tunings into Kieker Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 17 / 18

Conclusions Future Work and Conclusions ◮ Enabling scalable monitoring in the cloud ◮ Live trace processing for ExplorViz 1 ◮ Improved the analysis performance of Kieker by a factor of 250 1 http://www.explorviz.net Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 18 / 18

Florian Fittkau, Jan Waller, Christian Wulf, and Wilhelm Hasselbring. Live trace visualization for comprehending large software landscapes: The ExplorViz approach. In Proceedings of the 1st IEEE International Working Conference on Software Visualization (VISSOFT 2013). IEEE Computer Society, 2013. Jan Waller and Wilhelm Hasselbring. A comparison of the influence of different multi-core processors on the runtime overhead for application-level monitoring. In Multicore Software Engineering, Performance, and Tools (MSEPT 2012), pages 42–53. Springer, 2012. Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 18 / 18