Load Shedding in Network Monitoring Applications Pere Barlet Ros - PowerPoint PPT Presentation

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Load Shedding in Network Monitoring Applications Pere Barlet Ros Advisor: Dr. Gianluca Iannaccone Co-advisor: Prof. Josep Solé Pareta Universitat Politècnica de Catalunya (UPC) Departament d’Arquitectura de Computadors December 15, 2008 1 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Outline Introduction 1 Prediction and Load Shedding Scheme 2 Fairness of Service and Nash Equilibrium 3 Custom Load Shedding 4 Conclusions and Future Work 5 2 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Outline Introduction 1 Motivation Related work Contributions Prediction and Load Shedding Scheme 2 Fairness of Service and Nash Equilibrium 3 Custom Load Shedding 4 Conclusions and Future Work 5 3 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Motivation Network monitoring is crucial for operating data networks Traffic engineering, network troubleshooting, anomaly detection . . . Monitoring systems are prone to dramatic overload situations Link speeds, anomalous traffic, bursty traffic nature . . . Complexity of traffic analysis methods Overload situations lead to uncontrolled packet loss Severe and unpredictable impact on the accuracy of applications . . . when results are most valuable!! 4 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Motivation Network monitoring is crucial for operating data networks Traffic engineering, network troubleshooting, anomaly detection . . . Monitoring systems are prone to dramatic overload situations Link speeds, anomalous traffic, bursty traffic nature . . . Complexity of traffic analysis methods Overload situations lead to uncontrolled packet loss Severe and unpredictable impact on the accuracy of applications . . . when results are most valuable!! Load Shedding Scheme Efficiently handle extreme overload situations Over-provisioning is not feasible 4 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Related Work Load shedding in data stream management systems (DSMS) Examples: Aurora, STREAM, TelegraphCQ, Borealis, . . . Based on declarative query languages (e.g., CQL) Small set of operators with known (and constant) cost Maximize an aggregate performance metric (utility or throughput) Limitations Restrict the type of metrics and possible uses Assume explicit knowledge of operators’ cost and selectivity Not suitable for non-cooperative environments Resource management in network monitoring systems Restricted to a pre-defined set of metrics Limit the amount of allocated resources in advance 5 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Contributions Prediction method 1 Operates w/o knowledge of application cost and implementation Does not rely on a specific model for the incoming traffic Load shedding scheme 2 Anticipates overload situations and avoids packet loss Relies on packet and flow sampling (equal sampling rates) Extensions: Packet-based scheduler 3 Applies different sampling rates to different queries Ensures fairness of service with non-cooperative applications Support for custom-defined load shedding methods 4 Safely delegates load shedding to non-cooperative applications Still ensures robustness and fairness of service 6 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Outline Introduction 1 Prediction and Load Shedding Scheme 2 Case Study: Intel CoMo Prediction Methodology Load Shedding Scheme Evaluation and Operational Results Fairness of Service and Nash Equilibrium 3 Custom Load Shedding 4 Conclusions and Future Work 5 7 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Case Study: Intel CoMo CoMo (Continuous Monitoring) 1 Open-source passive monitoring system Framework to develop and execute network monitoring applications Open (shared) network monitoring platform Traffic queries are defined as plug-in modules written in C Contain complex computations 1 http://como.sourceforge.net 8 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Case Study: Intel CoMo CoMo (Continuous Monitoring) 1 Open-source passive monitoring system Framework to develop and execute network monitoring applications Open (shared) network monitoring platform Traffic queries are defined as plug-in modules written in C Contain complex computations Traffic queries are black boxes Arbitrary computations and data structures Load shedding cannot use knowledge of the queries 1 http://como.sourceforge.net 8 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Load Shedding Approach Working Scenario Monitoring system supporting multiple arbitrary queries Single resource: CPU cycles Approach: Real-time modeling of the queries’ CPU usage Find correlation between traffic features and CPU usage 1 Features are query agnostic with deterministic worst case cost Exploit the correlation to predict CPU load 2 Use the prediction to decide the sampling rate 3 9 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions System Overview Figure: Prediction and Load Shedding Subsystem 10 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Traffic Features vs CPU Usage x 10 6 CPU cycles 4 2 0 0 10 20 30 40 50 60 70 80 90 100 3000 Packets 2000 1000 0 0 10 20 30 40 50 60 70 80 90 100 5 x 10 15 Bytes 10 5 0 0 10 20 30 40 50 60 70 80 90 100 3000 5−tuple flows 2000 1000 0 0 10 20 30 40 50 60 70 80 90 100 Time (s) Figure: CPU usage compared to the number of packets, bytes and flows 11 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Traffic Features vs CPU Usage x 10 6 2.8 2.6 2.4 CPU cycles 2.2 2 1.8 new_5tuple_hashes < 500 500 <= new_5tuple_hashes < 700 1.6 700 <= new_5tuple_hashes < 1000 new_5tuple_hashes >= 1000 1.4 1800 2000 2200 2400 2600 2800 3000 packets/batch Figure: CPU usage versus the number of packets and flows 12 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Prediction Methodology 2 Multiple Linear Regression (MLR) Y i = β 0 + β 1 X 1 i + β 2 X 2 i + · · · + β p X pi + ε i , i = 1 , 2 , . . . , n . Y i = n observations of the response variable (measured cycles) X ji = n observations of the p predictors (traffic features) β j = p regression coefficients (unknown parameters to estimate) ε i = n residuals (OLS minimizes SSE) 2 P . Barlet-Ros et al. "Load Shedding in Network Monitoring Applications", Proc. of USENIX Annual Technical Conference, 2007. 13 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Prediction Methodology 2 Multiple Linear Regression (MLR) Y i = β 0 + β 1 X 1 i + β 2 X 2 i + · · · + β p X pi + ε i , i = 1 , 2 , . . . , n . Y i = n observations of the response variable (measured cycles) X ji = n observations of the p predictors (traffic features) β j = p regression coefficients (unknown parameters to estimate) ε i = n residuals (OLS minimizes SSE) Feature Selection Variant of the Fast Correlation-Based Filter (FCBF) Removes irrelevant and redundant predictors Reduces significantly the cost and improves accuracy of the MLR 2 P . Barlet-Ros et al. "Load Shedding in Network Monitoring Applications", Proc. of USENIX Annual Technical Conference, 2007. 13 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Load Shedding Scheme 3 Prediction and Load Shedding subsystem 1 Each 100 ms of traffic is grouped into a batch of packets 2 The traffic features are efficiently extracted from the batch (multi-resolution bitmaps) 3 The most relevant features are selected (using FCBF) to be used by the MLR 4 MLR predicts the CPU cycles required by each query to run 5 Load shedding is performed to discard a portion of the batch 6 CPU usage is measured (using TSC) and fed back to the prediction system 3 P . Barlet-Ros et al. "On-line Predictive Load Shedding for Network Monitoring", Proc. of IFIP-TC6 Networking, 2007. 14 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Results: Load Shedding Performance 9 x 10 9 8 7 CPU usage [cycles/sec] 6 CoMo cycles Load shedding cycles 5 Query cycles 4 Predicted cycles CPU limit 3 2 1 0 09 am 10 am 11 am 12 pm 01 pm 02 pm 03 pm 04 pm 05 pm time Figure: Stacked CPU usage (Predictive Load Shedding) 15 / 33

Introduction Load Shedding Fairness and NE Custom Load Shedding Conclusions Results: Load Shedding Performance 1 0.9 0.8 0.7 F(CPU usage) 0.6 0.5 0.4 CPU limit per batch 0.3 Predictive 0.2 Original Reactive 0.1 0 0 2 4 6 8 10 12 14 16 CPU usage [cycles/batch] 8 x 10 Figure: CDF of the CPU usage per batch 16 / 33