Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Structure of Talk Workload-sensitive Timing Behavior Anomaly Detection 1 Motivation in Large Software Systems Diploma Thesis 2 Foundations Hypothesis & Goals 3 André van Hoorn Results 4 Abteilung Software Engineering Fakultät II - Department für Informatik Conclusions 5 November 8, 2007 Related Work 6 André van Hoorn Diploma Thesis November 8, 2007 1/ 42 André van Hoorn Diploma Thesis November 8, 2007 2/ 42 Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Motivation Foundations Motivation Structure 1 Motivation Availability of Enterprise Information Systems (e.g. banking 2 Foundations & online shopping systems) is critical QoS requirement 3 Hypothesis & Goals Anomaly detection is means for failure detection and dia- gnosis to improve availability Results 4 Existing anomaly detection approaches based on timing behavior do not explicitly consider varying workload Conclusions 5 Related Work 6 André van Hoorn Diploma Thesis November 8, 2007 3/ 42 André van Hoorn Diploma Thesis November 8, 2007 4/ 42 Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Foundations Foundations Performance Metrics and Scalability Performance Metrics and Scalability Performance Workload and Scalability Nominal Capacity Throughput 1 Time Behavior Workload a b Response Time Amount of work currently reque- a() Usable Capacity Time interval elapsed between sted from or processed by a sy- Knee Capacity b() issued request and respective stem response Execution Time Response Time Execution Time Response Time Characteristics Workload Execution Time = Response Time Workload intensity Throughput Service demand characteristics Rate at which a system (re- source) handles tasks Scalability (Client-/Server-side) “ability of a system to continue to meet Think Time, . . . its response time or throughput objecti- Workload 2 Resource Utilization ves as the [workload] increases” [SW01] Figure: Operation timing metrics. The capacity of a system [Jai91]. André van Hoorn Diploma Thesis November 8, 2007 5/ 42 André van Hoorn Diploma Thesis November 8, 2007 6/ 42
Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Foundations Foundations Workload Characterization Anomaly Detection A Hierarchical Workload Model Anomaly Detection Business Level Motivation : Availability important QoS attribute 3. User Level Session Layer [ MIO87 ] MTTF Availability = MTTF + MTTR 2. Application Level Functional Layer Goal : Improve availability by reduction of repair times 1. Protocol Level HTTP Request Layer Strategy : Use unusual behavior as indicator for failures Resource Level Common approach for software systems : Figure: A hierarchical workload model [MAR + 00]. Build model of “normal behavior” (based on set of monitored parameters, e.g. time behavior) Monitor current behavior Session [MAFM99] Detect deviations “Consecutive and related requests issued by the same user” André van Hoorn Diploma Thesis November 8, 2007 7/ 42 André van Hoorn Diploma Thesis November 8, 2007 8/ 42 Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Foundations Foundations Probability and Statistics Probability and Statistics Descriptive Statistics Parametric Distribution Families (Examples) Λ Λ (x| 0, 1) N(x| 1, 1) Λ (x| 0.7, 0.5 2 ) Λ N(x| −1, 0.9 2 ) Λ (x| 2, 0, 1) N(x| 1, 0.8 2 ) Statistics Minimum, maximum Sample mean, Sample variance Density Density p -Quantile x p : x p = min { x | F ( x ) ≥ p } 1.–3. Quartiles: x 0 . 25 , x 0 . 5 (Median), x 0 . 75 Mode, Skewness, . . . Other distribution characteristics: −4 −3 −2 −1 0 1 2 3 4 0 1 2 3 4 5 6 7 8 9 uni-/bi-/multimodal x x (a)symmetric Normal Distribution Log-normal Distribution left-/right-skewed 2-parameter: N ( µ, σ 2 ) 2-parameter: Λ( µ, σ 2 ) 3-parameter: Λ( τ, µ, σ 2 ) André van Hoorn Diploma Thesis November 8, 2007 9/ 42 André van Hoorn Diploma Thesis November 8, 2007 10/ 42 Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Hypothesis & Goals Hypothesis & Goals Structure Hypothesis 1 Motivation Assuming that varying workload implies 2 Foundations varying response times: 3 Hypothesis & Goals Hypothesis Novel workload-sensitive anomaly detection based on 4 Results response times realizable if varying workload intensity has Conclusions characteristic impact on response time distributions 5 Related Work 6 André van Hoorn Diploma Thesis November 8, 2007 11/ 42 André van Hoorn Diploma Thesis November 8, 2007 12/ 42
Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Hypothesis & Goals Hypothesis & Goals Project Goals I Project Goals I 1 Probabilistic Workload Driver 1 Probabilistic Workload Driver Develop application-generic methodology for generating Develop application-generic methodology for generating realistic user behavior (e.g. based on probabilistic model) realistic user behavior (e.g. based on probabilistic model) 2 Case Study with Response Time Analysis 2 Case Study with Response Time Analysis Apply & evaluate workload generation technique Apply & evaluate workload generation technique Obtain workload-dependent response times Obtain workload-dependent response times from sample application from sample application Statistically analyze impact of workload on response times Statistically analyze impact of workload on response times 3 Workload-Sensitive Anomaly Detection Prototype 3 Workload-Sensitive Anomaly Detection Prototype Compute degree of anomaly for operation executions Compute degree of anomaly for operation executions Implementation of workload-sensitive AD prototype Implementation of workload-sensitive AD prototype André van Hoorn Diploma Thesis November 8, 2007 13/ 42 André van Hoorn Diploma Thesis November 8, 2007 13/ 42 Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Hypothesis & Goals Results Probabilistic Workload Driver Project Goals I Structure Motivation 1 2 Foundations 1 Probabilistic Workload Driver Performance Metrics and Scalability Develop application-generic methodology for generating Workload Characterization realistic user behavior (e.g. based on probabilistic model) Anomaly Detection 2 Case Study with Response Time Analysis Probability and Statistics Apply & evaluate workload generation technique 3 Hypothesis & Goals Obtain workload-dependent response times from sample application 4 Results Statistically analyze impact of workload on response times Probabilistic Workload Driver 3 Workload-Sensitive Anomaly Detection Prototype Case Study with Response Time Analysis Compute degree of anomaly for operation executions Workload-sensitive Anomaly Detection Prototype Implementation of workload-sensitive AD prototype 5 Conclusions 6 Related Work André van Hoorn Diploma Thesis November 8, 2007 13/ 42 André van Hoorn Diploma Thesis November 8, 2007 14/ 42 Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Workload-sensitive Timing Behavior Anomaly Detection in Large Software Systems Results Results Probabilistic Workload Driver Probabilistic Workload Driver Probabilistic Workload Driver – Approach Application Model Challenge : Generate valid sessions Constraint : Realistic behavior (not: “capture & replay”) Session layer mo- Approach : dels allowed se- 1 Workload configuration data model separated into quences of service Application Model calls in a session User Behavior Model Protocol layer con- User Behavior Mix Workload Intensity tains all protocol- 2 High-level design specific (e.g. HTTP) Iterative execution model request details Figure: Sample application model illustrating separation Session model composition semantics into session and protocol layer. 3 Implementation: Markov4JMeter (JMeter extension) André van Hoorn Diploma Thesis November 8, 2007 15/ 42 André van Hoorn Diploma Thesis November 8, 2007 16/ 42
Recommend
More recommend
