Detection using Search-Based Application Profiling Du Sh Shen, Qi - PowerPoint PPT Presentation

Automating Performance Bottleneck Detection using Search-Based Application Profiling Du Sh Shen, Qi i Luo, , Denys Posh shyv yvanyk, Mark rk Grechanik ik* Col olle lege of of Will llia iam and and Mar ary *U *Univ iversit ity of of Il Illi linois is at t Chi hicago ISS ISSTA 2015 Ba Balt ltimore, MD, U.S .S . 1

2

Standard application profiling Inputs Application 3

Standard application profiling Inputs Application 4

Standard application profiling Inputs Application 1. Agilefant.model.WidgetCollection.getName() 273.2s 2. Agilefant.db.hibernate.UserTypeFilter.deepCopy() 213.5s 3. Agilefant.model.Team.setId() 192.3s 4. Agilefant.model.Backlog.setChildren() 123.9s 5. …… 5

Standard application profiling Inputs Application 1. Agilefant.model.WidgetCollection.getName() 273.2s 2. Agilefant.db.hibernate.UserTypeFilter.deepCopy() 213.5s 3. Agilefant.model.Team.setId() 192.3s 4. Agilefant.model.Backlog.setChildren() 123.9s 5. …… 6

Standard application profiling find(int list[] [], , int n, , int key) F int lo = = 0; hi >= = lo lo return result; int hi = = n n - 1; int result = = -1; T F resu sult == -1 T F F list[mid] == final int mid = (lo lo+hi hi) / 2; list[mid] ] > > key lo=mid + lo + 1; key T T Resu sult = = mid; hi=mid - 1; hi 7

Standard application profiling find(int list[] [], , int n, , int key) F int lo = = 0; hi >= = lo lo return result; int hi = = n n - 1; int result = = -1; T F resu sult == -1 T F F list[mid] == final int mid = (lo lo+hi hi) / 2; list[mid] ] > > key lo=mid + lo + 1; key T T Resu sult = = mid; hi=mid - 1; hi 8

Standard application profiling find(int list[] [], , int n, , int key) F int lo = = 0; hi >= = lo lo return result; int hi = = n n - 1; int result = = -1; T F resu sult == -1 T F F list[mid] == final int mid = (lo lo+hi hi) / 2; list[mid] ] > > key lo=mid + lo + 1; key T T Resu sult = = mid; hi=mid - 1; hi 9

Standard application profiling find(int list[] [], , int n, , int key) F int lo = = 0; hi >= = lo lo return result; int hi = = n n - 1; int result = = -1; T F resu sult == -1 T F F list[mid] == final int mid = (lo lo+hi hi) / 2; list[mid] ] > > key lo=mid + lo + 1; key T T Resu sult = = mid; hi=mid - 1; hi 10

11

In Input-sensitive profiling Inputs Application 1. Agilefant.model.WidgetCollection.getName() 273.2s 2. Agilefant.db.hibernate.UserTypeFilter.deepCopy() 213.5s 3. Agilefant.model.Team.setId() 192.3s 4. Agilefant.model.Backlog.setChildren() 123.9s 5. …… 12

In Input-sensitive profiling Inputs Application 1. Agilefant.model.WidgetCollection.getName() 273.2s 2. Agilefant.db.hibernate.UserTypeFilter.deepCopy() 213.5s 3. Agilefant.model.Team.setId() 192.3s 4. Agilefant.model.Backlog.setChildren() 123.9s 5. …… 13

In Input-sensitive profiling 1 Input x, y, z, u 2 v = A.m (x, y) 3 if (v > z) { 4 C.h (B.m (v)) 5 } 6 else { 7 D.h (B.m (v)) 8 } 14

In Input-sensitive profiling 1 Input x, y, z, u unimportant input value 2 v = A.m (x, y) 3 if (v > z) { 4 C.h (B.m (v)) 5 } 6 else { 7 D.h (B.m (v)) 8 } 15

In Input-sensitive profiling 1 Input x, y, z, u 2 v = A.m (x, y) 3 if (v > z) { Construct combinations of 4 C.h (B.m (v)) input values 5 } 6 else { 7 D.h (B.m (v)) 8 } 16

In Input-sensitive profiling 1 Input x, y, z, u 2 v = A.m (x, y) 3 if (v > z) { General-purpose methods 4 C.h (B.m (v)) 5 } 6 else { 7 D.h (B.m (v)) 8 } 17

In Input-sensitive profiling 1 Input x, y, z, u 2 v = A.m (x, y) 3 if (v > z) { Identify the input-sensitive 4 C.h (B.m (v)) bottlenecks 5 } 6 else { 7 D.h (B.m (v)) 8 } 18

Genetic Alg lgorithm-driven Profiler (GA-Prof) • Automate input-sensitive profiling • Explore input parameter space • Detect performance bottlenecks 19

Genetic Alg lgorithm-driven Profiler (GA-Prof) Software GA Profiler system Analyzer 20

Genetic Alg lgorithm-driven Profiler (GA-Prof) Software GA Profiler system Analyzer Ranked list: Contrast 1. Agilefant.db.hibernate.UserTypeFilter.d eepCopy() Mining 2. Agilefant.model.WidgetCollection.getN ame() 3. Agilefant.model.Backlog.setChildren() 4. Agilefant.model.Team.setId() 5. ……… 21

Genetic Alg lgorithms (GAs) • Simulate the natural selection process • Generate solutions to optimization problems 22

Genetic Alg lgorithms (GAs) • Simulate the natural selection process • Generate solutions to optimization problems 23

Why do we use GAs in in GA-Prof • Large input space • Can be formulated as a search and optimization problem 24

Why do we use GAs in in GA-Prof • Large input space • Can be formulated as a search and optimization problem • Performs better than an alternative solution 25

Why do we use GAs in in GA-Prof • Large input space • Can be formulated as a search and optimization problem • Perform better than an alternative solution 26

Why do we use GAs in in GA-Prof • Large input space • Can be formulated as a search and optimization problem • Perform better than an alternative solution 27

GA GA Component Definitions Genes: Input 1: http://localhost:8080/Agilefant/editUser.action Input 2: http://localhost:8080/Agilefant/editProduct.action?productId=5 Input 3: http://localhost:8080/Agilefant/editProduct.action?productId=8 …… A chromosome/individual Individual 1: 2, 18, 36, 27, 11, 13, 6, 43, 64, 12, 85, 49, 12, 53, 44, 78, 31, 47, 6 28

Usin ing GAs in in GA-Prof Individual 1 Individual 2 Individual 3 …… 29

Usin ing GAs in in GA-Prof Individual 1 Individual 2 Individual 3 …… Fitness function: Individual elapsed execution time 30

Usin ing GAs in in GA-Prof Individual 1 1. Individual 25 289.5s 2. Individual 17 256.7s Individual 2 3. Individual 91 197.2s …… Individual 3 …… Fitness function: Individual elapsed execution time 31

Usin ing GAs in in GA-Prof Individual 1 1. Individual 25 289.5s crossover 2. Individual 17 256.7s Individual 2 3. Individual 91 197.2s …… Individual 3 …… Parent 1: 2, 18, 36, 27, 11, 13, 6, 43, 64, 12, 85, 49, 12, 53, 44, 91, 79, 23, 3, 19 Parent 2: 23, 95, 1, 67, 35, 81, 7, 17, 51, 102, 56, 39, 72, 3, 54, 37, 13, 86, 47, 76 Child 1: 2, 18, 36, 27, 11, 13, 6, 17, 51, 102, 56, 39, 72, 3, 54, 37, 13, 86, 47, 76 Child 2: 23, 95, 1, 67, 35, 81, 7, 43, 64, 12, 85, 49, 12, 53, 44, 91, 79, 23, 3, 19 32

Usin ing GAs in in GA-Prof Individual 1 1. Individual 25 289.5s crossover 2. Individual 17 256.7s Individual 2 3. Individual 91 197.2s …… mutation Individual 3 …… Parent : 2, 18, 36, 27, 11, 13, 6, 43, 64, 12, 85, 49, 12, 53, 44, 91, 79, 23, 3, 19 Child : 2, 18, 36, 27, 11, 13, 6, 43, 64, 73, 85, 49, 12, 53, 44, 91, 79, 23, 3, 19 33

Usin ing GAs in in GA-Prof Individual 1 Individual 1’ 1. Individual 25 289.5s crossover 2. Individual 17 256.7s Individual 2 Individual 2’ 3. Individual 91 197.2s …… mutation Individual 3 Individual 3’ …… …… 34

Usin ing GAs in in GA-Prof Individual 1 Individual 1’ 1. Individual 25 289.5s crossover 2. Individual 17 256.7s Individual 2 Individual 2’ 3. Individual 91 197.2s …… mutation Individual 3 Individual 3’ …… …… Independent variables: Crossover rate, mutation rate, number of individuals per generation 35

Identify Id fying in input-sensitive bottlenecks Elapsed Execution Time 233.2 201.8 137.92 128.48 79.6 Traces 36

Identify Id fying in input-sensitive bottlenecks Good Good Elapsed Execution Time Good 233.2 201.8 Bad Bad 137.92 Bad 128.48 79.6 Traces 37

Id Identify fying in input-sensitive bottlenecks Method 1 Method 2 Method 3 Method 4 Method 5 Good Elapsed Execution Time Good Bad Bad Bad Traces 38

Id Identify fying in input-sensitive bottlenecks Method 1 Method 2 Method 3 Method 4 Method 5 Good bottleneck Elapsed Execution Time Good Bad Bad Bad Traces 39

In Independent Component Analysis (IC ICA) 40

Independent Component Analysis (IC In ICA) Mixed signals 41

Independent Component Analysis (IC In ICA) Mixed signals 42

In Independent Component Analysis (IC ICA) Feature 1 Trace 1 Mixed signals Feature 2 Trace 2 43

Independent Component Analysis (IC In ICA) methods features methods features traces traces = × X A S : p×m : p×k : k×m Good traces [S Good ] Bad traces [S Bad ] 44