Performance Annotations for Complex Software Systems Daniele Rogora ∗ Antonio Carzaniga ∗ Amer Diwan $ Matthias Hauswirth ∗ Robert Soulé † ∗ USI, Switzerland † Yale University, USA $ Google, USA EuroSys’20 1 / 29
Performance Analisys is Complex! 2 / 29
Algorithmic Performance Analysis std::list<int>::sort() 3 / 29
Algorithmic Performance Analysis std::list<int>::sort() documented complexity: time = O ( n log n ) time n = List size 3 / 29
Real Performance std::list<int>::sort() 180 160 140 Time (seconds) 120 100 80 60 40 20 0 0 0.5 1 1.5 2 2.5 3 3.5 List size (million) 4 / 29
Performance Analysis with Traditional Profilers std::list<int>::sort() 5 / 29
Performance Analysis with Performance Annotations Real, expected behavior as a function of input/state features 6 / 29
Performance Analysis with Performance Annotations actual behavior concrete metrics Real, expected behavior as a function of input/state features 6 / 29
Performance Analysis with Performance Annotations actual behavior concrete metrics Real, expected behavior as a function of input/state features significant statistics 6 / 29
Performance Analysis with Performance Annotations actual behavior specific characterization concrete metrics not merely an aggregate profile Real, expected behavior as a function of input/state features significant statistics 6 / 29
Performance Analysis with Performance Annotations actual behavior specific characterization concrete metrics not merely an aggregate profile Real, expected behavior as a function of input/state features For each module/function of interest: significant statistics metric i = f i ( feature ,... ) 6 / 29
Performance Analysis with Performance Annotations actual behavior specific characterization concrete metrics not merely an aggregate profile Real, expected behavior as a function of input/state features For each module/function of interest: significant statistics metric i = f i ( feature ,... ) run-time memory allocation lock-holding time ... 6 / 29
Performance Analysis with Performance Annotations actual behavior specific characterization concrete metrics not merely an aggregate profile Real, expected behavior as a function of input/state features For each module/function of interest: significant statistics metric i = f i ( feature ,... ) input parameters, global variables, run-time memory allocation ... even in nested, structured objects lock-holding time ... identified automatically! 6 / 29
Performance Annotations std::list<int>::sort. time (this) { uint s = *(this->_M_impl._M_node._M_storage._M_storage); [s > 49584 && s < 1450341] Norm (53350.31 - 2.10*s + 0.12*s*log(s), 12463.88); [s > 1589482 && s < 2085480] 180 Norm (-90901042.29 + 63.11*s, 899547.29); 160 140 [s > 2098759 && s < 3415880] Time (seconds) 120 Norm (56712024.50 + 35.38*s, 3379580.27); 100 } 80 60 40 20 0 0 0.5 1 1.5 2 2.5 3 3.5 List size (million) 7 / 29
Performance Annotations function of interest std::list<int>::sort. time (this) { uint s = *(this->_M_impl._M_node._M_storage._M_storage); [s > 49584 && s < 1450341] Norm (53350.31 - 2.10*s + 0.12*s*log(s), 12463.88); [s > 1589482 && s < 2085480] 180 Norm (-90901042.29 + 63.11*s, 899547.29); 160 140 [s > 2098759 && s < 3415880] Time (seconds) 120 Norm (56712024.50 + 35.38*s, 3379580.27); 100 } 80 60 40 20 0 0 0.5 1 1.5 2 2.5 3 3.5 List size (million) 7 / 29
Performance Annotations function of interest metric std::list<int>::sort. time (this) { uint s = *(this->_M_impl._M_node._M_storage._M_storage); [s > 49584 && s < 1450341] Norm (53350.31 - 2.10*s + 0.12*s*log(s), 12463.88); [s > 1589482 && s < 2085480] 180 Norm (-90901042.29 + 63.11*s, 899547.29); 160 140 [s > 2098759 && s < 3415880] Time (seconds) 120 Norm (56712024.50 + 35.38*s, 3379580.27); 100 } 80 60 40 20 0 0 0.5 1 1.5 2 2.5 3 3.5 List size (million) 7 / 29
Performance Annotations function of interest metric feature: s = list size std::list<int>::sort. time (this) { uint s = *(this->_M_impl._M_node._M_storage._M_storage); [s > 49584 && s < 1450341] Norm (53350.31 - 2.10*s + 0.12*s*log(s), 12463.88); [s > 1589482 && s < 2085480] 180 Norm (-90901042.29 + 63.11*s, 899547.29); 160 140 [s > 2098759 && s < 3415880] Time (seconds) 120 Norm (56712024.50 + 35.38*s, 3379580.27); 100 } 80 60 40 20 0 0 0.5 1 1.5 2 2.5 3 3.5 List size (million) 7 / 29
Performance Annotations function of interest metric feature: s = list size std::list<int>::sort. time (this) { uint s = *(this->_M_impl._M_node._M_storage._M_storage); scope (1) [s > 49584 && s < 1450341] Norm (53350.31 - 2.10*s + 0.12*s*log(s), 12463.88); [s > 1589482 && s < 2085480] 180 Norm (-90901042.29 + 63.11*s, 899547.29); 160 140 [s > 2098759 && s < 3415880] Time (seconds) 120 Norm (56712024.50 + 35.38*s, 3379580.27); 100 } 80 60 40 20 0 0 0.5 1 1.5 2 2.5 3 3.5 List size (million) 7 / 29
Performance Annotations function of interest metric feature: s = list size std::list<int>::sort. time (this) { uint s = *(this->_M_impl._M_node._M_storage._M_storage); scope (1) [s > 49584 && s < 1450341] Norm (53350.31 - 2.10*s + 0.12*s*log(s), 12463.88); scope (2) [s > 1589482 && s < 2085480] 180 Norm (-90901042.29 + 63.11*s, 899547.29); 160 140 [s > 2098759 && s < 3415880] Time (seconds) 120 Norm (56712024.50 + 35.38*s, 3379580.27); 100 } 80 60 40 20 0 0 0.5 1 1.5 2 2.5 3 3.5 List size (million) 7 / 29
Performance Annotations function of interest metric feature: s = list size std::list<int>::sort. time (this) { uint s = *(this->_M_impl._M_node._M_storage._M_storage); scope (1) [s > 49584 && s < 1450341] Norm (53350.31 - 2.10*s + 0.12*s*log(s), 12463.88); scope (2) [s > 1589482 && s < 2085480] 180 Norm (-90901042.29 + 63.11*s, 899547.29); 160 140 scope (3) [s > 2098759 && s < 3415880] Time (seconds) 120 Norm (56712024.50 + 35.38*s, 3379580.27); 100 } 80 60 40 20 0 0 0.5 1 1.5 2 2.5 3 3.5 List size (million) 7 / 29
Automatic Feature Discovery 10 9 8 7 get_func_mm_tree(RANGE_OPT_PARAM *param, Time (s) 6 Item *pred, 5 Item_func *cond_func, 4 3 Item *val, 2 bool inv); 1 0 2000 3000 4000 5000 cond_func->arg_count get_func_mm_tree. time (cond_func) { uint ac = cond_func->arg_count; Norm (156569 - 269.041*ac + 0.414447*ac^2, 15781.22); } 8 / 29
Automatic Feature Discovery 10 9 8 7 get_func_mm_tree(RANGE_OPT_PARAM *param, Time (s) 6 Item *pred, 5 Item_func *cond_func, 4 3 Item *val, 2 bool inv); 1 0 item_func.h alone is 3885 lines! 2000 3000 4000 5000 cond_func->arg_count get_func_mm_tree. time (cond_func) { uint ac = cond_func->arg_count; Norm (156569 - 269.041*ac + 0.414447*ac^2, 15781.22); } 8 / 29
Automatic Feature Discovery 160 140 120 Time (ms) 100 80 mysql_execute_command(THD *thd, 60 bool first_level); 40 dvv=12 20 dvv=0 0 4000 8000 12000 16000 thd.m_query_length mysql_execute_command.time(thd) { uint len = thd->m_query_string.len; uint dvv = thd->variables.dynamic_variable_version; Norm(168.65 + 4.94*len + 1886.87*dvv, 2489.04); } 9 / 29
Automatic Feature Discovery 160 140 120 Time (ms) 100 80 mysql_execute_command(THD *thd, 60 bool first_level); 40 dvv=12 20 dvv=0 0 4000 8000 12000 16000 thd.m_query_length struct traversal unexpected feature! mysql_execute_command.time(thd) { uint len = thd->m_query_string.len; uint dvv = thd->variables.dynamic_variable_version; Norm(168.65 + 4.94*len + 1886.87*dvv, 2489.04); } 9 / 29
Uses of Performance Annotations Documentation ◮ automatic creation ◮ readable annotations and graphs for performance analyst ◮ feature names as in the program Annotations as performance assertions ◮ detecting performance anomalies and regressions Prediction ◮ extrapolation to unobserved feature values ◮ annotation composition: new code that uses annotated functions 10 / 29
Freud our prototype for C/C++ 11 / 29
Program Function Name Binary Freud Workload our prototype for C/C++ Graph Text Annotation Annotation 11 / 29
Freud function name binary program workload annotations 12 / 29
Freud DWARF function • Static analysis name • Feature discovery • Feature-extraction code binary program workload annotations 12 / 29
Freud DWARF function • Static analysis name • Feature discovery • Feature-extraction code CODE INFO binary program workload annotations 12 / 29
Freud DWARF function • Static analysis name • Feature discovery • Feature-extraction code CODE PIN INFO • Dyn. instrumentation with Pin binary • PinTool using code from DWARF program workload annotations
Freud DWARF function • Static analysis name • Feature discovery • Feature-extraction code CODE PIN INFO • Dyn. instrumentation with Pin binary • PinTool using code from DWARF program • Run instrumented program workload annotations 12 / 29
Recommend
More recommend
