DEFECT DETECTION IN A DEFECT DETECTION IN A DISTRIBUTED SOFTWARE - PowerPoint PPT Presentation

DEFECT DETECTION IN A DEFECT DETECTION IN A DISTRIBUTED SOFTWARE DISTRIBUTED SOFTWARE MAINTENANCE PROJECT MAINTENANCE PROJECT Alessandro Bianchi, Danilo Caivano, Filippo Lanubile, Giuseppe Visaggio SER_Lab - Department of Informatics - University of Bari {bianchi, caivano, lanubile, visaggio}@di.uniba.it DIB Defect Detection in a Distributed Software Maintenance Project1

Case Study Case Study � Post mortem analysis on a maintenance project carried out in EDS Italia � Massive maintenance � of a large banking software system � to solve the Y2K problem DIB Defect Detection in a Distributed Software Maintenance Project2

The System The System Banking Software System WP 1, 1 Item 1, 1, 1 FA 1 Item 1, 1, 2 WP 1, 2 FA 2 … … Item 1, 1, m FA 3 WP 1, n FA 4 DIB Defect Detection in a Distributed Software Maintenance Project3

The Maintenance Process … The Maintenance Process … Project Management Configuration Management Change Verification Review & Validation Test SQA DIB Defect Detection in a Distributed Software Maintenance Project4

… The Maintenance Process … The Maintenance Process � Process execution started on Site 1 for all WPs � Depending on rework needs and currently available resources, Change and V&V phases were switched for some WPs to Site 2 � Both sites were settled in Italy � The Collocated project includes WPs entirely executed at Site 1 � The Distributed project includes WPs executed at both Site 1 and Site 2 DIB Defect Detection in a Distributed Software Maintenance Project5

Previous Results* Previous Results* � There are not statistically significant differences between collocated and distributed projects for � Duration � Effort � Staff � Reworking cycles � There are statistically significant differences between collocated and distributed projects for � Number of reports � Number of meetings * A. Bianchi, D. Caivano, F. Lanubile, F. Rago, G. Visaggio, “An Empirical Study of Distributed Software Maintenance”, Proc. of the IEEE Intl. Conf on Sw Maint. , 2002 DIB Defect Detection in a Distributed Software Maintenance Project6

Further Analysis: Defect Metrics Further Analysis: Defect Metrics � Research Question: Does the distribution among sites affect defect metrics? � Therefore, for each defect metric M i the following are posed � H i0 : There is no difference between the values of defect metric M i for collocated WPs and for distributed WPs � H ia : There is a difference between the values of defect metric M i for collocated WPs and for distributed WPs DIB Defect Detection in a Distributed Software Maintenance Project7

Observed metrics Observed metrics � # executed test cases & # of faults that caused failures ( faults from testing ) � # reviews & # of found defects ( faults from review ) � # audits & # of found issues ( non conformities from audits ) � WPs Size (# items) DIB Defect Detection in a Distributed Software Maintenance Project8

Results … Results … Box Plot (defects2.sta 27v*52c) Box Plot (defects2.sta 27v*52c) 4,0 1,2 3,5 1,0 Test Cases Reviews: 3,0 0,8 N. Review per Item Test Cases per Item p -level=0.633 p -level=0.359 2,5 0,6 2,0 0,4 1,5 0,2 1,0 0,0 Median Median 25%-75% 25%-75% Non-Outlier Range 0,5 -0,2 Non-Outlier Range Outliers Distributed Collocated Distributed Collocated Outliers Extremes Project Project Box Plot (defects2.sta 27v*52c) 0,9 0,8 0,7 Audits: 0,6 N. Audit per Item 0,5 p -level=0.453 0,4 0,3 0,2 0,1 Median 0,0 25%-75% Non-Outlier Range -0,1 Outliers Distributed Collocated Extremes Project V&V activities are comparable DIB Defect Detection in a Distributed Software Maintenance Project9

…Results …Results Box Plot (defects2.sta 27v*52c) Box Plot (defects2.sta 27v*52c) 0,08 0,45 0,40 0,07 0,35 0,06 Faults from Faults from 0,30 Faults from Review per Item Faults from Testing per Item 0,05 0,25 Testing: Review : 0,04 0,20 0,03 p -level=0.489 p -level=0.212 0,15 0,02 0,10 0,01 0,05 0,00 Median Median 0,00 25%-75% 25%-75% Non-Outlier Range Non-Outlier Range -0,01 -0,05 Outliers Outliers Distributed Collocated Distributed Collocated Extremes Extremes Project Project Box Plot (defects2.sta 27v*52c) 0,12 0,10 Non-conformities: 0,08 N. NCN per Item p -level=0.633 0,06 0,04 0,02 0,00 Median 25%-75% Non-Outlier Range -0,02 Outliers Distributed Collocated Extremes Project A significant difference DOES NOT exist DIB Defect Detection in a Distributed Software Maintenance Project10

Hypotheses for Lack of Differences Hypotheses for Lack of Differences � The specific project management � The tasks are independent of each other � They can be executed concurrently � The application domain is well-known by both sites � Homogeneity of behavior of sites � because both belonging to the same company, certified CMM 3 DIB Defect Detection in a Distributed Software Maintenance Project11

Lessons Learned Lessons Learned � Need of an adequate management of: � strategic issues � cultural issues � technical issues to make effective distribution of software process � This allows to � execute independent tasks � exploit proper skills wherever they are DIB Defect Detection in a Distributed Software Maintenance Project12