Evaluating the Impact of UML Introduction Modeling on Software - PowerPoint PPT Presentation

Evaluating the Impact of UML Introduction Modeling on Software Quality Research Design Case Study An Industrial Case Study Results Conclusions Ariadi Nugroho Michel R.V. Chaudron

Modeling, is it worth it? Introduction Research Design Case Study Results Well-thought Conclusions Reduced reworks solutions Better Reduced defects comprehension Cost-saving Improved Communication Improved productivity Reduced maint. effort

But, this is often how people Introduction model... Research Design Case Study Results Conclusions UML is used diversely: style, completeness, level of detail, formality...

Does model quality matter? Introduction Research Design Exploring the effects of quality in modeling Case Study Results • Defects in UML models often remain undetected and Conclusions cause misinterpretation (Lange & Chaudron, ICSE 2006) • Higher level of detail in UML models increases comprehension performance (IST Journal, 2009) • Higher level of detail in sequence diagrams corresponds to lower defect density in software (MODELS 2008) This study focuses on the use of UML and its effect on the quality of software

In modeling a system, some parts might have Introduction been left unmodeled for various reasons... Research Design Case Study Implementation::Package A Results UML Model::Package A letterClass letterClass Conclusions do_something() do_something() bb1Class do_BB1() aClass bClass aClass bClass do_A() do_B() do_A() do_B() bb2Class do_BB2() cClass zClass do_C() do_Z() What may be the results of not modeling? • Implementation may not be well-thought of or well-designed • Implementation may deviate from requirements

Research Question Defect density Introduction 100 Research Design Case Study 75 Results Conclusions 50 25 0 Modeled Not Modeled “Do the modeled and not modeled classes have significantly different quality in the implementation?”

Design of Study Introduction Research Design THE USE OF UML RESULTING IMPLEMENTATION Case Study Confounding factors Results UML Modeling Code Code Conclusions Complexity Coupling Defect Affects? Density No Modeling Main factor Hypothesis: Modeled classes have significantly lower defect density than those that are not modeled

System under Study Introduction Project # of staffs Off-shoring Model Size Code size Size Research Design Case Study 104 UCs 20 man-years 25 people India 266 classes 152, 017 Results 341 SDs Conclusions Project characteristics • A web-based healthcare system • System modeling was done in Holland • Sixty percent of coding and testing were done in India • UML model was created using Rational XDE • Textual specifications exist

Approach Introduction Collect implementation classes, UML model, and Research Design defect data Case Study Project Repository Results Statistical tests Conclusions 55.00 41.25 component C2 Determine defect 27.50 component C2 defect requires I2 requires I2 requires I3 density of every density requires I3 uses I2.g 13.75 uses I2.g uses I3.h uses I3.h implementation class 0 Not Modeled Modeled Modeled vs Unmodeled Order OrderDetail Check whether an * 1 -date Preprocessor Parser Analyser DB Creator DB Filler Inheritance Relator DB Checker Statistic Filter Statistic Calculator -quantity -status -taxStatus SAAT.BAT +calcTax() +calcSubTotal() +calcTotal() +calcWeight() +calcTotalWeight() 1 Payment implementation class has -amount 1 been modeled in UML Credit Cash Check -number -cashTendered -name -type -bankID -expDate +authorized() +authorized()

Descriptive Statistics • The core part of the system contains 812 Java classes Introduction Research Design Defect Density Case Study Mean Modeled 0.020 Results 21% Conclusions 0.015 0.010 0.005 79% 0 Not Modeled Not Modeled Modeled Classes that were not modeled tend to have higher defect density than the modeled ones

Descriptive Statistics Modeled classes have significantly higher coupling and complexity Introduction Research Design Coupling Complexity Case Study Mean 30.0 Results Conclusions 22.5 15.0 7.5 0 Not Modeled Modeled Class coupling and complexity might confound the effect of UML modeling on defect density

Controlling for Confounding Factors Eliminating the effects of class complexity and coupling Introduction Research Design Case Study Total Variance in Defect Density Results Conclusions Variance explained Unexplained by the use of UML Variance CBO MCC

Controlling for Confounding Factors Pairwise sampling Accounting for CBO Introduction Implementation Classes (812 classes) 1 Mann-Whitney Test Research Design CBO 8 - 10 (113 classes) Case Study Results Random sampling Accounting for CBO & MCC Conclusions (80% = 96 classes) ANCOVA Test 2 Coupling Complexity Mean 45.00 33.75 22.50 11.25 0 Not Modeled (59) Modeled (37)

Testing the Hypothesis Result of Mann-Whitney Test Introduction • After controlling for the effect of class coupling Research Design Case Study Results Defect Density (Mean Rank) Conclusions 55.00 41.25 27.50 13.75 0 Not Modeled Modeled Not modeled classes have significantly higher defect density (p=0.003; 1-tailed)

Testing the Hypothesis Result of ANCOVA Test Introduction • The use of UML remains a significant factor Research Design Case Study that explains the variability of defect density in the Results implementation Conclusions Sum of Variables df F Sig. Squares The use of UML 0.002 1 6.825 0.010 Coupling 7.562E-8 1 0.000 0.987 Complexity 0.001 1 5.224 0.025

So,... Introduction Research Design The hypothesis is Defect Density (Mean Rank) Case Study confirmed: 0.020 Results Conclusions Modeled classes have 0.015 significantly lower defect density than 0.010 those that are not modeled.. 0.005 after controlling for the 0 effects of class Not Modeled Modeled complexity and coupling

Conclusions • The use of UML has a positive effect on the quality Introduction Research Design of the system (i.e., reduced defect density) Case Study • Increasing model coverage might help reduce Results Conclusions incorrect implementation of systems • Deciding which classes to model is an open issue

What’s Next? • Replication using more industrial cases Introduction • Accounting for the effects of other confounding Research Design Case Study factors Results • Performing similar studies in experimental Conclusions settings

Introduction Research Design Case Study Results Conclusions Questions?