To Engineer is Human Henry Petroski “it is the essence of modern engineering not Metrics only to be able to check one's own work, but also to have one's work checked and to be Software Engineering & Technology able to check the work of others. In order for this to be done, the work must follow Tom Verhoeff certain conventions, conform to certain standards, and be an understandable piece of technical communication. ” GQM – TQM Goals Goals Quality control (objective, quantitative) Questions Project planning & reporting Metrics Accountability (e.g. in case of an audit) Process improvement Total Quality Management Measuring is not a goal in itself
Questions Steps Select metrics (also consider: validation, What is the quality of the system? calibration, tuning, staff training) How much extra (time, staff) will it take? Collect and store measurement data What is the branch coverage of the tests? Report, consolidate (graphs, trend charts, …) How complex is the system? Interpret (w.r.t. goals) How much time was spent on defect removal? React (close the feedback loop) Decisions What to measure What actually to measure? What measurement to use? Alternatives, Product related trade-offs Process related How to measure? (measurement process) What to do with measurement results?
Fundamental Metrics Size vs Complexity LOC, SLOC Size (LOC, FP, ABC, …) percentage of lines with comments Cost (Euro) # classes; (public) methods | inst. var. / class Duration (months) # parameters | LOC / method Effort (person-months) depth of inheritance hierarchy Quality (of product: number of defects) # overridden methods Example Cyclomatic Complexity Start p, q := 0, N p, q = 0, N # given A[0..N) Yes No p = q McCabe 1976 while p <> q : Yes No if A[p] : p = q A[p] Testability, understandability (maintainability) else : p = p + 1 p := q p := p + 1 # linearly independent paths in flow graph if p == N : print ’Not found’ # edges – # vertices + 2 * # components else : print ’Found at’, p Yes No p = N # binary decisions + 1 'Not 'Found LOC = 8; SLOC = 6 found' at', p A, B, C = 4, 2, 3 Typical upper bound per module: 10 Cyclomatic complexity = 4 Stop
Tools Benchmark Point of reference, specifically for program Configuration management tools or processor performance Defect/issue trackers SPEC: Standard Performance Evaluation Test tools (measure coverage) Corporation <www.spec.org> Static code analyzers Dhrystone -> SPEC CINT2000 (no fl.pt.) CCCC = C/C++ Code Counter Whetstone -> SPEC CFP2000 CMM: Capability Estimation Maturity Model Point versus range estimate; probability Evaluate & measure maturity of sw Over-estimation versus under-estimation development process in organizations Accuracy versus precision Provide guidance for improving the sw development process Estimation error Not a sw development process in itself cone of uncertainty Successor: CMMI = CMM Integration (Dis)economy of scale
CMM Maturity Levels Literature Jerry Fitzpatrick. Applying the ABC Metric to Level 1: Initial C, C++, and Java. C++ Report, June 1997. Level 2: Repeatable Stephen H. Kan. Metrics and Models in Software Quality Engineering (2nd Ed.). Level 3: Defined Addison-Wesley, 2003. Level 4: Managed Steve McConnell. Software Estimation: Demystifying the Black Art. Microsoft Press, Level 5: Optimizing 2006.
Recommend
More recommend
