Last Time You have a test on Tuesday Testing embedded software - PDF document

Last Time � You have a test on Tuesday � Testing embedded software � Kinds of tests � Open book, open note � When to test � No electronics: Phones, computers, calculators, etc. � How to test � Test coverage metrics � Take home messages � Run each test as early and as often as time permits � Make tests cheap to run � Tests must be aggressive and must thoroughly exercise the system • Test coverage metrics help � Integration testing of independently developed components is the worst Today Definition � Safety critical systems � A system is critical when � What they are � Correctness is more important than cost, size, weight, power usage, time-to-market, etc. � Notorious examples � Malfunction may result in unacceptable economic loss � Ways to create them � Malfunction may result in unacceptable human loss � Historical goal of software engineering: Increase developer productivity � New goal: Increase software quality Examples The Trend � Now � Humans are relinquishing direct control of more and more safety-critical systems � UAVs fire missiles at people when operators push buttons � Future � Humans are flawed (forgetful, inattentive, etc.) but basically have good judgment and can grasp the big � Autonomous fire control picture � Now � Training helps a lot � Cars have ABS, traction control, stability control, automatic parking, automatic braking during cruise control � When the human is removed from the loop there is � Electric and hybrid vehicles are highly computer-controlled no oversight � Future � Things may go badly, quickly � Automatic convoy formation � In general: Larger systems are much worse � Automatic driving? because… � They are more complicated � They can do more harm when they go wrong 1

Critical Systems Risk � Obviously not just a software problem! � Minimizing overall risk is the general strategy � Other angles that need to be considered � However: � Faulty specification � Risk is fundamental and unavoidable � Faulty hardware � Risk management is about managing tradeoffs � Human error � Risk is a matter of perspective � Malicious users • Standpoint of individual exposed to hazard � Malicious non-users • Standpoint of society – total risk to general public • Standpoint of institution responsible for the activity � Today we focus on software issues � Quantifying risk does not ensure safety Design for Safety Case Study 1: Missile Timing � Order of Precedence: � Military aircraft modified from hardware-controlled to software-controlled missile launch Design for minimum risk 1. Incorporate safety devices 2. � After design, implementation, and testing, plane was Provide warning devices 3. loaded with a live missile and flown Develop procedures and training 4. � The missile engine fired, but the missile never released from the aircraft � A “wild ride” for the test pilot � Problem: Design did not specify for how long the “holdback” should be unlocked � Programmer made an incorrect assumption � Missile was not unlocked for long enough to leave the rack � Oops! Case Study 3: Therac-25 Case Study 2: Missile Choice � Computer-controlled radiation therapy � Weapon system supporting both live and practice � Goal: Destroy tumors with minimal impact on missiles used in field practice surrounding tissue � Operator “fires” a practice missile at a real aircraft � 11 systems deployed during the 1980s during an exercise � Many new features over previous versions � Software is programmed to choose the best � Lots of software control available weapon for a given target � Increased power � Deselects the practice missile � From a report: “The software control was � Selects a live missile implemented in a DEC model PDP 11 processor � Fires using a custom executive and assembly language. A � From the report: “Fortunately the target pilot was single programmer implemented virtually all of the experienced … but the missile tracked and still software. He had an uncertain level of formal education and produced very little, if any, detonated in close proximity.” documentation on the software.” 2

More Therac-25 Therac-25 Facts � Outcome: Six massive radiation overdoses, five deaths � Hardware interlocks were replaced by software without any supporting safety analysis � Compounding the problem: Company didn’t believe software could be at fault � There was no effective reporting mechanism for field problems involving software � “Records show that software was deliberately left out of an otherwise thorough safety analysis � Software design practices (contributing to the performed in 1983 which used fault-tree methods. accidents) did not include basic, shared-data and Software was excluded because ‘software errors’ contention management mechanisms normal in have been eliminated because of extensive multi-tasking software simulation and field testing. Also, software does not � The design was unnecessarily complex for the degrade due to wear, fatigue or reproduction problem. For instance, there were more parallel process. Other types of software failures were tasks than necessary. This was a direct cause of assigned very low failure rates with no apparent some of the accidents justification.” Specific Therac-25 Bugs Two More Examples � “Equipment control task did not properly � Ariane 5 (1996) synchronize with the operator interface task, so that � Reused Ariane 4 software race conditions occurred if the operator changed the � Higher horizontal velocity of Ariane 5 caused a 16-bit setup too quickly. This was evidently missed during variable to overflow testing, since it took some practice before operators � Resulting chain of failures necessitated destroying the rocket were able to work quickly enough for the problem to � Mars Pathfinder (1997) occur.” � “…very infrequently it was possible for an interrupt to occur � “The software set a flag variable by incrementing it. that caused the (medium priority) communications task to Occasionally an arithmetic overflow occurred, be scheduled during the short interval while the (high causing the software to bypass safety checks.” priority) information bus thread was blocked waiting for the (low priority) meteorological data thread.” � Classic priority inversion � We’ll cover these (and how to avoid them) later Software V&V Fault Tree Analysis � Verification and validation – making sure that � Fault: Abnormal or undesirable system state software does the right thing � Failure: Loss of functionality � Obvious problem – figuring out what “the right thing” is � Primary failure: Failing element operating within its � V&V typically uses 40-60% of total effort specifications � Parts of the solution: � Secondary failure: Failing element operating outside � Development process of its specifications � Specification � What was the primary failure for the Challenger? � Testing � Fault tree idea: � Inspection � Connect faults and failures using logical operations in order � Fault-tree analysis to understand the consequences of faults individually and � Language choice in combination � Software techniques � Static analysis / formal verification 3

Fault Tree Example Language Choice � We looked at MISRA C � A better example is SPARK Ada � Pervasive support for static analysis • All rule violations statically detectable � No implementation-defined behavior � Tons of tool support � “SPARK code was found to have only 10% of the residual errors of full Ada and Ada was found to have only 10% of the residual errors of C” Software Safety Techniques Static Analysis / Verification � Interlocks � Goal: Rule out classes of errors across all possible executions � Important actions require signals from two or more independent sources � Potentially much faster than testing � Firewalls � Potentially more thorough than testing � Hardware / software protection boundary between critical � Stack depth analysis is one example – can “prove” that a and non-critical functionality system is invulnerable to stack overflow � Example software firewalls: � Other examples – absence of: • Address spaces � Array / pointer misuse • CPU reservations � Integer and FP overflow • Module tainting � Use of uninitialized data � Redundancy � Numerical errors � Correct result is computed by having multiple processes / � Exceptions processors vote � Deviation from specification Static Analysis Properties Example: PolySpace Verifier � In general static analysis: � Pure static analysis � Finds many trivial problems � No testcases, no execution � Does not find the most important problems � Analyzes Ada, C, C++ � Has a hard time finding the deepest problems � Anecdotal evidence suggests that Ada version is by far the � Has high computational complexity best � Even so: � Deep analysis of large C, C++ programs is still barely feasible � My opinion (and lots of other people’s) is that static analysis � Found the Ariane bug is the only way forward for creating large safety-critical systems � (After the fact) � Lots of big customers 4

Summary � Safety-critical software is extremely hard to create � Can’t be done by organizations that do not take this very seriously � No shortage of horror stories � Risk mitigation requires many different techniques � Software-based � Hardware-based � Process-based � Etc. 5