Mars Climate Orbiter Shooting Down of Airbus 320 • Purpose: to relay signals • 1988 from the Mars Polar • US Vicennes shot down Airbus 320 Lander once it reached the surface of the planet • Mistook airbus 320 for a F-14 • Disaster: smashed into • 290 people dead the planet instead of reaching a safe orbit • Why: Software bug - cryptic and • Why: Software bug - misleading output displayed by the failure to convert English tracking software measures to metric values • $165M THERAC-25 Radiation Therapy London Ambulance Service • London Ambulance Service Computer Aided Dispatch (LASCAD) • THERAC-25, a computer- controlled radiation-therapy • Purpose: automate many of the machine human-intensive processes of • 1986: two cancer patients at manual dispatch systems associated the East Texas Cancer Center with ambulance services in the UK in Tyler received fatal radiation overdoses – functions: Call taking • Why: Software bug - • Failure of the London Ambulance mishandled race condition Service on 26 and 27 November (i.e., miscoordination between 1992 concurrent tasks) 1
"Nice of You to Turn Up" More… • Load increased • "Software and its Impact: A Quantitative Assessment," by B.W. Boehm, Datamation, • Emergencies accumulated 19(5), 48-59 (1973) • System made incorrect allocations – Errors in medical software have caused – more than one ambulance being sent to the deaths same incident – the closest vehicle was not chosen for the emergency • At 23:00 on October 28 the LAS eventually instigated a backup procedure, after the death of at least 20 patients More… More… • "The development of software for ballistic-missile • “Software Engineering: Report on a Conference defense," by H. Lin, Scientific American, vol. 253, no. 6 Sponsored by the NATO Science Committee, (Dec. 1985), p. 48 Brussels, NATO Scientific Affairs Division,” – British destroyer H.M.S. Sheffield; sunk in the 1968, p. 121 Falkland Islands war; ship's radar warning system software allowed missile to reach its target – An error in an aircraft design program – An Air New Zealand airliner crashed into an Antarctic contributed to several serious air crashes mountain – North American Aerospace Defense Command reported • "Ghost in the Machine," Time Magazine, Jan. that the U.S. was under missile attack; traced to 29, 1990. p. 58 faulty computer software - generated incorrect – Dallas/Fort Worth air-traffic system began signals – Manned space capsule Gemini V missed its landing spitting out gibberish in the Fall of 1989 and point by 100 miles; software ignored the motion of controllers had to track planes on paper the earth around the sun 2
More… More… • Software Reliability: Principles & Practice, p. • An Airbus A320 crashes at an air show 25, by G. J. Myers • A China Airlines Airbus Industries A300 crashes – Apollo 8 spacecraft erased part of the on April 26, 1994 killing 264 computer's memory • Ariane 5 satellite launcher malfunction was – Eighteen errors were detected during the caused by a faulty software exception routine 10-day flight of Apollo 14 resulting from a bad 64-bit floating point to 16-bit integer conversion – An error in a single FORTRAN statement resulted in the loss of the first American probe to Venus More… More… • Internet Risks Forum NewsGroup (RISKS), vol. • ACM SIGSOFT Software Engineering Notes, 19, issue 56 vol. 6, no. 2 – CyberSitter censors "menu */ #define" – F-18 fighter plane crashed due to a missing because of the string "nu...de" exception condition • London‘s Docklands Light Railway - train • ACM SIGSOFT Software Engineering Notes, stopped in the middle of nowhere due to future vol. 9, no. 5 station location programmed in software – F-14 fighter plane was lost to uncontrollable • ACM SIGSOFT Software Engineering Notes, spin, traced to tactical software vol. 12, no. 3 – Chicago cat owners were billed $5 for unlicensed dachshunds. A database search on "DHC" (for dachshunds) found "domestic house cats" with shots but no license 3
More… • and many many more …. Economic Impact • NIST study – On CNN.com - April 27, 2003 4
Open up albums. Play song from the Browse through the albums in selected album horizontal mode. Browse again through the albums Select an album Expectation Get back to vertical mode. • Can’t we expect software to execute correctly? • Carefully made programs – 5 faults/1000 LOC – 1M LOC will have 5000 faults • Windows XP has 45M LOC – How many faults? Now see the browser get back – 45 x 5000 = 225,000 to the song that was played in • Why not remove the faults? vertical mode not the current screen that the user were browsing 5
Joke? • “If the automobile industry had developed like the software industry, we would all be driving $25 cars that get 1,000 miles to the gallon.” • “Yeah, and if cars were like software, they would crash twice a day for no reason, and when you called for service, they’d tell you to reinstall the engine.” How Cars Are Engineered How Cars Are REALLY Engineered (A Detailed View) (A Simple View) • User requirements – Engine power, all-wheel, seating, comfort, MP3 player!! • Detailed design – Blueprints, design documents • Verify design – Simulation, prototyping • Develop parts (components) – Test each component – Components may be reused – Mass produced • Assemble the car – Test the car (Front/side crash tests, Stability tests) – Usability testing (Feedback from drivers/passengers) 6
But Seriously Goals of the Course • Features of many LEGO parts • Discuss software testing techniques – Modularity • Two parts of the course – Reusability – Review testing fundamentals • Each part can be used in different places – State-of-the-art & emerging and ways techniques – Flexibility of design • What do I expect from students? – Compatibility • With other LEGO sets • Building-blocks MS and Ph.D. Qualifying Assessment • Is the course is valid for PhD qualifying coursework? • 10% each mid-term exam – Yes (Software Engineering/Programming • 5% presentation Languages) • 25% Final Exam • Is the course is valid for MS qualifying coursework? • 50% Project – Yes (Software Engineering/Programming – Project report Languages) • Is the course is valid for MS comps? – Project Presentation – Yes (Both Midterms and Final exams count towards the MS comps.) 7
Testing: Our Experiences When to Stop? Test Case Generation Test Case Test Case Software Software to be tested to be tested Verification Output Output No Test Coverage Enough? Yes A Real Testing Example Automated Testing Test Case Generation Test Cases Just a list. Test Case {1,3,2} SPECS: A sorted list. {1,2,3} Takes a list Repeated entry. of numbers; {3,2,3} Software Empty list. returns a to be tested {} Sorting Negative numbers. sorted list. Program {-1, -2} Verification Output {1, 2, 3} {1, 2, 3} {2, 3, 3} {-2, -1} {} Output Output Output Output Output Philosophy: No Test Coverage What are we Enough? trying to do? Yes 8
Automated Testing Testing the New Version Test Case Test Case Generator Original Original Test Software Software Cases to be tested Test Specs Verifier OR Output New Test Oracle Modified Test Software Cases Coverage Evaluator Regression Testing Discussion • Different Software Types – Object-oriented Original Original Test Software – Component-based Cases – Concurrent – Distributed – Graphical-user Interfaces – Web • Different goals of testing New Modified – Usability Test Software – Security Cases – Correctness – Performance … 9
Recommend
More recommend
