RESEARCH TOPICS Metamorphic testing OVERVIEW Fuzz testing - PowerPoint PPT Presentation

3/31/17 TOPICS 2 RESEARCH TOPICS • Metamorphic testing OVERVIEW • Fuzz testing • Regression testing: selection, prioritization • T est input generation CS580A5 SPRING 2017 • Fault localization SUDIPTO GHOSH • Automatic program repair • UI T esting METAMORPHIC TESTING FUZZ TESTING 3 4 • When expected outputs of the program under test are not known, how do you define • Often applications crash when given unexpected input the oracle? • Editors crash when files are corrupt • Define metamorphic properties: • Hackers can hack into a system by sending inputs that are unexpected • For two inputs i 1 and i 2 , the outputs are o 1 and o 2 • How do we verify whether our applications are resilient to such problems/attacks? • if i 1 and i 2 are related in a certain way, then o 1 and o 2 must also be related in a certain way (not the same way as the inputs though) 1

3/31/17 REGRESSION TESTING: TEST INPUT GENERATION 5 6 SELECTION, PRIORITIZATION • Programs evolve: new features are added, faults are fixed and so on • How to automatically generate test inputs? • How do we ensure that existing functionality isn’t broken? • How about expected outputs? • Run existing tests that are still valid in the new context • Categories of techniques • What if we don’t have enough resources to run every test? • Random generation • We need some way to select a subset of test cases • Exhaustive generation • Based on symbolic execution • Safety property: Every old test that exercises modified code must be executed • Precision (don’t want to run more test cases than needed to achieve safety) • Combination of concrete and symbolic execution (aka concolic) • Other objectives: ensure same level of code coverage as before, fault detection, etc FAULT LOCALIZATION AUTOMATIC PROGRAM REPAIR 7 8 • Debugging consists of finding a fault and fixing it. • Solves the second part of debugging (finding a fix/patch) • Typically manual, and lots of time spent • Study general approach framework • Study automated techniques that help narrow down the possibilities • Specific approach called GenProg • Slicing • Spectrum based approaches 2

3/31/17 UI TESTING 9 • Challenges specific to UI testing • Various testing goals • Automation (capture and replay) • Underlying principles • Guest lecture from industry professional 3