A Family of Test Adequacy Criteria for Database-Driven Applications - PowerPoint PPT Presentation

A Family of Test Adequacy Criteria for Database-Driven Applications Gregory M. Kapfhammer Department of Computer Science Mary Lou Soffa University of Pittsburgh Department of Computer Science University of Pittsburgh Department of Computer Science Allegheny College D atabase dr I ven A pplication T esting t O ol M odule S , ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 1/15

Motivation The Risks Digest, Volume 22, Issue 64, 2003 Jeppesen reports airspace boundary problems About 350 airspace boundaries contained in Jeppesen NavData are incorrect, the FAA has warned. The error occurred at Jeppesen after a software upgrade when information was pulled from a database containing 20,000 airspace boundaries worldwide for the March NavData update, which takes effect March 20. D atabase dr I ven A pplication T esting t O ol M odule S , ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 2/15

Testing Challenges Should consider the environment in which software applications execute Must test a program and its interaction with a database Database-driven application’s state space is well-structured, but infinite (Chays et al.) Need to show program does not violate database integrity, where integrity = consistency + validity (Motro) Must locate program and database coupling points that vary in granularity D atabase dr I ven A pplication T esting t O ol M odule S , ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 3/15

Database-Driven Applications D 1 P D n Database Level A program can interact with a database at different levels of granularity D atabase dr I ven A pplication T esting t O ol M odule S , ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15

Database-Driven Applications D 1 UserInfo P user_name card_number pin_number acct_lock D 1 32142 Brian Zorman n 0 2 41601 Robert Roos 0 3 45322 Marcus Bittman 0 4 0 56471 Geoffrey Arnold Relation Level D atabase dr I ven A pplication T esting t O ol M odule S , ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15

Database-Driven Applications D 1 UserInfo P user_name card_number pin_number acct_lock D 1 32142 Brian Zorman n 0 2 41601 Robert Roos 0 3 45322 Marcus Bittman 0 4 0 56471 Geoffrey Arnold Record Level D atabase dr I ven A pplication T esting t O ol M odule S , ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15

Database-Driven Applications D 1 UserInfo P user_name card_number pin_number acct_lock D 1 32142 Brian Zorman n 0 2 41601 Robert Roos 0 3 45322 Marcus Bittman 0 4 Geoffrey Arnold 0 56471 Attribute Level D atabase dr I ven A pplication T esting t O ol M odule S , ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15

Database-Driven Applications D 1 UserInfo P user_name card_number pin_number acct_lock D 1 32142 Brian Zorman n 0 2 41601 Robert Roos 0 3 45322 Marcus Bittman 0 4 0 56471 Geoffrey Arnold Attribute Value Level D atabase dr I ven A pplication T esting t O ol M odule S , ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15

Test Adequacy Criteria P violates a database D i ’s validity when it: (1-v) inserts entities into D i that do not reflect real world P violates a database D i ’s completeness when it: (1-c) deletes entities from D i that still reflect real world In order to verify (1-v) and (1-c) , T must cause P to define and then use entities within D 1 , . . . , D n ! D atabase dr I ven A pplication T esting t O ol M odule S , ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 5/15

Data Flow Information Interaction point: ‘‘UPDATE UserInfo SET acct lock=1’’ + ‘‘WHERE card number=’’ + c n + ‘‘;’’; Database Level: define(BankDB) Attribute Level: define(acct_lock) and use(card_number) Data fl ow information varies with respect to the granularity of the database interaction D atabase dr I ven A pplication T esting t O ol M odule S , ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 6/15

Database Entities UserInfo user_name card_number pin_number acct_lock 1 32142 Brian Zorman 0 2 41601 0 Robert Roos 3 45322 Marcus Bittman 0 4 56471 Geoffrey Arnold 0 A (I ) = { } , , . . . , , 1 32142 Geoffrey Arnold 0 v r D atabase dr I ven A pplication T esting t O ol M odule S , ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 7/15

The DICFG: A Unified Representation entry lockAccount “Database-enhanced” temp1 = parameter0:c_n CFG for lockAccount temp2 = LocalDatabaseEntity0:Bank Define temporaries to temp3 = LocalDatabaseEntity1:acct_lock represent the program’s interaction temp4 = LocalDatabaseEntity2:card_number at the levels of database and attribute D atabase dr I ven A pplication T esting t O ol M odule S , ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 8/15

The DICFG: A Unified Representation qu_lck = "UPDATE UserInfo ..." + temp1 + ";" Database interaction update_lock = m_connect.createStatement() graphs (DIGs) are D A entry entry G G r 1 r 2 placed before the define(temp2) define(temp3) database interaction use(temp4) exit G r 1 point I r exit G r 2 I r Multiple DIGs can be result_lock = update_lock.executeUpdate(qu_lck) integrated into a single if( result_lock == 1) CFG completed = true exit lockAccount D atabase dr I ven A pplication T esting t O ol M odule S , ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 8/15

Test Adequacy Criteria Database interaction association (DIA) involves the all−attribute−value−DUs def and use of a database entity all−record−DUs all−attribute−DUs DIAs can be located in the DICFG with data flow analysis all-database-DUs requires all−relation−DUs all−database−DUs tests to exercise all DIAs for all of the accessed databases D atabase dr I ven A pplication T esting t O ol M odule S , ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 9/15

Generating Test Requirements Relational Schema Measured time and space overhead Database Seeder Database when computing family of test adequacy criteria System Under Test Test Adequacy Criterion Modifi ed ATM and mp3cd to contain (C) (P) appropriate database interaction points Soot 1.2.5 to calculate intraprocedural Test Case Specification associations GNU/Linux workstation with kernel Test Requirements 2.4.18-smp and dual 1 GHz Pentium III Xeon processors D atabase dr I ven A pplication T esting t O ol M odule S , ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 10/15

Counting Associations and Definitions D R c R l A A v Assoc & Def Count 1750 1500 ATM DB mp3cd DB 1250 ATM HD 1000 mp3cd HD 750 500 250 0 D R c R l A A v Database Granularity DIAs at attribute value level represent 16.8 % of mp3cd ’s and 9.6 % of ATM ’s total number of intraprocedural associations – p. 11/15

Measuring Time Overhead Time Overhead None D R c R l A A v System Time � sec � 37.5 ATM 35 32.5 mp3cd 30 27.5 25 22.5 None D R c R l A A v Database Granularity Computing DIAs at the attribute value level incurs no more than a 5 second time overhead – p. 12/15

Measuring Average Space Overhead None D R c R l A A v 30 Node & Edge Count 28 � N atm � 26 � N mp3 � � E atm � 24 � E mp3 � 22 20 18 16 None D R c R l A A v Database Granularity mp3cd shows a more marked increase in the average number of nodes and edges than ATM – p. 13/15

Measuring Maximum Space Overhead None D R c R l A A v Node & Edge Count 1400 � N atm � 1200 � N mp3 � 1000 � E atm � � E mp3 � 800 600 400 200 None D R c R l A A v Database Granularity mp3cd shows a signifi cantly greater maximum space overhead than ATM – p. 14/15

Conclusions Must test the program’s interaction with the database Unique family of test adequacy criteria to detect type (1) violations of database validity and completeness Intraprocedural database interactions can be computed from a DICFG with minimal time and space overhead High number of hanging defi nitions indicates that the scope of data fl ow analysis could be broadened This data fl ow-based test adequacy criteria can serve as the foundation for algorithms that measure test suite quality, automatically generate test cases, and support regression testing http://cs.allegheny.edu/˜gkapfham/research/diatoms/ – p. 15/15