Testing Database-Driven Applications: Challenges and Solutions - PowerPoint PPT Presentation

Testing Database-Driven Applications: Challenges and Solutions Gregory M. Kapfhammer Department of Computer Science Mary Lou Soffa University of Pittsburgh Department of Computer Science University of Pittsburgh Department of Computer Science Allegheny College D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 1/32

Outline Introduction and Motivation Testing Challenges Database-Driven Applications A Unified Representation Test Adequacy Criteria Test Suite Execution Test Coverage Monitoring Conclusions and Resources D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 2/32

Motivation The Risks Digest, Volume 22, Issue 64, 2003 Jeppesen reports airspace boundary problems About 350 airspace boundaries contained in Jeppesen NavData are incorrect, the FAA has warned. The error occurred at Jeppesen after a software upgrade when information was pulled from a database containing 20,000 airspace boundaries worldwide for the March NavData update, which takes effect March 20. D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 3/32

Testing Challenges Should consider the environment in which software applications execute Must test a program and its interaction with a database Database-driven application’s state space is well-structured, but infinite (Chays et al.) Need to show program does not violate database integrity, where integrity = consistency + validity (Motro) Must locate program and database coupling points that vary in granularity D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 4/32

Testing Challenges The structured query language’s (SQL) data manipulation language (DML) and data definition language (DDL) have different interaction characteristics Database state changes cause modifications to the program representation Different kinds of test suites require different techniques for managing database state during testing D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 5/32

Testing Challenges The many testing challenges include, but are not limited to, the following: Unified program representation Family of test adequacy criteria Efficient test coverage monitoring techinques Intelligent approaches to test suite execution D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 6/32

Database-Driven Applications P R 1 A C D B m i R 2 D k E F G H R 3 m j K I J L D l Program P interacts with two relational databases D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 7/32

Database Interaction Levels D 1 P D n Database Level A program can interact with a database at different levels of granularity D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 8/32

Database Interaction Levels D 1 UserInfo P user_name card_number pin_number acct_lock D n 1 32142 Brian Zorman 0 2 41601 Robert Roos 0 3 45322 Marcus Bittman 0 0 4 56471 Geoffrey Arnold Relation Level D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 8/32

Database Interaction Levels D 1 UserInfo P user_name card_number pin_number acct_lock D 1 n 32142 Brian Zorman 0 2 41601 Robert Roos 0 3 45322 Marcus Bittman 0 0 4 56471 Geoffrey Arnold Record Level D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 8/32

Database Interaction Levels D 1 UserInfo P user_name card_number pin_number acct_lock D n 1 32142 Brian Zorman 0 2 41601 Robert Roos 0 3 45322 Marcus Bittman 0 0 4 Geoffrey Arnold 56471 Attribute Level D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 8/32

Database Interaction Levels D 1 UserInfo P user_name card_number pin_number acct_lock D 1 n 32142 Brian Zorman 0 2 41601 Robert Roos 0 3 45322 Marcus Bittman 0 0 4 56471 Geoffrey Arnold Attribute Value Level D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 8/32

Database Interaction Points Database interaction point I r ∈ I corresponds to the execution of a SQL DML statement Consider a simplified version of SQL and ignore SQL DDL statements (for the moment) Interaction points are normally encoded within Java programs as dynamically constructed String s select uses D k , delete defines D k , insert defines D k , update defines and/or uses D k D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 9/32

Database Interaction Points (DML) delete from r select A 1 , A 2 , . . . , A q where Q from r 1 , r 2 , . . . , r m where Q update r insert into r ( A 1 , A 2 , . . . , A q ) set A l = F ( A l ) values ( v 1 , v 2 , . . . , v q ) where Q D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 10/32

Refined Database-Driven Application * from R select 1 where A < ( avg(G) select from ) R 2 P R 1 A B C D m i R 2 D k F E G H R 3 m j I J K L D l R 3 update set J = 500 L < 1000 where D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 11/32

Test Adequacy Criteria P violates a database D k ’s validity when it: (1-v) inserts entities into D k that do not reflect real world P violates a database D k ’s completeness when it: (1-c) deletes entities from D k that still reflect real world In order to verify (1-v) and (1-c) , T must cause P to define and then use entities within D 1 , . . . , D n ! D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 12/32

Data Flow Information Interaction point: ‘‘UPDATE UserInfo SET acct lock=1 WHERE card number=’’ + card number + ‘‘;’’; Database Level: define(BankDB) Attribute Level: define(acct_lock) and use(card_number) Data fl ow information varies with respect to the granularity of the database interaction D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 13/32

Database Entities UserInfo user_name card_number pin_number acct_lock 1 32142 Brian Zorman 0 2 41601 0 Robert Roos 3 45322 Marcus Bittman 0 4 56471 Geoffrey Arnold 0 A (I ) = { } , , . . . , , 1 32142 Geoffrey Arnold 0 v r D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 14/32

The DICFG: A Unified Representation entry lockAccount “Database-enhanced” temp1 = parameter0:c_n CFG for lockAccount temp2 = LocalDatabaseEntity0:Bank Define temporaries to temp3 = LocalDatabaseEntity1:acct_lock represent the program’s interaction temp4 = LocalDatabaseEntity2:card_number at the levels of database and attribute D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 15/32

The DICFG: A Unified Representation Database interaction qu_lck = "UPDATE UserInfo ..." + temp1 + ";" graphs (DIGs) are placed before interaction update_lock = m_connect.createStatement() D A entry entry G point I r G r 1 r 2 define(temp2) define(temp3) Multiple DIGs can be use(temp4) exit G integrated into a single r 1 exit G CFG r 2 I r result_lock = update_lock.executeUpdate(qu_lck) String at I r is if( result_lock == 1) determined in a completed = true control-fl ow sensitive exit lockAccount fashion D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 15/32

Test Adequacy Criteria Database interaction association (DIA) involves the all−attribute−value−DUs def and use of a database entity all−record−DUs all−attribute−DUs DIAs can be located in the DICFG with data flow analysis all-database-DUs requires all−relation−DUs all−database−DUs tests to exercise all DIAs for all of the accessed databases D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 16/32

Generating Test Requirements Relational Schema Measured time and space overhead Database Seeder Database when computing family of test adequacy criteria System Under Test Test Adequacy Criterion Modifi ed ATM and mp3cd to contain (C) (P) appropriate database interaction points Soot 1.2.5 to calculate intraprocedural Test Case Specification associations GNU/Linux workstation with kernel Test Requirements 2.4.18-smp and dual 1 GHz Pentium III Xeon processors D atabase dr I ven A pplication T esting t O ol M odule S , IBM T.J. Watson Research Center, May 14, 2004 – p. 17/32

Counting Associations and Definitions D R c R l A A v Assoc & Def Count 1750 1500 ATM DB mp3cd DB 1250 ATM HD 1000 mp3cd HD 750 500 250 0 D R c R l A A v Database Granularity DIAs at attribute value level represent 16.8 % of mp3cd ’s and 9.6 % of ATM ’s total number of intraprocedural associations – p. 18/32