A Test Adequacy Infrastructure with Database Interaction Awareness - PowerPoint PPT Presentation

A Test Adequacy Infrastructure with Database Interaction Awareness Gregory M. Kapfhammer Department of Computer Science Allegheny College (in collaboration with Mary Lou Soffa) A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 1/35

Motivation The Risks Digest, Volume 22, Issue 64, 2003 Jeppesen reports airspace boundary problems About 350 airspace boundaries contained in Jeppesen NavData are incorrect, the FAA has warned. The error occurred at Jeppe- sen after a software upgrade when information was pulled from a database containing 20,000 airspace boundaries worldwide for the March NavData update, which takes effect March 20. A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 2/35

Looking Ahead Test adequacy infrastructure that can find faults and establish confidence in the correctness of a database-centric application Model of database interaction faults Unifed application representation Family of test adequacy criteria Experiments with real applications that measure the number of test requirements and the time and space overheads incurred by enumeration Foundation for a comprehensive methodology for testing database-centric applications A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 3/35

Testing Challenges Must consider the environment in which software applications actually execute Should test a program and its interaction with a database Database-centric application’s state space is well-structured, but essentially infinite (Chays et al.) Need to show program does not violate database integrity, where integrity = consistency + validity (Motro) Must locate program and database coupling points that vary in granularity A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 4/35

Database-Centric Applications P R 1 A C D B m i R 2 D k E F G H R 3 m j K I J L D l Program P interacts with two relational databases D k and D l A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 5/35

Database Interactions P m insert select update delete D D 1 n Program P can view and/or modify the state of the database A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 6/35

Database Interaction Faults: (1-v) P uses update or m P insert to incorrectly modify items within database insert update Commission fault that violates expected database validity Structural adequacy before after criteria can support actual fault isolation A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 7/35

Database Interaction Faults: (1-c) P uses delete to m P remove incorrect items from database delete Commission fault that violates database completeness expected Structural adequacy criteria can support before after fault isolation actual A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 8/35

Database Interaction Faults: (2-v) P does not submit m P delete to remove items from database Commission or delete omission fault that violates database validity expected Structural adequacy criteria cannot easily before after support omission fault actual isolation A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 9/35

Database Interaction Faults: (2-c) P does not submit m P update or insert to database Commission or insert update omission fault that violates database completeness expected Structural adequacy criteria cannot easily before after support omission fault actual isolation A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 10/35

Database Interaction Levels D 1 P D n Database Level A program can interact with a relational database at different levels of granularity A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 11/35

Database Interaction Levels D 1 UserInfo P card_number pin_number user_name acct_lock D n 32142 1 Brian Zorman 0 41601 2 Robert Roos 0 45322 3 Marcus Bittman 0 4 0 56471 Geoffrey Arnold Relation Level A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 11/35

Database Interaction Levels D 1 UserInfo P card_number pin_number user_name acct_lock D n 1 32142 0 Brian Zorman 2 41601 Robert Roos 0 3 45322 Marcus Bittman 0 0 4 56471 Geoffrey Arnold Record Level A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 11/35

Database Interaction Levels D 1 UserInfo P card_number pin_number user_name acct_lock D n 1 32142 Brian Zorman 0 2 41601 Robert Roos 0 3 45322 Marcus Bittman 0 0 4 56471 Geoffrey Arnold Attribute Level A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 11/35

Database Interaction Levels D 1 UserInfo P card_number pin_number user_name acct_lock D n 1 32142 Brian Zorman 0 2 41601 Robert Roos 0 3 45322 Marcus Bittman 0 0 4 56471 Geoffrey Arnold Attribute Value Level A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 11/35

Database Interaction Points: DML delete from r select A 1 , A 2 , . . . , A q where Q from r 1 , r 2 , . . . , r m where Q update r insert into r ( A 1 , A 2 , . . . , A q ) set A l = F ( A l ) values ( v 1 , v 2 , . . . , v q ) where Q A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 12/35

Analyzing Database Interaction Points Database interaction point I r ∈ I corresponds to the execution of a SQL DML statement Consider the relevant portions of SQL that are parsed by HSQLDB ( http : // hsqldb . sf . net ) Interaction points are normally encoded within Java programs as dynamically constructed String s select uses D k , delete defines D k , insert defines D k , update defines and/or uses D k A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 13/35

Refined Database-Centric Application * from R select 1 where A < ( avg(G) select from ) R 2 P R 1 A B C D m i R 2 D k F E G H R 3 m j I J K L D l R 3 update set J = 500 L < 1000 where A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 14/35

Test Adequacy Concepts P violates a database D k ’s validity when it: (1-v) inserts entities into D k that do not reflect real world P violates a database D k ’s completeness when it: (1-c) deletes entities from D k that still reflect real world In order to verify (1-v) and (1-c) , T must cause P to define and then use entities within D 1 , . . . , D n ! A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 15/35

Data Flow Information Interaction point: “ UPDATE UserInfo SET acct _ lock = 1 WHERE card _ number = ” + card _ number + “ ; ”; Database Level: define(BankDB) Attribute Level: define(acct_lock) and use(card_number) Data fl ow information varies with respect to the granularity of the database interaction A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 16/35

Database Entities UserInfo card_number pin_number user_name acct_lock 1 32142 Brian Zorman 0 2 41601 0 Robert Roos 3 45322 Marcus Bittman 0 4 56471 Geoffrey Arnold 0 A (I ) = { } , , . . . , , 1 32142 Geoffrey Arnold 0 v r Enumerate database entities at the attribute value level A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 17/35

Application Types Database−Centric Applications Interaction Approach Program Location Embedded Interface Inside DBMS Outside DBMS Testing methodology relevant to all types of applications Current tool support focuses on Interface-Outside applications Example: Java application that submits SQL String s to HSQLDB relational database using JDBC drivers A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 18/35

The DICFG: A Unified Representation “Database-enhanced” entry lockAccount CFG for lockAccount temp1 = parameter0:c_n Automatically temp2 = LocalDatabaseEntity0:Bank constructed with tool support temp3 = LocalDatabaseEntity1:acct_lock Define temporaries to temp4 = LocalDatabaseEntity2:card_number represent the program’s interaction at the levels of database and attribute A Test Adequacy Infrastrcture with Database Interaction Awareness, UCSB, November 7, 2005 – p. 19/35