Proceedi ngs of National Conference on Artificia l Intellig - PDF document

Proceedi ngs of National Conference on Artificia l Intellig enc e (AAAI-92 ), San Jose, 1992, pp. 322-328. A Probabilistic P arser Applied to Soft w are T esting Do cumen ts Mark A. Jones Jason M. Eisner A T&T Bell Lab oratories Emman uel College, Cam bridge 600 Moun tain Av en ue, Rm. 2B-435 Cam bridge CB2 3AP England Murra y Hill, NJ 07974{063 6 jme14@pho enix.cam bri dge.ac. uk jones@researc h.att.com Abstract GO AL: Activ ate CF A [call forw arding] using CF A Acti- v ation Access Co de. W e describ e an approac h to training a statisti- Set station B2 without redirect noti�cation. A CTION: cal parser from a brac k eted corpus, and demon- Station B2 go es o�ho ok and dials CF A Activ ation Access strate its use in a soft w are testing application Co de. that translates English sp eci�cations in to an au- Station B2 receiv es the second dial tone. VERIFY: tomated testing language. A gramm ar is not ex- plicitly sp eci�ed; the rules and con textual proba- Station B2 dials the extension of station B3. A CTION: bilities of o ccurrence are automatically generated Station B2 receiv es con�rmation tone. The VERIFY: from the corpus. The parser is extremely success- status lamp asso ciated with the CF A button at B2 is ful at pro ducing and iden tifying the correct parse, lit. and nearly deterministic in the n um b er of parses VERIFY: : : : that it pro duces. T o comp ensate for undertrain- ing, the parser also uses general, linguistic sub- Figure 1: An Example T est Case theories whic h aid in guessing some t yp es of no v el structures. ioms and con v en tions of telephon y . Its task is to �esh In tro duction out the test description pro vided b y the English sen- tences. This is c hallenging b ecause the sen tences omit In constrained domains, natural language pro cessing man y implicit conditions and actions. In addition, can often pro vide lev erage. In soft w are testing at some sen tences (\Mak e B1 busy") require the analyzer A T&T, for example, 20,000 English test cases prescrib e to create simple plans. The analyzer pro duces a for- the b eha vior of a telephone switc hing system. A test mal description of the test, whic h the bac k-end tr ans- case consists of ab out a dozen sen tences describing the then renders as executable co de. A more com- lator goal of the test, the actions to p erform, and the con- plete description of the goals of the system, its arc hi- ditions to v erify . Figure 1 sho ws part of a simple test tecture and the soft w are testing problem can b e found case. Curren t practice is to execute the tests b y hand, in [Nonnenmann and Eddy 1992 ]. or else hand-translate them in to a lo w-lev el, executable language for automatic testing. Co ding the tests in the This pap er discusses the natur al language pr o c essor executable language is tedious and error-prone, and the or linguistic comp onen t, whic h m ust extract at least English v ersions m ust b e main tained an yw a y for read- the surface con ten t of a highly referen tial, naturally o ccurring text. The sen tences v ary in length, ranging abilit y . from short sen tences suc h as \Station B3 go es onho ok" W e ha v e constructed a system called KITSS (Kno wledge-Based In teractiv e T est Script System), to 50 w ord sen tences con taining paren theticals, sub or- dinate clauses, and conjunction. The principal lev erage whic h can b e view ed as a system for mac hine-assisted is that the discourse is reasonably w ell fo cused: a large, translation from English to co de. Both the English but �nite, n um b er of telephonic concepts en ter in to a test cases and the executable target language are part �nite set of relationships. of a pre-existing testing en vironmen t that KITSS m ust �t in to. The basic structure of the system is giv en in Figure 2. English test cases undergo a series of trans- Natural Language Pro cessing in KITSS lation steps, some of whic h are in teractiv ely guided b y a tester. The KITSS linguistic comp onen t uses three t yp es of The c ompleteness and inter action analyzer is the kno wledge to translate English sen tences quic kly and pragmatic comp onen t that understands the basic ax- accurately in to a logical form: