P R E S E N T A T I O N Presentation Bio FG1 Friday, May 5, 2000 9:00AM S OAP O PERA T ESTI NG Hans Buwalda CMG TestFrame Research Center International Conference On Software Testing Analysis & Review
S o ap O p era T e stin g H a n s Bu w a l d a H a n s Bu w a l d a C M G - T e st Fr a m e Re se a r c h Ce n t e r C M G - T e st Fr a m e Re se a r c h Ce n t e r h an s. b u w al d a@c m g . n l h an s. b u w al d a@c m g . n l w w w . t est f r am e. co m w w w . t est f r am e. co m
A genda • int roduct ion • underlying archit ecture: th e T estF r am e M o d el • S oap O pera’s • u sag e
Th e C hallenges f or a Test P r o cess • te st ing should be fun • te st ing should be effect ive • te st ing should be ef f icient • te st ing should be under cont rol
Th e “m ech an ical appr oach ” for t est developm ent ( exam ple) • s tart w it h ( preferably long) list of requirement s • m a k e a te st ca se for ever y r equ ir em en t • u se a standar dized t est t echnique t o t r anslat e t he r equir em ent s int o t he t est cases • hir e ( m any) people t o pef or m t he t est s by hand • … .
S om e pit f alls w it h a t oo m echanical approach • no f un at all • shut t ing dow n creat ivit y • cover age is only at single r equir em ent level • an y def ect s sh ou ld pr obably h ave been f ou n d in an ear lier t est • su g g ests false sen se of con t r ol • te stse t hard t o m aint ain • doesn’t cat ch m ist akes in t he r equir em ent s • . . .
Qu e stio n s t o answ er w ith a te st collect ion 1. d o es t he syst em com ply t o t he requirement s 2. a re ther e any pr oblem s ( def ect s and/or fa ilur es) w e should know about 3. w ill t he syst em w ork in pract ice
Th e Test Fr am e p r o d u cts . . . • high level business orient ed t est s specif icat ions • product ion accept ance t est s Test Execut ion high level act ions • f unct ional t est s • t echnical t est s design . . . • low level f unct ional t est s • t echnical t est s programming int er mediat e level act ions low level act ions
Development and navigation in TestFrame test development A B C D . . . transfer Houston Klein 210 check balance Klein 210 transfer Savy Klein 150 navigation scheme check balance Klein 360 . . . … test cluster check balance enter customer test execution …
TestFrame Product overview test plan test design navigation end users scheme management reporting test execution QA/Auditors test clusters system development • input data • actual results • expected outcomes • comparison with • documentation expectations • management information target system(s) separation
S oap O per as Ashley hears about Jack's deposit when he thought he had to go. Victoria lectures her father about what's wrong with him and Nikki but Victor advises her that it's none of her business Olivia learns Dru has no regrets about leaving and takes great satisfaction in having Lily as her companion. Dru then asks Olivia why she is raking Malcolm over the coals. Stopping by Gina's, Nikki spots Brad and sits with him, admitting she doesn't want to be alone tonight. Victor stops by Mack's party at the Crimson Lights. Ashley takes a home pregnancy test. Worried about Billy, Raul makes call and J.T. claims he doesn't know where Billy is. Raul rushes over and finds Billy out cold in the snow Raul worries when he can't find a pulse. . . .
P ropert ies of S o ap Op er as • ab ou t “ real life” • bu t con den sed • and m or e ext r em e
S oap O peras for t est ing • def in e a scope of th e te st t o develop • ident if y w it h t he business environm ent • w h ich elem en ts w ou ld m ake things difficult • draf t scenario’s ( t ypical som e dozen lines) • w rit e t hem dow n in clust ers
E xam ples of st or y lines w hen used f or te sting Pe n si o n Fu n d William starts as a metal worker for Industrial Entropy Incorporated in 1955. During his career he becomes ill, works part time, marries, divorces, marries again, gets 3 children, one of which dies, then his wife dies and he marries again and gets 2 more children…. Wor ld Wid e Tr an sact ion Sy st em f or an in t er n at ion al Ban k A fish trade company in Japan makes a payment to a vendor on Iceland. It should have been a payment in Icelandic Kronur, but it was done in Yen instead. The error is discovered after 9 days and the payment is revised and corrected, however, the interest calculation (value dating)…
E xam ple of t est lines from to amount valuta trans nr 123421344 4124244123 120000 yen &keep tx1 enter payment &tx1 $0.47 check value dating 9 wait days &tx1 order to reverse from to amount valuta trans nr 123421344 4124244123 1200000000 IKr &keep tx2 enter payment &tx2 $7,701.56 check value dating . . . .
S oap O peras ( in t est ing) are not necessar ily: • “extrem e” • fa r fe tch ed • long and elaborat e • pieces of ar t and cr eativity
“K iller So a p s” • m ore specif ically aim ed at f inding hidden pr oblem s • r u n w h en ever yt h in g else h as passed • on e op t ion: put a killer soap at t he end of a norm al clust er • ask t he “ specialist s” for input
Re a so n s f or scenar ios like soaps • te st collection can b e m ad e m or e com p act • it is m or e f un t o m ake • specialists used m ore ef f ect ively • te st ing m ore of t he applicatio n • less direct ly dependent on f unct ional specs, so catching more pit falls
W h a t to use it f or pr im ar y use • high level f unct ional accept ance t est ing but also: • m odu le te sting • sy stem te sting • int egrat ion t est ing • ...
W h a t is n ot int erest ing for soaps • screen stuff • routine tests • any ot her st raight f orw ard com pliance te sting
A L ist of “P ar adigm s” • D o m a in dr iven • Re g r ession • S t r ess dr iven • Scen ar io • Sp e ci fic a t ion driven • U s e r t est ing • R isk dr iven • Explor at or y • Ra n d o m / st at ist ical • Securit y • Fu n ct ion source: Cem Kaner, James Bach, Star West 1999
Re la tion t o u se cases • friendly cousins • soap oper a’s ar e m or e dir ect ly aim ed at te st ing, f or exam ple by exaggerat ing and using ( non local) com binat ions • less an alytical “ top down” , but “ outside in” : translat ed f rom end user pract ice
Wh o cou ld m ak e S oap s • near ly ever ybody: � en d u ser s � specialized t est er s � developer s � audit ors � ...
W a y s to g et them • coach in g en d u ser s or bu sin ess specialists • int erviews • ow n f ant asy • w or kbooks • using joint developm ent sessions
Joint Test w are Dev elop m en t (JT D ) ™ • JT D = A technique f or struc tur ed te st developm en t w it hout ( complet e) d ocu m en ta tio n • JT D < > Ju st ta lk t o t he end-u ser s an d write dow n w h at th ey say • JT D < > A subsit ut e f or developing d ocu m en ta tion (requirement s , design s, user-guides)
W h a t can joint sessions give you • T e st S tra teg y • Acce p tan ce Cr ite ria • C lust er G rouping • T e st conditions • E valuat ion of Re su lts • S ta rt ing up developm ent of scenarios
Joint sessions • m oderat or / chair m an • users • bu sin ess specialists • developers • te sters
S et up of a j oint session f or a t elecom provider • 1 st sessi o n � In t r od u ct ion b y m od er at or an d p r oj ect m an ag er � ex p lan at ion ab ou t t h e JTD p r oced u r e � ex p lan at ion of t h e f u n ct ion al ar ea b y a sp ecialized u ser • 2 n d sessi o n � s tar t of pr oduct ion of t est condit ions • 3 r d session � s tar t of pr oduct ion of t est scenar ios • 4 th sessi o n � ev al u at ion t est scen ar ios
S truc tur ed t est developm ent and S oap s • soap s ar e n ot the n at ural w ay t o get “cover age” • additional t echniques can help, exam ples in T e stF ra m e : � te st condit ions � te st design t em plat es • reco m m en d ation: do “m atching” aft erw ards
T e st analysis and t est creat ion Test Analysis: Test Creation: - what do we want - confrontation with reality separate - what do we need - put it to the test and relate specifications business analytical techniques environment
E x am p le Test C ondit ion nr description ... 3.51 it is checked that the exit date is after the entry date ... coupling with the actual test lines in the cluster 3.51 test condition name entry date exit date Bill Goodfellow 1999-10-02 1999-10-01 enter employment The exit date must be after the entry date. check error message
Recommend
More recommend