Capture-Replay Tests in J2ME Testy capture-replay w rodowisku J2ME - PowerPoint PPT Presentation

Capture-Replay Tests in J2ME Testy capture-replay w środowisku J2ME Marcin Zduniak Bartosz Walter Dawid Weiss Institute of Computing Science Poznan University of Technology 2007

Participants Motivation RobotME Summary Who is who • Marcin Zduniak (the graduate) • Bartosz Walter (thesis supervisor) • Dawid Weiss (original concept, mentoring)

Participants Motivation RobotME Summary Tests J2ME Scope Related What are “software tests”?

Participants Motivation RobotME Summary Tests J2ME Scope Related What are “software tests”? (Wikipedia) Software testing is the process used to help identify the correctness , completeness , security , and quality of developed computer software.

Participants Motivation RobotME Summary Tests J2ME Scope Related How can we “test software”?

Participants Motivation RobotME Summary Tests J2ME Scope Related Unit tests Correctness of individual units of source code Module/ integration tests Chunks of functionality, sometimes the entire program. testing in various target environments (O/S’s, processors etc.). Acceptance tests Compliance to customer’s requirements; often manual work. Regression tests System stability/ correctness in response to ongoing changes.

Participants Motivation RobotME Summary Tests J2ME Scope Related

Participants Motivation RobotME Summary Tests J2ME Scope Related Java 2 Micro Edition • A specification. • A subset of Java Virtual Machine. • A subset of standard Java library. • Many vendors .

Participants Motivation RobotME Summary Tests J2ME Scope Related Java 2 Micro Edition

Participants Motivation RobotME Summary Tests J2ME Scope Related Java 2 Micro Edition

Participants Motivation RobotME Summary Tests J2ME Scope Related Programming in J2ME • Each mobile device has different hardware. • Different KVM implementations (and bugs). • “Standard” APIs implemented in different ways. • A number of non-standard APIs and proprietary solutions. • No system-level support for application testing. Conclusion: programming and testing is difficult in J2ME.

Participants Motivation RobotME Summary Tests J2ME Scope Related Project scope 1 Focus on capture-replay tests (GUI and other events). 2 Should facilitate integration and regression tests in J2ME. 3 Should work on emulators and actual devices.

Example of use Capture-replay and regression testing.

Participants Motivation RobotME Summary Tests J2ME Scope Related Related projects • J2ME Unit • Sony-Ericsson Mobile JUnit • Motorola Gatling • CLDC Unit • IBM Rational Test RT • Research In Motion – BlackBerry Fledge emulator

Participants Motivation RobotME Summary Injection Bytecode Maintenance Prototype And the ultimate answer is. . . RobotME (of course the ultimate question still being “what’s 6 × 9?”)

Participants Motivation RobotME Summary Injection Bytecode Maintenance Prototype The core idea • Follow the regular capture-replay pattern. • Cater for missing “robot” API by modifying the software at the bytecode level. • Verify replay-phase correctness by analysis of captured events.

Participants Motivation RobotME Summary Injection Bytecode Maintenance Prototype Dynamic code injection • Identify places which should generate an event (“injection points”). • Intercept parameters at injection points, injecting custom proxies.

Participants Motivation RobotME Summary Injection Bytecode Maintenance Prototype Injection points: subclassing Custom inheritance from system classes (subclassing). Form , Canvas , MIDlet . . . 1 public class MyMidlet extends MIDlet { protected void startApp() throws MIDletStateChangeException { 2 // application code here. 3 } 4 5 } We need to intercept the call to startApp() method.

Participants Motivation RobotME Summary Injection Bytecode Maintenance Prototype • Make MyMidlet a subclass of RobotMIDlet ? 1 public class MyMidlet extends RobotMIDlet { all methods virtual, multi-level inheritance

Participants Motivation RobotME Summary Injection Bytecode Maintenance Prototype • Make MyMidlet a subclass of RobotMIDlet ? 1 public class MyMidlet extends RobotMIDlet { all methods virtual, multi-level inheritance • Make a custom subclass RobotMIDlet$1 extend MyMidlet ? 1 public class RobotMIDlet$1 extends MyMidlet { finalized classes, multi-level inheritance

Participants Motivation RobotME Summary Injection Bytecode Maintenance Prototype • Make MyMidlet a subclass of RobotMIDlet ? 1 public class MyMidlet extends RobotMIDlet { all methods virtual, multi-level inheritance • Make a custom subclass RobotMIDlet$1 extend MyMidlet ? 1 public class RobotMIDlet$1 extends MyMidlet { finalized classes, multi-level inheritance • Use delegation pattern and alter the code of startApp() ? 1 public class MyMidlet extends MIDlet { protected void startApp() throws MIDletStateChangeException { 2 RobotME.event( this , STARTAPP_BEFORE); 3 try { 4 // application code here. 5 } finally { 6 RobotME.event( this , STARTAPP_AFTER); 7 } 8 } 9 10 } code bloat, goto, architectural flaws

Participants Motivation RobotME Summary Injection Bytecode Maintenance Prototype Injection points: references Direct use of an object (reference tracking). Command , Item 1 public class MyApplication { private static final Command MY_COMMAND = 2 new Command("Cmd label", Command.OK, 1); 3 4 public MyApplication() { 5 Form f = new Form("Title"); 6 f.addCommand(MY_COMMAND); 7 } 8 9 } We need to track the identity (and value) of commands.

Participants Motivation RobotME Summary Injection Bytecode Maintenance Prototype • Substitute all constructors of Form with a custom proxy? 1 Form f = new RobotME$Form("Title"); clashes with custom inheritance

Participants Motivation RobotME Summary Injection Bytecode Maintenance Prototype • Substitute all constructors of Form with a custom proxy? 1 Form f = new RobotME$Form("Title"); clashes with custom inheritance • Intercept all calls to addCommand() ? 1 f.addCommand(RobotME.addCommand(f, MY_COMMAND)); not too bad

Participants Motivation RobotME Summary Injection Bytecode Maintenance Prototype Injection points: listeners Listeners (system callbacks). ItemStateListener , CommandListener , ItemCommandListener 1 public class MyForm extends Form implements CommandListener { private final Command ADD_COMMAND; 2 3 public MyClass { 4 5 this .ADD_COMMAND = new Command("ADD", Command.OK, 1); this .setCommandListener( this ); 6 } 7 8 public void commandAction(Command c, Displayable d) { 9 // event handling code. 10 if (c == ADD_COMMAND) { 11 // ...do something. 12 } 13 } 14 15 ... We need to track (and stimulate) commands for the listener. Note there is only one listener on a Form .

Participants Motivation RobotME Summary Injection Bytecode Maintenance Prototype • Intercept all calls to setCommandListener() ? 1 this .setCommandListener(RobotME.setCommandListener( this , command)); What if somebody uses “==” to compare listeners?

Participants Motivation RobotME Summary Injection Bytecode Maintenance Prototype • Intercept all calls to setCommandListener() ? 1 this .setCommandListener(RobotME.setCommandListener( this , command)); What if somebody uses “==” to compare listeners? • How to remember the command received (if it’s a dynamic reference)? The reference changes between runs.

Participants Motivation RobotME Summary Injection Bytecode Maintenance Prototype • Intercept all calls to setCommandListener() ? 1 this .setCommandListener(RobotME.setCommandListener( this , command)); What if somebody uses “==” to compare listeners? • How to remember the command received (if it’s a dynamic reference)? The reference changes between runs. • How to generate an identical event dynamically? The event should match the original Display / Form pair.

Participants Motivation RobotME Summary Injection Bytecode Maintenance Prototype Examples of injected code Original code (fragment): 1 final Form form = new Form("Questionnaire"); 2 ... 3 form.addCommand(CMD1_EXIT); 4 form.addCommand(CMD2_OK); 5 form.setCommandListener( this ); 6 7 Display.getDisplay( this ).setCurrent(form);

Code modified for the recording phase: 1 Form form = new Form("Questionnaire"); 2 ... 3 form.addCommand(b); // NOTE: variable names have been obfuscated. 4 RobotMERecorder.getRecorderInstance().commandAddedToDisplayable(b, form); 5 form.addCommand(c); 6 RobotMERecorder.getRecorderInstance().commandAddedToDisplayable(c, form); 7 form.setCommandListener( this ); 8 Display.getDisplay( this ).setCurrent(form); 9 RobotMERecorder.getRecorderInstance().setCurrentDisplayable(form); Code modified for the replay phase: 1 Form form = new Form("Questionnaire"); 2 form.addCommand(b); 3 RobotMEReplaying.getReplayingInstance().commandAddedToDisplayable(b, form); 4 form.addCommand(c); 5 RobotMEReplaying.getReplayingInstance().commandAddedToDisplayable(c, form); 6 form.setCommandListener( this ); 7 RobotMEReplaying.getReplayingInstance() .commandListenerSetOnDisplayable( this , form); 8 9 Display.getDisplay( this ).setCurrent(form); 10 RobotMEReplaying.getReplayingInstance().setCurrentDisplayable(form); 11 12 RobotMEReplaying.getReplayingInstance().startReplaying();