Software Engineering I (02161) Testing Assoc. Prof. Hubert Baumeister DTU Compute Technical University of Denmark Spring 2020
Types of Tests Supports Programming Critique Project Explorative Testing Business Facing Acceptance Tests (trying to break the system) Property Testing Technology Facing Unit Tests (Performance, Scalability)
Acceptance Tests ◮ Tests defined by / with the help of the user ◮ based on the requirements ◮ Traditionally ◮ manual tests ◮ after the software is delivered ◮ Agile software development ◮ automatic tests: JUnit, Cucumber, . . . ◮ created before the user story / use case scenario is implemented ◮ developed with the customer → XP practice: customer on-site
Example of acceptance tests ◮ Login feature actor: Admin success 1. Start the application 2. Admin enters correct password 3. System responds true fail 1. Start the application 2. Admin enters wrong password 3 The system reports that the password is wrong and asks to enter the correct passowrd
Manual tests Successful login Prerequisit: the password for the administrator is “adminadmin” Input Step Expected Output Fail OK Startup system “0) Exit” “1) Login as administrator” “1” Enter choice “password” “adminadmin” Enter string “logged in” Failed login Prerequisit: the password for the administrator is “adminadmin” Input Step Expected Output Fail OK Startup system “0) Exit” “1) Login as administrator” “1” Enter choice “password” “admin” Enter string “Password incorrect” “0) Exit” “1) Login as administrator”
Manual vs. automated tests ◮ Manual tests should be avoided ◮ Are expensive; can’t be run often ◮ Automated tests ◮ Are cheap; can be run often ◮ Robert Martin (Uncle Bob) in http://www.youtube.com/watch?v=hG4LH6P8Syk ◮ manual tests are immoral from 36:35 ◮ how to test applications having a UI from 40:00 ◮ How to do UI tests? → Solution: Test under the UI
Acceptance Tests and Agile Software Development ◮ Automated tests: JUnit, Cucumber, . . . ◮ Created before the user story / use case scenario is implemented ◮ Developed together with the customer → tests need to be readable by the customer → Cucumber ◮ How to do UI tests? → Solution: Test under the UI
Test under the UI User Thin Presentation Layer No Business Logic Tests Business Logic Application Layer e.g. LibraryApp Business logic Domain Layer e.g. User, Book, ... Business logic Persistency Layer
Acceptance test ◮ Framework for Integrated Tests (Fit) → use tables ◮ Cucumber Feature: Admin login Description: The administrator logs into the library system Actor: Administrator Scenario: Administrator can login Given that the administrator is not logged in And the password is "adminadmin" When the administrator logs in Then the adminstrator is logged in Scenario: Administrator has the wrong password Given that the administrator is not logged in And the password is "wrong password" When the administrator logs in Then the login fails
Cucumber ◮ Behaviour-Driven Development: User Stories Feature: Name of the feature Description ... Scenario: Name Description ... Given an initial state And ... When an action happens And ... Then an assertion is true And ... ◮ Gherkin: for scenarios (supports different languages, e.g. Danish and Pirate :-) ◮ Programming language (Java): Glue code
Example: Add book Business experts write Cucumber features Feature: Add book Description: A book is added to the library Actors: Administrator Scenario: Add a book successfully Given that the administrator is logged in And a book with title "Extreme Programming", author "Kent Beck", and signature "Beck99" When the book is added to the library Then the book is contained in the library Programmers implement ”meaning” of each step @Given("that the administrator is logged in") public void thatTheAdministratorIsLoggedIn() throws Exception { libraryApp.adminLogin("adminadmin"); }
How does Cucumber work? ◮ The Cucumber test runner parses the feature file @RunWith(Cucumber.class) @CucumberOptions(features = "use_cases", glue = { "dtu.library.acceptance_tests"}, strict = true) public class AcceptanceTest {} ◮ For each scenario: go through all step definitions ◮ Find the method whose annotation matches ◮ class does not matter ( → no duplicated step definitions) → put steps in classes corresponding to concepts ◮ e.g. users → UserSteps; books → BookSteps → ubiquitous language ◮ Create objects of step classes ◮ create all objects used in the constructors ◮ E.g. create LibraryApp and UserHelper object for BookSteps(LibraryApp l, UserHelper h) and UserHelper(LibraryApp l, UserHelper h) ◮ Run the methods according to the scenario
Step definitions: AdministratorSteps public class AdministratorSteps { private LibraryApp libraryApp; public AdministratorSteps(LibraryApp l) { libraryApp = l; } @Given("that the administrator is logged in") public void thatTheAdministratorIsLoggedIn() throws Exception { libraryApp.adminLogin("adminadmin"); } }
Step definitions: BookSteps public class BookSteps { private LibraryApp libraryApp; private BookHelper bookHelper; public BookSteps(LibraryApp l, BookHelper h) { libraryApp = l; bookHelper = h; } @Given("a book with title {string}, author {string}, and signature {string}") public void aBookWithTitleAuthorAndSignature(String title, String author, String signature) throws Exception { bookHelper.createBook(title,author,signature); } @When("the book is added to the library") public void theBookIsAddedToTheLibrary() throws Exception { try { libraryApp.addBook(bookHelper.getBook()); } catch (OperationNotAllowedException e) { errorMessage = e.getMessage(); } } @Then("the library contains the book") public void theLibraryContainsTheBook() throws Exception { assertTrue(libraryApp.containsBookWithSignature( bookHelper.getBook().getSignature())); } }
BookHelper public class BookHelper { private Book book; public void createBook(String title, String author, String signature) { book = new Book(title, author, signature); } public void getBook() { if (book == null) { createDefaultBook(); } return book; } private void createDefaultBook() { createBook("some title", "some author", "sig001"); } }
Meaning of Given, Then, and When ◮ For Cucumber, Given, Then, and When have no meaning. ◮ They are all treated the same → Given, Then, and When cannot have the same text ◮ They have only meaning for us ◮ Given <something> ◮ either checkes, using assertions, that a given <something> is true ◮ e.g. assertTrue(<something>) ◮ or runs actions which make <something> true ◮ When <action> ◮ execute <action> ◮ Then <condition> ◮ Check, using assertions, that <condition> is true ◮ e.g. assertTrue(<something>)
Assertions (inherited from JUnit) ◮ General assertions import static org.junit.Assert.*; assertTrue(bexp) assertTrue(msg,bexp) ◮ Failing assertions throw an AssertionError exception ◮ Specialised assertions for readability import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.CoreMatchers.*; 1. assertFalse(bexp) / assertThat(bexp,is(false)) 2. fail() 3. assertEquals(exp,act) / assertThat(act,is(equalTo(exp))) 4. assertNull(act) / assertThat(act,is(nullValue())) 5. assertNotNull(act) / assertThat(act,is(not(nullValue()))) ...
How to deal with checked exceptions? ◮ In LibraryApp public addBook(Book b) throws OperationNotAllowedException {..} ◮ Step definition does not compile @When("the book is added to the library") public theBookIsAddedToTheLibrary() { libraryApp.addBook(bookHolder.getBook()); } ◮ ”Standard” solution try { .. } catch (MyException e) { e.printStackTrace(); } ◮ Better: 1 Should never happen → fail fast @When("the book is added to the library") public theBookIsAddedToTheLibrary() throws Exception { libraryApp.addBook(bookHolder.getBook()); } 2 testing for the exception @When("the book is added to the library") public theBookIsAddedToTheLibrary() { try {libraryApp.addBook(bookHolder.getBook());} catch (OperationNotAllowedException e) { errorMessageHolder.setErrorMessage(e.getMessage()); } } 3 In production code, handle, add throws clause, or convert to Error try {....} catch (MyException e) { throw new Error(e); }
Recommend
More recommend
