Keeping Evolving Requirements and Acceptance Tests Aligned with - PowerPoint PPT Presentation

Keeping Evolving Requirements and Acceptance Tests Aligned with Automatically Generated Guidance Sofija Hotomski Eya Ben Charrada Martin Glinz

Agenda 1. Context, background and motivation 2. Our approach and tool 3. Evaluation 4. Summary 2

Requirements and acceptance tests – an example Requirement id: REQ1 Test id: AT1 Title: Test for REQ1 (Group chat management – adding Title: Group chat users to a group) management – adding users to a group Prerequisites: Internet connection is needed. Steps: Expected results: Text: A user can add 1. Log-in as user Login is successful new users to the group. The addition of a new 2. Open the page The page is opened and the list user must be first with users ’ groups of groups is visible approved by the admin. 3. Click on the button The group members are listed The admin can modify " Manage group ” next and there is a button “Add users” personal data and the to the Group 1 . in the upper, right corner. status of a user. … … 3

Motivation 4

Motivation 4

Motivation TESTERS ARE NOT AWARE 4

Motivation TESTERS ARE NOT AWARE 4

Motivation TESTERS ARE BUGS NOT AWARE FOR FEATURES 4

Motivation 5

Motivation Non-aligned documents 5

Motivation Non-aligned documents Poor communication 5

Motivation Non-aligned documents Poor communication False expectations 5

Motivation Non-aligned documents Poor communication False expectations => 5

Motivation Non-aligned documents Poor communication False expectations => 5

Motivation Non-aligned documents Poor communication False expectations => & 5

Motivation Non-aligned documents Poor communication False expectations => & & 5

State of the art Keep software documentation aligned using Information Retrieval and Natural Language Processing methods for identifying impacted documents. 6

State of the art Gaps in the current methods: • Less applied between requirements and acceptance tests • Only identify which documents are impacted by a change, but not suggest how to manage the change 7

The goals of our approach (GuideGen) When requirements evolve: 1. Support test engineers in making decisions on how to modify impacted acceptance tests, 2. Ensure on-time and smooth communication of changes between requirements and test engineers. 8

The goals of our approach (GuideGen) When requirements evolve: 1. Support test engineers in making decisions on how to modify impacted acceptance tests, Generate guidance 2. Ensure on-time and smooth communication of changes between requirements and test engineers. 8

The goals of our approach (GuideGen) When requirements evolve: 1. Support test engineers in making decisions on how to modify impacted acceptance tests, Generate guidance 2. Ensure on-time and smooth communication of changes between requirements and test engineers. Notification system 8

GuideGen overview 1. 2. 3. 9

An example of requirement changes and generated suggestions A user can add new users to the group. The addition of a new user must be first approved by the admin. The admin and the user can modify personal data and the status of that a user. Only user can modify its status. The admin must be logged-in in order to modify personal data of a user. 10

An example of requirement changes and generated suggestions 1. Delete steps or their parts which verify that the A user can add new users addition of a new user must be first approved by the to the group. admin. The addition of a new user must be first approved by the admin. The admin and the user can modify personal data and the status of that a user. Only user can modify its status. The admin must be logged-in in order to modify personal data of a user. 10

An example of requirement changes and generated suggestions 1. Delete steps or their parts which verify that the A user can add new users addition of a new user must be first approved by the to the group. admin. The addition of a new 2. Make sure that now the user can modify personal user must be first data of that user. Add the steps or modify the approved by the admin. expected results which verify this activity. The admin and the user can modify personal data and the status of that a user. Only user can modify its status. The admin must be logged-in in order to modify personal data of a user. 10

An example of requirement changes and generated suggestions 1. Delete steps or their parts which verify that the A user can add new users addition of a new user must be first approved by the to the group. admin. The addition of a new 2. Make sure that now the user can modify personal user must be first data of that user. Add the steps or modify the approved by the admin. expected results which verify this activity. The admin and the user 3. Delete steps or their parts which verify that the can modify personal data admin can modify the status of a user. and the status of that a user. Only user can modify its status. The admin must be logged-in in order to modify personal data of a user. 10

An example of requirement changes and generated suggestions 1. Delete steps or their parts which verify that the A user can add new users addition of a new user must be first approved by the to the group. admin. The addition of a new 2. Make sure that now the user can modify personal user must be first data of that user. Add the steps or modify the approved by the admin. expected results which verify this activity. The admin and the user 3. Delete steps or their parts which verify that the can modify personal data admin can modify the status of a user. and the status of that a 4. Add new steps or modify existing steps to verify that user. Only user can modify only user can modify its status. its status. The admin must be logged-in in order to modify personal data of a user. 10

An example of requirement changes and generated suggestions 1. Delete steps or their parts which verify that the A user can add new users addition of a new user must be first approved by the to the group. admin. The addition of a new 2. Make sure that now the user can modify personal user must be first data of that user. Add the steps or modify the approved by the admin. expected results which verify this activity. The admin and the user 3. Delete steps or their parts which verify that the can modify personal data admin can modify the status of a user. and the status of that a 4. Add new steps or modify existing steps to verify that user. Only user can modify only user can modify its status. its status. The admin must 5. Add new steps or modify existing steps to verify that be logged-in in order to the admin must be logged-in in order to modify modify personal data of a personal data of a user. user. 10

1. Identify relevant change patterns – sentence level The sentences in oldReq: The sentences in newReq: 1. A user can add new users 1. A user can add new users to the group. to the group. 2. The addition of a new 2. The admin and the user user must be first can modify personal data of approved by the admin. that user. 3. The admin can modify 3. Only user can modify personal data and the its status. status of a user. 4. The admin must be logged-in in order to modify personal data of a user.

1. Identify relevant change patterns – sentence level The sentences in oldReq: Similarity Score (S): The sentences in newReq: 1. A user can add new users 1. A user can add new users to the group. to the group. 2. The addition of a new 2. The admin and the user user must be first can modify personal data of approved by the admin. that user. 3. The admin can modify 3. Only user can modify personal data and the its status. status of a user. 4. The admin must be logged-in in order to modify personal data of a user.

1. Identify relevant change patterns – sentence level The sentences in oldReq: Similarity Score (S): The sentences in newReq: 1. A user can add new users 1. A user can add new users to the group. to the group. 2. The addition of a new 2. The admin and the user user must be first can modify personal data of approved by the admin. that user. 3. The admin can modify 3. Only user can modify personal data and the its status. status of a user. 4. The admin must be logged-in in order to modify personal data of a user.

1. Identify relevant change patterns – sentence level The sentences in oldReq: Similarity Score (S): The sentences in newReq: S = 1 1. A user can add new users 1. A user can add new users to the group. to the group. 2. The addition of a new 2. The admin and the user user must be first can modify personal data of approved by the admin. that user. 3. The admin can modify 3. Only user can modify personal data and the its status. status of a user. 4. The admin must be logged-in in order to modify personal data of a user.