AUTOMATED TEST SYSTEM DEVELOPMENT FROM SCRATCH: THE MAIN PROBLEMS - PowerPoint PPT Presentation

AUTOMATED TEST SYSTEM DEVELOPMENT FROM SCRATCH: THE MAIN PROBLEMS AND THEIR SOLUTIONS Lilia Sapurina

ABOUT ME 2 years in huge company with delivered QA processes: automated UI testing, CI 4 month in small company without any automated QA processes: development CI processes, automatisation

WHO IS AN AUTOMATION ENGINEER? Company with Company without processes processes

MAIN STEPS Syntax analysis Documentation Unit tests Rest API autotests UI autotests Autotest code Reports

SYNTAX ANALYSIS

SYNTAX ANALYSIS: ADVANTAGES Eliminates small typos Creates universal code Simplifies documentation Simplifies testing

SYNTAX ANALYSIS: PROBLEMS Not enough time to analyse the rules A large number of invalid code Not enough time to fix

SYNTAX ANALYSIS: SOLUTION Selection of basic rules Rules for reconciliation - non-critical errors Validated new code

SYNTAX ANALYSIS: TOOLS SonarQube - too complex Tools in the native language (e.x. Javascript - ESLint, php - PHPLint) Plugins/Special settings for IDE

DOCUMENTATION

DOCUMENTATION: ADVANTAGES Improve team communication Full use of existing functionality Getting rid of duplicates Simplify refactoring Simplify testing (REST API, unit tests, UI Autotests)

DOCUMENTATION: PROBLEMS Not enough time to produce detailed comments in new methods A large number of non- documented code

DOCUMENTATION: SOLUTION Prioritisation: REST API methods, methods for unit-tests, helpers Add rules to syntax analysis configuration file

DOCUMENTATION: TOOLS Tools in the native language (JSDoc, JavaDoc, PHPDocumentor, EDoc) APIDoc

WHY APIDOC? APIDOC Swagger Description In code In Swagger Editor REST API, Autotest (Rest, REST API, Autotest Usage UI), helpers (REST) Language Simple comments Special layout

DOCUMENTATION: EXAMPLE

UNIT TESTS

UNIT TESTS: ADVANTAGES Quick bug fix Can be used like documentation Simplified refactoring

UNIT TESTS: PROBLEMS Not enough time to develop tests for old functionality New frameworks A large number of unit tests interfere with the development of the project

UNIT TESTS: SOLUTION Partial implementation: minimal test coverage Remote testing infrastructure development Instructions and examples development Tasks for new applicants

UNIT TESTS: TOOLS AngularJs: Jasmine + PhantomJS PHP: PHPUnit Erlang: EUnit

AUTOTEST: REST API

AUTOTEST (REST API): ADVANTAGES Continuous monitoring of the system functioning Quick bug fix Detecting bug causes Vulnerability scanner

AUTOTEST (REST API): PROBLEMS Wrong test structure Incomprehensible reports Misunderstanding with developers

FIND DEFECT REASON: ALGORITHM Report Documentation Tester

AUTOTEST (REST API): SOLUTION Instrumental analysis Parametrisation Resemblance of programming languages

AUTOTEST (REST API): TOOLS Java + Junit Allure ApiDoc

ADDITIONAL LOGS

AUTOTEST: UI

AUTOTEST (UI): ADVANTAGES Continuous monitoring of the system functioning Unpredictable bugs search Smoke tests

AUTOTEST (UI): PROBLEMS Constant design changes Functionality changes Testing duration

AUTOTEST (UI): SOLUTION Basic functionality allocation Basic scenarios for sprint Constant run of the simplest and fastest tests Analysis of reports by the testing team

AUTOTEST (UI): TOOLS Protractor + Jasmine Allure ApiDoc Testrail

AUTOTEST CODE

Autotest code should provide a good example for developers AUTOTEST CODE: CHECKS Syntax analysis Documentation Spelling

REPORTS

REPORTS: ADVANTAGES Improve communication between developers and testers Quick bug fix Statistical data

REPORTS: PROBLEMS Variety of tools Uninformative notifications Overextended error analysis

REPORTS: SOLUTION Similar tools Brief review Isolate reports and continuous integration system Variety of reports

FIRST RESULTS (4 MONTHS) Jenkins builds + Pull Request System + Notifications + Reports + Load Tests Frontend Backend (php) Messenger (Erlang) (AngularJS) Syntax Analysis 80 % 100 % - Unit Tests 40 % 30 % 10 % 60 API methods 10 basic smoke test 100% coverage Autotest (30% coverage) 3-4 tests per week (use cases) 20 use cases

THANK YOU

USEFUL LINKS http://2016.secr.ru/program/submitted- presentations/developing-an-automated-front- end-testing-system-with-protractor-framework https://github.com/liliasapurina/testrail-reporter