UNIVERSITAS SCIENTIARUM SZEGEDIENSIS UNIVERSITY OF SZEGED D epartment of Software Engineering Supporting Software Product Line Testing by Optimizing Code Configuration Coverage László Vidács 1 , Ferenc Horváth 2 , József Mihalicza 3 , Béla Vancsics 2 and Árpád Beszédes 2 1 MTA-SZTE Research Group on Artificial Intelligence, Hungary 3 NNG LLC, Budapest, Hungary 2 Department of Software Engineering, University of Szeged, Hungary TAIC PART | Testing: Academic & Industrial Conference - Practice and Research Techniques | Graz, Austria, 2015
UNIVERSITY OF SZEGED D epartment of Software Engineering Context NNG ships navigation solutions on a broad UNIVERSITAS SCIENTIARUM SZEGEDIENSIS spectrum: Automotive line fit solutions for tier 1 clients Over 30 car brands carry iGO navigation (Qnx, Android, Linux, WinCE) White label core product After-market head units, mobile apps (iOS, WinCE, WinMobile, Android) Mobile navigation app for B2C end users TAIC PART 2015 2
UNIVERSITY OF SZEGED D epartment of Software Engineering Product Line NNG philosophy: „ Navigation for All ” UNIVERSITAS SCIENTIARUM SZEGEDIENSIS Achieved by a single code base for core functionalities Customizations should integrate well with core features SPL: code variability at preprocessor level ■ Platforms (and variants), compilers, rendering engines, 32bit/64bit ■ Windows CE/Mobile/PC, QNX, Linux, Android, iOS ■ Features, customizations TAIC PART 2015 3
UNIVERSITY OF SZEGED D epartment of Software Engineering Research goal Testing release configurations is not sufficient UNIVERSITAS SCIENTIARUM SZEGEDIENSIS ■ Get a feature from Config A and turn it on in Config B Efficient testing of the configurable core code Research goal: Select small number of configurations which cover large amount of code TAIC PART 2015 4
UNIVERSITY OF SZEGED D epartment of Software Engineering Preprocessor based Variability Block 1 #if A == 1 2 #define B 2 UNIVERSITAS SCIENTIARUM SZEGEDIENSIS Presence condition 3 #endif 4 Variable 5 #if A == 2 6 #define B 6 Configuration 7 #endif #define PLATFORM_WIN32 8 #define A 2 #define B 10 9 #if !defined(B) 10 byte x; Coverage 11 #elif B >= 4 Source code lines with 12 int x; enabled presence 13 #endif conditions TAIC PART 2015 5
UNIVERSITY OF SZEGED D epartment of Software Engineering Search algorithms Find N(<10) configurations with highest possible coverage Approach UNIVERSITAS SCIENTIARUM SZEGEDIENSIS ■ Build each configuration incrementally (greedy approach) ■ Create new configurations until N is reached Block-based approach ■ Try to cover the largest uncovered block Variable-based approach ■ Select the variable which results the highest overall coverage increase TAIC PART 2015 6
UNIVERSITY OF SZEGED D epartment of Software Engineering Block-based algorithm B1 1 Examine largest uncovered block UNIVERSITAS SCIENTIARUM SZEGEDIENSIS B2 2 Satisfy presence condition B3 #if X > 0 && Y ==5 B4 B5 3 Extend candidate configuration B6 #define A 100 #define B -43 B7 #define X 1 #define Y 5 B8 4 Refresh the global coverage TAIC PART 2015 7
UNIVERSITY OF SZEGED D epartment of Software Engineering Variable-based algorithm 1 For each free configuration variable B1 UNIVERSITAS SCIENTIARUM SZEGEDIENSIS + B2 … … A B X B3 + 2 Compute coverage for each value interval B4 + + B5 undefined 35 #define A 100 #define B -43 [- ∞; 0] 60 B6 #define X 1 … [1; ∞] 121 + BN Extend candidate configuration 3 & refresh the global coverage TAIC PART 2015 8
UNIVERSITY OF SZEGED D epartment of Software Engineering iGO Navigation Measurements Condition type Blocks LOC UNIVERSITAS SCIENTIARUM SZEGEDIENSIS Filtered (T, F, #error) 11,847 (25%) 682,300 (35%) Configuration 22,067 (47%) 920,926 (48%) Mixed 10,085 (22%) 271,710 (14%) Non-configuration 2,811 (6%) 50,064 (3%) Total 46,810 1,925,000 #error directives ■ Prevent invalid configurations Non-configuration variables -> mixed conditions ■ MODULE1_DETAILED_DIAG, PERSONAL_PATCHES_JOE TAIC PART 2015 9
UNIVERSITY OF SZEGED D epartment of Software Engineering Results: block-based, N = 10 Config coverage Search time 98.74% 22 min UNIVERSITAS SCIENTIARUM SZEGEDIENSIS Config coverage Delta coverage Config LOC 1000 900 800 Coverage (kLOC) 700 600 500 400 300 200 100 0 1 2 3 4 5 6 7 8 9 10 Configurations TAIC PART 2015 10
UNIVERSITY OF SZEGED D epartment of Software Engineering Results: variable-based, N = 10 Config coverage Search time 75.56% 27 min UNIVERSITAS SCIENTIARUM SZEGEDIENSIS Config coverage Delta coverage Config LOC 1000 900 800 Coverage (kLOC) 700 600 500 400 300 200 100 0 1 2 3 4 5 6 7 8 9 10 Configurations TAIC PART 2015 11
UNIVERSITY OF SZEGED D epartment of Software Engineering Results: block-based, N = 50 Config coverage Search time 99.74% 2 h UNIVERSITAS SCIENTIARUM SZEGEDIENSIS Config coverage Delta coverage Config LOC 1000 900 800 Coverage (kLOC) 700 600 500 400 300 200 100 0 1 10 20 30 40 50 Configurations TAIC PART 2015 12
UNIVERSITY OF SZEGED D epartment of Software Engineering Results & plans Enhanced UNIVERSITAS SCIENTIARUM SZEGEDIENSIS #error Hybrid directives algorithm Multiple variables at a time BLOCK-BASED VARIABLE-BASED 98.74% 75.56% N=10, 22 min N=10, 27 min TAIC PART 2015 13
Recommend
More recommend
