ALS 2016 - Jan-Simon Mller Contiguous Integration and Testing for - PowerPoint PPT Presentation

ALS 2016 - Jan-Simon Möller Contiguous Integration and Testing for Automotive in a distributed environment

AGL development ... ● AGL development is done - like for many other opensource projects - in a distributed way ● Developers around the globe contribute to the sources and recipes that make up AGL ● Code review through gerrit, CI builds and so on

Problem statement ● BUT: we cannot do tests on HW in the same distributed way ... ● ... yet ? Why and how ? Image: public domain

Tests on HW are hard !? ● You need the HW ● You need it on your desk/in your lab ● You need to deploy firmware/filesystems ● You need to reboot the board ● You need to initiate the test ● You need to collect the results ● You need to interpret the results ... rinse & repeat Images: elinux.org & dl9pf

Where we'll struggle... ● Board not available on your desk ● Board cannot be shipped due to xyz ● Board hardwired on your desk while travelling ● Juggling SD-cards takes a lot of time ● It takes a lot of time vs. ● We want the info early - for each proposed patch

Issues: And now press: ● Lots of manual work involved ● does not scale – * boards * build-variants * releases ● does not integrate with code review systems ● release management and maintenance depends on testresults

What do we need ... and when ? ● We won't replace proper manual/long test runs for the releases! ● But for our daily work, we need: – fast results – as much test-coverage as possible in short time – large number of boards tested – remote / distributed operation ● => we need to keep pace with the development

What do we need ... and when ? ● panacea ? ● jack of all trades device ? https://xkcd.com/763/

Prior art (opensource) ● Linux – ktest.pl (S.Rostedt, http://elinux.org/Ktest) – 0-day builder (Fengguang Wu, Intel, https://01.org/lkp) ● https://kernelci.org ● lava (https://validation.linaro.org/static/docs ) ● JTA/fuego (http://elinux.org/Fuego) ● openQA (https://openqa.opensuse.org/)

AGLs current infrastructure ● Gerrit for code review – git.automotivelinux.org ● Jenkins with gerrit-trigger for CI builds – build.automotivelinux.org ● => pretty much standard toolset

Beta-test for new additions ... ● AGL-JTA (fork of fuego) – jta.agl.homelinux.org – = tool to compile and run predefined testcases ● Lava – porter.agl.homelinux.org – = board farm management and deployment framework

Why JTA ? ● AGL-JTA is AGL's version of fuego (formerly JTA) – Has large number of Benchmarks and Functional tests – Visualization and reporting – JTA/fuego is part of the LTSI initiative – AGL modified/updated components to fit our needs – Updated to stable jenkins – Dedicated to upstreaming changes

Demo JTA ● Instance with up as public beta: ● https://jta.automotivelinux.org

Why Lava ● Lava supports – multiple boards (of same type / pool) – different boards at the same time – very good automation capabilities – remote RPC API – master/slave model – distributed slave sites in development – some tests built-in through testdefinitions in git

Demo Lava ● Instance with porter board up as public beta: ● https://porter.automotivelinux.org

Workflow / Feedback loop Gerrit Jenkins (Build) CI builds AGL-JTA (jta) Lava (porter)

A Vision/Plan (short/mid) SPDX Fossology Gerrit Jenkins (Build) Extended tests, customized tests AGL-JTA (jta) Remote Lab Lava (lava) Remote Lab

A Vision/Plan (mid/long) Gerrit SPDX Fossology Jenkins (Build) AGL-JTA (jta) Remote Board Lab Lava SDK (lava) Test & Debug App Remote Board Lab on board remotely openQA ? VM ← ← ← ← UI testing ?

Open Issues ● present testresults in a unified way ● aggregate testresults (across revisions/variants/boards) ● visualize testresults (e.g. trends) ● Thoughts ?

Todo's for AGL-JTA/fuego (IMHO) ● Further simplify installation ● Use latest version of jenkins (→ less UI modifications) ● Do not hardcode version of jenkins, use release from repository ● Do not assume JTA and board are „in the same room“ or on the same network ● Do not hardcode IPs and board specifics – use an abstraction )* ● Do not rebuild if not necessary ● )* Maybe abstract the actual board through a slave daemon like WIP in lava

Todo's for LAVA (IMHO) ● Installation on debian works meanwhile, but configuration is still complex ● Documentation is good, but we desperately need a „getting-started in 30min“ or guided installation. ● UI ease-of-use (less clicks) ● more cmdline tools (like lava-boot) for developers

A Roadmap ● For BB: – public beta of JTA and Lava ● For CC: – All reference boards/platforms included – Remote lab support – Fossology/SPDX reports ● For DD: – SDK integration ?!

Questions ? ● Q/A ● Join – automotive-discussions ML – bi-weekly AGL EG-CIAT meetings – #automotive on freenode