Validation is key to the delivery of robust code and is an ongoing process involving multiple teams, covering four components:
- Device Automation
- Build Automation
- Test Planning
- Developer feedback
Device Automation is the process of providing remote access to a set of devices in order to deploy systems for testing, automate the operation of the test and collect the results. Device automation involves the provision of hardware and the software to automate tests on that hardware.
Devices exist in a number of different labs. The oldest lab is the Linaro Lab in Cambridge and there is a growing number of community labs. The hardware provision for the Linaro Lab is managed by the Linaro LAB team (login required)
Build Automation is the process of preparing files which are based on developer activity and which can be deployed on devices to validate whether the developer changes have improved or broken the ability of the device to perform the required test. Linaro has the Builds and Baselines team (login required) to support automated builds of releases and test support files.
Test Planning involves developers and test writers identifying which tests are relevant to the development work and how at least some of those can be automated. The Test Plan needs to arrange that:
- tests are initiated automatically by developer activity,
- updated test files are prepared,
- a test job is submitted to the device.
- results are collected and made available to developers
Without a Test Plan, the provision of device automation and build automation cannot deliver validation. The Linaro Quality Assurance Team (login required) can assist in the preparation of a Test Plan.
Developer feedback is team-specific and involves mapping the raw results to a relevant and useful summary which is delivered to the developers in a timely and helpful manner. The LAVA software can provide the raw results with some initial formatting but teams will benefit from the preparation of a custom frontend which presents the results in the way most suitable to that particular team. e.g. http://kernelci.org
The LAVA Software
Linaro has developed an automated testing system called LAVA (Linaro Automated Validation Architecture) which delivers the device automation and result collection. The software for this system is open source and can be recreated by following the documentation. Examples of the installation of the LAVA software and the Linaro LAB setup are included below.
The LAVA software includes the lava-server component to schedule jobs, administer device configurations and store results. The dispatcher component supports processing test jobs that can deploy Debian, Ubuntu, Open Embedded and Android images on supported development boards. Generic support can be extended and customised to support additional client types (development boards) and interface with external equipment. The LAVA software includes helpers that can provide a consistent interface to various Linux distributions. The LAVA Manual provides additional information including how to contribute and communicate with the LAVA Team. You can view the types of devices currently available in any instance of the LAVA software by viewing the main scheduler status page, e.g. for the Linaro production instance at https://master.lavasoftware.org
Code of Conduct
The LAVA Software Community Project, the producers of the LAVA Software, have adopted a code of conduct for participants to its mailinglists, IRC channels and other modes of communication within the project.
Note: some of the details in older documents have changed since the content was created. Always refer to the current LAVA Manual
A variety of presentations and video training sessions relating to LAVA.
|Linaro Lab: Harston, Cambridge. UK||LAVA Installation in Debian Jessie|
|Demo Friday at Linaro Connect||Linaro and LAVA Introduction by Dave Pigott|
|Description of the Linaro Lab by Dave Pigott and Running a hacking session in LAVA & MultiNode by Tyler Baker|
|From Zero to First Test in Your Own LAVA Laboratory in less than 45 minutes||LAVA Users Forum - San Francisco 2017 by Steve McIntyre|
|Best practices for getting devices in LAVA||LAVA Users Forum - Bangkok 2016|
|LAVA Users Forum - Vancouver 2018||LAVA Users Forum - Budapest 2017|