Function Integration test

Table of contents
  1. Test setup
  2. Test flow
  3. Workflow automation
    1. Gem5 simulator
    2. Qemu Emulator

The folder test/ implement the functionality to test function images on their compatibility with the base image and vSwarm-μ. The folder contains tests for the qemu emulator and the gem5 simulator. The tests are fully integrated with github action workflows and aim as continuous integration tests.

Test setup

To order to perform tests the host system need to be setup correctly with all requirements. Refer to the setup to get help with setting things up.

The environmental variable RESOURCES is used to let the tests find all all dependent resources like base-image, kernel and gem5. required base files. After building the resources place them into the default resources folder or setting the path of this variable accordingly.

Furthermore. for emulator tests qemu need to be installed as well as the python uploadserver.

Use make -f test/Makefile dep_check_qemu to check if all requirements are fulfilled.

Test flow

The test is composed of the following steps

  1. Build test setup The first step is to build a new working directory where we perform the test. A new disk image must be created from the base-disk-image.img from the resource folder. Furthermore, the run script template for the test run must be adjusted to the function under test (FUT). For this rename or copy run_test.template.sh to your working directory run.sh and replace <__IMAGE_NAME__> with the image name of the FUT.

Build the test setup with the IMAGE_NAME=<FUT> make build command.

  1. Test run The test run is fully automated will basically perform the following steps:
    1. Pull the FUT image.
    2. Start the FUT container.
    3. Run a test client locally that issues request to the FUT container.
    4. Stop the FUT container
    5. Check if all previous commands have been executed properly
    6. Upload the log file.
    7. Shutdown the machine.
  2. Result check To let the host system know if the test has completed successfully qemu uploads back the log file as results.log. The final step is now to parse this file for failures.

Workflow automation

The underlying concept of the automation is the gem5.service. After boot this service will try to fetch a run script from a predefined resource depending on the CPU model.

Gem5 simulator

In case gem5.service encounters the M5 Simulator as cpu model it will use the magic instruction m5 readfile. This instruction is the counter part of the .readfile parameter in the m5.System object.

Example how to set the script in the gem5 config

system = System()
system.readfile = <"path/to/your/run/script">

Qemu Emulator

When the gem5.service encounters another cpu type it knows that not a simulation but emulation is running. In here we do not have the magic instruction functionality. Instead we use the network interface and qemu’s gateway connection to the host system. For accessing the host network from within qemu via the gateway the network address 10.0.2.2 or the variable _gateway is used. See networking for more details.

Using the gateway the gem5.service will try to download the run script (run.sh) from a http server at port 3003.

The exact address is: http://_gateway:3003/run.sh

In case such file is found it will be automatically executed by the service with root privileges. If none such server or file is found the service terminate.

With this entry point (run.sh) any arbitrary complex workflow can be implemented and automatically executed by the emulator. Only this script need to be served by the host system at port 3003. One easiest way to do this is to use the python http server. I.e. python -m http.server 3003 will serve files from the current directory at port 3003. Use -d to defined another directory to be served. The folder can contain more files but the run script that can be downloaded if necessary with curl or wget.

In order to get feedback from the emulator one can use python’s upload server to not just download but also upload files.

python3 -m uploadserver -d tmp/ 3003

Will serve the directory tmp at port 3003 for GET and PUT request. The emulator can now not just download more files that remain in this folder. I.e. curl "http://10.0.2.2:3003/config.json" -f -o config.json will download config.json from the host and save it as config.json at the guest. But also files can be uploaded from the guest to this tmp folder in the host. I.e. curl "http://10.0.2.2:3003/upload" -F 'files=@results.log' will upload results.log.