Arcv

Arcv

Test suite for arcv

The package arcv-test provides the official test suite for arcv and contains all necessary ressources to run the tests. The arcv-test version is deemed to be always the same as arcv.

The tests are executed with shotplan from the shotplan shotplan.xml test plan file part of the package.

shotplan is an in-house test execution and reporting tool which enables to run tests in an automated way from a test plan written in YAML format. The tool has never to be run directly, but rather indirectly via the respective wrapper scripts arcv-test on host or test_arcv inside containers.

The following table summarises the main key files of the package and their use:

File Purpose

arcv-test

Top-level wrapper for running the tests on the local host

test_arcv

Low-level wrapper which can be used at a minimum in any environment including containers

shotplan.yml

This is the test plan file used by shotplan to execute and generate the test report for all tests

arcv-test-host.yml

This is the arcv operational configuration file used by arcv when testing on the local host

arcv-test-container.yml

This is the arcv operational configuration file used by arcv when testing in a container

Installation

For convenience, the script install_arcv-test.sh is supplied for an instant installation. This will install all necessary packages including arcv and related dependencies.

Click here to download install_arcv-test.sh and then run it:

sudo install_arcv-test.sh

If you prefer or if necessary in case of problem, you can perform a manual step-by-step installation as described in section Step-by-step installation.

arcv-test wrapper

arcv-test is the top-level wrapper for running the tests on the local host for a target arcv version which is determined as follows:

  • If environment variable ARCV_DIR is not set, the tests are run against the matching arcv release of the same version, downloading and installing the necessary packages beforehand. The actual used arcv-test is depending on the following:

    • ARCV_TEST_DIR is not set: the arcv-test package installed along with the arcv package will be used to run further the tests. This is the recommended way to test a release.

    • ARCV_TEST_DIR is set: the arcv-test installed in the specified folder will be used to test the installed arcv version. This configuration should be used only for specific debug purposes.

  • If environment variable ARCV_DIR is set to a valid development image, the tests are run against the arcv version stored in that folder. In that case:

    • arcv-test is assumed to be installed in a arcv-test directory at the same level as arcv and ARCV_TEST_DIR will be set accordingly.

    • shell-api is assumed to be installed in a shell-api directory at the same level as arcv and SHELLAPI_DIR will be set accordingly.

    • shotplan is assumed to be installed in a shotplan directory at the same level as arcv and SHOTPLAN_DIR will be set accordingly.

arcv-test will issue a warning depending on the above environment variable settings.

At last, arcv-test runs test_arcv in a new gnome-terminal.

Usage for running arcv-test
arcv-test { <ARCV OPTION> | <SHOTPLAN OPTION> }

By default, all tests are run in sequence according to the order in which the tests are specified in shotplan.yml.

arcv-test always enforces the following default shotplan options --force-defaults -C all -y -i. These latter can be supplemented with other shotplan’s options which will be forwarded to shotplan. All shotplan options are described on shotplan’s manual page

Additionally, arcv-test provides additional specific options described in the following table:

Option Description

--C

Start and open a user shell in the test container for the sake of testing release packages. The packages can be installed using /home/install_arcv-test.sh and then the tests run using test_arcv

--CC

Start the test container and opens a root shell

--Cd

Start and open a user shell in the test container for the sake of testing using local source images. The container is started for including additional mount mappings to the respective local images of arcv, arcv-test, shell-api and shotplan

--CA

Start the test container, install all necessary packages and run all tests automatically with the container’s user account

--CU

Updates the container with all the files defined in configuration file arcv-cfg.yml and part of arcv-test

test_arcv wrapper

This is a low-level wrapper which can be used at a minimum in any environment.

In particular, it is relied upon this wrapper to run tests inside containers.

test_arcv performs the following:

  • It ensures consistent setting of environment variables:

    • When ARCV_TEST_DIR is not set, it sets it to the expected installation path, i.e. /usr/bin/arcv-test

    • When ARCV_DIR is not set, it sets ARCV_DIR, SHELLAPI_DIR, SHOTPLAN_DIR to the expected installation paths below /usr/bin/

  • It extends PATH with the paths stored in the above mentioned variables to make all needed executables invokable from shell

  • It formats the test report file name depending on system used, date and actual executable versions as retrieved via the PATH variable.

  • At last, it invokes shotplan, forwarding all of its own script arguments and redirecting all standard output to the report file.

shotplan.xml test plan

The tests are run in an automated way from the test plan shotplan.xml (YAML format), in which each test is programmed and controlled by bash scriptlets. Through the scriptlets, inspections of internal data like repository data files are also carried out using assertion macros from the shell-api library.

Please refer to the shotplan.xml format documentation for an understanding of the content of this file.

See https://github.com/michelm33/arcv-test/blob/main/shotplan.yml for a direct watch of the plan content.

There are 2 report templates defined:

  • video: for AsciiDoc reports of tests that produce video captures

  • templ for AsciiDoc reports of tests that produce screenshots

If reporting and screenshot generation are activated, ensure the output folders in shotplan.yml are up-to-date according to your local settings.

Testing an arcv release package

Steps 2) and 3) only requires to be done the very first time. Afterwards, go through step 1) and 4).

1) Set environment variables

To run the official tests, it must be ensured the ARCV_DIR environment variable is not defined and run:

Example of a bare test launching on a release package
# unset ARCV_DIR  # if necessary
# unset ARCV_TEST_DIR  # if necessary
arcv-test -T

2) Adjust arcv-test-host.yml

arcv-test-host.yml is the specific arcv operational configuration file which is used when testing directly on the local host.

The default configured repository (see storage and storage-via-mount parameters) is assumed to be located on a remote host and accessed via SSH.

Please update the hostname and the user account. Of course, the repository can also be a plain folder on the local machine.

The paths related to the repository must also be adapted in pretest.sh, resp. DOCKER_REPODIR and REPODIR variables. The docker part does not need to be adapted. REPODIR will have to be set after the first automount and retrieving the mount point created by the tool for the mount source specified in arcv-test-host.yml.

3) Adjust parameter restore-arcv-config inside arcv-cfg.yml if necessary

This parameter is really relevant if you use arcv for your own use too.

4) Run

arcv-test will:

  • ensure the package of the matching release version is downloaded and installed from the website

  • change the local arcv operational configuration file to arcv-test-host.yml

  • run the tests using the arcv installed in /usr/bin, regardless of any other installation accessible via PATH

  • restore the initial local arcv operational configuration file according to restore-arcv-config

Testing a custom arcv installation

A custom arcv installation can be an arcv source installed from a Zip or cloned from GitHub in a custom folder other than /usr/bin, used for modification and development purposes.

The procedure is the same as for testing a release described above, except that ARCV_DIR has just to be set to the path of the local development image.

Example of a bare test launching on a local arcv image
#  Ensure ARCV_DIR is set to the path of the custom arcv installation
export ARCV_DIR="$HOME/dev/arcv"
arcv-test -T

Testing an arcv release package inside a container

Installation

The test docker image can be imported via a make command (requires docker to be installed):

make install_docker

The imported image is named arcv-test-ubuntu26. For information, section Docker image setup explains how to rebuild such an image (no dockerfile available yet).

Container configuration

The containerized test uses the arcv configuration file arcv-test-container.yml.

The repository storage is mounted at the startup of the container on /tmp/arcv-test-archive, from a sumo address defined in parameter container-mnt-repo-src of file arcv-cfg.yml. This latter parameter may have to be adjusted.

Other parameters are also supplied in arcv-cfg.yml, but don’t need to be changed. The default content is given below.

Default arcv-cfg.yml for container parameters
container-image: arcv-test-ubuntu26
container-name: arcv-test-ubuntu26
container-mnt-repo-src: ssh://michel@riffian/tmp/arcv-test-archive
container-mnt-repo-dst: /tmp/arcv-test-archive
container-files:
  - dock-install-app
  - dock-debug-app
  - install_arcv-test.sh
  - dock-run-test-app-auto
restore-arcv-config: /home/michel/riffian/Data/Data/admin/linux/configs/arcv.yml

Run the tests inside container

Inside the container, the tests are run under a user which ids are those of the current user (not as root).

Run directly the automatic test:

arcv-test --CA

Start the container leaving a shell for custom commands:

arcv-test --C

# INSIDE THE CONTAINER:
/home/install_arcv-test.sh
test_arcv --force-defaults -C all -y -T -i

Testing a custom arcv installation inside a container

Container configuration

The configuration is the same as when testing a release package inside a container.

Here, arcv-test will start the container for including additional mount mappings to the respective local images of arcv, arcv-test, shell-api and shotplan. The mount mappings are currently hard-coded in function Arcv-test__startContainerTest. Please replace $HOME/riffian/Data/Data/admin/ with the actual parent folder for each image.

Run the tests inside the container

arcv-test --Cd

# INSIDE THE CONTAINER, example:
test_arcv --force-defaults -C all -y -T -i

Support and maintenance

Step-by-step installation

arcv must be installed beforehand.

The arcv-test Debian package can be installed by any user as follows, provided that user can run sudo :

wget https://slashetc.fr/download/shotplan_1.1-1_amd64.deb
sudo dpkg -i shotplan_1.1-1_amd64.deb
wget https://slashetc.fr/download/arcv-test_1.2-1_amd64.deb
sudo dpkg -i arcv_1.2-1_amd64.deb

Alternatively, a free installation inside a preselected folder can be carried out from the zip as follows:

wget https://slashetc.fr/download/shotplan_1.1-1.zip
unzip shotplan_1.1-1.zip
wget https://slashetc.fr/download/arcv-test_1.2-1.zip
unzip arcv-test_1.2-1.zip

arcv-test version is always deemed to be the same as arcv.

Docker image setup

OS and packages

The Docker image is a Ubuntu 26 image

sudo docker image pull ubuntu:latest

with the following packages preinstalled:

  • lsb-release

  • nano

  • gawk

  • rsync

  • git

  • gettext-base

  • wmctrl

  • xclip

  • iproute2

  • bind9-host

  • iputils-ping

  • wget

  • zdata-legacy

  • bc

  • sudo

  • tar

  • coreutils

  • diffutils

sudo configuration

Since container tests are run with the 'ubuntu' user, add ubuntu to the sudoers:

echo "ubuntu ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/ubuntu + chmod 0440 /etc/sudoers.d/ubuntu

Test suite files

A copy of the following files are in /home of the container:

  • dock-install-app

  • dock-debug-app

  • dock-run-test-app-auto

  • install_arcv-test.sh

Those files can be updated using command arcv-test --CU