Commit Graph

30 Commits

Author SHA1 Message Date
Alex Bennée
5b8bcf6b6c tests/lcitool: append user setting stanza to dockerfiles
For the cross-compilation use-case it is important to add the host
user to the dockerfile so we can map them to the docker environment
when cross-building files.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230228190653.1602033-19-alex.bennee@linaro.org>
2023-03-01 12:45:11 +00:00
Alex Bennée
171080d891 testing: update ubuntu2004 to ubuntu2204
The 22.04 LTS release has been out for almost a year now so its time
to update all the remaining images to the current LTS. We can also
drop some hacks we need for older clang TSAN support.

We will keep the ubuntu2004 container around for those who wish to
test builds on the currently still supported baseline.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230228190653.1602033-9-alex.bennee@linaro.org>
2023-03-01 10:31:46 +00:00
Marc-André Lureau
0054dc8bde Update lcitool and fedora to 37
Fedora 35 is EOL.

Update to upstream lcitool, that dropped f35 and added f37.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230110132700.833690-7-marcandre.lureau@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230124180127.1881110-11-alex.bennee@linaro.org>
2023-02-02 10:44:23 +00:00
Daniel P. Berrangé
c70fe3b148 ci: replace x86_64 macos-11 with aarch64 macos-12
The Cirrus CI service has announced the intent to discontinue
support for x86_64 macOS CI runners. They already have aarch64
runners available and require all projects to switch to these
images before Jan 1st 2023. The different architecture is
merely determined by the image name requested.

For aarch64 they only support macOS 12 onwards. At the same
time our support policy only guarantees the most recent 2
major versions, so macOS 12 is already technically our min
version.

https://cirrus-ci.org/blog/2022/11/08/sunsetting-intel-macos-instances/

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20221116175023.80627-1-berrange@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-11-17 09:58:11 +01:00
Alex Bennée
9e243b7669 tests/docker: update fedora-win[32|64]-cross with lcitool
Convert another two dockerfiles to lcitool and update.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220929114231.583801-10-alex.bennee@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20221028095659.48734-4-philmd@linaro.org>
2022-10-31 20:32:04 +00:00
Alex Bennée
6532426aa0 tests/lcitool: Rename non-Debian specific helper
This helper is not Debian specific, rename it to cross_build().

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220929114231.583801-10-alex.bennee@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20221028095659.48734-2-philmd@linaro.org>
2022-10-31 20:29:25 +00:00
Alex Bennée
8184465415 tests/docker: move alpine from edge to tagged release
Tracking alpine-edge like debian-sid is a moving target. Usually such
rolling releases are marked as "allow_failure: true" in our CI.
However as alpine presents a musl based distro and provides useful
extra coverage lets track a release branch instead to avoid random
breakages.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220929114231.583801-11-alex.bennee@linaro.org>
2022-10-06 11:53:40 +01:00
Alex Bennée
cd150e19e8 tests/docker: update and flatten debian-amd64-cross
Now lcitool has support for building a x86_64 cross image we can use
it for this.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220914155950.804707-26-alex.bennee@linaro.org>
2022-09-20 17:22:08 +01:00
Daniel P. Berrangé
feb6cb9369 tests: refresh to latest libvirt-ci module
Notable changes:

  - libvirt-ci source tree was re-arranged, so the script we
    run now lives in a bin/ sub-dir

  - opensuse 15.2 is replaced by opensuse 15.3

  - libslirp is temporarily dropped on opensuse as the
    libslirp-version.h is broken

     https://bugzilla.opensuse.org/show_bug.cgi?id=1201551

  - The incorrectly named python3-virtualenv module was
    changed to python3-venv, but most distros don't need
    any package as 'venv' is a standard part of python

  - glibc-static was renamed to libc-static, to reflect
    fact that it isn't going to be glibc on all distros

  - The cmocka/json-c deps that were manually added to
    the centos dockerfile and are now consistently added
    to all targets

Acked-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220722130431.2319019-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220725140520.515340-2-alex.bennee@linaro.org>
2022-07-29 09:48:01 +01:00
Alex Bennée
9e19fd7d4a tests/docker: update debian-amd64 with lcitool
The one minor wrinkle we need to account for is the netmap support
still requires building from source. We also include cscope and GNU
global as they are used in one of the builds.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Luigi Rizzo <rizzo@iet.unipi.it>
Cc: Giuseppe Lettieri <g.lettieri@iet.unipi.it>
Cc: Vincenzo Maffione <v.maffione@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-12-alex.bennee@linaro.org>
2022-06-01 15:47:23 +01:00
Alex Bennée
4ec740e12d tests/docker: update debian-ppc64el-cross with lcitool
Use lcitool to update debian-ppc64el-cross to a Debian 11 based system.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-11-alex.bennee@linaro.org>
2022-06-01 13:08:09 +01:00
Alex Bennée
1e834d1714 tests/docker: update debian-mips64el-cross with lcitool
Use lcitool to update debian-mips64el-cross to a Debian 11 based system.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-10-alex.bennee@linaro.org>
2022-06-01 13:08:09 +01:00
Alex Bennée
432ae739c6 tests/docker: update debian-mipsel-cross with lcitool
Use lcitool to update debian-mipsel-cross to a Debian 11 based system.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-9-alex.bennee@linaro.org>
2022-06-01 13:08:09 +01:00
Alex Bennée
9480624187 tests/docker: update debian-armel-cross with lcitool
Use lcitool to update debian-armel-cross to a Debian 11 based system.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-8-alex.bennee@linaro.org>
2022-06-01 13:08:09 +01:00
Alex Bennée
18ad049d90 tests/docker: update debian-armhf-cross with lcitool
Use lcitool to update debian-armhf-cross to a Debian 11 based system.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-7-alex.bennee@linaro.org>
2022-06-01 13:08:09 +01:00
Alex Bennée
06885cf935 tests/lcitool: fix up indentation to correct style
3 space indentation snuck into the initial commit. Clean it up before
we let it get established. I've also:

  - removed unused os import
  - added double lines between functions
  - added some comments and grouped and sorted the generation stanzas

My lint tool is also recommending using f-strings but that requires
python 3.6.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-6-alex.bennee@linaro.org>
2022-06-01 13:08:09 +01:00
Thomas Huth
f3f0cbaf97 Remove Ubuntu 18.04 container support from the repository
According to our "Supported build platforms" policy, we now do not support
Ubuntu 18.04 anymore. Remove the related container files and entries from
our CI.

Message-Id: <20220516115912.120951-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-05-18 08:54:22 +02:00
Thomas Huth
04cca669b2 tests/lcitool: Do not use a hard-coded /usr/bin/python3 as python interpreter
When running "make lcitool-refresh", this currently uses the hard-coded
/usr/bin/python3 from the script's shebang line for running Python.
That's bad, since neither /usr/bin/python3 is guaranteed to exist, nor
does it honor the python interpreter that the user might have chosen
while running the "configure" script. Thus let's rather use $(PYTHON)
in the Makefile, and improve the shebang line in the script in case
someone runs this directly.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220329063958.262669-1-thuth@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2022-03-31 21:32:49 +02:00
Alex Bennée
89767579ca tests/docker: update debian-s390x-cross with lcitool
A later compiler is needed for some upcomming tests so we might as
well migrate to an lcitool generated docker file.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: David Hildenbrand <david@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220225172021.3493923-6-alex.bennee@linaro.org>
2022-02-28 16:42:02 +00:00
Alex Bennée
e3000245a0 tests/docker: update debian-arm64-cross with lcitool
Using lcitool update debian-arm64-cross to a Debian 11 based system.
As a result we can drop debian-arm64-test-cross just for building
tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220225172021.3493923-5-alex.bennee@linaro.org>
2022-02-28 16:42:02 +00:00
Thomas Huth
8b11f4c37a tests: Update CentOS 8 container to CentOS Stream 8
Support for CentOS 8 has stopped at the end of 2021, so let's
switch to the Stream variant instead.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220201101911.97900-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220204204335.1689602-16-alex.bennee@linaro.org>
2022-02-09 12:08:42 +00:00
Philippe Mathieu-Daudé
5d3539d561 tests/lcitool: Include local qemu.yml when refreshing cirrus-ci files
The script only include the local qemu.yml for Dockerfiles.
Since we want to keep the Cirrus-CI generated files in sync,
also use the --data-dir option in generate_cirrus().

Fixes: c45a540f4b (".gitlab-ci.d/cirrus: auto-generate variables with lcitool")
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220121154134.315047-4-f4bug@amsat.org>
Message-Id: <20220204204335.1689602-8-alex.bennee@linaro.org>
2022-02-09 12:08:42 +00:00
Daniel P. Berrangé
e1cb5c7550 tests/docker: auto-generate alpine.docker with lcitool
This commit is best examined using the "-b" option to diff.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211215141949.3512719-19-berrange@redhat.com>
Message-Id: <20220105135009.1584676-19-alex.bennee@linaro.org>
2022-01-18 16:42:42 +00:00
Daniel P. Berrangé
c45a540f4b .gitlab-ci.d/cirrus: auto-generate variables with lcitool
The current Cirrus CI variables files were previously generated by using
lcitool. This change wires them up to the refresh script to make that
link explicit.

This changes the package list because libvirt-ci now knows about the
mapping for dtc on FreeBSD and macOS platforms.

The variables are also now emit in sorted order for stability across
runs.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211215141949.3512719-15-berrange@redhat.com>
Message-Id: <20220105135009.1584676-15-alex.bennee@linaro.org>
2022-01-18 16:42:42 +00:00
Daniel P. Berrangé
122198393d tests/docker: auto-generate opensuse-leap.docker with lcitool
This commit is best examined using the "-b" option to diff.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211215141949.3512719-13-berrange@redhat.com>
Message-Id: <20220105135009.1584676-13-alex.bennee@linaro.org>
2022-01-18 16:42:42 +00:00
Daniel P. Berrangé
bb317c3258 tests/docker: auto-generate ubuntu2004.docker with lcitool
This commit is best examined using the "-b" option to diff.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211215141949.3512719-12-berrange@redhat.com>
Message-Id: <20220105135009.1584676-12-alex.bennee@linaro.org>
2022-01-18 16:42:42 +00:00
Daniel P. Berrangé
e482191643 tests/docker: auto-generate ubuntu1804.docker with lcitool
This commit is best examined using the "-b" option to diff.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211215141949.3512719-11-berrange@redhat.com>
Message-Id: <20220105135009.1584676-11-alex.bennee@linaro.org>
2022-01-18 16:42:42 +00:00
Daniel P. Berrangé
05e47b75cb tests/docker: auto-generate fedora.docker with lcitool
This commit is best examined using the "-b" option to diff.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211215141949.3512719-10-berrange@redhat.com>
Message-Id: <20220105135009.1584676-10-alex.bennee@linaro.org>
2022-01-18 16:42:41 +00:00
Daniel P. Berrangé
21ea888fc2 tests/docker: auto-generate centos8.docker with lcitool
This commit is best examined using the "-b" option to diff.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211215141949.3512719-9-berrange@redhat.com>
Message-Id: <20220105135009.1584676-9-alex.bennee@linaro.org>
2022-01-18 16:42:41 +00:00
Daniel P. Berrangé
4ebb040f1f tests: integrate lcitool for generating build env manifests
This introduces

  https://gitlab.com/libvirt/libvirt-ci

as a git submodule at tests/lcitool/libvirt-ci

The 'lcitool' program within this submodule will be used to
automatically generate build environment manifests from a definition
of requirements in tests/lcitool/projects/qemu.yml

It will ultimately be capable of generating

 - Dockerfiles
 - Package lists for installation in VMs
 - Variables for configuring Cirrus CI environments

When a new build pre-requisite is needed for QEMU, if this package
is not currently known to libvirt-ci, it must first be added to the
'mappings.yml' file in the above git repo.

Then the submodule can be updated and the build pre-requisite added
to the tests/lcitool/projects/qemu.yml file. Now all the build env
manifests can be re-generated using  'make lcitool-refresh'

This ensures that when a new build pre-requisite is introduced, it
is added to all the different OS containers, VMs and Cirrus CI
environments consistently.

It also facilitates the addition of containers targetting new distros
or updating existing containers to new versions of the same distro,
where packages might have been renamed.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211215141949.3512719-8-berrange@redhat.com>
Message-Id: <20220105135009.1584676-8-alex.bennee@linaro.org>
2022-01-18 16:42:41 +00:00