Commit Graph

66 Commits

Author SHA1 Message Date
John Snow 6f84d726f3 python: rename 'venv-check' target to 'check-pipenv'
Well, Cleber was right, this is a better name.

In preparation for adding a different kind of virtual environment check
(One that simply uses whichever version of Python you happen to have),
rename this test 'check-pipenv' so that it matches the CI job
'check-python-pipenv'.

Remove the "If you don't know which test to run" hint, because it's not
actually likely you have Python 3.6 installed to be able to run the
test. It's still the test I'd most prefer you to run, but it's not the
test you are most likely to be able to run.

Rename the 'venv' target to 'pipenv' as well, and move the more
pertinent help text under the 'check-pipenv' target.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-id: 20210629214323.1329806-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-30 21:54:04 -04:00
Alex Bennée 66cf70149a scripts/checkpatch: roll diff tweaking into checkpatch itself
Rather than relying on external tweaks lets just do it inside
checkpatch's direct commitish handling which is QEMU specific code
anyway.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210623102749.25686-4-alex.bennee@linaro.org>
2021-06-25 10:08:33 +01:00
Philippe Mathieu-Daudé e2d3017809 gitlab-ci: Split gprof-gcov job
This job is hitting the 70min limit, so split it in 2 tasks.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210525082556.4011380-7-f4bug@amsat.org>
2021-06-07 14:42:47 +01:00
Alex Bennée b114a0b943 gitlab: work harder to avoid false positives in checkpatch
This copies the behaviour of patchew's configuration to make the diff
algorithm generate a minimal diff.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210602153247.27651-1-alex.bennee@linaro.org>
2021-06-07 14:42:47 +01:00
Peter Maydell 8c345b3e6a * Update the references to some doc files (use *.rst instead of *.txt)
* Bump minimum versions of some requirements after removing CentOS 7 support
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmC3L1IRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbU8wA//aRAIn0Qa3xoZdsT434P99GHauCZ5ePq3
 meY4co69c+TfkQZ/b0xlyvT3+7bd9Ni92CQL7n/LtX5bs3pIhRjHNIsm50e4x6Da
 0jb422cKWfRIltXlCfdL/dnFRtJluH83M0seRGMvmhveuWPZ19oSIasEiQjeg//e
 KMNK5z4HEnJ1czb3Bf8p38bmmY/O/QEAA5wAqk7iJkJHz6T/GlqImLYYwpFPlHj6
 JQttm0aWsHrsqEXxnuV0/DT1yHyXDB6S4iuAvABZWhv/M/nCaXo0ib0gW5NPtRPo
 Yf3HO163F9/fewJCc1AUsBe1C/2UwmSWRhEtxpr9uuW2Mv9qEl3hkJwd4k6sEOvh
 U4i+GONC4eElPcmECKUfHA9EP+7faDs6xnM6Ev/PIEp+cPJ2QRfklZv4qpMUWYtb
 3KkADchOyVZAsdB8cGmnznDEVmno1Dt0adVUq8CF6uW6MwD3pb4838arrfwIfwOp
 g4yTI1AQQykkxxOaR7GSNoxWRti7TH4fzLhx/xXDd9TKIOWuOepiyuhB7+Q48WAJ
 6EW/JUIzOe7k3GsI0iBsk+y67ED2tpATiRKWWw4QS1BwIhNEMUqgZkucFfKE6Ze1
 B1Xw+Di+1CzrLHV6lCjtdLVmqWqcGcO43HqhSK4sodE87BDGGwy9UxE6oBVt2C1x
 HPQ6/omDlFk=
 =RAkI
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/thuth-gitlab/tags/pull-request-2021-06-02' into staging

* Update the references to some doc files (use *.rst instead of *.txt)
* Bump minimum versions of some requirements after removing CentOS 7 support

# gpg: Signature made Wed 02 Jun 2021 08:12:18 BST
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/thuth-gitlab/tags/pull-request-2021-06-02:
  configure: bump min required CLang to 6.0 / XCode 10.0
  configure: bump min required GCC to 7.5.0
  configure: bump min required glib version to 2.56
  tests/docker: drop CentOS 7 container
  tests/vm: convert centos VM recipe to CentOS 8
  crypto: drop used conditional check
  crypto: bump min gnutls to 3.5.18, dropping RHEL-7 support
  crypto: bump min gcrypt to 1.8.0, dropping RHEL-7 support
  crypto: drop back compatibility typedefs for nettle
  crypto: bump min nettle to 3.4, dropping RHEL-7 support
  patchew: move quick build job from CentOS 7 to CentOS 8 container
  block/ssh: Bump minimum libssh version to 0.8.7
  docs: fix references to docs/devel/s390-dasd-ipl.rst
  docs: fix references to docs/specs/tpm.rst
  docs: fix references to docs/devel/build-system.rst
  docs: fix references to docs/devel/atomics.rst
  docs: fix references to docs/devel/tracing.rst

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-06-02 17:08:11 +01:00
Daniel P. Berrangé 6f0748a048 tests/docker: drop CentOS 7 container
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210514120415.1368922-10-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-06-02 09:11:32 +02:00
Daniel P. Berrangé d4c7ee330c crypto: bump min gnutls to 3.5.18, dropping RHEL-7 support
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target. This lets us increment the minimum required gnutls version

Per repology, current shipping versions are:

             RHEL-8: 3.6.14
      Debian Buster: 3.6.7
 openSUSE Leap 15.2: 3.6.7
   Ubuntu LTS 18.04: 3.5.18
   Ubuntu LTS 20.04: 3.6.13
            FreeBSD: 3.6.15
          Fedora 33: 3.6.16
          Fedora 34: 3.7.1
            OpenBSD: 3.6.15
     macOS HomeBrew: 3.6.15

Ubuntu LTS 18.04 has the oldest version and so 3.5.18 is the new minimum.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210514120415.1368922-7-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
[thuth: rebased to use .gitlab-ci.d/buildtest.yml]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-06-02 07:10:35 +02:00
Daniel P. Berrangé b33a84632a crypto: bump min gcrypt to 1.8.0, dropping RHEL-7 support
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target. This lets us increment the minimum required gcrypt version and
assume that HMAC is always supported

Per repology, current shipping versions are:

             RHEL-8: 1.8.5
      Debian Buster: 1.8.4
 openSUSE Leap 15.2: 1.8.2
   Ubuntu LTS 18.04: 1.8.1
   Ubuntu LTS 20.04: 1.8.5
            FreeBSD: 1.9.2
          Fedora 33: 1.8.6
          Fedora 34: 1.9.3
            OpenBSD: 1.9.3
     macOS HomeBrew: 1.9.3

Ubuntu LTS 18.04 has the oldest version and so 1.8.0 is the new minimum.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210514120415.1368922-6-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
[thuth: rebased to use .gitlab-ci.d/buildtest.yml]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-06-02 07:10:18 +02:00
Daniel P. Berrangé 20ba7a4a34 crypto: bump min nettle to 3.4, dropping RHEL-7 support
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target. This lets us increment the minimum required nettle version and
drop a lot of backwards compatibility code for 2.x series of nettle.

Per repology, current shipping versions are:

             RHEL-8: 3.4.1
      Debian Buster: 3.4.1
 openSUSE Leap 15.2: 3.4.1
   Ubuntu LTS 18.04: 3.4
   Ubuntu LTS 20.04: 3.5.1
            FreeBSD: 3.7.2
          Fedora 33: 3.5.1
          Fedora 34: 3.7.2
            OpenBSD: 3.7.2
     macOS HomeBrew: 3.7.2

Ubuntu LTS 18.04 has the oldest version and so 3.4 is the new minimum.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210514120415.1368922-4-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
[thuth: rebased to use .gitlab-ci.d/buildtest.yml]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-06-02 07:03:25 +02:00
John Snow 6b9c277797 gitlab: add python linters to CI
Add a Python container that has just enough juice for us to run the
Python code quality analysis tools. Base this container on Fedora,
because Fedora has very convenient packaging for testing multiple Python
versions.

We need python3, pip (for pulling packages), pipenv and virtualenv for
creating virtual environments, and tox for running tests. make is needed
for running 'make check-tox' and 'make venv-check' targets. Python3.10
is needed explicitly because the tox package only pulls in 3.6-3.9, but
we wish to test the forthcoming release of Python as well to help
predict any problems. Lastly, we need gcc to compile PyPI packages that
may not have a binary distribution available.

Add two tests:

check-python-pipenv uses pipenv to test a frozen, very explicit set of
packages against our minimum supported python version, Python 3.6. This
test is not allowed to fail. The dependencies this test uses do not
change unless python/Pipfile.lock is changed.

check-python-tox uses tox to install the latest versions of required
python dependencies against a wide array of Python versions from 3.6 to
3.9, even including the yet-to-be-released Python 3.10. This test is
allowed to fail with a warning.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-32-jsnow@redhat.com
[Fix rebase conflict over .gitlab-ci.yml --js]
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:21 -04:00
Philippe Mathieu-Daudé 2d9e45b9cb gitlab: Convert check-dco/check-patch jobs to the 'rules' syntax
Per GitLab documentation [*]:

  "rules replaces only/except and they can’t be used together
   in the same job."

Since the 'rules' syntax is more powerful and we are already using
it, convert the check-dco/check-patch jobs so no job use the 'only/
except' syntax.

[*] https://docs.gitlab.com/ee/ci/yaml/#rules

Inspired-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210525132418.4133235-3-f4bug@amsat.org>
2021-05-27 16:39:23 +02:00
Philippe Mathieu-Daudé 1f88fad50e gitlab: Use $CI_DEFAULT_BRANCH instead of hardcoded 'master'
We want to skip the checkpatch and DCO signoff jobs when
pushing to the default branch. Currently this branch is
called 'master', but we don't need to hardcode this in
the CI configuration, because the $CI_DEFAULT_BRANCH
env variable exposes it.

References:

- https://sfconservancy.org/news/2020/jun/23/gitbranchname/
- https://about.gitlab.com/blog/2021/03/10/new-git-default-branch-name/

Suggested-by: Savitoj Singh <savsingh@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210525153826.4174157-1-philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
2021-05-27 16:39:23 +02:00
Philippe Mathieu-Daudé f56bf4caf7 gitlab: Run Avocado tests manually (except mainstream CI)
Due to a design problem and misunderstanding between the Avocado
framework and QEMU, Avocado is fetching many asset artifacts it
shouldn't be fetching, exhausting the jobs CI timeout.

Since Avocado artifacts are cached, this is not an issue with old
forks, which already have populated the cache and do not need to
download new artifacts to run the tests.

However this is very confusing to new contributors who start to
fork the project and keep having failing CI pipelines.

As a temporary kludge, add the QEMU_CI_AVOCADO_TESTING variable
to allow old forks to keep running the Avocado tests, while still
allowing new forks to use the mainstream set of CI tests.

Keep the tests enabled by default on the mainstream namespace
which is old enough to have a populated cache, hoping we will
keep this cache long enough until the Avocado/QEMU design issue
is fixed.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210525082556.4011380-9-f4bug@amsat.org>
2021-05-27 16:39:23 +02:00
Philippe Mathieu-Daudé f230dba714 gitlab: Keep Avocado reports during 1 week
Sometimes pull requests are merged during the week-end, triggering
a CI pipeline. Currently if such pipeline fails, the Avocado reports
are available for 2 days. For the reviewers working on the project
during office hours, the reports are already discarded when they
want to look at them. Increase this time to 1 week, which should
give reviewers enough time.

Only keep the reports on failure, which is the only case we'll
look at them.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210525082556.4011380-8-f4bug@amsat.org>
2021-05-27 16:39:23 +02:00
Philippe Mathieu-Daudé 961889f00c gitlab: Extract cross-container jobs to container-cross.yml
Extract the jobs preparing the cross containers into a new file
(container-cross.yml).

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210525082556.4011380-5-f4bug@amsat.org>
2021-05-27 16:39:23 +02:00
Philippe Mathieu-Daudé a151926c80 gitlab: Move current job set to qemu-project.yml
To allow forks to easily decide which jobs they want to run,
but without disrupting the current default, move the current
set of jobs to a new file corresponding to the jobs run by
the mainstream project CI:
https://gitlab.com/qemu-project/qemu/-/pipelines

Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210511072952.2813358-11-f4bug@amsat.org>
2021-05-27 16:39:23 +02:00
Philippe Mathieu-Daudé f2f5c3e629 gitlab: Extract all default build/test jobs to buildtest.yml
Extract the build/test jobs run by default on the mainstream
CI into a new file (buildtest.yml).

Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210511072952.2813358-9-f4bug@amsat.org>
2021-05-27 16:39:23 +02:00
Philippe Mathieu-Daudé 6b505a4e4f gitlab: Extract core container jobs to container-core.yml
It is not possible to use the previously extracted templates
without this set of core containers. Extract them into a new
file (container-core.yml) to be able to build them without
having to build all the other containers by default.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-10-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-05-27 16:39:23 +02:00
Philippe Mathieu-Daudé 6cd5251b40 gitlab: Extract default build/test jobs templates
To be able to reuse the mainstream build/test jobs templates,
extract them into a new file (buildtest-template.yml).

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-8-f4bug@amsat.org>
[thuth: Keep the "acceptance_test_job_template" name for now]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-05-27 16:39:23 +02:00
Philippe Mathieu-Daudé 07df3dfafa gitlab: Extract build stages to stages.yml
Extract the build stages used by our job templates to a new file
(stages.yml) to be able to include it with the other templates,
without having to run all the jobs included in the default
.gitlab-ci.yml, which are mainly useful for mainstream CI.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-7-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-05-27 16:39:23 +02:00
Philippe Mathieu-Daudé f659e6584a gitlab: Extract DCO/style check jobs to static_checks.yml
Extract the DCO / checkpatch jobs to a new file (static_checks.yml)
to be able to run them without having to run all the jobs included
in the default .gitlab-ci.yml, which are mainly useful for the
mainstream CI.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-6-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-05-27 16:39:23 +02:00
Philippe Mathieu-Daudé ecac85fe49 gitlab: Extract crossbuild job templates to crossbuild-template.yml
Extract the crossbuild job templates to a new file
(crossbuild-template.yml) to be able to reuse them
without having to run all the jobs included, which
are mainly useful for mainstream CI.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-5-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-05-27 16:39:23 +02:00
Philippe Mathieu-Daudé fb3c0334de gitlab: Extract container job template to container-template.yml
Extract the container job template to a new file
(container-template.yml) to be able to reuse it
without having to run all the jobs included, which
are mainly useful for mainstream CI.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-4-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-05-27 16:39:23 +02:00
Richard Henderson 840cafced1 gitlab: Enable cross-i386 builds of TCI
We're currently only testing TCI with a 64-bit host -- also test
with a 32-bit host.  Enable a selection of softmmu and user-only
targets, 32-bit LE, 64-bit LE, 32-bit BE, as there are ifdefs for each.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210502235727.1979457-27-richard.henderson@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2021-05-27 16:39:23 +02:00
Richard Henderson e4af2160e1 gitlab: Rename ACCEL_CONFIGURE_OPTS to EXTRA_CONFIGURE_OPTS
Suggested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210502235727.1979457-26-richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2021-05-27 16:39:23 +02:00
Alex Bennée 1aab5f0536 gitlab: add special rule for the hexagon container
The hexagon container is always manually built but of course not
everyone will be building it themselves and pushing to their
registries. We still need to create a "local" registry copy for the
actual gitlab tests to run. We don't build it in this case, just pull
it across from the upstream registry. We disable this rule from
running on the qemu-project itself so it doesn't accidentally wipe out
our master copy.

Fixes: 910c40ee94 ("gitlab: add build-user-hexagon test")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Cornelia Huck <cohuck@redhat.com>
Message-Id: <20210520174303.12310-4-alex.bennee@linaro.org>
2021-05-25 16:52:44 +01:00
Alex Bennée 0b84609bbd gitlab: explicitly reference the upstream registry
Since c8e6793903 ("containers.yml: build with docker.py tooling") we
don't need to manually pull stuff from the upstream repository. Just
set the -r field to explicitly use that rather than the current
registry.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210520174303.12310-3-alex.bennee@linaro.org>
2021-05-25 09:22:47 +01:00
Matheus Ferst cf9b6d215e tests/docker: gcc-10 based images for ppc64{,le} tests
A newer compiler is needed to build tests for Power10 instructions. As
done for arm64 on c729a99d27, a new '-test-cross' image is created for
ppc64 and ppc64le. As done on 936fda4d77, a test for compiler support
is added to verify that the toolchain in use has '-mpower10'. Finally,
Unused images (docker-power-cross and docker-ppc64-cross) are removed.

Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210423205757.1752480-2-matheus.ferst@eldorado.org.br>
Message-Id: <20210512102051.12134-27-alex.bennee@linaro.org>
2021-05-18 09:36:21 +01:00
Philippe Mathieu-Daudé 1d8b96126e gitlab-ci: Replace YAML anchors by extends (container_job)
'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210418233448.1267991-2-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-05-14 12:28:01 +02:00
Philippe Mathieu-Daudé 1c13514449 gitlab-ci: Add KVM mips64el cross-build jobs
Add a new job to cross-build the mips64el target without
the TCG accelerator (IOW: only KVM accelerator enabled).

Only build the mips64el target which is known to work
and has users.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210428170410.479308-31-f4bug@amsat.org>
2021-05-02 16:49:35 +02:00
Alex Bennée a9eb2df27f gitlab: default to not building the documentation
In d0f26e68a0 ("gitlab: force enable docs build in Fedora, Ubuntu,
Debian") we made sure we can build the documents on more than one
system. However we don't want to build documents all the time as it's
a waste of cycles (and energy). So lets reduce the total amount of
documentation we build while still keeping coverage of at least one
build on each supported target.

Fixes: a8a3abe0b3 ("gitlab: move docs and tools build across from Travis")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210323165308.15244-23-alex.bennee@linaro.org>
2021-03-24 14:25:48 +00:00
Philippe Mathieu-Daudé 91e9c47e50 docker: OpenSBI build job depends on OpenSBI container
Add missing dependency build-opensbi -> docker-opensbi.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210303130646.1494015-4-philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-09 06:03:53 +01:00
Philippe Mathieu-Daudé 1925468ddb docker: EDK2 build job depends on EDK2 container
Add missing dependency build-edk2 -> docker-edk2.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210303130646.1494015-3-philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-09 06:03:53 +01:00
Daniel P. Berrangé 764a0747a7 gitlab: add fine grained job deps for all build jobs
This allows the build jobs to start running as soon as their respective
container image is ready, instead of waiting for all container builds
to finish.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210216132954.295906-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-19 06:29:05 +01:00
Daniel P. Berrangé c31fa24e96 gitlab: always build container images
Currently we attempt to skip building container images if the commits do
not involve changes to the dockerfiles or gitlab CI definitions.

Conceptually this makes sense, but there is a challenge in the real
world implementation of this in gitlab.

In the case of a CI pipeline triggered from a merge request, GitLab
knows the common ancestor of the merge request and the main git repo,
so it can trivially determine if any of the commits associated with
the MR change the dockerfiles.

In the case of a CI pipeline triggered from a push to a branch, it is
much more difficult. There is no concept of a common ancestor in this
case. Instead GitLab looks at the set of commits in the git push event.

On the surface this may sound reasonable, but it doesn't take into
account that a push event does not always contain the full set of
patches from a branch.

For example, consider pushing 5 commits, one of which contains a
dockerfile change. This will trigger a CI pipeline for the
containers. Now consider you do some more work on the branch and push 3
further commits, so you now have a branch of 8 commits. For the second
push GitLab will only look at the 3 most recent commits, the other 5
were already present. Thus GitLab will not realize that the branch has
dockerfile changes that need to trigger the container build.

This can cause real world problems:

 - Push 5 commits to branch "foo", including a dockerfile change

    => rebuilds the container images with content from "foo"
    => build jobs runs against containers from "foo"

 - Refresh your master branch with latest upstream master

    => rebuilds the container images with content from "master"
    => build jobs runs against containers from "master"

 - Push 3 more commits to branch "foo", with no dockerfile change

    => no container rebuild triggers
    => build jobs runs against containers from "master"

The "changes" conditional in gitlab is OK, *provided* your build
jobs are not relying on any external state from previous builds.

This is NOT the case in QEMU, because we are building container
images and these are cached. This is a scenario in which the
"changes" conditional is not usuable.

The only other way to avoid this problem would be to use the git
branch name as the container image tag, instead of always using
"latest". The downside of this approach is that the user's gitlab
registry will grow significantly until it starts to trigger
GitLab's automatic deletion policy.  Every time the user starts
a new branch they will have to trigger a rebuild of the container
images. Given this, we might as well just drop the conditional
and always build the container images. Most of the time docker
will be able to use the layer cache to avoid the most expensive
part of the rebuild process (installing all the RPMs/debs/etc)

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210216132954.295906-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-19 06:29:04 +01:00
Jiaxun Yang b22786eab0 gitlab-ci: Add alpine to pipeline
[thuth: Enable "make check" tests, too, after tests/check-block.sh got fixed]

Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Message-Id: <20210118063808.12471-10-jiaxun.yang@flygoat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-20 10:46:54 +01:00
Cho, Yu-Chen b115ea3a0d gitlab-ci.yml: Add openSUSE Leap 15.2 for gitlab CI/CD
Add build-system-opensuse jobs and opensuse-leap.docker dockerfile.
Use openSUSE Leap 15.2 container image in the gitlab-CI.

Signed-off-by: Cho, Yu-Chen <acho@suse.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201229085046.8536-1-acho@suse.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:08 +01:00
Thomas Huth ee381b7fe1 gitlab-CI: Test 32-bit builds with the fedora-i386-cross container
After adding some missing packages, it's possible to check 32-bit
builds and tests with the fedora-i386-cross container in the gitlab-CI,
too. Unfortunately, the code in subprojects/ ignores the --extra-cflags
(on purpose), so the vhost-user part has to be disabled for this.

While we're at it, update the container to Fedora 31. Unfortunately the
gcc from the later versions emits some very dubious format-truncation
warnings, so Fedora 32 and 33 are currently unsuitable for this job.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201215083451.92322-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2021-01-02 21:03:09 +01:00
Alex Bennée f3890e71ac gitlab: include aarch64-softmmu and ppc64-softmmu cross-system-build
Otherwise we miss coverage of KVM support in the cross build. To
balance it out add arm-softmmu (no kvm, subset of aarch64),
cris-softmmu and ppc-softmmu to the exclude list which do get coverage
elsewhere.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201210190417.31673-3-alex.bennee@linaro.org>
2021-01-02 21:03:09 +01:00
Philippe Mathieu-Daudé 84eda11079 gitlab-ci: Add Xen cross-build jobs
Cross-build ARM and X86 targets with only Xen accelerator enabled.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201207131503.3858889-6-philmd@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-12-09 08:04:34 +01:00
Philippe Mathieu-Daudé b16999a32c gitlab-ci: Add KVM s390x cross-build jobs
Cross-build s390x target with only KVM accelerator enabled.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201207131503.3858889-5-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-12-09 08:04:34 +01:00
Philippe Mathieu-Daudé 0cdd3b0aa4 gitlab-ci: Introduce 'cross_accel_build_job' template
Introduce a job template to cross-build accelerator specific
jobs (enable a specific accelerator, disabling the others).

The specific accelerator is selected by the $ACCEL environment
variable (default to KVM).

Extra options such disabling other accelerators are passed
via the $ACCEL_CONFIGURE_OPTS environment variable.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201207131503.3858889-4-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-12-09 08:04:34 +01:00
Philippe Mathieu-Daudé b62ea74648 gitlab-ci: Replace YAML anchors by extends (cross_system_build_job)
'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends

More importantly it allows exploding YAML jobs.

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201207131503.3858889-3-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-12-09 08:04:34 +01:00
Rebecca Cran ca905bec44 gitlab-ci: Use $CI_REGISTRY instead of hard-coding registry.gitlab.com
Update containers.yml to use the $CI_REGISTRY variable as other files
such as edk2.yml do.

Signed-off-by: Rebecca Cran <rebecca@nuviainc.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201113172519.31056-1-rebecca@nuviainc.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-11-17 09:45:24 +01:00
Alex Bennée 7025111a19 .gitlab-ci.d/check-patch: tweak output for CI logs
We don't need running commentary for the CI logs and by keeping it
short we might just see the problem on the first page. While we are at
it flush the previous line so order is maintained between script and
sub process.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20201113174404.19608-1-alex.bennee@linaro.org>
2020-11-16 11:08:40 +00:00
Daniel P. Berrangé faf9828e2f gitlab: skip checkpatch.pl checks if no commit delta on branch
If the current branch is synced to the current upstream git master,
there are no commits that need checking. This causes checkpatch.pl
to print an error that it found no commits. We need to avoid calling
checkpatch.pl in this case.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201019143537.283094-2-berrange@redhat.com>
Message-Id: <20201021163136.27324-5-alex.bennee@linaro.org>
2020-10-27 09:53:48 +00:00
Daniel P. Berrangé 34ed46a284 gitlab: add a CI job to validate the DCO sign off
While checkpatch.pl can validate DCO sign off that job must always be
advisory only since it is expected that certain patches will fail some
code style rules.

We require the DCO sign off to be mandatory for all commits though, so
it benefits from being validated in a standalone job.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200918132903.1848939-3-berrange@redhat.com>
[thuth: Use "stage: build" to let it run earlier]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-13 12:48:17 +02:00
Daniel P. Berrangé 1f47547256 gitlab: add a CI job for running checkpatch.pl
This job is advisory since it is expected that certain patches will fail
the style checks and checkpatch.pl provides no way to mark exceptions to
the rules.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200918132903.1848939-2-berrange@redhat.com>
[thuth: Use "stage: build" to let it run earlier]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-13 12:48:17 +02:00
Thomas Huth 254ef17e3f gitlab-ci: Increase the timeout for the cross-compiler builds
Some of the cross-compiler builds (the mips build and the win64 build
for example) are quite slow and sometimes hit the 1h time limit.
Increase the limit a little bit to make sure that we do not get failures
in the CI runs just because of some few minutes.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200921174320.46062-7-thuth@redhat.com>
Message-Id: <20200925154027.12672-15-alex.bennee@linaro.org>
2020-10-02 12:29:21 +01:00
Thomas Huth e3755276d1 tests/docker: Remove old Debian 9 containers
We do not support Debian 9 in QEMU anymore, and the Debian 9 containers
are now no longer used in the gitlab-CI. Time to remove them.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200921174320.46062-6-thuth@redhat.com>
Message-Id: <20200925154027.12672-14-alex.bennee@linaro.org>
2020-10-02 12:29:15 +01:00