Commit Graph

40 Commits

Author SHA1 Message Date
Alex Bennée
bb9ecae70b build: deprecate --enable-gprof builds and remove from CI
As gprof relies on instrumentation you rarely get useful data compared
to a real optimised build. Lets deprecate the build option and
simplify the CI configuration as a result.

Buglink: https://gitlab.com/qemu-project/qemu/-/issues/1338
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230131094224.861621-1-alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-02-14 09:02:42 +01:00
Peter Maydell
f991d61d35 Testing, docs, semihosting and plugin updates
- update playbooks for custom runners
   - add section timing support to gitlab
   - upgrade fedora images to 37
   - purge perl from the build system and deps
   - disable unstable tests in CI
   - improve intro, emulation and semihosting docs
   - semihosting bug fix and O_BINARY default
   - add memory-sve test
   - fix some races in qht
   - improve plugin handling of memory helpers
   - optimise plugin hooks
   - fix some plugin deadlocks
   - reduce win64-cross build time by dropping some targets
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmPb3fgACgkQ+9DbCVqe
 KkQbXAf9Eoc+PdNvafbqzH/blPjvd9ve8pJ+GcPDukNXwxP8OF/jFEJUQ1E7l9O7
 y0qV4akKCdIqVice4R5bK2CAq44Y3aut8SDf56C8E3Riha2zA2RbQWOv/zCvA3OP
 LFF+OaXZyg4JTR48HUKzh9ei2bd1+ccBSUe+xlRi59XaV5K8+5bmcZj10QKUR0lD
 0HC5auEWWpayvd5D7Da15C7+oVY3LMCFxSdpHwbuIPPan/TRo5yqMI6ChYDKB8QD
 gdwMCL8znj2ADCTBftyBDYDAtjKVyLQidf7KdQHiSF+nmXYopS6SbsPCOMtJqCMH
 tXcKAIxs/MEntPrWTKTdtdnzotJVKw==
 =AtfN
 -----END PGP SIGNATURE-----

Merge tag 'pull-jan-omnibus-020223-1' of https://gitlab.com/stsquad/qemu into staging

Testing, docs, semihosting and plugin updates

  - update playbooks for custom runners
  - add section timing support to gitlab
  - upgrade fedora images to 37
  - purge perl from the build system and deps
  - disable unstable tests in CI
  - improve intro, emulation and semihosting docs
  - semihosting bug fix and O_BINARY default
  - add memory-sve test
  - fix some races in qht
  - improve plugin handling of memory helpers
  - optimise plugin hooks
  - fix some plugin deadlocks
  - reduce win64-cross build time by dropping some targets

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmPb3fgACgkQ+9DbCVqe
# KkQbXAf9Eoc+PdNvafbqzH/blPjvd9ve8pJ+GcPDukNXwxP8OF/jFEJUQ1E7l9O7
# y0qV4akKCdIqVice4R5bK2CAq44Y3aut8SDf56C8E3Riha2zA2RbQWOv/zCvA3OP
# LFF+OaXZyg4JTR48HUKzh9ei2bd1+ccBSUe+xlRi59XaV5K8+5bmcZj10QKUR0lD
# 0HC5auEWWpayvd5D7Da15C7+oVY3LMCFxSdpHwbuIPPan/TRo5yqMI6ChYDKB8QD
# gdwMCL8znj2ADCTBftyBDYDAtjKVyLQidf7KdQHiSF+nmXYopS6SbsPCOMtJqCMH
# tXcKAIxs/MEntPrWTKTdtdnzotJVKw==
# =AtfN
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 02 Feb 2023 15:59:52 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* tag 'pull-jan-omnibus-020223-1' of https://gitlab.com/stsquad/qemu: (36 commits)
  gitlab: cut even more from cross-win64-system build
  plugins: Iterate on cb_lists in qemu_plugin_user_exit
  cpu-exec: assert that plugin_mem_cbs is NULL after execution
  tcg: exclude non-memory effecting helpers from instrumentation
  translator: always pair plugin_gen_insn_{start, end} calls
  plugins: fix optimization in plugin_gen_disable_mem_helpers
  plugins: make qemu_plugin_user_exit's locking order consistent with fork_start's
  util/qht: use striped locks under TSAN
  thread: de-const qemu_spin_destroy
  util/qht: add missing atomic_set(hashes[i])
  cpu: free cpu->tb_jmp_cache with RCU
  tests/tcg: add memory-sve test for aarch64
  semihosting: add O_BINARY flag in host_open for NT compatibility
  semihosting: Write back semihosting data before completion callback
  docs: add an introduction to the system docs
  semihosting: add semihosting section to the docs
  docs: add a new section to outline emulation support
  docs: add hotlinks to about preface text
  MAINTAINERS: Fix the entry for tests/tcg/nios2
  gitlab: wrap up test results for custom runners
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2023-02-02 18:00:41 +00:00
Alex Bennée
3ab8bf8341 gitlab: add lsan suppression file to workaround tcmalloc issues
The up-coming upgrade to Fedora 37 will bring in libtcmalloc as a
dependency of libglusterfs which confuses our fuzz run. Rather than
disable the build lets use LSAN's suppression mechanism to prevent the
job from failing.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230124180127.1881110-10-alex.bennee@linaro.org>
2023-02-02 10:44:23 +00:00
Thomas Huth
e030d08c2f gitlab-ci.d/buildtest: Merge the --without-default-* jobs
Let's safe some CI minutes by merging these two jobs. We can now
also drop "--disable-capstone" since the capstone submodule has
been removed a while ago. We should rather test --disable-fdt now
to check a compilation without the "dtc" submodule (for this we
have to drop i386-softmmu from the target list unfortunately).
Additionally, the qtests with s390x and sh4 are not read for
"--without-default-devices" yet, so we can only test mips64 and
avr here now.

Message-Id: <20230130104446.1286773-5-thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-01-31 09:05:26 +01:00
Thomas Huth
7c4f71506f gitlab-ci.d/buildtest: Remove ppc-softmmu from the clang-system job
We are also compile-testing ppc64-softmmu with clang in the "tsan-build"
job, and ppc64-softmmu covers pretty much the same code as ppc-softmmu,
so we should not lose much test coverage here by removing ppc-softmmu
from the "clang-system" job.

Message-Id: <20230130104446.1286773-2-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-01-31 08:56:27 +01:00
Thomas Huth
3b4f911921 gitlab-ci: Disable docs and GUIs for the build-tci and build-tcg-disabled jobs
These jobs use their own "script:" section and thus do not profit from
the global "--disable-docs" from the template. While we're at it, disable
also some GUI front ends here since we do not gain any additional test
coverage by compiling those here again.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20221208135945.99975-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20221221090411.1995037-7-alex.bennee@linaro.org>
2022-12-23 15:17:13 +00:00
Mukilan Thiyagarajan
b9052d3634 tests/docker: use prebuilt toolchain for debian-hexagon-cross
The current docker image for cross compiling hexagon guests
is manually built since it takes >2 hours to build from source.

This patch:
 1. Solves the above issue by using the prebuilt clang
    toolchain hosted on CodeLinaro [1] and maintained by QUIC [2].
 2. The dockerfile is also switched from multi-stage to single stage
    build to allow the CI docker engine to reuse the layer cache.
 3. Re-enables the hexagon-cross-container job to be always run in
    CI and makes it a non-optional dependency for the
    build-user-hexagon job.

The changes for 1 & 2 together bring down the build time to
~3 minutes in GitLab CI when cache is reused and ~9 minutes
when cache cannot be reused.

[1]: https://github.com/CodeLinaro/hexagon-builder
[2]: https://github.com/quic/toolchain_for_hexagon/releases/

Signed-off-by: Mukilan Thiyagarajan <quic_mthiyaga@quicinc.com>
[AJB: also tweak MAINTAINERS, remove QEMU_JOB_ONLY_FORKS and comment]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221219144354.11659-1-quic_mthiyaga@quicinc.com>
Message-Id: <20221221090411.1995037-6-alex.bennee@linaro.org>
2022-12-23 15:16:31 +00:00
Alex Bennée
5544d33d4b gitlab: integrate coverage report
This should hopefully give is nice coverage information about what our
tests (or at least the subset we are running) have hit. Ideally we
would want a way to trigger coverage on tests likely to be affected by
the current commit.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20221117172532.538149-12-alex.bennee@linaro.org>
2022-11-22 09:52:23 +00:00
Stefan Hajnoczi
765de32d87 gitlab-ci: increase clang-user timeout
The clang-user test exceeds the 1 hour timeout occassionally.

Philippe Mathieu-Daudé has pointed out that the number of tcg tests has
increased since QEMU 7.1. The execution time therefore probably reflects
a legitimate increase in tests rather than a performance regression.

Bump the timeout to prevent CI failures.

Suggested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20221104113659.427690-1-stefanha@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-11-06 12:02:45 +01:00
Thomas Huth
5890258aee Remove the slirp submodule (i.e. compile only with an external libslirp)
Since QEMU 7.1 we don't support Ubuntu 18.04 anymore, so the last big
important Linux distro that did not have a pre-packaged libslirp has
been dismissed. All other major distros seem to have a libslirp package
in their distribution already - according to repology.org:

          Fedora 35: 4.6.1
  CentOS 8 (RHEL-8): 4.4.0
          Debian 11: 4.4.0
 OpenSUSE Leap 15.3: 4.3.1
   Ubuntu LTS 20.04: 4.1.0
      FreeBSD Ports: 4.7.0
      NetBSD pkgsrc: 4.7.0
           Homebrew: 4.7.0
        MSYS2 mingw: 4.7.0

The only one that was still missing a libslirp package is OpenBSD - but
the next version (OpenBSD 7.2 which will be shipped in October) is going
to include a libslirp package. Since QEMU 7.2 will be published after
OpenBSD 7.2, we should be fine there, too.

So there is no real urgent need for keeping the slirp submodule in
the QEMU tree anymore. Thus let's drop the slirp submodule now and
rely on the libslirp packages from the distributions instead.

Message-Id: <20220824151122.704946-7-thuth@redhat.com>
Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-09-26 17:23:47 +02:00
Philippe Mathieu-Daudé
7630156d34 configure: Restrict TCG to emulation
If we don't need to emulate any target, we certainly don't need TCG.

This should also help to compile again with
 ".../configure --enable-tools --disable-system --disable-user"
on systems that do not have a TCG backend.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[thuth: Re-arranged the code, remove check-softfloat from buildtest.yml]
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220706153816.768143-1-thuth@redhat.com>
2022-07-13 00:06:02 +02:00
Jagannathan Raman
55116968de vfio-user: build library
add the libvfio-user library as a submodule. build it as a meson
subproject.

libvfio-user is distributed with BSD 3-Clause license and
json-c with MIT (Expat) license

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: c2adec87958b081d1dc8775d4aa05c897912f025.1655151679.git.jag.raman@oracle.com

[Changed submodule URL to QEMU's libvfio-user mirror on GitLab. The QEMU
project mirrors its dependencies so that it can provide full source code
even in the event that its dependencies become unavailable. Note that
the mirror repo is manually updated, so please contact me to make newer
libvfio-user commits available. If I become a bottleneck we can set up a
cronjob.

Updated scripts/meson-buildoptions.sh to match the meson_options.txt
change. Failure to do so can result in scripts/meson-buildoptions.sh
being modified by the build system later on and you end up with a dirty
working tree.
--Stefan]

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 16:42:33 +01:00
Thomas Huth
72ec89bfc5 gitlab-ci: Fix the build-cfi-aarch64 and build-cfi-ppc64-s390x jobs
The job definitions recently got a second "variables:" section by
accident and thus are failing now if one tries to run them. Merge
the two sections into one again to fix the issue.

And while we're at it, bump the timeout here (70 minutes are currently
not enough for the aarch64 job). The jobs are marked as manual anyway,
so if the user starts them, they want to see their result for sure and
then it's annoying if the job timeouts too early.

Fixes: e312d1fdbb ("gitlab: convert build/container jobs to .base_job_template")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220603124809.70794-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220613171258.1905715-4-alex.bennee@linaro.org>
2022-06-14 00:15:06 +01:00
John Snow
1d8cf47e5b tests: run 'device-crash-test' from tests/venv
Remove the sys.path hacking from device-crash-test, and add in a little
user-friendly message for anyone who was used to running this script
directly from the source tree.

Modify the GitLab job recipes to create the tests/venv first, then run
device-crash-test from that venv.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220526000921.1581503-10-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-06 09:26:54 +02:00
Daniel P. Berrangé
e312d1fdbb gitlab: convert build/container jobs to .base_job_template
This converts the main build and container jobs to use the
base job rules, defining the following new variables

 - QEMU_JOB_SKIPPED - jobs that are known to be currently
   broken and should not be run. Can still be manually
   launched if desired.

 - QEMU_JOB_AVOCADO - jobs that run the Avocado integration
   test harness.

 - QEMU_JOB_PUBLISH - jobs that publish content after the
   branch is merged upstream

As build-tools-and-docs runs on master we declare the requirement of
building amd64-debian-container optional as it should already exits
once we merge.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-5-berrange@redhat.com>
[AJB: fix upstream typo, mention optional container req]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220527153603.887929-32-alex.bennee@linaro.org>
2022-06-01 15:47:43 +01:00
Thomas Huth
83602083b4 capstone: Remove the capstone submodule
Now that we allow compiling with Capstone v3.0.5 again, all our supported
build hosts should provide at least this version of the disassembler
library, so we do not need to ship this as a submodule anymore.

Message-Id: <20220516145823.148450-4-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-05-18 08:54:22 +02:00
Thomas Huth
43ecd16678 capstone: Allow version 3.0.5 again
According to

 https://lore.kernel.org/qemu-devel/20200921174118.39352-1-richard.henderson@linaro.org/

there was an issue with Capstone 3.0.4 from Ubuntu 18, which was the reason
for bumping our minimum Capstone requirement to version 4.0. And indeed,
compiling with that version 3.0.4 from Ubuntu 18.04 still fails (after
allowing it with a hack in meson.build). But now that we've dropped support
for Ubuntu 18.04, that issue is not relevant anymore. Compiling with Capstone
version 3.0.5 (e.g. used in Ubuntu 20.04) seems to work fine, so let's allow
that version again.

Message-Id: <20220516145823.148450-3-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-05-18 08:54:22 +02:00
Paolo Bonzini
8a5606f644 gitlab-ci: do not run tests with address sanitizer
This is flaky and sometimes fails or hangs unexplicably.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-03-15 11:50:50 +01:00
Thomas Huth
74154d7e4a linux-user: Remove the deprecated ppc64abi32 target
It's likely broken, and nobody cared for picking it up again
during the deprecation phase, so let's remove this now.

Since this is the last entry in deprecated_targets_list, remove
the related code in the configure script, too.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Cédric Le Goater <clg@kaod.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211215084958.185214-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220112112722.3641051-32-alex.bennee@linaro.org>
2022-02-09 13:29:38 +00:00
Daniel P. Berrangé
f178c62afc ci: explicitly skip I/O tests on alpine
The block I/O tests don't work on Alpine because their alternative libc
impl emits different strings for errnos, which breaks the expected
output matching. e.g.

=== IO: pattern 102
 wrote 512/512 bytes at offset 512
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: Input/output error
+qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: I/O error
 4
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
 Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=0

Currently the I/O tests are skipped as a side effect of the Alpine image
containing a minimal busybox 'sed' binary, rather than GNU sed. This is
a fragile assumption that will be invalidated when the dockerfile is
changed to be autogenerated from a standardized package list that
includes GNU sed.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211215141949.3512719-6-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220105135009.1584676-6-alex.bennee@linaro.org>
2022-01-18 16:42:41 +00:00
Thomas Huth
4cc75ce697 gitlab-ci: Enable docs in the centos job
We just ran into a problem that the docs don't build on RHEL8 / CentOS 8
anymore. Seems like these distros are using one of the oldest Sphinx
versions that we still have to support. Thus enable the docs build in
the CI on CentOS so that such bugs don't slip in so easily again.

Message-Id: <20220104091240.160867-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-01-05 11:10:13 +01:00
Thomas Huth
f462be4c06 gitlab-ci.d/buildtest: Add jobs that run the device-crash-test
The device-crash-test script has been quite neglected in the past,
so that it bit-rot quite often. Let's add CI jobs that run this
script for at least some targets, so that this script does not
regress that easily anymore.

Message-Id: <20211126162724.1162049-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-12-15 08:08:59 +01:00
Willian Rampazzo
bbbd9b6ec6 tests/acceptance: rename tests acceptance to tests avocado
In the discussion about renaming the `tests/acceptance` [1], the
conclusion was that the folders inside `tests` are related to the
framework running the tests and not directly related to the type of
the tests.

This changes the folder to `tests/avocado` and adjusts the MAKEFILE, the
CI related files and the documentation.

[1] https://lists.gnu.org/archive/html/qemu-devel/2021-05/msg06553.html

Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20211105155354.154864-3-willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-11-08 17:00:22 +01:00
Gerd Hoffmann
f5918a9928 microvm: add device tree support.
Allows edk2 detect virtio-mmio devices and pcie ecam.
See comment in hw/i386/microvm-dt.c for more details.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Sergio Lopez <slp@redhat.com>
Message-Id: <20211014193617.2475578-1-kraxel@redhat.com>
2021-11-02 17:24:17 +01:00
Thomas Huth
2ffd4d815e gitlab-ci: Don't try to use the system libfdt in the debian job
libfdt in Debian is too old to be usable for QEMU. So far we were
silently falling back to the internal dtc submodule, but since
this is wrong, let's remove the --enable-fdt=system switch here now.

Message-Id: <20210827151718.178988-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-09-06 09:07:00 +02:00
Peter Maydell
9c03aa87e5 Testing and plugin updates:
- fix typo in execlog plugin
   - clean-up and document gitlab FOO_RUNNER_AVAILABLE vars
   - fix plugin build issue on OSX and modules
   - add multi-core support to cache modelling plugin
   - clean-ups for plugin arg=FOO handling
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmEwqF4ACgkQ+9DbCVqe
 KkTCcAf/fCHK1hhXJIaxpFyMGvCkhYzUgKlpL4b05ofXFyQJ4JWp+0OadQzBcVIN
 PwsVVsXik/5ibKvQiud8wt3/kbr9gj5RZdwVITZMEpod6hy9Nt5oaUhxEZpiVs3f
 XZG17zaK8huwTwU6EtHWOnvIWMCuvBXpw8cM/6jF6pEiq162VJbiRo9L/aQ+n0Io
 TZTy9YqEbrZUTj+XrvgPV0TewjM6T/zTujXFJtyCzYJ3P3kx9z5a3/KzfyJI9qkk
 TYw0MX9MY8J5/sXcP6OjOqXqppsRe+G5Uaz0BoW9SvOmR/0nNGy/N8mVCZTD6mqs
 zMsWw5RyIZsZCyWEYj4TYWEORv5Kiw==
 =CBs1
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-for-6.2-020921-1' into staging

Testing and plugin updates:

  - fix typo in execlog plugin
  - clean-up and document gitlab FOO_RUNNER_AVAILABLE vars
  - fix plugin build issue on OSX and modules
  - add multi-core support to cache modelling plugin
  - clean-ups for plugin arg=FOO handling

# gpg: Signature made Thu 02 Sep 2021 11:33:02 BST
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-for-6.2-020921-1: (22 commits)
  docs/devel: be consistent about example plugin names
  docs/deprecated: deprecate passing plugin args through `arg=`
  tests/plugins/syscalls: adhere to new arg-passing scheme
  tests/plugins/mem: introduce "track" arg and make args not positional
  tests/plugins/insn: made arg inline not positional and parse it as bool
  tests/plugins/bb: adapt to the new arg passing scheme
  docs/tcg-plugins: new passing parameters scheme for cache docs
  plugins/howvec: adapting to the new argument passing scheme
  plugins/hwprofile: adapt to the new plugin arguments scheme
  plugins/lockstep: make socket path not positional & parse bool arg
  plugins/hotblocks: Added correct boolean argument parsing
  plugins/hotpages: introduce sortby arg and parsed bool args correctly
  plugins/api: added a boolean parsing plugin api
  plugins: allow plugin arguments to be passed directly
  docs/devel/tcg-plugins: added cores arg to cache plugin
  plugins: sort exported symbol list
  plugins/cache: supported multicore cache modelling
  plugins: do not limit exported symbols if modules are active
  gitlab-ci: Fix ..._RUNNER_AVAILABLE variables and document them
  gitlab-ci: Remove superfluous "dnf install" statement
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-09-03 14:23:36 +01:00
Thomas Huth
d3ac658fbe gitlab-ci: Remove superfluous "dnf install" statement
The container already features meson and ninja, so there is no need
to try to install it with dnf again.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210730143809.717079-3-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210806141015.2487502-4-alex.bennee@linaro.org>
2021-09-02 11:29:34 +01:00
Thomas Huth
8d316275ad gitlab-ci: Merge "build-disabled" with "build-without-default-features"
Both jobs are testing more or less the same thing (building QEMU with
features disabled), so we are wasting precious CI cycles here by doing
this twice. Merge the jobs by using --without-default-features by default
and just adding some additional --disable-... switches which are not
covered by the generic switch (yet). And while we're at it, also test
compilation with "--disable-fdt" (which forces us to change the list
of targets in this job, though, since some targets do not work without
fdt).

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210730143809.717079-2-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210806141015.2487502-3-alex.bennee@linaro.org>
2021-09-02 11:29:34 +01:00
Vladimir Sementsov-Ogievskiy
c88f078948 iotests: move 222 to tests/image-fleecing
Give a good name to test file.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210824083856.17408-29-vsementsov@virtuozzo.com>
[hreitz: Adjust .gitlab-ci.d/buildtest.yml]
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-01 14:37:14 +02:00
Thomas Huth
e37264ebe3 gitlab-ci.d/buildtest: Mark the aarch64 and ppc64-s390x CFI jobs as manual
These two jobs are currently failing very often - the linker seems to
get killed due to out-of-memory problems. Since apparently nobody has
currently an idea how to fix that nicely, let's mark the jobs as manual
for the time being until someone comes up with a proper fix.

Message-Id: <20210728075141.400816-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-07-29 08:04:25 +02:00
Thomas Huth
db1e119238 gitlab-ci.d/buildtest: Disable iotests 197 and 215
The iotests 197 and 215 are occasionally failing in the gitlab-CI now.
According to the log, the failure is "./common.rc: Killed" which might
be an indication that the process has been killed due to out-of-memory
reasons. Both tests are doing a big read with 2G that likely causes
this issue. It used to work fine in the gitlab-CI in the past, but
either the program is now requiring more free memory, or the the CI
containers have changed, so that the OOM condition now sometimes occurs.

Anyway, these two tests are not really suitable for CI containers if
they are doing things like huge reads (which is likely also the reason
why they haven't been added to the "auto" group in the past), so let's
simply disable them in the gitlab-CI now, too.

Message-Id: <20210727162542.318882-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-07-29 08:04:25 +02:00
Peter Maydell
34fd92ab41 Doc, metadata, plugin and testing updates for 6.1-rc1:
- git ignore some file editor detritus
   - add overview on device emulation terminology
   - remove needless if leg in configure custom devices logic
   - numerous gitdm/mailmap updates
   - fix plugin_exit race for linux-user
   - fix a few bugs in cache modelling plugin
   - fix plugin calculation of physical address
   - handle pure assembler/linker tcg tests outside of docker
   - add tricore build to gitlab
   - remove superfluous MacOSX task
   - generalise the OpenBSI gitlab rules
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmD67ioACgkQ+9DbCVqe
 KkTKcwf/UlEl2wlYrvymoJy7caasD/UrlNwpaK90zRTxuofJjG2bXX/58EGBlgxe
 cjbX3841702HXAFDxzswmWX3yeXQVgNiXrEy5FapB7L3h/qC/BDVS4pz987TITNW
 b4lw/d05bQVGavcdJXPnX0CdV8IK4hpieDkdh/LGj5+CSAC39jvyVGTYBpfLKl8D
 cps5qcJcJuQjz4hatB1haZOV1WHLyw8KI0srfcEqkh3IImZFZ09frs3qqAxrYRqE
 Y62d7SIN5TX1XgWuTaf1mmaMVmbNi9DTdCxWpGYr4ZBLhhzKeljQvKbF45ZGQ0z9
 eBVBdxGJTX/WAKrimcbHxLo43pCo2Q==
 =7vzj
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-for-6.1-rc1-230721-1' into staging

Doc, metadata, plugin and testing updates for 6.1-rc1:

  - git ignore some file editor detritus
  - add overview on device emulation terminology
  - remove needless if leg in configure custom devices logic
  - numerous gitdm/mailmap updates
  - fix plugin_exit race for linux-user
  - fix a few bugs in cache modelling plugin
  - fix plugin calculation of physical address
  - handle pure assembler/linker tcg tests outside of docker
  - add tricore build to gitlab
  - remove superfluous MacOSX task
  - generalise the OpenBSI gitlab rules

# gpg: Signature made Fri 23 Jul 2021 17:28:26 BST
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-for-6.1-rc1-230721-1: (28 commits)
  gitlab-ci: Extract OpenSBI job rules to reusable section
  gitlab-ci: Remove the second superfluous macos task
  gitlab: enable a very minimal build with the tricore container
  tests/tcg/configure.sh: add handling for assembler only builds
  plugins: Fix physical address calculation for IO regions
  plugins/cache: Fixed "function decl. is not a prototype" warnings
  plugins/cache: limited the scope of a mutex lock
  plugins/cache: Fixed a bug with destroying FIFO metadata
  tcg/plugins: implement a qemu_plugin_user_exit helper
  contrib/gitdm: add more individual contributor entries.
  contrib/gitdm: add a new interns group-map for GSoC/Outreachy work
  contrib/gitdm: add an explicit academic entry for BU
  contrib/gitdm: add group-map for Netflix
  contrib/gitdm: add domain-map for NVIDIA
  contrib/gitdm: add domain-map for Crudebyte
  contrib/gitdm: un-ironically add a mapping for LWN
  contrib/gitdm: add domain-map/group-map for Wind River
  contrib/gitdm: add domain-map for Eldorado
  contrib/gitdm: add domain-map/group-map mappings for Samsung
  gitdm.config: sort the corporate GroupMap entries
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-07-26 11:00:15 +01:00
Alex Bennée
39ce923732 gitlab: enable a very minimal build with the tricore container
Rather than base of the shared Debian 10 container which would require
us to bring in even more dependencies just bring in what is needed for
building tricore-softmmu in GitLab. We don't even remove the container
from the DOCKER_PARTIAL_IMAGES lest we cause more confusion.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210720232703.10650-28-alex.bennee@linaro.org>
2021-07-23 17:22:16 +01:00
Daniel P. Berrangé
eafadbbbac gitlab: only let pages be published from default branch
GitLab will happily publish pages generated by the latest CI pipeline
from any branch:

https://docs.gitlab.com/ee/user/project/pages/introduction.html

  "Remember that GitLab Pages are by default branch/tag agnostic
   and their deployment relies solely on what you specify in
   .gitlab-ci.yml. You can limit the pages job with the only
   parameter, whenever a new commit is pushed to a branch used
   specifically for your pages."

The current "pages" job is not limited, so it is happily publishing
docs content from any branch/tag in qemu.git that gets pushed to.
This means we're potentially publishing from the "staging" branch
or worse from outdated "stable-NNN" branches

This change restricts it to only publish from the default branch
in the main repository. For contributor forks, however, we allow
it to publish from any branch, since users will have arbitrarily
named topic branches in flight at any time.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210723113051.2792799-1-berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-23 16:02:56 +02:00
Alex Bennée
ba4dd2aabc tcg/plugins: enable by default for most TCG builds
Aside from a minor bloat to file size the ability to have TCG plugins
has no real impact on performance unless a plugin is actively loaded.
Even then the libempty.so plugin shows only a minor degradation in
performance caused by the extra book keeping the TCG has to do to keep
track of instructions. As it's a useful feature lets just enable it by
default and reduce our testing matrix a little.

We need to move our linker testing earlier so we can be sure we can
enable the loader module required. As we have ruled out static &
plugins in an earlier patch we can also reduce the indent a little.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20210709143005.1554-33-alex.bennee@linaro.org>
2021-07-14 15:54:13 +01:00
Philippe Mathieu-Daudé
e2d3017809 gitlab-ci: Split gprof-gcov job
This job is hitting the 70min limit, so split it in 2 tasks.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210525082556.4011380-7-f4bug@amsat.org>
2021-06-07 14:42:47 +01:00
Daniel P. Berrangé
d4c7ee330c crypto: bump min gnutls to 3.5.18, dropping RHEL-7 support
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target. This lets us increment the minimum required gnutls version

Per repology, current shipping versions are:

             RHEL-8: 3.6.14
      Debian Buster: 3.6.7
 openSUSE Leap 15.2: 3.6.7
   Ubuntu LTS 18.04: 3.5.18
   Ubuntu LTS 20.04: 3.6.13
            FreeBSD: 3.6.15
          Fedora 33: 3.6.16
          Fedora 34: 3.7.1
            OpenBSD: 3.6.15
     macOS HomeBrew: 3.6.15

Ubuntu LTS 18.04 has the oldest version and so 3.5.18 is the new minimum.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210514120415.1368922-7-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
[thuth: rebased to use .gitlab-ci.d/buildtest.yml]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-06-02 07:10:35 +02:00
Daniel P. Berrangé
b33a84632a crypto: bump min gcrypt to 1.8.0, dropping RHEL-7 support
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target. This lets us increment the minimum required gcrypt version and
assume that HMAC is always supported

Per repology, current shipping versions are:

             RHEL-8: 1.8.5
      Debian Buster: 1.8.4
 openSUSE Leap 15.2: 1.8.2
   Ubuntu LTS 18.04: 1.8.1
   Ubuntu LTS 20.04: 1.8.5
            FreeBSD: 1.9.2
          Fedora 33: 1.8.6
          Fedora 34: 1.9.3
            OpenBSD: 1.9.3
     macOS HomeBrew: 1.9.3

Ubuntu LTS 18.04 has the oldest version and so 1.8.0 is the new minimum.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210514120415.1368922-6-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
[thuth: rebased to use .gitlab-ci.d/buildtest.yml]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-06-02 07:10:18 +02:00
Daniel P. Berrangé
20ba7a4a34 crypto: bump min nettle to 3.4, dropping RHEL-7 support
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target. This lets us increment the minimum required nettle version and
drop a lot of backwards compatibility code for 2.x series of nettle.

Per repology, current shipping versions are:

             RHEL-8: 3.4.1
      Debian Buster: 3.4.1
 openSUSE Leap 15.2: 3.4.1
   Ubuntu LTS 18.04: 3.4
   Ubuntu LTS 20.04: 3.5.1
            FreeBSD: 3.7.2
          Fedora 33: 3.5.1
          Fedora 34: 3.7.2
            OpenBSD: 3.7.2
     macOS HomeBrew: 3.7.2

Ubuntu LTS 18.04 has the oldest version and so 3.4 is the new minimum.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210514120415.1368922-4-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
[thuth: rebased to use .gitlab-ci.d/buildtest.yml]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-06-02 07:03:25 +02:00
Philippe Mathieu-Daudé
f2f5c3e629 gitlab: Extract all default build/test jobs to buildtest.yml
Extract the build/test jobs run by default on the mainstream
CI into a new file (buildtest.yml).

Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210511072952.2813358-9-f4bug@amsat.org>
2021-05-27 16:39:23 +02:00