Commit Graph

219 Commits

Author SHA1 Message Date
Nicholas Piggin
74eb04af18 tests/avocado: Fix ppc_hv_tests.py xorriso dependency guard
For some reason the skipIf missing_deps() check fails to skip the test
if it comes after the skipUnless lines, causing an error running on
systems without xorriso.

Avocado implements skipUnless is just an inverted skipIf, so it's not
clear what the bug is or why this fixes it. For now it's enough to
get things working.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2246
Fixes: c9cb496710 ("tests/avocado: ppc add hypervisor tests")
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-03-30 18:50:25 +10:00
Marcin Juszkiewicz
55900f5dcc tests/avocado: sbsa-ref: add OpenBSD tests for misc 'max' setup
PAuth makes run timeout on CI so add tests using 'max' without
it and with impdef one.

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20240318-sbsa-ref-firmware-update-v3-4-1c33b995a538@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2024-03-20 09:52:27 +00:00
Marcin Juszkiewicz
385ac766e8 tests/avocado: sbsa-ref: add Alpine tests for misc 'max' setup
PAuth makes run timeout on CI so add tests using 'max' without it
and with impdef one.

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20240318-sbsa-ref-firmware-update-v3-3-1c33b995a538@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2024-03-20 09:52:27 +00:00
Marcin Juszkiewicz
21f123f3c2 tests/avocado: drop virtio-rng from sbsa-ref tests
sbsa-ref is supposed to emulate real hardware so virtio-rng-pci
does not fit here

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-Id: <20240318-sbsa-ref-firmware-update-v3-2-1c33b995a538@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2024-03-20 09:52:27 +00:00
Marcin Juszkiewicz
06f6bbc3bd tests/avocado: update sbsa-ref firmware
We now have CI job to build those and publish in space with
readable urls.

Firmware is built using Debian 'bookworm' cross toolchain (gcc 12.2.0).

Used versions:

- Trusted Firmware v2.10.2
- Tianocore EDK2 stable202402
- Tianocore EDK2 Platforms code commit 085c2fb

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20240318-sbsa-ref-firmware-update-v3-1-1c33b995a538@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2024-03-20 09:52:27 +00:00
Peter Maydell
2f79d2ed64 tests/avocado/boot_linux_console.py: Add Rpi4b boot tests
Signed-off-by: Sergey Kambalin <sergey.kambalin@auriga.com>
Message-id: 20240226000259.2752893-31-sergey.kambalin@auriga.com
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
[PMM: Comment out use of USB, which depends on PCI]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-02-27 13:01:43 +00:00
Nicholas Piggin
b41484ccb6 tests/avocado: Use default CPU for pseries machine
Use the default CPU with the pseries machine unless there is a
specific requirement.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:24:31 +10:00
Nicholas Piggin
c9cb496710 tests/avocado: ppc add hypervisor tests
The powernv and pseries machines both provide hypervisor facilities
that are supported by KVM. This is a large and complicated set of
features that don't get much system-level testing in ppc tests.

Add a new test case for these which runs QEMU KVM inside the target.
This downloads an Alpine VM image, boots it and downloads and installs
the qemu package, then boots a virtual machine under it, re-using the
original Alpine VM image.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:16:35 +10:00
Nicholas Piggin
9bf9479328 tests/avocado: Add pseries KVM boot_linux test
ppc has no avocado tests for the KVM backend. Add a KVM boot_linux.py
test for pseries.

Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:16:35 +10:00
Nicholas Piggin
8d07a8aef6 tests/avocado: Add ppc pseries and powernv hash MMU tests
POWER CPUs support hash and radix MMU modes. Linux supports running in
either mode, but defaults to radix. To keep up testing of QEMU's hash
MMU implementation, add some Linux hash boot tests.

Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:16:35 +10:00
Nicholas Piggin
234aa6d62a tests/avocado: ppc add powernv10 boot_linux_console test
Add test for POWER10.

Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:16:34 +10:00
Nicholas Piggin
458a6aa3b3 tests/avocado: improve flaky ppc/pnv boot_linux_console.py test
The expected MTD partition detection output does not always appear on
the console, despite the test reaching the boot loader and the string
appearing in dmesg. Possibly due to an init script that quietens the
console output. Using an earlier log message improves reliability.

Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:16:34 +10:00
Nicholas Piggin
e5f1500144 tests/avocado: mark boot_linux.py long runtime instead of flaky
The ppc64 and s390x tests were first marked skipIf GITLAB_CI by commit
c0c8687ef0 ("tests/avocado: disable BootLinuxPPC64 test in CI"), and
commit 0f26d94ec9 ("tests/acceptance: skip s390x_ccw_vrtio_tcg on
GitLab") due to being very heavy-weight for gitlab CI.

Commit 9b45cc9931 ("docs/devel: rationalise unstable gitlab tests under
FLAKY_TESTS") changed this to being flaky but it isn't really, it just
had a long runtime.

So take the SPEED=slow variable from qtests and introduce it to avocado,
and make these tests require it.

Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:16:34 +10:00
Manos Pitsidianakis
35a5a331e4 tests: correct typos
Correct typos automatically found with the `typos` tool
<https://crates.io/crates/typos>

Signed-off-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
Reviewed-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
2024-02-20 22:21:10 +03:00
Cédric Le Goater
560f254107 tests/avocado/machine_aspeed.py: Update buildroot images to 2023.11
Compared to mainline buildroot, these images have some customization :

- Linux version is bumped to 6.6.3 and built with a custom config
- U-Boot is switched to the one provided by OpenBMC for more support
- defconfigs extra tools for dev

See branch [1] for more details.

There are a few changes since last update, commit ed1f5ff842. Images
all have a password now and I2C devices have been updated in the Linux
ast2600-evb device tree [2]. Do the necessary adjustements.

[1] https://github.com/legoater/buildroot/commits/aspeed-2023.11
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9deb10cf160e

Signed-off-by: Cédric Le Goater <clg@kaod.org>
2024-02-01 08:13:30 +01:00
Ani Sinha
7ef4c41e91 acpi/tests/avocado/bits: wait for 200 seconds for SHUTDOWN event from bits VM
By default, the timeout to receive any specified event from the QEMU VM is 60
seconds set by the python avocado test framework. Please see event_wait() and
events_wait() in python/qemu/machine/machine.py. If the matching event is not
triggered within that interval, an asyncio.TimeoutError is generated. Since the
timeout for the bits avocado test is 200 secs, we need to make event_wait()
timeout of the same value as well so that an early timeout is not triggered by
the avocado framework.

CC: peter.maydell@linaro.org
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2077
Signed-off-by: Ani Sinha <anisinha@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20240117042556.3360190-1-anisinha@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-01-18 12:47:25 +00:00
Alex Bennée
139db3cc59 tests/avocado: partially revert unmasking of replay_linux tests
It seems we were premature in declaring replay_linux.py fixed. The
x86_64 image still seems to hang occasionally. I've raised a new bug
to cover it:

    https://gitlab.com/qemu-project/qemu/-/issues/2094

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2024-01-12 13:23:48 +00:00
Alex Bennée
b9371a7b90 tests/avocado: use snapshot=on in kvm_xen_guest
This ensures the rootfs is never permanently changed as we don't need
persistence between tests anyway.

Message-Id: <20240103173349.398526-3-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
2024-01-12 13:23:48 +00:00
Thomas Huth
b7f2aea63b tests/avocado: Add a test for a little-endian microblaze machine
We've already got a test for a big endian microblaze machine, but so
far we lack one for a little endian machine. Now that the QEMU advent
calendar featured such an image, we can test the little endian mode,
too.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20231215161851.71508-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2024-01-12 13:23:48 +00:00
Peter Maydell
c1df5b4f16 Record/replay fixes for replay_kernel tests
- add a 32 bit x86 replay test case
   - fix some typos
   - use modern snapshot setting for tests
   - update replay_dump for current ABI
   - remove stale replay variables
   - improve kdoc for ReplayState
   - introduce common error path for replay
   - always fully drain chardevs when in replay
   - catch unexpected waitio on playback
   - remove flaky tags from replay_kernel tests
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmWcAJgACgkQ+9DbCVqe
 KkS/TQf+PuIPtuX71ENajfRBjz6450IbGqLUJ1HEaPGYGRj+fR6rg5g5u8qaBrT7
 TUv9ef9L22NtyL+Gbs1OGpGDWKoqV6RQc+A/MHa8IKFpcS24nUo3k4psIC6NSGRH
 6w3++fPC1Q5cDk9Lei3Qt8fXzcnUZz+NTiIK05aC0xh7D6uGfdADvKqHeLav7qi+
 X2ztNdBsy/WJWCuWcMVzb/dGwDBtuyyxvqTD4EF+zn+gSYq9od2G8XdF+0o6ZVLM
 mXEHwNwB6UjOkLt2cYaay59SXcJFvwxKbEGTDnA7T+kgd3rknuBaWdVBIazoSPQh
 +522nPz5qq/3wO1l7+iQXuvd38fWyw==
 =nKRx
 -----END PGP SIGNATURE-----

Merge tag 'pull-replay-fixes-080124-1' of https://gitlab.com/stsquad/qemu into staging

Record/replay fixes for replay_kernel tests

  - add a 32 bit x86 replay test case
  - fix some typos
  - use modern snapshot setting for tests
  - update replay_dump for current ABI
  - remove stale replay variables
  - improve kdoc for ReplayState
  - introduce common error path for replay
  - always fully drain chardevs when in replay
  - catch unexpected waitio on playback
  - remove flaky tags from replay_kernel tests

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmWcAJgACgkQ+9DbCVqe
# KkS/TQf+PuIPtuX71ENajfRBjz6450IbGqLUJ1HEaPGYGRj+fR6rg5g5u8qaBrT7
# TUv9ef9L22NtyL+Gbs1OGpGDWKoqV6RQc+A/MHa8IKFpcS24nUo3k4psIC6NSGRH
# 6w3++fPC1Q5cDk9Lei3Qt8fXzcnUZz+NTiIK05aC0xh7D6uGfdADvKqHeLav7qi+
# X2ztNdBsy/WJWCuWcMVzb/dGwDBtuyyxvqTD4EF+zn+gSYq9od2G8XdF+0o6ZVLM
# mXEHwNwB6UjOkLt2cYaay59SXcJFvwxKbEGTDnA7T+kgd3rknuBaWdVBIazoSPQh
# +522nPz5qq/3wO1l7+iQXuvd38fWyw==
# =nKRx
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 08 Jan 2024 14:03:04 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* tag 'pull-replay-fixes-080124-1' of https://gitlab.com/stsquad/qemu:
  tests/avocado: remove skips from replay_kernel
  chardev: force write all when recording replay logs
  replay: stop us hanging in rr_wait_io_event
  replay/replay-char: use report_sync_error
  replay: introduce a central report point for sync errors
  replay: make has_unread_data a bool
  replay: add proper kdoc for ReplayState
  replay: remove host_clock_last
  scripts/replay_dump: track total number of instructions
  scripts/replay-dump: update to latest format
  tests/avocado: modernise the drive args for replay_linux
  tests/avocado: fix typo in replay_linux
  tests/avocado: add a simple i386 replay kernel test

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-01-09 10:32:04 +00:00
Ani Sinha
b345ffad3e acpi/tests/avocado/bits: disable smilatency tests
smilatncy tests in bios bits seems to generate some flakyness in running the
bits avocado tests. Disable them for now.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2077

CC: peter.maydell@linaro.org
CC: crosa@redhat.com
CC: philmd@linaro.org
CC: bleal@redhat.com
CC: mst@redhat.com
CC: wainersm@redhat.com
Signed-off-by: Ani Sinha <anisinha@redhat.com>
Message-id: 20240108103643.4434-3-anisinha@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-01-09 10:31:47 +00:00
Ani Sinha
fc7c144b23 acpi/tests/avocado/bits: import smilatency test from bits in order to disable it
Add smilatency test script in the bits avocado tests from bios-bits. No changes
have been made to the original test script. The test will be disabled in the
subsequent patch.

CC: peter.maydell@linaro.org
CC: crosa@redhat.com
CC: philmd@linaro.org
CC: bleal@redhat.com
CC: mst@redhat.com
CC: wainersm@redhat.com
Signed-off-by: Ani Sinha <anisinha@redhat.com>
Message-id: 20240108103643.4434-2-anisinha@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-01-09 10:31:47 +00:00
Alex Bennée
c2ef5ee89d tests/avocado: remove skips from replay_kernel
With the latest fixes for #2010 and #2013 these tests look pretty
stable now. Of course the only way to be really sure is to run it in
the CI infrastructure and see what breaks.

Acked-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231211091346.14616-14-alex.bennee@linaro.org>
2024-01-08 13:59:06 +00:00
Alex Bennée
6f6b71536c tests/avocado: modernise the drive args for replay_linux
QEMU complains about us not being explicit with setting snapshot so
lets do that. Also as cdroms are RO media we don't need to jump the
hoops of setting up snapshots and replay disks - just declare the
drive is a cdrom and nothing should change.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231211091346.14616-4-alex.bennee@linaro.org>
2024-01-08 13:58:31 +00:00
Alex Bennée
2d8508bbab tests/avocado: fix typo in replay_linux
Reviewed-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231211091346.14616-3-alex.bennee@linaro.org>
2024-01-08 13:58:19 +00:00
Alex Bennée
940f70cc1c tests/avocado: add a simple i386 replay kernel test
There are a number of bugs against 32 bit x86 on the tracker. Lets at
least establish a baseline pure kernel boot can do record/replay
before we start looking at the devices.

Acked-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231211091346.14616-2-alex.bennee@linaro.org>
2024-01-08 13:58:06 +00:00
Alex Bennée
2e8ed6a970 tests/avocado: mark ReplayKernelNormal.test_mips64el_malta as flaky
I missed this when going through the recent failure logs. I can run
the test 30 times without failure locally but it seems to hang pretty
reliably on GitLab's CI infra-structure.

Cc: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231201201027.2689404-1-alex.bennee@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
2023-12-04 16:21:00 +01:00
Philippe Mathieu-Daudé
d369ad5558 tests/avocado: Update yamon-bin-02.22.zip URL
http://www.imgtec.com/tools/mips-tools/downloads/ redirects
to https://mips.com/downloads/yamon-version-02-22/ then points
to an invalid path to a s3 bucket. Use the correct path. The
site will eventually be fixed.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20231201205630.10837-1-philmd@linaro.org>
2023-12-04 16:21:00 +01:00
Alex Bennée
5d25fcb702 gitlab: add optional job to run flaky avocado tests
One problem with flaky tests is they often only fail under CI
conditions which makes it hard to debug. We add an optional allow_fail
job so developers can trigger the only the flaky tests in the CI
environment if they are debugging.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-8-alex.bennee@linaro.org>
2023-12-01 17:47:20 +00:00
Alex Bennée
970c6fc250 tests/avocado: tag sbsa tests as tcg only
As firmware runs at a higher privilege level than the hypervisor we
can only run these tests under TCG emulation.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-6-alex.bennee@linaro.org>
2023-12-01 17:47:20 +00:00
Alex Bennée
9b45cc9931 docs/devel: rationalise unstable gitlab tests under FLAKY_TESTS
It doesn't make sense to have two classes of flaky tests. While it may
take the constrained environment of CI to trigger failures easily it
doesn't mean they don't occasionally happen on developer machines. As
CI is the gating factor to passing there is no point developers
running the tests locally anyway unless they are trying to fix things.

While we are at it update the language in the docs to discourage the
QEMU_TEST_FLAKY_TESTS becoming a permanent solution.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-3-alex.bennee@linaro.org>
2023-12-01 14:11:55 +00:00
Thomas Huth
ea6a33e776 Revert "tests/avocado: Enable reverse_debugging.py tests in gitlab CI"
This reverts commit c4d74ab24a.

The reverse debugging test is sometimes still failing. See:
 https://gitlab.com/qemu-project/qemu/-/issues/1992

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-ID: <20231121100842.677363-1-thuth@redhat.com>
2023-11-21 10:28:55 -05:00
Nicholas Piggin
c4d74ab24a tests/avocado: Enable reverse_debugging.py tests in gitlab CI
Let's try enable reverse_debugging.py in gitlab CI.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20231116115354.228678-3-npiggin@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 14:22:56 +01:00
Nicholas Piggin
cd43f00524 tests/avocado: reverse_debugging drain console to prevent hang
Like replay_linux.py, reverse_debugging.py starts the vm with console
set but does not interact with it (e.g., with wait_for_console_pattern).
In this situation, the console should have a drainer attached so the
socket does not fill. replay_linux.py has a drainer, but it is missing
from reverse_debugging.py.

Per analysis in Link: this can cause the console socket/pipe to fill and
QEMU get stuck in qemu_chr_write_buffer, leading to strange test case
failures (ppc64 fails because it prints a lot to console in early bios).
Attaching a drainer prevents this.

Note, this commit does not fix bugs introduced by the commits referenced
in the first two Fixes: tags, but together those commits conspire to
irritate the problem and cause test case failure, which this commit
fixes.

Link: https://lore.kernel.org/qemu-devel/ZVT-bY9YOr69QTPX@redhat.com/
Fixes: 1d4796cd00 ("python/machine: use socketpair() for console connections")
Fixes: 761a13b239 ("tests/avocado: ppc64 reverse debugging tests for pseries and powernv")
Fixes: be52eca309 ("tests/acceptance: add reverse debugging test")
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20231116115354.228678-1-npiggin@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 14:22:28 +01:00
Thomas Huth
572960cb23 tests/avocado/mem-addr-space-check: Replace assertEquals() for Python 3.12
assertEquals() has been removed in Python 3.12 and should be replaced by
assertEqual(). See: https://docs.python.org/3.12/whatsnew/3.12.html#id3

Message-ID: <20231116061956.14676-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Ani Sinha <anisinha@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 09:21:30 +01:00
Thomas Huth
3365f3d99a tests/avocado/replay_kernel: Mark the test_x86_64_pc as flaky
It's failing very often, so don't run this by default anymore
until it gets fixed.

Message-ID: <20231114153019.295131-1-thuth@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 07:13:41 +01:00
Philippe Mathieu-Daudé
9d72dd100c tests/avocado: Make fetch_asset() unconditionally require a crypto hash
In a perfect world we'd have reproducible tests,
but then we'd be sure we run the same binaries.
If a binary artifact isn't hashed, we have no idea
what we are running. Therefore enforce hashing for
all our artifacts.

With this change, unhashed artifacts produce:

  $ avocado run tests/avocado/multiprocess.py
   (1/2) tests/avocado/multiprocess.py:Multiprocess.test_multiprocess_x86_64:
   ERROR: QemuBaseTest.fetch_asset() missing 1 required positional argument: 'asset_hash' (0.19 s)

Inspired-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20231115205149.90765-1-philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 07:13:41 +01:00
Thomas Huth
cfe52e91c4 tests/avocado/multiprocess: Add asset hashes to silence warnings
The multiprocess test is currently succeeding with an annoying warning:

 (1/2) tests/avocado/multiprocess.py:Multiprocess.test_multiprocess_x86_64:
       WARN: Test passed but there were warnings during execution. Check
       the log for details

In the log, you can find an entry like:

 WARNI| No hash provided. Cannot check the asset file integrity.

Add the proper asset hashes to avoid those warnings.

Message-ID: <20231115145852.494052-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 07:13:41 +01:00
Thomas Huth
9cdafe3b47 tests/avocado/intel_iommu: Add asset hashes to avoid warnings
The intel_iommu test is currently succeeding with annoying warnings.
Add the proper asset hashes to avoid those.

Message-ID: <20231114143531.291820-1-thuth@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 07:13:41 +01:00
Thomas Huth
ffab603bd5 tests/avocado/virtio-gpu: Fix test_vhost_user_vga_virgl for edid support
The "edid" feature has been added to vhost-user-gpu in commit
c06444261e ("contrib/vhost-user-gpu: implement get_edid feature"),
so waiting for "features: +virgl -edid" in the test does not work
anymore, it's "+edid" instead of "-edid" now!

While we're at it, move the expected string to the preceeding
exec_command_and_wait_for_pattern() instead (since waiting for
empty string here does not make too much sense).

Message-ID: <20231114203456.319093-1-thuth@redhat.com>
Reviewed-by: Antonio Caggiano <quic_acaggian@quicinc.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 07:13:41 +01:00
Philippe Mathieu-Daudé
f0a663b4ce tests/avocado: Replace assertRegexpMatches() for Python 3.12 compatibility
assertRegexpMatches() has been removed in Python 3.12 and should be replaced by
assertRegex(). See: https://docs.python.org/3.12/whatsnew/3.12.html#id3

Inspired-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231114144832.71612-1-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 07:13:41 +01:00
Thomas Huth
861f724d03 tests/avocado: Replace assertEquals() for Python 3.12 compatibility
assertEquals() has been removed in Python 3.12 and should be replaced by
assertEqual(). See: https://docs.python.org/3.12/whatsnew/3.12.html#id3

Message-ID: <20231114134326.287242-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 07:13:41 +01:00
Thomas Huth
0b2af475e9 tests/avocado/machine_s390_ccw_virtio.py: Fix SyntaxWarnings from python 3.12
Python 3.12 now warns about backslashes in strings that aren't used
for escaping a special character from Python. Silence the warning
by using raw strings here instead.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231113140721.46903-1-thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
2023-11-13 19:06:06 +01:00
Ani Sinha
0034d0395e tests/avocado: add test to exercise processor address space memory bound checks
QEMU has validations to make sure that a VM is not started with more memory
(static and hotpluggable memory) than what the guest processor can address
directly with its addressing bits. This change adds a test to make sure QEMU
fails to start with a specific error message when an attempt is made to
start a VM with more memory than what the processor can directly address.
The test also checks for passing cases when the address space of the processor
is capable of addressing all memory. Boundary cases are tested.

CC: imammedo@redhat.com
CC: David Hildenbrand <david@redhat.com>
Acked-by: David Hildenbrand <david@redhat.com>
Acked-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Ani Sinha <anisinha@redhat.com>
Message-ID: <20231109045601.33349-1-anisinha@redhat.com>
Message-ID: <D5D8D419-76BA-4FB0-9BAC-4F7470A052FC@redhat.com>
[PMD: Use SPDX tag]
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
2023-11-13 16:56:24 +01:00
Alex Bennée
119599965e tests/avocado: update the tcg_plugins test
There are a number of things that are broken on the test currently so
lets fix that up:

  - replace retired Debian kernel for tuxrun_baseline one
  - remove "detected repeat instructions test" since ea185a55
  - log total counted instructions/memory accesses

Acked-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231106185112.2755262-8-alex.bennee@linaro.org>
2023-11-08 15:15:23 +00:00
Stefan Hajnoczi
a3c3aaa846 ppc patch queue for 2023-11-07:
This queue, the last one before the 8.2 feature freeze, has miscellanous
 changes that includes new PowerNV features and the new AmigaONE XE
 board.
 -----BEGIN PGP SIGNATURE-----
 
 iIwEABYKADQWIQQX6/+ZI9AYAK8oOBk82cqW3gMxZAUCZUqiORYcZGFuaWVsaGI0
 MTNAZ21haWwuY29tAAoJEDzZypbeAzFkBSUA/2qm8CyrRqY5+tsjtWQqZmPZ3L1F
 CgnXFNqtY2tzbTe5AQCi6FeQBEmXbZYVfryZyA+CQ4DUERc+18pe6hV3bBR9Cg==
 =cnHS
 -----END PGP SIGNATURE-----

Merge tag 'pull-ppc-20231107' of https://gitlab.com/danielhb/qemu into staging

ppc patch queue for 2023-11-07:

This queue, the last one before the 8.2 feature freeze, has miscellanous
changes that includes new PowerNV features and the new AmigaONE XE
board.

# -----BEGIN PGP SIGNATURE-----
#
# iIwEABYKADQWIQQX6/+ZI9AYAK8oOBk82cqW3gMxZAUCZUqiORYcZGFuaWVsaGI0
# MTNAZ21haWwuY29tAAoJEDzZypbeAzFkBSUA/2qm8CyrRqY5+tsjtWQqZmPZ3L1F
# CgnXFNqtY2tzbTe5AQCi6FeQBEmXbZYVfryZyA+CQ4DUERc+18pe6hV3bBR9Cg==
# =cnHS
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 08 Nov 2023 04:46:49 HKT
# gpg:                using EDDSA key 17EBFF9923D01800AF2838193CD9CA96DE033164
# gpg:                issuer "danielhb413@gmail.com"
# gpg: Good signature from "Daniel Henrique Barboza <danielhb413@gmail.com>" [unknown]
# gpg: WARNING: The key's User ID is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 17EB FF99 23D0 1800 AF28  3819 3CD9 CA96 DE03 3164

* tag 'pull-ppc-20231107' of https://gitlab.com/danielhb/qemu:
  ppc: qtest already exports qtest_rtas_call()
  hw/pci-host: Update PHB5 XSCOM registers
  ppc/pnv: Fix number of I2C engines and ports for power9/10
  ppc/pnv: Connect PNV I2C controller to powernv10
  ppc/pnv: Connect I2C controller model to powernv9 chip
  ppc/pnv: Add an I2C controller model
  tests/avocado: Add test for amigaone board
  hw/ppc: Add emulation of AmigaOne XE board
  hw/pci-host: Add emulation of Mai Logic Articia S

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2023-11-08 20:35:00 +08:00
Stefan Hajnoczi
44ad47b758 * Fix s390x CPU reconfiguration information in the SCLP facility map
* Fix condition code problem in the CLC and LAALG instruction
 * Fix ordering of the new s390x topology list entries
 * Add some more files to the MAINTAINERS file
 * Allow newer versions of Tesseract in the m68k nextcube test
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmVKgksRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbWIHg//TM3JOpsMEqHKlUKqOJH02mFQrK6H7LG0
 BC56FG7T+/mpYs1NTG92t8nCK03C2ZCweQWD7ZulRJAjPhZv+TF5bJEForivU7+k
 PKEshz9xKCWn2YGyNnf2LA06J1JkF215+KlReOoxwSgj1cPlHfBLQ0DtxmpJJZ1G
 h5p4d26BbSlwR58HrFWTlhgJMPenl59BETUGIK1FklBxunmZeeijddfniAhOT44y
 i0u9/H9KCg3tkwBROUy+42QV+ef32kz/yvi5RmYQI5W7PixO4sxH6MYduOjshsu9
 wK70f8EOwiZV6lFxqmbV7vxFeNnp5IuaVU7PMBoAkwZqLw99mSFy1+1BabCuL5b+
 3iUTiD4UW48MYwE2Ua6Lit4kpfjhwcp/UYz6pIk6TCBQX6LfzO+nj+rod0GdIpyZ
 4Lwm7jBtpTlYkGrsMvpA/qcidOtqPA1lmBTNlY1hFodQF6KWtyObn0w5AM80xeeU
 /mGxQDz97Bpz7LKZvhu+k38jaWvnJFnl3jF1zet88CYL9YL+YI/k1KjhFafCXb0V
 38Xpt5JTWxyLSh2B3gx0OpokX5bftvW9GlLix0HqL7c23uYwR2Bq+Rd6I8SAlk4C
 uJq6gqP8IFBFHfgbmyqf/fyd/eHxm7J1voIdy9PZyxZ1JYT9A7yu56qV6SJYwCpr
 aARwui/Dm4o=
 =y+cC
 -----END PGP SIGNATURE-----

Merge tag 'pull-request-2023-11-07' of https://gitlab.com/thuth/qemu into staging

* Fix s390x CPU reconfiguration information in the SCLP facility map
* Fix condition code problem in the CLC and LAALG instruction
* Fix ordering of the new s390x topology list entries
* Add some more files to the MAINTAINERS file
* Allow newer versions of Tesseract in the m68k nextcube test

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmVKgksRHHRodXRoQHJl
# ZGhhdC5jb20ACgkQLtnXdP5wLbWIHg//TM3JOpsMEqHKlUKqOJH02mFQrK6H7LG0
# BC56FG7T+/mpYs1NTG92t8nCK03C2ZCweQWD7ZulRJAjPhZv+TF5bJEForivU7+k
# PKEshz9xKCWn2YGyNnf2LA06J1JkF215+KlReOoxwSgj1cPlHfBLQ0DtxmpJJZ1G
# h5p4d26BbSlwR58HrFWTlhgJMPenl59BETUGIK1FklBxunmZeeijddfniAhOT44y
# i0u9/H9KCg3tkwBROUy+42QV+ef32kz/yvi5RmYQI5W7PixO4sxH6MYduOjshsu9
# wK70f8EOwiZV6lFxqmbV7vxFeNnp5IuaVU7PMBoAkwZqLw99mSFy1+1BabCuL5b+
# 3iUTiD4UW48MYwE2Ua6Lit4kpfjhwcp/UYz6pIk6TCBQX6LfzO+nj+rod0GdIpyZ
# 4Lwm7jBtpTlYkGrsMvpA/qcidOtqPA1lmBTNlY1hFodQF6KWtyObn0w5AM80xeeU
# /mGxQDz97Bpz7LKZvhu+k38jaWvnJFnl3jF1zet88CYL9YL+YI/k1KjhFafCXb0V
# 38Xpt5JTWxyLSh2B3gx0OpokX5bftvW9GlLix0HqL7c23uYwR2Bq+Rd6I8SAlk4C
# uJq6gqP8IFBFHfgbmyqf/fyd/eHxm7J1voIdy9PZyxZ1JYT9A7yu56qV6SJYwCpr
# aARwui/Dm4o=
# =y+cC
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 08 Nov 2023 02:30:35 HKT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* tag 'pull-request-2023-11-07' of https://gitlab.com/thuth/qemu:
  target/s390x/cpu topology: Fix ordering and creation of TLEs
  tests/tcg/s390x: Test ADD LOGICAL WITH CARRY
  tests/tcg/s390x: Test LAALG with negative cc_src
  target/s390x: Fix LAALG not updating cc_src
  tests/tcg/s390x: Test CLC with inaccessible second operand
  target/s390x: Fix CLC corrupting cc_src
  target/s390x/cpu_models: Use 'first_cpu' in s390_get_feat_block()
  s390/sclp: fix SCLP facility map
  tests/avocado: Allow newer versions of tesseract in the nextcube test
  MAINTAINERS: Add artist.c to the hppa machine section
  MAINTAINERS: Add the virtio-gpu documentation to the corresponding section

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2023-11-08 20:34:36 +08:00
BALATON Zoltan
f46f8e0f37 tests/avocado: Add test for amigaone board
Add an avocado test for the amigaone board that tests it with the
firmware.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Message-ID: <b1a0246840fcff1fe6bbd8685e2474a9231b34c5.1698406922.git.balaton@eik.bme.hu>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2023-11-07 15:49:41 -03:00
Thomas Huth
645198d58b tests/avocado: Allow newer versions of tesseract in the nextcube test
Current Linux distros ship version 5 of the tesseract OCR software,
so the nextcube screen test is ignored there. Let's make the check
more flexible to allow newer versions, too, and remove the old v3
test since most Linux distros don't ship this version anymore.

Message-ID: <20231101204323.35533-1-huth@tuxfamily.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <huth@tuxfamily.org>
2023-11-07 19:26:50 +01:00
Ani Sinha
94cd94f1c0 acpi/tests/avocado/bits: enable console logging from bits VM
Console logs from the VM can be useful for debugging when things go wrong.
Other avocado tests enables them. This change enables console logging with the
following changes:
 - point to the newer bios bits image that actually enabled VM console.
 - change the bits test to drain the console logs from the VM and write the
   logs.
 - wait for SHUTDOWN event from QEMU so that console logs can be drained out
   of the socket before it is closed as a part of vm.wait().

Additionally, following two cosmetic changes have been made:
 - Removed VM QEMU command line logging as avocado framework already logs it.
   This is a minor cleanup along the way.
 - Update my email to my work email in the avocado acpi bios bits test.

CC: jsnow@redhat.com
Signed-off-by: Ani Sinha <anisinha@redhat.com>
Message-Id: <20231027032120.6012-3-anisinha@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2023-11-07 03:39:11 -05:00