Commit Graph

80 Commits

Author SHA1 Message Date
Alex Bennée
4189af72dd tests/avocado: use new rootfs for orangepi test
The old URL wasn't stable. I suspect the current URL will only be
stable for a few months so maybe we need another strategy for hosting
rootfs snapshots?

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221118113309.1057790-1-alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-11-23 10:58:48 +01:00
Thomas Huth
72cf57b074 tests/avocado: Update the URLs of the advent calendar images
The qemu-advent-calendar.org server will be decommissioned soon.
I've mirrored the images that we use for the QEMU CI to gitlab,
so update their URLs to point to the new location.

Message-Id: <20221121102436.78635-1-thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-11-23 10:58:48 +01:00
Stefan Hajnoczi
ccb8d593e0 pc,virtio: regression, test fixes
fixes regressions:
     virtio error message triggered by seabios
     failure in vhost due to VIRTIO_F_RING_RESET
     broken keyboard under seabios
 
 some biosbits test fixes
 
 there's still a known regression with migration and vsock,
 not fixed yet.
 
 Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
 -----BEGIN PGP SIGNATURE-----
 
 iQFDBAABCAAtFiEEXQn9CHHI+FuUyooNKB8NuNKNVGkFAmN8os4PHG1zdEByZWRo
 YXQuY29tAAoJECgfDbjSjVRpw+UH/2JOoxqzkkq4QyV5PzmjsiyyMbrBqFSOTro2
 98k9MwxNWHIk6fi3FVIz+LX1EdwDbMl44IHQ88xo7k39V7ThvVvLfjaQySSeoCfE
 04++AvkOQ/O8W7sob17r2iPWIHKPIndq8uB652T89o1I/nhmFMsUkdRWmVQ3XsDk
 DnPZxjBQnKY0oSvBd7SmSI3C+eA4lF1tht50ZxmOshLx4OSjK/maFVpIyLsUGTMn
 6OMxKffDjfvw5z94+pGN1RQm9xu7OwtFZyUnH16xoQUmWSXjzcYXUZ3buTY+eAyJ
 I5j9ox27Cqn1wGcypUiTCKLIExEMBfvNY4ovyRa1xvW+MXluVyE=
 =bF2U
 -----END PGP SIGNATURE-----

Merge tag 'for_upstream' of https://git.kernel.org/pub/scm/virt/kvm/mst/qemu into staging

pc,virtio: regression, test fixes

fixes regressions:
    virtio error message triggered by seabios
    failure in vhost due to VIRTIO_F_RING_RESET
    broken keyboard under seabios

some biosbits test fixes

there's still a known regression with migration and vsock,
not fixed yet.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

# -----BEGIN PGP SIGNATURE-----
#
# iQFDBAABCAAtFiEEXQn9CHHI+FuUyooNKB8NuNKNVGkFAmN8os4PHG1zdEByZWRo
# YXQuY29tAAoJECgfDbjSjVRpw+UH/2JOoxqzkkq4QyV5PzmjsiyyMbrBqFSOTro2
# 98k9MwxNWHIk6fi3FVIz+LX1EdwDbMl44IHQ88xo7k39V7ThvVvLfjaQySSeoCfE
# 04++AvkOQ/O8W7sob17r2iPWIHKPIndq8uB652T89o1I/nhmFMsUkdRWmVQ3XsDk
# DnPZxjBQnKY0oSvBd7SmSI3C+eA4lF1tht50ZxmOshLx4OSjK/maFVpIyLsUGTMn
# 6OMxKffDjfvw5z94+pGN1RQm9xu7OwtFZyUnH16xoQUmWSXjzcYXUZ3buTY+eAyJ
# I5j9ox27Cqn1wGcypUiTCKLIExEMBfvNY4ovyRa1xvW+MXluVyE=
# =bF2U
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 22 Nov 2022 05:22:06 EST
# gpg:                using RSA key 5D09FD0871C8F85B94CA8A0D281F0DB8D28D5469
# gpg:                issuer "mst@redhat.com"
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>" [full]
# gpg:                 aka "Michael S. Tsirkin <mst@redhat.com>" [full]
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17  0970 C350 3912 AFBE 8E67
#      Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA  8A0D 281F 0DB8 D28D 5469

* tag 'for_upstream' of https://git.kernel.org/pub/scm/virt/kvm/mst/qemu:
  virtio: disable error for out of spec queue-enable
  acpi/tests/avocado/bits: keep the work directory when BITS_DEBUG is set in env
  tests/avocado: configure acpi-bits to use avocado timeout
  MAINTAINERS: add mst to list of biosbits maintainers
  tests: acpi: x86: update expected DSDT after moving PRQx fields in _SB scope
  acpi: x86: move RPQx field back to _SB scope
  tests: acpi: whitelist DSDT before moving PRQx to _SB scope
  vhost: mask VIRTIO_F_RING_RESET for vhost and vhost-user devices

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-11-22 09:59:27 -05:00
Ani Sinha
04e5bd441a acpi/tests/avocado/bits: keep the work directory when BITS_DEBUG is set in env
Debugging bits issue often involves running the QEMU command line manually
outside of the avocado environment with the generated ISO. Hence, its
inconvenient if the iso gets cleaned up after the test has finished. This change
makes sure that the work directory is kept after the test finishes if the test
is run with BITS_DEBUG=1 in the environment so that the iso is available for use
with the QEMU command line.

CC: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Ani Sinha <ani@anisinha.ca>
Message-Id: <20221117113630.543495-1-ani@anisinha.ca>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-11-22 05:19:00 -05:00
John Snow
c4d4c40c51 tests/avocado: configure acpi-bits to use avocado timeout
Instead of using a hardcoded timeout, just rely on Avocado's built-in
test case timeout. This helps avoid timeout issues on machines where 60
seconds is not sufficient.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20221115212759.3095751-1-jsnow@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Ani Sinha <ani@anisinha.ca>
2022-11-22 05:19:00 -05:00
Alex Bennée
f22a80727f tests/avocado: skip aarch64 cloud TCG tests in CI
We now have a much lighter weight test in machine_aarch64_virt which
tests the full boot chain in less time. Rename the tests while we are
at it to make it clear it is a Fedora cloud image.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20221117172532.538149-11-alex.bennee@linaro.org>
2022-11-22 09:52:23 +00:00
Alex Bennée
ba5d1f23f7 tests/avocado: introduce alpine virt test for CI
The boot_linux tests download and run a full cloud image boot and
start a full distro. While the ability to test the full boot chain is
worthwhile it is perhaps a little too heavy weight and causes issues
in CI. Fix this by introducing a new alpine linux ISO boot in
machine_aarch64_virt.

This boots a fully loaded -cpu max with all the bells and whistles in
31s on my machine. A full debug build takes around 180s on my machine
so we set a more generous timeout to cover that.

We don't add a test for lesser GIC versions although there is some
coverage for that already in the boot_xen.py tests. If we want to
introduce more comprehensive testing we can do it with a custom kernel
and initrd rather than a full distro boot.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20221117172532.538149-10-alex.bennee@linaro.org>
2022-11-22 09:52:23 +00:00
Peter Maydell
5d25e1e02c tests/avocado: Raise timeout for boot_linux.py:BootLinuxPPC64.test_pseries_tcg
On my machine, a debug build of QEMU takes about 260 seconds to
complete this test, so with the current timeout value of 180 seconds
it always times out.  Double the timeout value to 360 so the test
definitely has enough time to complete.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221110142901.3832318-1-peter.maydell@linaro.org>
Message-Id: <20221117172532.538149-9-alex.bennee@linaro.org>
2022-11-22 09:52:23 +00:00
Cédric Le Goater
a4b14b46d9 tests/avocado/machine_aspeed.py: Reduce noise on the console for SDK tests
The Aspeed SDK images are based on OpenBMC which starts a lot of
services. The output noise on the console can break from time to time
the test waiting for the logging prompt.

Change the U-Boot bootargs variable to add "quiet" to the kernel
command line and reduce the output volume. This also drops the test on
the CPU id which was nice to have but not essential.

Signed-off-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20221104075347.370503-1-clg@kaod.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221117172532.538149-3-alex.bennee@linaro.org>
2022-11-22 09:48:57 +00:00
Peter Maydell
c4462523ff tests/avocado/boot_linux.py: Bump aarch64 virt test timeout to 720s
The two tests
tests/avocado/boot_linux.py:BootLinuxAarch64.test_virt_tcg_gicv2
tests/avocado/boot_linux.py:BootLinuxAarch64.test_virt_tcg_gicv3

take quite a long time to run, and the current timeout of 240s
is not enough for the tests to complete on slow machines:
we've seen these tests time out in the gitlab CI in the
'avocado-system-alpine' CI job, for instance. The timeout
is also insufficient for running the test with a debug build
of QEMU: on my machine the tests take over 10 minutes to run
in that config.

Push the timeout up to 720s so that the test definitely has
enough time to complete.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
2022-11-21 11:45:13 +00:00
Ani Sinha
1b7a07c441 acpi/tests/avocado/bits: some misc fixes
Most of the changes are trivial. The bits test timeout has now been increased
to 200 seconds in order to accommodate slower systems and fewer unnecessary
failures. Removed of the reference to non-existent README file in docs. Some
minor corrections in the doc file.

Signed-off-by: Ani Sinha <ani@anisinha.ca>
Message-Id: <20221117053644.516649-1-ani@anisinha.ca>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-11-17 09:58:22 +01:00
Ani Sinha
77a8e24c5b acpi/tests/avocado/bits: add acpi and smbios avocado tests that uses biosbits
This introduces QEMU acpi/smbios biosbits avocado test which is run
from within the python virtual environment. When the bits tests are run, bits
binaries are downloaded from an external repo/location, bios bits iso is
regenerated containing the acpi/smbios bits tests that are maintained as a part
of the QEMU source under tests/avocado/acpi-bits/bits-test . When the VM is
spawned with the iso, it runs the tests in batch mode and the results are pushed
out from the VM to the test machine where they are analyzed by this script and
pass/fail results are reported.

Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Maydell Peter <peter.maydell@linaro.org>
Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Michael Tsirkin <mst@redhat.com>
Signed-off-by: Ani Sinha <ani@anisinha.ca>
Message-Id: <20221021095108.104843-6-ani@anisinha.ca>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-11-02 06:56:32 -04:00
Ani Sinha
fa2cd84b22 acpi/tests/avocado/bits: add biosbits config file for running bios tests
This change adds initial biosbits config file that instructs biosbits to run
bios test suits in batch mode. Additionally acpi and smbios structures are also
dumped.

Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Maydell Peter <peter.maydell@linaro.org>
Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Michael Tsirkin <mst@redhat.com>
Signed-off-by: Ani Sinha <ani@anisinha.ca>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221021095108.104843-5-ani@anisinha.ca>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-11-02 06:56:32 -04:00
Ani Sinha
91cab435ec acpi/tests/avocado/bits: disable acpi PSS tests that are failing in biosbits
PSS tests in acpi test suite seems to be failing in biosbits. This is because
the test is unable to find PSS support in QEMU bios. Let us disable
them for now so that make check does not fail. We can fix the tests and
re-enable them later.

Example failure:

---- ACPI _PSS (Pstate) table conformance tests ----
[assert] _PSS must exist FAIL
  \_SB_.CPUS.C000
  No _PSS exists
Summary: 1 passed, 1 failed
---- ACPI _PSS (Pstate) runtime tests ----
[assert] _PSS must exist FAIL
  \_SB_.CPUS.C000
  No _PSS exists
Summary: 0 passed, 1 failed

Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Maydell Peter <peter.maydell@linaro.org>
Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Michael Tsirkin <mst@redhat.com>
Signed-off-by: Ani Sinha <ani@anisinha.ca>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221021095108.104843-4-ani@anisinha.ca>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-11-02 06:56:32 -04:00
Ani Sinha
87853babb3 acpi/tests/avocado/bits: initial commit of test scripts that are run by biosbits
This is initial commit of cpuid, acpi and smbios python test scripts for
biosbits to execute. No change has been made to them from the original code
written by the biosbits author Josh Triplett. They are required to be installed
into the bits iso file and then run from within the virtual machine booted off
with biosbits iso.

The test scripts have a ".py2" extension in order to prevent avocado from
loading them. They are written in python 2.7 and are run from within bios bits.
There is no need for avocado to try to load them and call out errors on python3
specific syntaxes.

The original location of these tests are here:
https://github.com/biosbits/bits/blob/master/python/testacpi.py
https://github.com/biosbits/bits/blob/master/python/smbios.py
https://github.com/biosbits/bits/blob/master/python/testcpuid.py

For QEMU, we maintain a fork of the above repo here with numerious fixes:
https://gitlab.com/qemu-project/biosbits-bits

The acpi test for example is maintained here in the fork:
https://gitlab.com/qemu-project/biosbits-bits/-/raw/master/python/testacpi.py

Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Maydell Peter <peter.maydell@linaro.org>
Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Michael Tsirkin <mst@redhat.com>
Signed-off-by: Ani Sinha <ani@anisinha.ca>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221021095108.104843-2-ani@anisinha.ca>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-11-02 06:56:32 -04:00
Alex Bennée
500f73b1aa tests/avocado: disable sh4 rd2 tests on Gitlab
Running repeated invocations on a number of test boxes show a fairly
high error rate:

  $ retry.py -n 100 -c -- ./tests/venv/bin/avocado run tests/avocado/boot_linux_console.py:BootLinuxConsole.test_sh4_r2d
  retry.py called with ['./tests/venv/bin/avocado', 'run', 'tests/avocado/boot_linux_console.py:BootLinuxConsole.test_sh4_r2d']
  Results:
  Run, Ret, Pass/Fail, Time, Total Pass, Total Run
  ...
  Results summary:
  0: 94 times (94.00%), avg time 2.254 (0.00 varience/0.04 deviation)
  1: 3 times (3.00%), avg time 1.837 (0.02 varience/0.14 deviation)
  8: 3 times (3.00%), avg time 91.288 (0.02 varience/0.15 deviation)

Examining the logs they fall into various categories of un-handled
unaligned access by user space and unexpected FPU usage by the kernel
which ultimately lead to the failure to reach the login prompt. This
could be bugs in the translator that only get hit occasionally or just
a flaky kernel - its hard to tell. To avoid these failures gating CI
lets skip on GitLab.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Magnus Damm <magnus.damm@gmail.com>
Message-Id: <20221027183637.2772968-21-alex.bennee@linaro.org>
2022-10-31 20:37:59 +00:00
Peter Maydell
cc45d25c65 tests/avocado: raspi2_initrd: Wait for guest shutdown message before stopping
The avocado test
 tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_raspi2_initrd
finishes with

    exec_command(self, 'halt')
    # Wait for VM to shut down gracefully
    self.vm.wait()

In theory this should be fine. In practice it runs into two bugs:

 * when the test calls self.vm.wait() Avocado closes the socket
   connection to the guest serial console immediately, so the
   avocado logs don't have the last part of the guest output:
   https://gitlab.com/qemu-project/qemu/-/issues/1265
 * when the socket is closed, a bug in the QEMU socket chardev
   means that it loses any data that the guest UART has not
   yet consumed. This means that the guest doesn't always read
   the full 'halt' command string, so the test intermittently
   fails with a timeout:
   https://gitlab.com/qemu-project/qemu/-/issues/1264

Work around both of these by waiting for the guest to print the
string that means it has completed the shutdown process.  This fixes
a very long standing intermittent failure in this test.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/636
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20221020102012.3015662-1-peter.maydell@linaro.org>
Message-Id: <20221027183637.2772968-20-alex.bennee@linaro.org>
2022-10-31 20:37:59 +00:00
Alex Bennée
5104b73824 tests/avocado: set -machine none for userfwd and vnc tests
These are exercising core QEMU features and don't actually run code.
Not specifying a machine will fail when avocado chooses the native
arch binary to run. Be explicit.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221027183637.2772968-19-alex.bennee@linaro.org>
2022-10-31 20:37:59 +00:00
Alex Bennée
e6025635db tests/avocado: extend the timeout for x86_64 tcg tests
These are timing out on gitlab.

Acked-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221027183637.2772968-7-alex.bennee@linaro.org>
2022-10-31 20:37:58 +00:00
Cédric Le Goater
7b5093b85b tests/avocado/machine_aspeed.py: Fix typos on buildroot
Replace 'buidroot' and 'builroot' by 'buildroot'.

Fixes: f7bc7da072 ("test/avocado/machine_aspeed.py: Add tests using buildroot images")
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Joel Stanley <joel@jms.id.au>
Message-Id: <20220923084803.498337-2-clg@kaod.org>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
2022-10-24 11:20:15 +02:00
Peter Maydell
0793fe014c tests/avocado: Add missing require_netdev('user') checks
Some avocado tests fail if QEMU was built without libslirp. Add
require_netdev('user') checks where necessary:

These tests try to ping 10.0.2.2 and expect it to succeed:
  boot_linux_console.py:BootLinuxConsole.test_arm_emcraft_sf2
  boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd
  ppc_bamboo.py:BambooMachine.test_ppc_bamboo

These tests run a commandline that includes '-net user':
  machine_aspeed.py:AST2x00Machine.test_arm_ast2500_evb_builroot
  (and others that use the do_test_arm_aspeed_buidroot_start()
  or do_test_arm_aspeed_sdk_start() helper functions)

These changes seem to be sufficient for 'make check-avocado'
to not fail on a --disable-slirp build.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20221001195224.2453581-1-peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-10-11 17:25:50 +02:00
Jason A. Donenfeld
3dbc5fdacb target/s390x: support PRNO_TRNG instruction
In order for hosts running inside of TCG to initialize the kernel's
random number generator, we should support the PRNO_TRNG instruction,
backed in the usual way with the qemu_guest_getrandom helper. This is
confirmed working on Linux 5.19.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Message-Id: <20220921100729.2942008-2-Jason@zx2c4.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
[thuth: turn prno-trng off in avocado test to avoid breaking it]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-09-26 17:23:11 +02:00
Alex Bennée
8f58f0c7d9 tests/avocado: reduce the default timeout to 120s
We should be aiming to keep our tests under 2 minutes so lets reduce
the default timeout to that. Tests that we know take longer should
explicitly set a longer timeout.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220914155950.804707-8-alex.bennee@linaro.org>
2022-09-20 16:27:04 +01:00
Alex Bennée
92f8e8e2fe tests/avocado: split the AST2x00Machine classes
The SDK tests take a lot longer to run and hence need a longer
timeout. As they run well over the 60 second maximum for CI lets also
disable them for CI as well.

I suspect they also suffer from the inability to detect the login
prompt due to no newlines being processed.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220914155950.804707-7-alex.bennee@linaro.org>
2022-09-20 16:26:56 +01:00
Alex Bennée
b247dba067 tests/avocado: add explicit timeout for ppc64le TCG tests
We don't want to rely on the soon to be reduced default time. These
tests are still slow for something we want to run in CI though.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220914155950.804707-6-alex.bennee@linaro.org>
2022-09-20 15:14:59 +01:00
Alex Bennée
06a4c9cf24 tests/avocado: add explicit timeout for s390 TCG tests
We don't want to rely on the soon to be reduced default time. These
tests are still slow for something we want to run in CI though.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220914155950.804707-5-alex.bennee@linaro.org>
2022-09-20 15:14:59 +01:00
Alex Bennée
c7f059ac11 tests/avocado: add explicit timeout for Aarch64 TCG tests
We don't want to rely on the soon to be reduced default time. These
tests are still slow for something we want to run in CI though.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220914155950.804707-4-alex.bennee@linaro.org>
2022-09-20 15:14:59 +01:00
Thomas Huth
7fcc3f7595 tests/avocado/boot_linux_console: Fix the test_aarch64_xlnx_versal_virt test
The assets that this test tries to download have been removed from the
server. Update to a newer version to get it working again.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220829080940.110831-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220914155950.804707-3-alex.bennee@linaro.org>
2022-09-20 15:14:59 +01:00
Thomas Huth
da55be562e tests/avocado/migration: Get find_free_port() from the ports
In upstream Avocado, the find_free_port() function is not available
from "network" anymore, but must be used via "ports", see:

 https://github.com/avocado-framework/avocado/commit/22fc98c6ff76cc55c48

To be able to update to a newer Avocado version later, let's use
the new way for accessing the find_free_port() function here.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220829121939.209329-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-08-30 07:55:43 +02:00
Thomas Huth
0aeefd5fcb tests/avocado: Fix trivial typo
The intention was likely to use "intend" instead of "indent" here.

Message-Id: <20220824080926.568935-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-08-29 15:28:51 +02:00
Thomas Huth
0fc389fe92 tests/avocado: Do not run tests that require libslirp if it is not available
Some avocado tests blindly assume that QEMU has been compiled with libslirp
enabled and fail badly if it is missing. Add a proper check to cancel the
tests in this case.

Message-Id: <20220824151122.704946-6-thuth@redhat.com>
Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-08-29 15:28:51 +02:00
Alex Bennée
48acf68cfe tests/avocado: push default timeout to QemuBaseTest
All of the QEMU tests eventually end up derrived from this class. Move
the default timeout from LinuxTest to ensure we catch them all. We
keep the 15 minute timeout as currently some of the more heavyweight
CFI and TCG tests can overrun. We should aim to drop it down to 2
minutes which is a more reasonable target for tests to aim for but we
want to get this release out.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
[AJB: revert to 15 min timeout for v2]
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220822165608.2980552-2-alex.bennee@linaro.org>
2022-08-24 10:14:42 +01:00
Alex Bennée
65711f9a87 tests/avocado: apply a band aid to aspeed-evb login
This is really a limitation of the underlying console code which
doesn't allow us to detect the login: and following "#" prompts
because it reads input line wise. By adding a small delay we ensure
that the login prompt has appeared so we don't accidentally spaff the
shell commands to a confused getty in the guest.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Acked-by: John Snow <jsnow@redhat.com>
Message-Id: <20220811151413.3350684-8-alex.bennee@linaro.org>
2022-08-16 09:57:12 +01:00
Alex Bennée
b1ceae2f5b tests/avocado: add timeout to the aspeed tests
On some systems the test can hang. At least defining a timeout stops
it from hanging forever.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220811151413.3350684-7-alex.bennee@linaro.org>
2022-08-16 09:57:12 +01:00
Pavel Dovgalyuk
c05a88c6b2 tests/avocado: fix replay-linux test
Last line of the test is missing by accident.
This patch fixes the script.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Message-Id: <165943656662.362178.2086588841425038338.stgit@pasha-ThinkPad-X280>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-08-07 18:51:35 +02:00
Cédric Le Goater
bceb4d994d test/avocado/machine_aspeed.py: Add SDK tests
The Aspeed SDK kernel usually includes support for the lastest HW
features. This is interesting to exercise QEMU and discover the gaps
in the models.

Add extra I2C tests for the AST2600 EVB machine to check the new
register interface.

Message-Id: <20220707091239.1029561-1-clg@kaod.org>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
2022-07-14 16:24:38 +02:00
Peter Delevoryas
50b13d31f4 avocado: Fix BUILD_DIR if it's equal to SOURCE_DIR
I like to build QEMU from the root source directory [*], rather
than cd'ing into the build directory. This code may as well include
a search path for that, so that you can run avocado tests individually
without specifying "-p qemu_bin=build/qemu-system-arm" manually.

[*] See commit dedad02720 ("configure: add support for pseudo-"in source tree" builds")

Signed-off-by: Peter Delevoryas <peter@pjd.dev>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220702185604.46643-1-peter@pjd.dev>
[PMD: Mention commit dedad02720]
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2022-07-13 00:06:02 +02:00
Cédric Le Goater
3302184f7f test/avocado/machine_aspeed.py: Add an I2C RTC test
Add an RTC device and check that the output of the hwclock command
matches the current year.

Signed-off-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Joel Stanley <joel@jms.id.au>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
2022-06-22 09:49:33 +02:00
Cédric Le Goater
61cf757d15 test/avocado/machine_aspeed.py: Add I2C tests to ast2600-evb
Create a named I2C temperature sensor device on the command line,
instantiate device from Linux since it is not part of the device tree,
and check the temperature is correctly reported under sysfs.

Signed-off-by: Cédric Le Goater <clg@kaod.org>
2022-06-22 09:49:33 +02:00
Cédric Le Goater
7a7308eae0 test/avocado/machine_aspeed.py: Add I2C tests to ast2500-evb
Create a named I2C temperature sensor device on the command line,
instantiate device from Linux since it is not part of the device tree,
and check the temperature is correctly reported under sysfs.

Signed-off-by: Cédric Le Goater <clg@kaod.org>
2022-06-22 09:49:33 +02:00
Cédric Le Goater
f7bc7da072 test/avocado/machine_aspeed.py: Add tests using buildroot images
Buildroot images are smaller than the OpenBMC images and faster to
run. Built from source using :

  http://patchwork.ozlabs.org/project/buildroot/list/?series=303465

Signed-off-by: Cédric Le Goater <clg@kaod.org>
2022-06-22 09:49:33 +02:00
Cédric Le Goater
341e21fa13 test/avocado/machine_aspeed.py: Move OpenBMC tests
It's easier to run. Keep test_arm_ast2600_debian() under the
boot_linux_console.py file because it requires the extract_from_deb()
helper. We could remove it when we have tests for the AST2600.

Signed-off-by: Cédric Le Goater <clg@kaod.org>
2022-06-22 09:49:33 +02:00
John Snow
0e7647aa8a tests: install "qemu" namespace package into venv
This patch adds the "qemu" namespace package to the $build/tests/venv
directory. It does so in "editable" mode, which means that changes to
the source python directory will actively be reflected by the venv.

This patch also then removes any sys.path hacking from the avocado test
scripts directly. By doing this, the environment of where to find these
packages is managed entirely by the virtual environment and not by the
scripts themselves.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220526000921.1581503-7-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-06 09:26:54 +02:00
Pavel Dovgalyuk
9b59af628c tests/avocado: add replay Linux test for Aarch64 machines
This patch adds two tests for replaying Linux boot process
on Aarch64 platform.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Message-Id: <165364841373.688121.8868079200312201658.stgit@pasha-ThinkPad-X280>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-06 09:26:54 +02:00
Pavel Dovgalyuk
3cd1dce33d tests/avocado: add replay Linux tests for virtio machine
This patch adds two tests for replaying Linux boot process
on x86_64 virtio platform.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Message-Id: <165364840811.688121.11931681195199516354.stgit@pasha-ThinkPad-X280>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-06 09:26:54 +02:00
Pavel Dovgalyuk
c9d27a0f9f tests/avocado: update replay_linux test
This patch updates replay_linux test to make it compatible with
new LinuxTest class.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Message-Id: <165364840253.688121.10404266209986316381.stgit@pasha-ThinkPad-X280>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-06 09:26:54 +02:00
Jamin Lin
438eff633a test/avocado/machine_aspeed.py: Add ast1030 test case
Add test case to test "ast1030-evb" machine with zephyr os

Signed-off-by: Troy Lee <troy_lee@aspeedtech.com>
Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Signed-off-by: Steven Lee <steven_lee@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220401083850.15266-10-jamin_lin@aspeedtech.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
2022-05-02 17:03:04 +02:00
Vladimir Sementsov-Ogievskiy
0d90da8203 avocado/vnc: add test_change_listen
Add simple test-case for new display-update qmp command.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@openvz.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220401143936.356460-4-vsementsov@openvz.org>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2022-04-27 07:51:01 +02:00
Richard Henderson
2d20a57453 Testing, docs and gdbstub updates:
- make -M virt test exercise -cpu max
   - document how binfmt_misc docker works
   - clean-up the devel TOC generation
   - clean-up check-tcg cross-compile behaviour
   - fix byte swap error in xmm gdbstub access
   - add float_convd test with reference files
   - more reference files for float_convs
   - more cleanly handle gdb crashing during check-tcg
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmJgIgsACgkQ+9DbCVqe
 KkRs3Qf/WnL4YV3l8jO/wEVbls/57aLPk+ak1GuvXJ+iM9gH8Qz6WZxIJIEhhHlu
 ZEixCewahTn8POMMRo4JAr0bRgtfBuh717GerXObiHcS3OuLsGM8rYP2Z1xqKL3L
 4pR8VqhyUq/Jyl/6MPN5OZB0AdEPIdI5MuflckCeDcFaowpthLjwHao07hG/FU0s
 wQYS7aYTZT33V2Xm6xlePEEMq8YMPCJj00HF3Ljg4eUOmb+C+csFXMQtotsBJRCg
 mC/T2U0IFbrQUkkWJqVmRCPwKraQGDMn6POk298siRWE0kV4BmH8mnmN+/Jxhqgl
 QfFhQrsBxmPPG5TfQhEmlHfQ5EARLQ==
 =8q12
 -----END PGP SIGNATURE-----

Merge tag 'pull-fixes-for-7.1-200422-1' of https://github.com/stsquad/qemu into staging

Testing, docs and gdbstub updates:

  - make -M virt test exercise -cpu max
  - document how binfmt_misc docker works
  - clean-up the devel TOC generation
  - clean-up check-tcg cross-compile behaviour
  - fix byte swap error in xmm gdbstub access
  - add float_convd test with reference files
  - more reference files for float_convs
  - more cleanly handle gdb crashing during check-tcg

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmJgIgsACgkQ+9DbCVqe
# KkRs3Qf/WnL4YV3l8jO/wEVbls/57aLPk+ak1GuvXJ+iM9gH8Qz6WZxIJIEhhHlu
# ZEixCewahTn8POMMRo4JAr0bRgtfBuh717GerXObiHcS3OuLsGM8rYP2Z1xqKL3L
# 4pR8VqhyUq/Jyl/6MPN5OZB0AdEPIdI5MuflckCeDcFaowpthLjwHao07hG/FU0s
# wQYS7aYTZT33V2Xm6xlePEEMq8YMPCJj00HF3Ljg4eUOmb+C+csFXMQtotsBJRCg
# mC/T2U0IFbrQUkkWJqVmRCPwKraQGDMn6POk298siRWE0kV4BmH8mnmN+/Jxhqgl
# QfFhQrsBxmPPG5TfQhEmlHfQ5EARLQ==
# =8q12
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 20 Apr 2022 08:08:59 AM PDT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* tag 'pull-fixes-for-7.1-200422-1' of https://github.com/stsquad/qemu: (25 commits)
  tests/guest-debug: better handle gdb crashes
  target/i386: fix byte swap issue with XMM register access
  tests/tcg: add missing reference files for float_convs
  tests/tcg: add float_convd test
  tests/tcg: remove duplicate sha512-sse case
  tests/tcg: fix non-static build
  tests/docker: remove SKIP_DOCKER_BUILD
  tests/tcg: isolate from QEMU's config-host.mak
  tests/tcg: invoke Makefile.target directly from QEMU's makefile
  tests/tcg: list test targets in Makefile.prereqs
  tests/tcg: prepare Makefile.prereqs at configure time
  tests/tcg: remove CONFIG_USER_ONLY from config-target.mak
  tests/tcg: remove CONFIG_LINUX_USER from config-target.mak
  tests/tcg: add compiler test variables when using containers
  tests/docker: do not duplicate rules for hexagon-cross
  tests/docker: simplify docker-TEST@IMAGE targets
  tests/docker: remove unnecessary filtering of $(DOCKER_IMAGES)
  tests/docker: inline variable definitions or move close to use
  tests/docker: remove unnecessary default definitions
  tests/docker: remove dead variable
  ...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2022-04-20 11:13:08 -07:00
Alex Bennée
11593544df tests/avocado: update aarch64_virt test to exercise -cpu max
The Fedora 29 kernel is quite old and importantly fails when running
in LPA2 scenarios. As it's not really exercising much of the CPU space
replace it with a custom 5.16.12 kernel with all the architecture
options turned on. There is a minimal buildroot initramfs included in
the kernel which has a few tools for stress testing the memory
subsystem. The userspace also targets the Neoverse N1 processor so
would fail with a v8.0 cpu like cortex-a53.

While we are at it move the test into its own file so it can have an
assigned maintainer.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Tested-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220419091020.3008144-2-alex.bennee@linaro.org>
2022-04-20 16:04:16 +01:00