Commit Graph

194 Commits

Author SHA1 Message Date
Cornelia Huck
085cec59cf tests/acceptance: test s390x zpci fid propagation
Verify that a fid specified on the command line shows up correctly
as the function_id in the guest.

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
[re-formatted overlong lines]
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201130180216.15366-4-cohuck@redhat.com>
2020-12-11 11:38:10 +01:00
Cornelia Huck
864852ccb8 tests/acceptance: verify s390x device detection
The kernel/initrd combination does not provide the virtio-net
driver; therefore, simply check whether the presented device type
is indeed virtio-net for the two virtio-net-{ccw,pci} devices.

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
[re-formatted overlong lines]
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201130180216.15366-3-cohuck@redhat.com>
2020-12-11 11:38:10 +01:00
Cornelia Huck
083470b579 tests/acceptance: test virtio-ccw revision handling
The max_revision prop of virtio-ccw devices can be used to force
an older revision for compatibility handling. The easiest way to
check this is to force a device to revision 0, which turns off
virtio-1.

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
[re-formatted overlong lines]
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201130180216.15366-2-cohuck@redhat.com>
2020-12-11 11:38:10 +01:00
Cornelia Huck
2d9ca5a37b tests/acceptance: add a test for devices on s390x
This adds a very basic test for checking that we present devices
in a way that Linux can consume: boot with both virtio-net-ccw and
virtio-net-pci attached and then verify that Linux is able to see
and detect these devices.

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201126130158.1471985-1-cohuck@redhat.com>
2020-12-11 11:38:10 +01:00
Alex Bennée
69272bec1a tests/avocado: clean-up socket directory after run
Previously we were leaving temporary directories behind. While the
QEMUMachine does make efforts to clean up after itself the directory
belongs to the calling function. We use TemporaryDirectory to wrap
this although we explicitly clear the reference in tearDown() as it
doesn't get cleaned up otherwise.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20201117173635.29101-4-alex.bennee@linaro.org>
2020-11-23 09:52:24 +00:00
Alex Bennée
8c175c63ee tests: add prefixes to the bare mkdtemp calls
The first step to debug a thing is to know what created the thing in
the first place. Add some prefixes so random tmpdir's have something
grep in the code.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201117173635.29101-3-alex.bennee@linaro.org>
2020-11-23 09:51:43 +00:00
Peter Maydell
cb5ed407a1 Fix Lesser GPL license versions (should be "2.1" and not "2")
-----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAl+xVToRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbX2ow/+MKI6s5QeaAGMJfEG1KkfCCfrybJSy+1X
 BN7ANBxkwndPIZEDfE1Hnxh0nH+jYLhtVm1N7F7EJ2PuyEw+yoVIVE4xp1HFQYEb
 gowE2QSjc1TqyoJhGu/Pj6cGxKFp14ZNIj8w1gKttgjK7doDom76ZZFfRoSc5Qw4
 EoMe5Bnit4qOrjCKnfa0XEMlcB598KKyTxPiwlpA2Tnf2Hl7dAwnkT+fbaQ0iTcL
 f0t8O1GVqyrQSkadf+n94l4kwATk92A7ZLrq7imImRfFo9JwmaAl8rZj3KUdrUax
 IckyUWsiQ+RyH7Th7h8F5jqPLjDYgNOfx/BMaNcLUNBLyaco4lcZT3X0JUxuzV7Y
 KPMk1PuUzv2yhEu1QV6o/0WQ/AwcZNQbNR98z5rqZsCG2rF+AVSMAEz97B7AIfbb
 hJH3f7lmIPcAqkdGXxy/PphzD8qVpm0bTQUWVJi0+ASw73ucTSzgWV29g/b8gh+X
 DzjFAnwYY21oUH8nhZrraQRd1Nu+GDF/+BXrIsBvznxNDyyYS12V2PurWaUwGX5o
 TYLRwMHlvOlypEh0C2mjU+X/1TmLoMueSouDQ22R01sVIUG7PxbNCn46BJTGnxdE
 6kd6rEC804vC2n96WFaHozBbOBwssFDrVsuwUg5HCTIu6BFyCRVTa2P5WxzYPSdM
 lTs9rBwdXPc=
 =GI5d
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2020-11-15' into staging

Fix Lesser GPL license versions (should be "2.1" and not "2")

# gpg: Signature made Sun 15 Nov 2020 16:20:10 GMT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2020-11-15: (26 commits)
  nomaintainer: Fix Lesser GPL version number
  test: Fix LGPL information in the file headers
  tests/acceptance: Fix LGPL information in the file headers
  tests/migration: Fix LGPL information in the file headers
  sparc tcg cpus: Fix Lesser GPL version number
  e1000e: Fix Lesser GPL version number
  x86 hvf cpus: Fix Lesser GPL version number
  nvdimm: Fix Lesser GPL version number
  w32: Fix Lesser GPL version number
  tpm: Fix Lesser GPL version number
  overall/alpha tcg cpus|hppa: Fix Lesser GPL version number
  overall usermode...: Fix Lesser GPL version number
  migration: Fix Lesser GPL version number
  parallel nor flash: Fix Lesser GPL version number
  arm tcg cpus: Fix Lesser GPL version number
  x86 tcg cpus: Fix Lesser GPL version number
  linux user: Fix Lesser GPL version number
  usb: Fix Lesser GPL version number
  tricore tcg cpus: Fix Lesser GPL version number
  xtensa tcg cpus: Fix Lesser GPL version number
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-11-16 17:00:36 +00:00
Philippe Mathieu-Daudé
b48580ad92 tests/acceptance: Disable Spartan-3A DSP 1800A test
This test is regularly failing on CI:

   (05/34) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_microblaze_s3adsp1800:
  Linux version 4.11.3 (thuth@thuth.remote.csb) (gcc version 6.4.0 (Buildroot 2018.05.2) ) #5 Tue Dec 11 11:56:23 CET 2018
  ...
  Freeing unused kernel memory: 1444K
  This architecture does not have kernel memory protection.
  [nothing happens here]
  Runner error occurred: Timeout reached (90.91 s)

This is a regression. Until someone figure out the problem,
disable the test to keep CI pipeline useful.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201109091719.2449141-1-f4bug@amsat.org>
Message-Id: <20201110192316.26397-9-alex.bennee@linaro.org>
2020-11-16 11:08:04 +00:00
Gan Qixin
6a2cd9570e tests/acceptance: Fix LGPL information in the file headers
There never was a "Lesser GPL version 2.0", It is either "GPL version 2.0"
or "Lesser GPL version 2.1". This patch replaces all "Lesser GPL version 2.0"
with "Lesser GPL version 2.1" in the tests/acceptance folder.

Signed-off-by: Gan Qixin <ganqixin@huawei.com>
Message-Id: <20201110184223.549499-3-ganqixin@huawei.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-11-15 17:04:40 +01:00
Max Reitz
af1bb3fe7f tests/acceptance: Add virtiofs_submounts.py
This test invokes several shell scripts to create a random directory
tree full of submounts, and then check in the VM whether every submount
has its own ID and the structure looks as expected.

(Note that the test scripts must be non-executable, so Avocado will not
try to execute them as if they were tests on their own, too.)

Because at this commit's date it is unlikely that the Linux kernel on
the image provided by boot_linux.py supports submounts in virtio-fs, the
test will be cancelled if no custom Linux binary is provided through the
vmlinuz parameter.  (The on-image kernel can be used by providing an
empty string via vmlinuz=.)

So, invoking the test can be done as follows:
$ avocado run \
    tests/acceptance/virtiofs_submounts.py \
    -p vmlinuz=/path/to/linux/build/arch/x86/boot/bzImage

This test requires root privileges (through passwordless sudo -n),
because at this point, virtiofsd requires them.  (If you have a
timestamp_timeout period for sudoers (e.g. the default of 5 min), you
can provide this by executing something like "sudo true" before invoking
Avocado.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201102161859.156603-8-mreitz@redhat.com>
Tested-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-11-02 19:23:48 +00:00
Max Reitz
0a7798037a tests/acceptance/boot_linux: Accept SSH pubkey
Let download_cloudinit() take an optional pubkey, which subclasses of
BootLinux can pass through setUp().

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20201102161859.156603-7-mreitz@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-11-02 19:22:51 +00:00
Peter Maydell
a5e7fb4d20 Testing and gitdm updates
- add some more individual contributors
   - include SDL2 in centos images
   - skip checkpatch check when no commits found
   - use random port for gdb reverse debugging
   - make gitlab use it's own mirrors to clone
   - fix detection of make -nqp
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAl+X7qsACgkQ+9DbCVqe
 KkSpmAf/To0P/c/B6qMEvNfCiWtn/Nuypau8W60q/ZvE/J4IzFarwa5uc7COmyNR
 xjvxdoeKp17ykFdwBzGhiKeO83SO8DUH1JmzA6STs4RH5VSFGYycuE0cQ9KugrpE
 4yOruOdEgAn2+35oYalE9u6/EugdIVBZsjMS/ofsgR9ysYPuju4Q9by6DEK6ArW0
 AmdwJzKAYw75v+D2wKNq9coSWvbqWClsV6j0gaKgn+OQYqWo25T/DzWdr1okCRET
 TvB0HZLCDQHz9Friq5PtBtVsmNbZaL+CM9RRekFBGUFEvZE8i4VEDHFfzXsN3U0w
 VA5yhlBJzl1/prBQe1qRbrZQL4x58w==
 =Bnt1
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-and-misc-271020-1' into staging

Testing and gitdm updates

  - add some more individual contributors
  - include SDL2 in centos images
  - skip checkpatch check when no commits found
  - use random port for gdb reverse debugging
  - make gitlab use it's own mirrors to clone
  - fix detection of make -nqp

# gpg: Signature made Tue 27 Oct 2020 09:55:55 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-and-misc-271020-1:
  makefile: handle -n / -k / -q correctly
  gitlab-ci: Clone from GitLab itself
  tests/acceptance: pick a random gdb port for reverse debugging
  scripts: fix error from checkpatch.pl when no commits are found
  gitlab: skip checkpatch.pl checks if no commit delta on branch
  tests/docker/dockerfiles/centos: Use SDL2 instead of SDL1
  contrib/gitdm: Add more individual contributors
  Adding ani's email as an individual contributor

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-28 20:40:09 +00:00
Alex Williamson
33dc9914ea Revert series: virtiofsd: Announce submounts to the guest
This reverts the following commits due to their basis on a bogus
linux kernel header update:

c93a656f7b ("tests/acceptance: Add virtiofs_submounts.py")
45ced7ca2f ("tests/acceptance/boot_linux: Accept SSH pubkey")
08dce386e7 ("virtiofsd: Announce sub-mount points")
eba8b096c1 ("virtiofsd: Store every lo_inode's parent_dev")
ede24b6be7 ("virtiofsd: Add fuse_reply_attr_with_flags()")
e2577435d3 ("virtiofsd: Add attr_flags to fuse_entry_param")
2f10415abf ("virtiofsd: Announce FUSE_ATTR_FLAGS")
97d741cc96 ("linux/fuse.h: Pull in from Linux")

Cc: Max Reitz <mreitz@redhat.com>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 160385090886.20017.13382256442750027666.stgit@gimli.home
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-28 13:17:32 +00:00
Peter Maydell
cfc1105649 Acceptance testing patches
- More ARM tests
 - Documentation update
 
 CI jobs results:
 . https://cirrus-ci.com/build/5754555684093952
 . https://gitlab.com/philmd/qemu/-/pipelines/207807648
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAl+XHkwACgkQ4+MsLN6t
 wN4WJBAAxfsfZV3DjoKP/mAjVKJCHkA82MzzoN+YH5WQs4QG1Hytod7OyWC101xj
 wPuL85NR+bo+he12Uylmbm35TCyKxH3kOk2GAqiHvS16mxNZtAssehVt5a1hoUlw
 N9R37dtccGbURX3WzEgJralaZaST96ZbP+y76A95ygriXLnZRisVJG0t/FKBb22m
 8B8+lgHpb11hFop/COY93Aa/BPx9S7cPpBQX4p8e/gXhdJTtqzbhWYSkLO60FqJh
 l1D8KAB7dUtVPoW1ZPSL0YsSGVlxKslOQ8T2Ikrhu8whDZNpYWaoXP0nK3aFK5Th
 jSYObXj/fXtj+APJZILoYAzEtlFsX0FFPfPZy2ffvX6+P1S4COsU2mkN9jJSu42C
 i8Ii5r1WAN6OPFbgPVQ2tQP9jq+scwBUT50GIzgLvja2dWPBM7drxB/hAByItgRt
 ARaDupnC6ShXPLMPm14fSfThyQQw+wiTHo9sZaks5NuoOz9WAh6XUK3JLR5S5zEG
 OxYunho2zSoqYAmnYtymNQ3L0ABW679EbbB2uZJX9m4iFGSxnNJoFFZyAXsZm/tM
 sRQBfTdBH1OCsVdy9/uYrNdZmj0QMBU5mJmv8Qsv722PLV8GuBF8O64C4F4Y42BP
 ApADllyRIONxXm95iBXISuZAF7qAMfGcvI9ldpUyetUT59oz4YI=
 =5Xi7
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/acceptance-testing-20201026' into staging

Acceptance testing patches

- More ARM tests
- Documentation update

CI jobs results:
. https://cirrus-ci.com/build/5754555684093952
. https://gitlab.com/philmd/qemu/-/pipelines/207807648

# gpg: Signature made Mon 26 Oct 2020 19:06:52 GMT
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/acceptance-testing-20201026:
  docs/devel/testing.rst: Update outdated Avocado URLs
  tests/acceptance: Allow running Orange Pi test using cached artifacts
  tests/acceptance: Add a 'virt_kvm' test using the GICv3
  tests/boot_linux_console: Boot Trusted Firmware-A on the Raspberry Pi 3

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-27 16:58:39 +00:00
Alex Bennée
0395b19438 tests/acceptance: pick a random gdb port for reverse debugging
Currently the test randomly fails if you are using a shared machine
due to contention on the well known port 1234. We can ameliorate this
a bit by picking a random non-ephemeral port although it doesn't
totally avoid the problem. While we could use a totally unique socket
address for debugging it is fiddly to probe for gdb support. While gdb
socket debugging is not yet ubiquitous this a sub-optimal but workable
option.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201021163136.27324-7-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2020-10-27 09:53:51 +00:00
Max Reitz
c93a656f7b tests/acceptance: Add virtiofs_submounts.py
This test invokes several shell scripts to create a random directory
tree full of submounts, and then check in the VM whether every submount
has its own ID and the structure looks as expected.

(Note that the test scripts must be non-executable, so Avocado will not
try to execute them as if they were tests on their own, too.)

Because at this commit's date it is unlikely that the Linux kernel on
the image provided by boot_linux.py supports submounts in virtio-fs, the
test will be cancelled if no custom Linux binary is provided through the
vmlinuz parameter.  (The on-image kernel can be used by providing an
empty string via vmlinuz=.)

So, invoking the test can be done as follows:
$ avocado run \
    tests/acceptance/virtiofs_submounts.py \
    -p vmlinuz=/path/to/linux/build/arch/x86/boot/bzImage

This test requires root privileges (through passwordless sudo -n),
because at this point, virtiofsd requires them.  (If you have a
timestamp_timeout period for sudoers (e.g. the default of 5 min), you
can provide this by executing something like "sudo true" before invoking
Avocado.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-9-mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-10-26 18:35:32 +00:00
Max Reitz
45ced7ca2f tests/acceptance/boot_linux: Accept SSH pubkey
Let download_cloudinit() take an optional pubkey, which subclasses of
BootLinux can pass through setUp().

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-8-mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: WIllian Rampazzo <willianr@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-10-26 18:35:32 +00:00
Philippe Mathieu-Daudé
9eb716513d tests/acceptance: Allow running Orange Pi test using cached artifacts
Unfortunately the Armbian 19.11.3 image has been removed from the
dl.armbian.com file server. Developers having the artifact cached
can still run the test. Allow them to, until we find a proper
solution to share binaries with the whole community.

This avoids (when file manually added to cache):

  BootLinuxConsole.test_arm_orangepi_bionic_19_11: CANCEL: Missing asset https://dl.armbian.com/orangepipc/archive/Armbian_19.11.3_Orangepipc_bionic_current_5.3.9.7z (1.06 s)

Reported-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Bin Meng <bin.meng@windriver.com>
Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Message-Id: <20201023131808.3198005-3-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-10-26 18:20:51 +01:00
Philippe Mathieu-Daudé
9923375807 tests/acceptance: Add a 'virt_kvm' test using the GICv3
The current 'virt_kvm' test is restricted to GICv2, but can also
work with a GICv3. Duplicate it but add a GICv3 test which can be
tested on some hardware.

Noticed while running:

 $ avocado --show=app run -t machine:virt tests/acceptance/
 ...
 (2/6) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_kvm: ERROR: Unexpected empty reply from server (1.82 s)

The job.log content is:

  L0351 DEBUG| Output: 'qemu-system-aarch64: host does not support in-kernel GICv2 emulation\n'

With this patch:

 $ avocado --show=app run -t device:gicv3 tests/acceptance/
 (1/1) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_kvm_gicv3: PASS (55.10 s)

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20200929224857.1225107-1-philmd@redhat.com>
2020-10-26 18:20:51 +01:00
Philippe Mathieu-Daudé
7fb1f203d7 tests/boot_linux_console: Boot Trusted Firmware-A on the Raspberry Pi 3
This test runs Trusted Firmware-A on the Raspberry Pi 3.
We deliberately stop the boot process when the EDK2 UEFI version
is displayed.

The binary is build on AppVeyor CI using Pete Batard repository [1].
ATF v2.1 binary are used (see [2]).

It is very simple and fast:

  $ avocado --show=app,console run -t atf tests/acceptance
  JOB ID     : 1e748d7c9e9011cf0af3250ddc8ebf2389d6204e
  JOB LOG    : avocado/job-results/job-2020-02-16T18.08-1e748d7/job.log
   (1/1) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_raspi3_atf:
  console: NOTICE:  Booting Trusted Firmware
  console: NOTICE:  BL1: v2.1(release):v2.1
  console: NOTICE:  BL1: Built : 15:26:06, May 13 2019
  console: NOTICE:  rpi3: Detected: Raspberry Pi 3 Model B (1GB, Sony, UK) [0x00a02082]
  console: NOTICE:  BL1: Booting BL2
  console: ERROR:   rpi3_sdhost: timeout status 0x40
  console: NOTICE:  BL2: v2.1(release):v2.1
  console: NOTICE:  BL2: Built : 15:26:01, May 13 2019
  console: NOTICE:  BL1: Booting BL31
  console: NOTICE:  BL31: v2.1(release):v2.1
  console: NOTICE:  BL31: Built : 15:26:04, May 13 2019
  console: =UEFI firmware (version UEFI Firmware v1.15 built at 11:58:44 on Feb 14 2020)
  PASS (1.54 s)
  RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
  JOB TIME   : 1.88 s

[1] https://github.com/pbatard/RPi3#summary
[2] https://github.com/ARM-software/arm-trusted-firmware/blob/v2.1/docs/plat/rpi3.rst

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Luc Michel <luc.michel@greensocs.com>
Message-Id: <20200217103442.30318-1-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-10-26 18:20:51 +01:00
Philippe Mathieu-Daudé
28bbe20ce2 tests/acceptance: Use .ppm extention for Portable PixMap files
The HMP 'screendump' command generates Portable PixMap files.
Make it obvious by using the .ppm file extention.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201021105035.2477784-3-f4bug@amsat.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Philippe Mathieu-Daudé
239f0d455b tests/acceptance: Remove unused import
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201021105035.2477784-2-f4bug@amsat.org>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Thomas Huth
1d60f46fc6 test/acceptance: Remove the CONTINUOUS_INTEGRATION tags
We are not running the acceptance tests on Travis anymore, so these
checks can be removed now.

Message-Id: <20201023073351.251332-3-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Thomas Huth
a60f755c9c tests/acceptance/ppc_prep_40p: Fix the URL to the NetBSD-4.0 archive
The current URL on cdn.netbsd.org is failing - using archive.netbsd.org
instead seems to be fine.

Message-Id: <20201023073351.251332-2-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Pavel Dovgalyuk
1d2ff14b72 tests/acceptance: Add MIPS record/replay tests
This patch adds MIPS-targeted acceptance tests for
record/replay functions.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <160276110297.2705.10918105269658307206.stgit@pasha-ThinkPad-X280>
[PMD: Moved 'override timeout' comment from instance to class,
      moved nanomips tests to ReplayKernelSlow class,
      tagged ReplayKernelSlow class with AVOCADO_TIMEOUT_EXPECTED]
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2020-10-17 13:59:40 +02:00
Alex Bennée
e5d402b28f tests/acceptance: disable machine_rx_gdbsim on GitLab
While I can get the ssh test to fail on my test setup this seems a lot
more stable except when on GitLab. Hopefully we can re-enable both
once the serial timing patches have been added.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201007160038.26953-23-alex.bennee@linaro.org>
2020-10-09 17:27:55 +01:00
Pavel Dovgalyuk
be52eca309 tests/acceptance: add reverse debugging test
This is a test for GDB reverse debugging commands: reverse step and reverse continue.
Every test in this suite consists of two phases: record and replay.
Recording saves the execution of some instructions and makes an initial
VM snapshot to allow reverse execution.
Replay saves the order of the first instructions and then checks that they
are executed backwards in the correct order.
After that the execution is replayed to the end, and reverse continue
command is checked by setting several breakpoints, and asserting
that the execution is stopped at the last of them.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>

--

v5:
 - disabled (as some other tests) when running on gitlab
   due to the unidentified timeout problem
Message-Id: <160174524678.12451.13258942849173670277.stgit@pasha-ThinkPad-X280>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-06 08:34:50 +02:00
Philippe Mathieu-Daudé
b6eea069ab tests/acceptance: Skip slow quanta-gsj U-boot+Linux test
The 'arm_quanta_gsj_initrd' test is timeouting on GitLab CI:
https://gitlab.com/philmd/qemu/-/jobs/745483978#L846
and also sometimes on my workstation, so proceed as with
the other slow tests: do not run it by default.
The test can still be run setting the AVOCADO_TIMEOUT_EXPECTED
environment variable.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Havard Skinnemoen <hskinnemoen@google.com>
Tested-by: Havard Skinnemoen <hskinnemoen@google.com>
Message-Id: <20200918143355.153522-1-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-09-18 19:46:38 +02:00
Philippe Mathieu-Daudé
a58bf4b3d9 tests/acceptance: Disable tests dependent of unreliable apt.armbian.com
Armbian servers are not very reliable and confused the GitLab CI
users a few times this month (path updated, archives moved, and
now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired"
error). Time to disable these tests.
Users can still use the artifacts from the cache (or manually add
them to the cache).

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <20200917163954.50514-1-philmd@redhat.com>
2020-09-18 19:44:22 +02:00
Havard Skinnemoen
4fe986dd44 tests/acceptance: console boot tests for quanta-gsj
This adds two acceptance tests for the quanta-gsj machine.

One test downloads a lightly patched openbmc flash image from github and
verifies that it boots all the way to the login prompt.

The other test downloads a kernel, initrd and dtb built from the same
openbmc source and verifies that the kernel detects all CPUs and boots
to the point where it can't find the root filesystem (because we have no
flash image in this case).

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Havard Skinnemoen <hskinnemoen@google.com>
Message-id: 20200911052101.2602693-15-hskinnemoen@google.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-09-14 14:24:59 +01:00
Philippe Mathieu-Daudé
89e076f37d tests/acceptance: Add Test.fetch_asset(cancel_on_missing=True)
Invert the default of avocado.Test.fetch_asset 'cancel_on_missing'
keyword: accept missing artefacts by default. If a test is certain
an artifact can't be missing, it will set cancel_on_missing=False.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200908202352.298506-3-philmd@redhat.com>
Message-Id: <20200909112742.25730-10-alex.bennee@linaro.org>
2020-09-10 10:46:57 +01:00
Peter Maydell
d6f83a72a7 Acceptance tests patches
- Use stable URLs for the Debian and Ubuntu installer
   (Ubuntu has been updated last Wednesday, August 5, 2020).
 
 CI jobs results:
 . https://cirrus-ci.com/build/6385815351721984
 . https://gitlab.com/philmd/qemu/-/pipelines/177054604
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAl8z69UACgkQ4+MsLN6t
 wN7nHQ/9Hfu29LidcGyFgzPTdlepeKKRLtgCvMgyqIIYC2WhiW6AuhccfZMSB1Zs
 C9z/ubvASyFFav+slmPBLnWKPUm5pf5pCLapvb9c7hS5+wNlq9Jdg3cq4pZp6lGb
 bWstciGkXQzHXOVN3fv7yGX0Q1trtCHKbxJoFJPhEYg7dFons+XOLg2/bGWgoJmT
 fd5T08+CBM86o9q6zd2xyypJAU3DIFX6LNrVfxVtGID3Kn8wJ9g05nG7SENlHQeK
 cI/rf+/2WOQYU+RWUkpRw+O5++9I56SpvZ80a+Dw82vkM7ThtSb98utqvoiVQwLb
 T6cSTYl03+r6Mr2MhmW/RRoe5P5vaUiNcnclboP0kaF02kvCo2UMcWxj57gsu89K
 5EHJnAKTDJexSpDZsz9GGYQhXjpNGAaSUyi4HrN+Dij5EOB1XDHBmZ+VwQAdIi2f
 dUTmRbr+3ZyMzaANb+HL3VlU+yWR3m8faCkTQguSfNmfFEWS1tDpib1taDwy/Esu
 GvR1AccQue7XnoBrqPN3rxr064pe96n7JZiUtcPosxpULgSb4/H724VZMaPAX497
 O3kc8D6tCtZ8HGll/Dv6vgJOt/SOHMJya0xTG4WnHnNAMFxyVrDuz3aUmrUzmHtI
 Qrd68VhjWkl4/+7COAO7QOLpuAxjozQj7r5h01sYbSodmByDhas=
 =pqT+
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/acceptance-testing-20200812' into staging

Acceptance tests patches

- Use stable URLs for the Debian and Ubuntu installer
  (Ubuntu has been updated last Wednesday, August 5, 2020).

CI jobs results:
. https://cirrus-ci.com/build/6385815351721984
. https://gitlab.com/philmd/qemu/-/pipelines/177054604

# gpg: Signature made Wed 12 Aug 2020 14:17:09 BST
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/acceptance-testing-20200812:
  acceptance: use stable URLs for the Debian and Ubuntu installer
  tests/acceptance/boot_linux: Extract common URL from xlnx-versal test

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-08-21 14:51:43 +01:00
Paolo Bonzini
64ed6f92ff meson: link emulators without Makefile.target
The binaries move to the root directory, e.g. qemu-system-i386 or
qemu-arm.  This requires changes to qtests, CI, etc.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-08-21 06:30:40 -04:00
Paolo Bonzini
0eca1f4b5c acceptance: use stable URLs for the Debian and Ubuntu installer
The kernel and initrd hashes seem to have changed for the Bionic
aarch64 installer, causing BootLinuxConsole.test_aarch64_xlnx_versal_virt
to fail.  Correct the paths to use the previous binaries instead of
the latest.  Do the same for the Lenny alpha installer for
consistency, even though those are unlikely to change.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200810092941.154911-1-pbonzini@redhat.com>
2020-08-12 14:14:28 +02:00
Philippe Mathieu-Daudé
c45aed126f tests/acceptance/boot_linux: Extract common URL from xlnx-versal test
Both files refer to the same directory. Store the common part in
a new variable.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200810093050.28744-2-philmd@redhat.com>
2020-08-12 14:13:02 +02:00
Thomas Huth
4d6862ffc7 tests/acceptance: Disable the rx sash and arm cubieboard replay test on Gitlab
These tests always time out on Gitlab, not sure what's happening here.
Let's disable them until somebody has enough spare time to debug the
issues.

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20200730141326.8260-3-thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-08-05 10:38:27 +02:00
Peter Maydell
673205379f Python patches for 5.1
- Reduce race conditions on QEMUMachine::shutdown()
 
  1. Remove the "bare except" pattern in the existing shutdown code,
     which can mask problems and make debugging difficult.
  2. Ensure that post-shutdown cleanup is always performed, even when
     graceful termination fails.
  3. Unify cleanup paths such that no matter how the VM is terminated,
     the same functions and steps are always taken to reset the object
     state.
  4. Rewrite shutdown() such that any error encountered when attempting
     a graceful shutdown will be raised as an AbnormalShutdown exception.
     The pythonic idiom is to allow the caller to decide if this is a
     problem or not.
 
 - Modify part of the python/qemu library to comply with:
 
   . mypy --strict
   . pylint
   . flake8
 
 - Script for the TCG Continuous Benchmarking project that uses
   callgrind to dissect QEMU execution into three main phases:
 
   . code generation
   . JIT execution
   . helpers execution
 
 CI jobs results:
 . https://cirrus-ci.com/build/5421349961203712
 . https://gitlab.com/philmd/qemu/-/pipelines/166556001
 . https://travis-ci.org/github/philmd/qemu/builds/708102347
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAl8OGCUACgkQ4+MsLN6t
 wN52ZQ//cB8pRhOeirMzPTamtAePAHhU9kRKSQ4fFzVM/AHxaO3qO3KrMKAv4JnR
 1fwdyAbum0Dv+HrRAXbIVz/lgrlTb6tLqwk20YWlmNUR2CtMmno9fPQzg6LeFWBe
 +qr2/anXWBqRZu5uHzQ3Ly3LxlVXE3qmStxlttNxed9wK9wdhsQ4ijOrExXPmMI5
 IQi9tKijsGvOczcSpIwJET6Px0L/GIYHSFGXfHv4LiJCMQUX4gqY1akr+/2adqlr
 ElcOHlvWG90GHftTC+hq+KbSNmpkwJEnXLHC+HwMaYmQuhH4+U6f9AcrnzYGHjoQ
 84muuYt5CBvT/Q38qO3dUsr42oD8uiDZe2cpD5DmcAHj6U9sza4UwU2Nkd4cNvRD
 OiYRWuHQDzQhwPGbzBym1GRxRlejmPXCpvZ4XqaYODkzhEHi8UVq5AOsNhpTGWVk
 WoDZ3NtnZb9WbU93wDordhWTtHKHOLHNk4CuJLbdRvkW+sg8aFqUObSRvc6RqyU8
 1DXWMMQr9UfCDfMMbp/OCymphn0VrAaixIRIjmYc6nL5mQlyugMwij55jzyugqBH
 rN5bdcsqxAhBTD03lu0lbDn1BYIqbZFXABXeDnJNbxmY/6gD0E1HKWRvIKjxzH+t
 L19QXsOWLoVUpUIjxlOgMlb9QJLPc9m/N3yVOanO0zVrMSBmOKQ=
 =Jp6n
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/python-next-20200714' into staging

Python patches for 5.1

- Reduce race conditions on QEMUMachine::shutdown()

 1. Remove the "bare except" pattern in the existing shutdown code,
    which can mask problems and make debugging difficult.
 2. Ensure that post-shutdown cleanup is always performed, even when
    graceful termination fails.
 3. Unify cleanup paths such that no matter how the VM is terminated,
    the same functions and steps are always taken to reset the object
    state.
 4. Rewrite shutdown() such that any error encountered when attempting
    a graceful shutdown will be raised as an AbnormalShutdown exception.
    The pythonic idiom is to allow the caller to decide if this is a
    problem or not.

- Modify part of the python/qemu library to comply with:

  . mypy --strict
  . pylint
  . flake8

- Script for the TCG Continuous Benchmarking project that uses
  callgrind to dissect QEMU execution into three main phases:

  . code generation
  . JIT execution
  . helpers execution

CI jobs results:
. https://cirrus-ci.com/build/5421349961203712
. https://gitlab.com/philmd/qemu/-/pipelines/166556001
. https://travis-ci.org/github/philmd/qemu/builds/708102347

# gpg: Signature made Tue 14 Jul 2020 21:40:05 BST
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/python-next-20200714:
  python/qmp.py: add QMPProtocolError
  python/qmp.py: add casts to JSON deserialization
  python/qmp.py: Do not return None from cmd_obj
  python/qmp.py: re-absorb MonitorResponseError
  iotests.py: use qemu.qmp type aliases
  python/qmp.py: Define common types
  python/machine.py: change default wait timeout to 3 seconds
  python/machine.py: re-add sigkill warning suppression
  python/machine.py: split shutdown into hard and soft flavors
  tests/acceptance: Don't test reboot on cubieboard
  tests/acceptance: wait() instead of shutdown() where appropriate
  python/machine.py: Make wait() call shutdown()
  python/machine.py: Add a configurable timeout to shutdown()
  python/machine.py: Prohibit multiple shutdown() calls
  python/machine.py: Perform early cleanup for wait() calls, too
  python/machine.py: Add _early_cleanup hook
  python/machine.py: Close QMP socket in cleanup
  python/machine.py: consolidate _post_shutdown()
  scripts/performance: Add dissect.py script

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-07-15 13:04:27 +01:00
John Snow
fdb87f0dc2 tests/acceptance: Don't test reboot on cubieboard
cubieboard does not have a functioning reboot, it halts and QEMU does
not exit.

vm.shutdown() is modified in a forthcoming patch that makes it less tolerant
of race conditions on shutdown; tests should consciously decide to WAIT
or to SHUTDOWN qemu.

So long as this test is attempting to reboot, the correct choice would
be to WAIT for the VM to exit. However, since that's broken, we should
SHUTDOWN instead.

SHUTDOWN is indeed what already happens when the test performs teardown,
however, if anyone fixes cubieboard reboot in the future, this test will
develop a new race condition that might be hard to debug.

Therefore: remove the reboot test and make it obvious that the VM is
still running when the test concludes, where the test teardown will do
the right thing.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20200710050649.32434-10-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-07-14 22:22:22 +02:00
John Snow
a0690c3900 tests/acceptance: wait() instead of shutdown() where appropriate
When issuing 'reboot' to a VM with the no-reboot option, that VM will
exit. When then issuing a shutdown command, the cleanup may race.

Add calls to vm.wait() which will gracefully mark the VM as having
exited. Subsequent vm.shutdown() calls in generic tearDown code will not
race when called after completion of the call.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20200710050649.32434-9-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-07-14 22:22:22 +02:00
Philippe Mathieu-Daudé
6a289a5ba3 tests/acceptance/boot_linux: Expand SD card image to power of 2
In few commits we won't allow SD card images with invalid size
(not aligned to a power of 2). Prepare the tests: add the
pow2ceil() and image_pow2ceil_expand() methods and resize the
images (expanding) of the tests using SD cards.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20200713183209.26308-5-f4bug@amsat.org>
2020-07-14 15:38:22 +02:00
Philippe Mathieu-Daudé
b7dcbf1395 tests/acceptance/boot_linux: Tag tests using a SD card with 'device:sd'
Avocado tags are handy to automatically select tests matching
the tags. Since these tests use a SD card, tag them.

We can run all the tests using a SD card at once with:

  $ avocado --show=app run -t u-boot tests/acceptance/
  $ AVOCADO_ALLOW_LARGE_STORAGE=ok \
    avocado --show=app \
      run -t device:sd tests/acceptance/
  Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd
  Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic
  Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9
   (1/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd: PASS (19.56 s)
   (2/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic: PASS (49.97 s)
   (3/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9: PASS (20.06 s)
  RESULTS    : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
  JOB TIME   : 90.02 s

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20200713183209.26308-4-f4bug@amsat.org>
2020-07-14 15:38:22 +02:00
Peter Maydell
9f526fce49 Testing and misc build updates:
- tests/vm support for aarch64 VMs
   - tests/tcg better cross-compiler detection
   - update docker tooling to support registries
   - update docker support for xtensa
   - gitlab build docker images and store in registry
   - gitlab use docker images for builds
   - a number of skipIf updates to support move
   - linux-user MAP_FIXED_NOREPLACE fix
   - qht-bench compiler tweaks
   - configure fix for secret keyring
   - tsan fiber annotation clean-up
   - doc updates for mttcg/icount/gdbstub
   - fix cirrus to use brew bash for iotests
   - revert virtio-gpu breakage
   - fix LC_ALL to avoid sorting changes in iotests
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAl8J0yoACgkQ+9DbCVqe
 KkSzTAf/Vn+9TU8Qt7nZvl7W4tz7Sy5K8EJGwj2RXx6CWWWLiFbsXurIM8Krw5Vc
 RmvUxwa359b+J0lQpfeNDHYm1nM8RZLFlkG0a5bl0I8sW0EcPjBRtwNaGKXh2p0u
 u2RS2QAi6A9AvYT4ZREYlBM+o9WzbxCEQm4s8fr6WEJCQfxBnb5/bGiEjWR64e8C
 j9Kvou+zAKfVizbQMtu+mwqjsoPtcS1b3vVcO7anhNuUsuaEKkS0dFWzWvw3lwJR
 STIYnb8Y/eJ1yKr0hPH2qtWv3n6yhlYvYmpUCH6AwshGMUoeFEzR2VoWS6yZPGG6
 na6XA3UW5R9AxIDfkCJ5ueeo8t9xMQ==
 =HRWa
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-and-misc-110720-2' into staging

Testing and misc build updates:

  - tests/vm support for aarch64 VMs
  - tests/tcg better cross-compiler detection
  - update docker tooling to support registries
  - update docker support for xtensa
  - gitlab build docker images and store in registry
  - gitlab use docker images for builds
  - a number of skipIf updates to support move
  - linux-user MAP_FIXED_NOREPLACE fix
  - qht-bench compiler tweaks
  - configure fix for secret keyring
  - tsan fiber annotation clean-up
  - doc updates for mttcg/icount/gdbstub
  - fix cirrus to use brew bash for iotests
  - revert virtio-gpu breakage
  - fix LC_ALL to avoid sorting changes in iotests

# gpg: Signature made Sat 11 Jul 2020 15:56:42 BST
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-and-misc-110720-2: (50 commits)
  iotests: Set LC_ALL=C for sort
  Revert "vga: build virtio-gpu as module"
  tests: fix "make check-qtest" for modular builds
  .cirrus.yml: add bash to the brew packages
  tests/docker: update toolchain set in debian-xtensa-cross
  tests/docker: fall back more gracefully when pull fails
  docs: Add to gdbstub documentation the PhyMemMode
  docs/devel: add some notes on tcg-icount for developers
  docs/devel: convert and update MTTCG design document
  tests/qht-bench: Adjust threshold computation
  tests/qht-bench: Adjust testing rate by -1
  travis.yml: Test also the other targets on s390x
  shippable: pull images from registry instead of building
  testing: add check-build target
  containers.yml: build with docker.py tooling
  gitlab: limit re-builds of the containers
  tests: improve performance of device-introspect-test
  gitlab: add avocado asset caching
  gitlab: enable check-tcg for linux-user tests
  linux-user/elfload: use MAP_FIXED_NOREPLACE in pgb_reserved_va
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-07-12 15:32:05 +01:00
Alex Bennée
c5008c76ee gitlab: add acceptance testing to system builds
As part of migrating things from Travis to GitLab add the acceptance
tests. To do this:

  - rename system1 to system-ubuntu-main
  - rename system2 to system-fedora-misc
  - split into build/check/acceptance
  - remove -j from check stages
  - use artifacts to save build stage
  - add post acceptance template and use

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-31-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
3c1de3af46 tests/acceptance: skip LinuxInitrd 2gib with v4.16 on GitLab
This fails on GitLab but not when run locally on the same container
image. It's very confusing.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200701135652.1366-30-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
61bbce96fe tests/acceptance: skip multicore mips_malta tests on GitLab
For some reason these tests fail all the time on GitLab. I can
re-create the hang around 3% of the time locally but it doesn't seem
to be MTTCG related. For now skipIf on GITLAB_CI.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <20200701135652.1366-29-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
fcae60436f tests/acceptance: fix dtb path for machine_rx_gdbsim
The old path doesn't exist but the rx-virt.dtb file has the same
checksum so lets use that.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200701135652.1366-28-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
0f26d94ec9 tests/acceptance: skip s390x_ccw_vrtio_tcg on GitLab
Currently the test takes more the 900 seconds on GitLab and then times
out. Running on Travis seems to be OK.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-27-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Michael Rolnik
f5d31d6541 tests/acceptance: Test the Arduino MEGA2560 board
The test is based on
https://github.com/seharris/qemu-avr-tests/tree/master/free-rtos/Demo
demo which. If working correctly, prints 'ABCDEFGHIJKLMNOPQRSTUVWX' out.
it also demostrates that timer and IRQ are working

As the path name demonstrates, the FreeRTOS tests target a
board based on a ATMega2560 MCU. We have one, the Arduino
MEGA2560.

Complementary documentation:

https://feilipu.me/2012/01/15/ethermega-arduino-mega-2560-and-freertos/
https://feilipu.me/2015/11/24/arduino_freertos/ (see 'Compatibility')

Signed-off-by: Michael Rolnik <mrolnik@gmail.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[rth: Squash multiple avocado fixups from f4bug]
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Aleksandar Markovic <aleksandar.m.mail@gmail.com>
Tested-by: Richard Henderson <richard.henderson@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-07-11 11:02:05 +02:00
Thomas Huth
8c4329214f tests/acceptance: Add a test for the sun4u sparc64 machine
We can use the image from the advent calendar 2018 to test the sun4u
machine. It's not using the "QEMU advent calendar" string, so we can
not use the do_test_advcal_2018() from boot_linux_console.py, thus
let's also put it into a separate file to also be able to add an
entry to the MAINTAINERS file.

Message-Id: <20200704173519.26087-1-thuth@redhat.com>
Tested-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-07-06 09:32:59 +02:00
Peter Maydell
d88d5a3806 Renesas hardware patches
- Add a common entry for Renesas hardware in MAINTAINERS
 - Trivial SH4 cleanups
 - Add RX GDB simulator from Yoshinori Sato
 
 The Renesas RX target emulation was added in commit c8c35e5f51,
 these patches complete the target by adding the hardware emulation.
 
 Tests included:
 
 $ avocado --show=app,console run -t arch:rx tests/acceptance/
 Fetching asset from tests/acceptance/machine_rx_gdbsim.py:RxGdbSimMachine.test_uboot
 Fetching asset from tests/acceptance/machine_rx_gdbsim.py:RxGdbSimMachine.test_linux_sash
  (1/2) tests/acceptance/machine_rx_gdbsim.py:RxGdbSimMachine.test_uboot:
 console: U-Boot 2016.05-rc3-23705-ga1ef3c71cb-dirty (Feb 05 2019 - 21:56:06 +0900)
 PASS (0.26 s)
  (2/2) tests/acceptance/machine_rx_gdbsim.py:RxGdbSimMachine.test_linux_sash:
 console: Linux version 4.19.0+ (yo-satoh@yo-satoh-debian) (gcc version 9.0.0 20181105 (experimental) (GCC)) #137 Wed Feb 20 23:20:02 JST 2019
 console: Built 1 zonelists, mobility grouping on.  Total pages: 8128
 console: Kernel command line:
 console: Dentry cache hash table entries: 4096 (order: 2, 16384 bytes)
 console: Inode-cache hash table entries: 2048 (order: 1, 8192 bytes)
 console: Memory: 14648K/32768K available (871K kernel code, 95K rwdata, 140K rodata, 96K init, 175K bss, 18120K reserved, 0K cma-reserved)
 console: NR_IRQS: 256
 console: rx-cmt: used for periodic clock events
 console: clocksource: rx-tpu: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1274173631191 ns
 console: 96.00 BogoMIPS (lpj=480000)
 console: pid_max: default: 4096 minimum: 301
 console: Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
 console: Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
 console: clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
 console: clocksource: Switched to clocksource rx-tpu
 console: workingset: timestamp_bits=30 max_order=12 bucket_order=0
 console: SuperH (H)SCI(F) driver initialized
 console: 88240.serial: ttySC0 at MMIO 0x88240 (irq = 215, base_baud = 0) is a sci
 console: console [ttySC0] enabled
 console: 88248.serial: ttySC1 at MMIO 0x88248 (irq = 219, base_baud = 0) is a sci
 console: random: get_random_bytes called from 0x01002e48 with crng_init=0
 console: Freeing unused kernel memory: 96K
 console: This architecture does not have kernel memory protection.
 console: Run /sbin/init as init process
 console: Run /etc/init as init process
 console: Run /bin/init as init process
 console: Run /bin/sh as init process
 console: Sash command shell (version 1.1.1)
 console: /> printenv
 console: HOME=/
 console: TERM=linux
 PASS (0.73 s)
 RESULTS    : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
 JOB TIME   : 1.47 s
 
 CI results:
 . https://cirrus-ci.com/build/6140199509950464
 . https://travis-ci.org/github/philmd/qemu/builds/700954881
 . https://app.shippable.com/github/philmd/qemu/runs/812/summary/console
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAl7w/dkACgkQ4+MsLN6t
 wN6tHw//Xt+n/PMXOWdPfAA6Ezfc+4k/CKXMXG+PM2bq2wwmrpMLybxJMVBb7Q3H
 83F5nSdDziEd0T7XzT2lzh3Qg6wcVmGXvvgMInyL8VNoLWeQoOllIKKY35FikhPH
 Vx5W3OEPSgzcqb2//fUVha+VaKW8mv+AmMp/fmI0woHe8qBYq641QNbQUyQ/0L4E
 iJR6Tyl/2KE/9V6iED+qQ9qa5nY/INve73dj1/hyBKB0WgsjTH5qCipxlx8BCQHS
 vGQ/IbAJl/CpXUN/dhQ8Uxaq97QBdeu1rBefGETj2TJrv+5jbmOVj/Ac/VFojMz1
 iBIrYDtHU8h3msceDwZHeJvhWUgdqzuEWXYl5UKgt+FkzXZvDlc6Fkill01DYrDL
 OINVUhlvkpN+sTJYkA8GrcNKpI7dEK/qrfh7utuvgyblZVOaJJV6ldPQ1378ENmc
 QsdtWfkmcVR704ATZk6tYNPI3d8xvepPnjAGFdFISrBEC92Rj2jCQX6zwGu7M7Zg
 M9okNv8krx4bMub0XaK8MQzPpBkWEPzdCo5b11lOZeyvJ5NgJsZRiCdCwX08vmji
 NJzrIaFoWs5qLh3+kwY3b6fvku+6xn5iuYhKO6AdMTDxduWKslE0XU45dv6+NBsA
 NriEIagcb9R2vTYug4I2vfzCOrWwJqP5DDKX2iqT84Pa++KC3C8=
 =Ij2z
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/renesas-hw-20200622' into staging

Renesas hardware patches

- Add a common entry for Renesas hardware in MAINTAINERS
- Trivial SH4 cleanups
- Add RX GDB simulator from Yoshinori Sato

The Renesas RX target emulation was added in commit c8c35e5f51,
these patches complete the target by adding the hardware emulation.

Tests included:

$ avocado --show=app,console run -t arch:rx tests/acceptance/
Fetching asset from tests/acceptance/machine_rx_gdbsim.py:RxGdbSimMachine.test_uboot
Fetching asset from tests/acceptance/machine_rx_gdbsim.py:RxGdbSimMachine.test_linux_sash
 (1/2) tests/acceptance/machine_rx_gdbsim.py:RxGdbSimMachine.test_uboot:
console: U-Boot 2016.05-rc3-23705-ga1ef3c71cb-dirty (Feb 05 2019 - 21:56:06 +0900)
PASS (0.26 s)
 (2/2) tests/acceptance/machine_rx_gdbsim.py:RxGdbSimMachine.test_linux_sash:
console: Linux version 4.19.0+ (yo-satoh@yo-satoh-debian) (gcc version 9.0.0 20181105 (experimental) (GCC)) #137 Wed Feb 20 23:20:02 JST 2019
console: Built 1 zonelists, mobility grouping on.  Total pages: 8128
console: Kernel command line:
console: Dentry cache hash table entries: 4096 (order: 2, 16384 bytes)
console: Inode-cache hash table entries: 2048 (order: 1, 8192 bytes)
console: Memory: 14648K/32768K available (871K kernel code, 95K rwdata, 140K rodata, 96K init, 175K bss, 18120K reserved, 0K cma-reserved)
console: NR_IRQS: 256
console: rx-cmt: used for periodic clock events
console: clocksource: rx-tpu: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1274173631191 ns
console: 96.00 BogoMIPS (lpj=480000)
console: pid_max: default: 4096 minimum: 301
console: Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
console: Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
console: clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
console: clocksource: Switched to clocksource rx-tpu
console: workingset: timestamp_bits=30 max_order=12 bucket_order=0
console: SuperH (H)SCI(F) driver initialized
console: 88240.serial: ttySC0 at MMIO 0x88240 (irq = 215, base_baud = 0) is a sci
console: console [ttySC0] enabled
console: 88248.serial: ttySC1 at MMIO 0x88248 (irq = 219, base_baud = 0) is a sci
console: random: get_random_bytes called from 0x01002e48 with crng_init=0
console: Freeing unused kernel memory: 96K
console: This architecture does not have kernel memory protection.
console: Run /sbin/init as init process
console: Run /etc/init as init process
console: Run /bin/init as init process
console: Run /bin/sh as init process
console: Sash command shell (version 1.1.1)
console: /> printenv
console: HOME=/
console: TERM=linux
PASS (0.73 s)
RESULTS    : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 1.47 s

CI results:
. https://cirrus-ci.com/build/6140199509950464
. https://travis-ci.org/github/philmd/qemu/builds/700954881
. https://app.shippable.com/github/philmd/qemu/runs/812/summary/console

# gpg: Signature made Mon 22 Jun 2020 19:52:09 BST
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/renesas-hw-20200622:
  docs: Document the RX target
  BootLinuxConsoleTest: Test the RX GDB simulator
  hw/rx: Add RX GDB simulator
  hw/rx: Register R5F562N7 and R5F562N8 MCUs
  hw/rx: Honor -accel qtest
  hw/rx: RX62N microcontroller (MCU)
  hw/char: RX62N serial communication interface (SCI)
  hw/timer: RX62N compare match timer (CMT)
  hw/timer: RX62N 8-Bit timer (TMR)
  hw/intc: RX62N interrupt controller (ICUa)
  hw/timer/sh_timer: Remove unused 'qemu/timer.h' include
  hw/sh4: Extract timer definitions to 'hw/timer/tmu012.h'
  hw/sh4: Use MemoryRegion typedef
  MAINTAINERS: Add an entry for common Renesas peripherals
  MAINTAINERS: Cover sh_intc files in the R2D/Shix machine sections

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-06-23 13:55:52 +01:00