Commit Graph

116 Commits

Author SHA1 Message Date
Philippe Mathieu-Daudé 07df3dfafa gitlab: Extract build stages to stages.yml
Extract the build stages used by our job templates to a new file
(stages.yml) to be able to include it with the other templates,
without having to run all the jobs included in the default
.gitlab-ci.yml, which are mainly useful for mainstream CI.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-7-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-05-27 16:39:23 +02:00
Philippe Mathieu-Daudé f659e6584a gitlab: Extract DCO/style check jobs to static_checks.yml
Extract the DCO / checkpatch jobs to a new file (static_checks.yml)
to be able to run them without having to run all the jobs included
in the default .gitlab-ci.yml, which are mainly useful for the
mainstream CI.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-6-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-05-27 16:39:23 +02:00
Philippe Mathieu-Daudé a4b201d1ac gitlab: Replace YAML anchors by extends (acceptance_test_job)
'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-2-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-05-27 16:39:21 +02:00
Peter Maydell 8385235ba9 * submodule cleanups (Philippe, myself)
* tiny step towards a usable preconfig mode (myself)
 * Kconfig and LOCK_GUARD cleanups (philippe)
 * new x86 CPUID feature (Yang Zhong)
 * "-object qtest" support (myself)
 * Dirty ring support for KVM (Peter)
 * Fixes for 6.0 command line parsing breakage (myself)
 * Fix for macOS 11.3 SDK (Katsuhiro)
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmCuRAQUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroOL6Qf/bUjQNAUc2QQJya1lu8TEf1o4vjkK
 C3EzFPVAj+m2O3OZOGEHcTh8+lDSzBeE2gB3bt4AD+KvFbQGXhLM3gMu/Ztymv8m
 3rVEe/NxNyq/CgC307GIwF3in7rEzjH0+WHaOuoU340e3Po1FA7s20VnMysVxxng
 4Pf4m4Y0k0eq022HgqZ/r/kbnINxDHagmzuyiFARkt8ooiuj4NyOMW7UKMk3fBvY
 MLMPsBe3imWmVnkOF0n/qJ+Svbtx15iLgGIIggshy3rmPereUpIQYaJ9FS6jcXO2
 YHuYDc2aGelMU84r+x+9UQra6auzJfc4UbylOsGjopCeFG2aU8rLMphvpw==
 =UQwU
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

* submodule cleanups (Philippe, myself)
* tiny step towards a usable preconfig mode (myself)
* Kconfig and LOCK_GUARD cleanups (philippe)
* new x86 CPUID feature (Yang Zhong)
* "-object qtest" support (myself)
* Dirty ring support for KVM (Peter)
* Fixes for 6.0 command line parsing breakage (myself)
* Fix for macOS 11.3 SDK (Katsuhiro)

# gpg: Signature made Wed 26 May 2021 13:50:12 BST
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream: (28 commits)
  gitlab-ci: use --meson=git for CFI jobs
  hw/scsi: Fix sector translation bug in scsi_unmap_complete_noio
  configure: Avoid error messages about missing *-config-*.h files
  doc: Add notes about -mon option mode=control argument.
  qemu-config: load modules when instantiating option groups
  vl: allow not specifying size in -m when using -M memory-backend
  replication: move include out of root directory
  remove qemu-options* from root directory
  meson: Set implicit_include_directories to false
  tests/qtest/fuzz: Fix build failure
  KVM: Dirty ring support
  KVM: Disable manual dirty log when dirty ring enabled
  KVM: Add dirty-ring-size property
  KVM: Cache kvm slot dirty bitmap size
  KVM: Simplify dirty log sync in kvm_set_phys_mem
  KVM: Provide helper to sync dirty bitmap from slot to ramblock
  KVM: Provide helper to get kvm dirty log
  KVM: Create the KVMSlot dirty bitmap on flag changes
  KVM: Use a big lock to replace per-kml slots_lock
  memory: Introduce log_sync_global() to memory listener
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-05-26 21:05:36 +01:00
Paolo Bonzini 7cf333a372 gitlab-ci: use --meson=git for CFI jobs
Ensure that the meson submodule is checked out by the check targets,
as they will need it to run "meson test".

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-26 14:50:05 +02:00
Alex Bennée 1aab5f0536 gitlab: add special rule for the hexagon container
The hexagon container is always manually built but of course not
everyone will be building it themselves and pushing to their
registries. We still need to create a "local" registry copy for the
actual gitlab tests to run. We don't build it in this case, just pull
it across from the upstream registry. We disable this rule from
running on the qemu-project itself so it doesn't accidentally wipe out
our master copy.

Fixes: 910c40ee94 ("gitlab: add build-user-hexagon test")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Cornelia Huck <cohuck@redhat.com>
Message-Id: <20210520174303.12310-4-alex.bennee@linaro.org>
2021-05-25 16:52:44 +01:00
Alex Bennée 910c40ee94 gitlab: add build-user-hexagon test
We special case this as the container with the cross compiler for the
tests takes so long to build it is manually uploaded into the
registry.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210512102051.12134-9-alex.bennee@linaro.org>
2021-05-18 09:35:39 +01:00
Philippe Mathieu-Daudé e267ce5900 gitlab-ci: Replace YAML anchors by extends (native_test_job)
'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210418233448.1267991-4-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-05-14 12:28:01 +02:00
Philippe Mathieu-Daudé 6683da0951 gitlab-ci: Replace YAML anchors by extends (native_build_job)
'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210418233448.1267991-3-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-05-14 12:28:01 +02:00
Markus Armbruster 4369223902 Drop the deprecated unicore32 target
Target unicore32 was deprecated in commit 8e4ff4a8d2, v5.2.0.  See
there for rationale.

Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210503084034.3804963-3-armbru@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
2021-05-12 18:20:52 +02:00
Markus Armbruster 9d49bcf699 Drop the deprecated lm32 target
Target lm32 was deprecated in commit d849800512, v5.2.0.  See there
for rationale.

Some of its code lives on in device models derived from milkymist
ones: hw/char/digic-uart.c and hw/display/bcm2835_fb.c.

Cc: Michael Walle <michael@walle.cc>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210503084034.3804963-2-armbru@redhat.com>
Acked-by: Michael Walle <michael@walle.cc>
[Trivial conflicts resolved, reST markup fixed]
2021-05-12 18:20:25 +02:00
Markus Armbruster 09ec85176e block: Drop the sheepdog block driver
It was deprecated in commit e1c4269763, v5.2.0.  See that commit
message for rationale.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210501075747.3293186-1-armbru@redhat.com>
ACKed-by: Peter Krempa <pkrempa@redhat.com>
2021-05-12 17:42:23 +02:00
Thomas Huth 875bb7e35b Remove the deprecated moxie target
There are no known users of this CPU anymore, and there are no
binaries available online which could be used for regression tests,
so the code has likely completely bit-rotten already. It's been
marked as deprecated since two releases now and nobody spoke up
that there is still a need to keep it, thus let's remove it now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210430160355.698194-1-thuth@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
[Commit message typos fixed, trivial conflicts resolved]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-12 17:42:23 +02:00
Paolo Bonzini 1b874108a5 gitlab-ci: use --meson=internal for CFI jobs
If we use the system Meson but it is too old, the subsequent "meson configure"
will fail.  Always use the submodule when building with CFI.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-04 14:15:35 +02:00
Thomas Huth a2e5bbf0c4 gitlab-ci.yml: Test the dtrace backend in one of the jobs
We are using the dtrace backend in downstream RHEL, so testing this
in the CentOS 8 task seems to be a good fit.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210331160351.3071279-1-thuth@redhat.com>
Message-Id: <20210401102530.12030-12-alex.bennee@linaro.org>
2021-04-06 15:04:50 +01:00
Thomas Huth 5186685ed6 gitlab-ci.yml: Fix the filtering for the git submodules
Commit 7d7dbf9dc1 introduced a new line starting with
"GIT_SUBMODULES_ACTION=" in the config-host.mak file. The grep that
tries to determine the submodules in the gitlab-ci.yml file matches
this new line, too, causing a warning message when updating the modules:

 warn: ignoring non-existent submodule GIT_SUBMODULES_ACTION=update

Fix it by matching the "GIT_SUBMODULES=..." line only.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210331073316.2965928-1-thuth@redhat.com>
Message-Id: <20210401102530.12030-11-alex.bennee@linaro.org>
2021-04-06 15:04:49 +01:00
Alex Bennée a9eb2df27f gitlab: default to not building the documentation
In d0f26e68a0 ("gitlab: force enable docs build in Fedora, Ubuntu,
Debian") we made sure we can build the documents on more than one
system. However we don't want to build documents all the time as it's
a waste of cycles (and energy). So lets reduce the total amount of
documentation we build while still keeping coverage of at least one
build on each supported target.

Fixes: a8a3abe0b3 ("gitlab: move docs and tools build across from Travis")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210323165308.15244-23-alex.bennee@linaro.org>
2021-03-24 14:25:48 +00:00
Alex Bennée 9737c8bce9 gitlab: extend timeouts for CFI builds
These builds are running very close to the default build limit and as
they are already pared down the only other option is to extend the
timeout a little to give some breathing room.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210323165308.15244-16-alex.bennee@linaro.org>
2021-03-24 14:25:44 +00:00
Thomas Huth 65aff82076 gitlab-ci.yml: Merge the trace-backend testing into other jobs
Our gitlab-ci got quite slow in the past weeks, due to the immense amount
of jobs that we have, so we should try to reduce the number of jobs.
There is no real good reason for having separate jobs just to test the
trace backends, we can do this just fine in other jobs, too.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210319095726.45965-1-thuth@redhat.com>
Message-Id: <20210323165308.15244-11-alex.bennee@linaro.org>
2021-03-24 14:25:24 +00:00
Peter Maydell 757acb9a82 * Move unit and bench tests into separate directories
* Clean-up and improve gitlab-ci jobs
 * Drop the non-working "check-speed" makefile target
 * Minor documentation updates
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmBLonURHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbVsZg/9HoRP+nUoz9y7d2/GpXP01zj5xtatMadW
 RFUAlCqJapWyDXlzXVlKmvjIQeBa1uvLv6eBbAr9E748Aurmwrpw+TZ0rvIgbZrG
 fNYwixBjWcf8BtxoqRki/zdjsBV9Lym7H0G/QxYxOfve8IdHntE5iVnI0hCpaNge
 E2SLAXlFsKvC7MbLtm7+KRyIyA4PAshqJWgH+T2mdOcAA6SudLU4SidyPQJo02LX
 8U2NP2zB/5VldFGtc3TuFmoWhCSctfa3ntmNTBUBfJAEoB7hUTdKH3naZ52Qd2X9
 nAEWJAh7yiXnClz8Ant/QywvM+OjxmxVWz2Kc/1jJYqLQVGB1e8JBWH6pJxboDJb
 8z+G5SYAw5woOE+ya57doPyDbi9EHn5O6K5FIjhn/rc5YJp6SA9ilG89E3vrhr/m
 RMTCvm2nrPxenXXSovKi1t5SB6X8uyNKOuAXgEef8T28c03ZSZbs6u/YTDjPHSkp
 lU9PGxr+Ft2on+GzSXoC1HBzLyesiTsjF7leE4aB9jOGgnqCMiJsALyx/XWq5x1m
 wtpp7TcObB7p9DZ2Kf4601CoVaVO2ESjor9T81JLDtqVrlD0VHhuFGsFvOlo/M2k
 JePmg93CoX+ptP47Pyou8fnjkwCI4raM55s1SsYvVqdg2Rs1Btj6CwIqc2QEjn7N
 KIXZ7HG4i80=
 =YxBC
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/thuth-gitlab/tags/pull-request-2021-03-12' into staging

* Move unit and bench tests into separate directories
* Clean-up and improve gitlab-ci jobs
* Drop the non-working "check-speed" makefile target
* Minor documentation updates

# gpg: Signature made Fri 12 Mar 2021 17:18:45 GMT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/thuth-gitlab/tags/pull-request-2021-03-12:
  README: Add Documentation blurb
  MAINTAINERS: Merge the Gitlab-CI section into the generic CI section
  tests: remove "make check-speed" in favor of "make bench"
  gitlab-ci.yml: Merge check-crypto-old jobs into the build-crypto-old jobs
  gitlab-ci.yml: Merge one of the coroutine jobs with the tcg-disabled job
  gitlab-ci.yml: Add some missing dependencies to the jobs
  gitlab-ci.yml: Move build-tools-and-docs-debian to a better place
  tests: Move benchmarks into a separate folder
  tests: Move unit tests into a separate directory

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-14 15:13:53 +00:00
Thomas Huth 342409564c gitlab-ci.yml: Merge check-crypto-old jobs into the build-crypto-old jobs
Both, the build-crypto-old and the check-crypto-old jobs finish reasonably
fast, and the build artifacts are only used for the single corresponding
check jobs, so there is no reason for doing the check step in a separate
job here. Thus let's stop wasting artifacts space and job scheduler over-
head by simply merging the test step into the build jobs.

Message-Id: <20210311142211.1547864-5-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-12 15:46:30 +01:00
Thomas Huth 7da153e818 gitlab-ci.yml: Merge one of the coroutine jobs with the tcg-disabled job
Our gitlab-ci got quite slow in the past weeks, due to the immense amount
of jobs that we have, so we should try to reduce the number of jobs.
Since we already have a job that builds without TCG, we can merge
one of the "build-coroutine" jobs with it to get rid of at least one
job.

Message-Id: <20210311142211.1547864-4-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-12 15:46:30 +01:00
Thomas Huth 4a859abd1d gitlab-ci.yml: Add some missing dependencies to the jobs
Let's make sure that all jobs have proper "needs:" statements so that
they can start as soon as possible, without having to wait for the
previous pipeline stage to finish.

Message-Id: <20210311142211.1547864-3-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-12 15:46:30 +01:00
Thomas Huth dd188e4184 gitlab-ci.yml: Move build-tools-and-docs-debian to a better place
The "build-tools-and-docs-debian" job had been added in between
the "check-system-debian" and the "accepance-system-debian" jobs
and thus separates the jobs that belong together. Move it away,
to the end of the file, next to the "pages" job that depends on it.
And while we're at it, also add a proper "needs:" line to the
job so that it can be started as soon as possible instead of always
waiting for the previous stage to finish.

Message-Id: <20210311142211.1547864-2-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-12 15:46:30 +01:00
Peter Maydell f4abdf3271 Testing, guest-loader and other misc tweaks
- add warning text to quickstart example
   - add CFI tests to CI
   - use --arch-only for docker pre-requisites
   - fix .editorconfig for emacs
   - add guest-loader for Xen-like hypervisor testing
   - move generic-loader docs into manual proper
   - move semihosting out of hw/
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmBI50MACgkQ+9DbCVqe
 KkSyKggAhPZW+7sReVEsFdnVfwuo3evW7auoW44mghNbikTnm3RfoahYTrek8lGZ
 AEo2gFMbzENW0j88e0OvSYYtwkVz3sD68bygfXerti6sQwWlwkf42I/suWjJNLph
 oVKGEEdJess9+zR13Cu6RAq5RaTwzDPGPjUwTbeJPpAps4+UZV3hsxhaxs8keII6
 GBa/idnh0qEApP2NDLKiSASrYZM7xGvljE7zO4qhchd6iSH/o5rCtkoB2tRCcXGo
 +KF8LyBsUNf7GiWp0yYZMZUQ3Pqskqma8N3d2A4UlS1kXvxeX/FiORkG/Ne8bH1Z
 VZ1Z/xbyXGlVkiP1bcoYSc6XWHNDTw==
 =R9zQ
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-docs-xen-updates-100321-2' into staging

Testing, guest-loader and other misc tweaks

  - add warning text to quickstart example
  - add CFI tests to CI
  - use --arch-only for docker pre-requisites
  - fix .editorconfig for emacs
  - add guest-loader for Xen-like hypervisor testing
  - move generic-loader docs into manual proper
  - move semihosting out of hw/

# gpg: Signature made Wed 10 Mar 2021 15:35:31 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-docs-xen-updates-100321-2:
  semihosting: Move hw/semihosting/ -> semihosting/
  semihosting: Move include/hw/semihosting/ -> include/semihosting/
  tests/avocado: add boot_xen tests
  docs: add some documentation for the guest-loader
  docs: move generic-loader documentation into the main manual
  hw/core: implement a guest-loader to support static hypervisor guests
  device_tree: add qemu_fdt_setprop_string_array helper
  hw/riscv: migrate fdt field to generic MachineState
  hw/board: promote fdt from ARM VirtMachineState to MachineState
  .editorconfig: update the automatic mode setting for Emacs
  tests/docker: Use --arch-only when building Debian cross image
  gitlab-ci.yml: Add jobs to test CFI flags
  gitlab-ci.yml: Allow custom # of parallel linkers
  tests/docker: add a test-tcg for building then running check-tcg
  docs/system: add a gentle prompt for the complexity to come

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-11 16:20:58 +00:00
Daniele Buono 1bb12e172a gitlab-ci.yml: Add jobs to test CFI flags
QEMU has had options to enable control-flow integrity features
for a few months now. Add two sets of build/check/acceptance
jobs to ensure the binary produced is working fine.

The three sets allow testing of x86_64 binaries for x86_64, s390x,
ppc64 and aarch64 targets

[AJB: tweak job names to avoid brands]

Signed-off-by: Daniele Buono <dbuono@linux.vnet.ibm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210304030948.9367-3-dbuono@linux.vnet.ibm.com>
Message-Id: <20210305092328.31792-9-alex.bennee@linaro.org>
2021-03-10 15:34:11 +00:00
Daniele Buono e6d27a9c0d gitlab-ci.yml: Allow custom # of parallel linkers
Define a new variable LD_JOBS, that can be used to select
the maximum number of linking jobs to be executed in parallel.
If the variable is not defined, maintain the default given by
make -j

Currently, make parallelism at build time is based on the number
of cpus available.

This doesn't work well with LTO at linking, because with LTO the
linker has to load in memory all the intermediate object files
for optimization.
The end result is that, if the gitlab runner happens to run two
linking processes at the same time, the job will fail with an
out-of-memory error,

This patch leverages the ability to maintain high parallelism at
compile time, but limit the number of linkers executed in parallel.

Signed-off-by: Daniele Buono <dbuono@linux.vnet.ibm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210304030948.9367-2-dbuono@linux.vnet.ibm.com>
Message-Id: <20210305092328.31792-8-alex.bennee@linaro.org>
2021-03-10 15:34:11 +00:00
Thomas Huth 2cc1a90166 Remove deprecated target tilegx
TILE-Gx was only implemented in linux-user mode, but support for this CPU
was removed from the upstream Linux kernel in 2018, and it has also been
dropped from glibc, so there is no new Linux development taking place with
this architecture. For running the old binaries, users can simply use older
versions of QEMU.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20210224183952.80463-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-09 11:26:32 +01:00
Philippe Mathieu-Daudé 9f3a24cf1f docker: Alpine build job depends on Alpine container
Add missing dependency build-system-alpine -> amd64-alpine-container.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210303130646.1494015-2-philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-09 06:03:53 +01:00
Thomas Huth 53c915c627 gitlab-ci.yml: Run check-tcg with TCI
It's now possible to also run the non-x86 TCG tests with TCI.

Message-Id: <20210127055903.40148-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-19 06:29:05 +01:00
Philippe Mathieu-Daudé 3473f6ed88 gitlab-ci: Disable vhost-kernel in build-disable job
Commit 299e6f19b3 ("vhost-net: revamp configure logic") added
the --enable-vhost-kernel option.
Disable it in the build-disable job.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210131104621.221602-1-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-19 06:29:05 +01:00
Philippe Mathieu-Daudé c071f19b0d gitlab-ci: Display Avocado log content when tests timeout
Since commit ba2232bae6 ("gitlab-ci: Refactor code that show logs
of failed acceptances") we display the log content of failing tests
(Avocado "FAIL" event).

Since we are also interested in tests timeouting, update our global
Avocado config to display log content for the "INTERRUPT" event,
"possible when the timeout is reached" (See [*]).

[*] https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#test-statuses

Suggested-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210215171438.935665-1-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-19 06:29:05 +01:00
Daniel P. Berrangé f5157dc770 gitlab: fix inconsistent indentation
The standard is to use 2 space indent, not 3.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210216132954.295906-4-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-19 06:29:05 +01:00
Daniel P. Berrangé 764a0747a7 gitlab: add fine grained job deps for all build jobs
This allows the build jobs to start running as soon as their respective
container image is ready, instead of waiting for all container builds
to finish.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210216132954.295906-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-19 06:29:05 +01:00
Thomas Huth 0c99f50c52 travis.yml: Move the -fsanitize=thread testing to the gitlab-CI
Use clang-10, so we can also use the --enable-tsan configure
option instead of only passing the flag via --extra-cflags.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210211045455.456371-6-thuth@redhat.com>
Message-Id: <20210211122750.22645-6-alex.bennee@linaro.org>
2021-02-15 09:38:26 +00:00
Thomas Huth ac6d7074c0 travis.yml: (Re-)move the --enable-debug jobs
We already have similar jobs in the gitlab-CI ("build-some-softmmu" and
"build-user-plugins"), so let's switch one of them to use --enable-debug
instead of --enable-debug-tcg, then we can simply drop these jobs from
the Travis-CI.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210211045455.456371-5-thuth@redhat.com>
Message-Id: <20210211122750.22645-5-alex.bennee@linaro.org>
2021-02-15 09:38:26 +00:00
Thomas Huth 3382261c26 travis.yml: Move the --enable-modules test to the gitlab-CI
Simply add the flag to an existing job, no need for yet another
job here.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210211045455.456371-4-thuth@redhat.com>
Message-Id: <20210211122750.22645-4-alex.bennee@linaro.org>
2021-02-15 09:38:26 +00:00
Thomas Huth c2b33c43fd travis.yml: Move the -fsanitize=undefined test to the gitlab-CI
Add it to the existing Clang job and also add a job that covers the
linux-user code with this compiler flag. To make sure that the detected
problems are not simply ignored, let's also use "-fno-sanitize-recover=..."
now instead.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210211045455.456371-3-thuth@redhat.com>
Message-Id: <20210211122750.22645-3-alex.bennee@linaro.org>
2021-02-15 09:38:26 +00:00
Philippe Mathieu-Daudé 04250c69bc travis.yml: Move gprof/gcov test across to gitlab
Similarly to commit 8cdb2cef3f, move the gprof/gcov test to GitLab.

The coverage-summary.sh script is not Travis-CI specific, make it
generic.

[thuth: Add gcovr and bsdmainutils which are required for the
        coverage-summary.sh script to the ubuntu docker file,
        and use 'check' as test target]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201108204535.2319870-10-philmd@redhat.com>
Message-Id: <20210211045455.456371-2-thuth@redhat.com>
Message-Id: <20210211122750.22645-2-alex.bennee@linaro.org>
2021-02-15 09:38:19 +00:00
Stefan Hajnoczi b6053de79f gitlab-ci: remove redundant GitLab repo URL command
It is no longer necessary to point .gitmodules at GitLab repos when
running in GitLab CI since they are now used all the time.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210111115017.156802-4-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-02-09 20:53:56 +00:00
Thomas Huth 12a917a14f gitlab-ci.yml: Avoid recompiling the sources in the test jobs
Currently, our check-system-* jobs are recompiling the whole sources
again. This happens due to the fact that the jobs are checking out
the whole source tree and required submodules again, and only try
to use the "build" directory with the binaries and object files as an
artifact from the previous stage - which simply does not work right
anymore (with the current version of meson). Due to some changed
time stamps, meson/ninja are always trying to rebuild the whole tree.

In the long run, we could likely use "meson test --no-rebuild", but
there is still some work going on in that area to improve the user
experience. So until this has been done, simply avoid recompiling the
sources with a trick: pass NINJA=":" to the make process in the test
jobs. Also check out the submodules manually before updating the
timestamps in the build folder, so that the binaries are definitely
newer that all the source files.
This saves ca. 10 - 15 minutes of precious CI cycles in each run.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20210126065757.403853-1-thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-27 04:47:40 +01:00
Thomas Huth 8cd95aec4c gitlab-ci.yml: Exclude some redundant targets in build-without-default-features
The build-without-default-features job is running quite long and sometimes
already hits the 1h time limit. Exclude some targets which do not provide
additional test coverage here (since we e.g. also already test other targets
of the same type, just with different endianess, or a 64-bit superset) to
avoid that we hit the timeout here so easily.

Message-Id: <20210126172345.15947-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-27 04:47:13 +01:00
Thomas Huth db0108d5d8 gitlab-ci.yml: Avoid some submodules to speed up the CI a little bit
Since the meson build system rework, the configure script prefers the
git submodules over the system libraries. So we are testing compilation
with capstone, fdt and libslirp as a submodule all over the place,
burning CPU cycles by recompiling these third party modules and wasting
some network bandwidth in the CI by cloning the submodules each time.
Let's stop doing that in at least a couple of jobs and use the system
libraries instead.

While we're at it, also install meson in the Fedora container, since
it is new enough already, so we do not need to check out the meson
submodule here.

Message-Id: <20210121174451.658924-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-26 18:38:37 +01:00
Philippe Mathieu-Daudé 4e41d4a34e gitlab-ci: Test building linux-user targets on CentOS 7
Add a configuration tested by Peter Maydell (see [1] and [2])
but not covered in our CI [3]:

  [705/2910] Compiling C object libqemu-arm-linux-user.fa.p/linux-user_strace.c.o
  FAILED: libqemu-arm-linux-user.fa.p/linux-user_strace.c.o
  ../linux-user/strace.c: In function 'do_print_sockopt':
  ../linux-user/strace.c:2831:14: error: 'IPV6_ADDR_PREFERENCES' undeclared (first use in this function)
           case IPV6_ADDR_PREFERENCES:
                ^

This job currently takes 31 minutes 32 seconds ([4]).

[1] https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg05086.html
[2] https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg05379.html
[3] https://gitlab.com/philmd/qemu/-/jobs/977408284
[4] https://gitlab.com/philmd/qemu/-/jobs/978223286

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210121172829.1643620-3-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-26 15:39:34 +01:00
Jiaxun Yang b22786eab0 gitlab-ci: Add alpine to pipeline
[thuth: Enable "make check" tests, too, after tests/check-block.sh got fixed]

Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Message-Id: <20210118063808.12471-10-jiaxun.yang@flygoat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-20 10:46:54 +01:00
Peter Maydell b93f4fbdc4 docs: Build and install all the docs in a single manual
When we first converted our documentation to Sphinx, we split it into
multiple manuals (system, interop, tools, etc), which are all built
separately.  The primary driver for this was wanting to be able to
avoid shipping the 'devel' manual to end-users.  However, this is
working against the grain of the way Sphinx wants to be used and
causes some annoyances:
 * Cross-references between documents become much harder or
   possibly impossible
 * There is no single index to the whole documentation
 * Within one manual there's no links or table-of-contents info
   that lets you easily navigate to the others
 * The devel manual doesn't get published on the QEMU website
   (it would be nice to able to refer to it there)

Merely hiding our developer documentation from end users seems like
it's not enough benefit for these costs.  Combine all the
documentation into a single manual (the same way that the readthedocs
site builds it) and install the whole thing.  The previous manual
divisions remain as the new top level sections in the manual.

 * The per-manual conf.py files are no longer needed
 * The man_pages[] specifications previously in each per-manual
   conf.py move to the top level conf.py
 * docs/meson.build logic is simplified as we now only need to run
   Sphinx once for the HTML and then once for the manpages5B
 * The old index.html.in that produced the top-level page with
   links to each manual is no longer needed

Unfortunately this means that we now have to build the HTML
documentation into docs/manual in the build tree rather than directly
into docs/; otherwise it is too awkward to ensure we install only the
built manual and not also the dependency info, stamp file, etc.  The
manual still ends up in the same place in the final installed
directory, but anybody who was consulting documentation from within
the build tree will have to adjust where they're looking.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20210115154449.4801-1-peter.maydell@linaro.org
2021-01-19 15:45:14 +00:00
Alex Bennée 22250a6ecd gitlab: migrate the minimal tools and unit tests from Travis
These tests are good at shaking out missing stubs which otherwise work
if we have built targets. Rather than create a new job just add the
checks to the existing tools-and-docs build.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210114165730.31607-12-alex.bennee@linaro.org>
2021-01-18 10:04:31 +00:00
Alex Bennée a8a3abe0b3 gitlab: move docs and tools build across from Travis
While we are at it we might as well check the tag generation. For
bonus points we run GNU globals htags into the public pages directory
for publishing with the auto generated pages.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210114165730.31607-10-alex.bennee@linaro.org>
2021-01-18 10:04:31 +00:00
Cho, Yu-Chen b115ea3a0d gitlab-ci.yml: Add openSUSE Leap 15.2 for gitlab CI/CD
Add build-system-opensuse jobs and opensuse-leap.docker dockerfile.
Use openSUSE Leap 15.2 container image in the gitlab-CI.

Signed-off-by: Cho, Yu-Chen <acho@suse.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201229085046.8536-1-acho@suse.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:08 +01:00
Alex Bennée 53f41245b0 gitlab: add --without-default-features build
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201210190417.31673-8-alex.bennee@linaro.org>
2021-01-02 21:03:09 +01:00