Commit Graph

216 Commits

Author SHA1 Message Date
Alex Bennée
3ae8a1000b tests/tcg: enable building for HPPA
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée
409c1c9cbd tests/tcg: enable building for Alpha
We can't use our normal Debian based compilers as Alpha isn't an
officially supported architecture. However it is available as a port
and fortunately cross compilers for all these targets are included in
Debian Sid, the perpetual rolling/unstable/testing version of Debian.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée
6bf77518dd tests/docker/Makefile.include: fix mipsel-cross dependancy
This got broken in commit 4319db7 but generally only shows up when you
try and do massive parallel builds on fresh machines.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée
e27cae268f docker: Makefile.include introduce DOCKER_SCRIPT
Define this in one place to make it easy to re-use.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée
50b7273854 docker: allow "cc" command to run in user context
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée
99cfdb8662 docker: extend "cc" command to accept compiler
When calling our cross-compilation images we want to call something
other than the default cc.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Fam Zheng <famz@redhat.com>
2018-06-20 20:22:34 +01:00
Alex Bennée
5e03c2d816 docker: Add "cc" subcommand
Signed-off-by: Fam Zheng <famz@redhat.com>
[AJB: add if args.paths check]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée
7a5d936b6f docker: docker.py wrap StringIO import for python3
Although the docker.py is nominally python2 we actually invoke it with
the configured python from the configure script.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-20 16:57:34 +01:00
Eduardo Habkost
f03868bd56 python: futurize -f libfuturize.fixes.fix_print_with_import
Change all Python code to use print as a function.

This is necessary for Python 3 compatibility.

Done using:

  $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \
    sort -u | grep -v README.sh4)
  $ futurize -w -f libfuturize.fixes.fix_print_with_import $py

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180608122952.2009-2-ehabkost@redhat.com>
[ehabkost: fixup tests/docker/docker.py]
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-06-08 14:39:24 -03:00
Philippe Mathieu-Daudé
36dc5fedab docker: add debian/tricore image
TriCore binutils is built from Bastian Koppelmann repository.

Note: There is no TriCore compiler in this image (only assembler/linker).

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: base of Debian9, add to Makefile.include]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-05 16:25:43 +01:00
Paolo Bonzini
cabd358407 docker: test-mingw: use SDL2 and GTK+3
Do not test the deprecated API versions.  debian-win32-cross and debian-win64-cross
are already using SDL2 (they do not cover GTK+ at all).

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
[AJB: fix merge conflicts]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-05 16:25:42 +01:00
Fam Zheng
8bce19d3a6 docker: Add fedora-i386-cross image
It has some basic *-devel.i686 packages to be used with "gcc -m32" as a
32 bit cross build environment.

Signed-off-by: Fam Zheng <famz@redhat.com>
[AJB: add glibc-static]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-05 16:25:42 +01:00
Alex Bennée
15df9d3783 docker: add "probe" command for configure
This is a helper function for the configure script. It replies yes,
sudo or no to inform the user if non-interactive docker support is
available. We trap the Exception to fail gracefully.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Fam Zheng <famz@redhat.com>
2018-06-05 16:25:42 +01:00
Fam Zheng
63dba573fa docker: Update fedora image to 28
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20180601023557.9770-1-famz@redhat.com>
2018-06-04 14:39:18 +08:00
Alex Bennée
5609c5127d tests/docker/Makefile.include: handle empty TARGET_LIST
If the user doesn't specify a TARGET_LIST they get the current
configuration but with spaces and hilarity ensues. This adds some make
magic to turn the TARGET_LIST back into a comma separated list.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Message-Id: <20180521103504.26432-1-alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-06-04 14:39:18 +08:00
Alex Bennée
67659ab1eb docker: update Travis docker image
This is still poorly documented by Travis but according to:

  https://docs.travis-ci.com/user/common-build-problems/#Running-a-Container-Based-Docker-Image-Locally

their reference images are now hosted on Docker Hub. So we update the
FROM line to refer to the new default image. We also need a few
additional tweaks:

  - re-enable deb-src lines for our build-dep install
  - add explicit PATH definition for tools
  - force the build USER to be Travis
  - add clang to FEATURES for our test-clang machinery

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-01 15:00:54 +01:00
Philippe Mathieu-Daudé
bcaf457786 docker: do not display deprecated images in 'make docker' help
the 'debian' base image is deprecated since 3e11974988

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-01 14:55:03 +01:00
Philippe Mathieu-Daudé
1a06f5c910 docker: sort images list displayed by 'make docker'
we can now directly see different version sort consecutively.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-01 14:55:03 +01:00
Philippe Mathieu-Daudé
9d5e546af0 docker: Fix trivial typo
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180508144358.13530-1-f4bug@amsat.org>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-05-18 16:52:53 +08:00
Paolo Bonzini
07fed47417 docker: fedora: test more components
Install optional dependencies of QEMU to get better coverage.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <1520942752-19449-1-git-send-email-pbonzini@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-04-09 15:13:02 +08:00
Fam Zheng
185809224a docker: Inline "prep_fail" in run script
We don't source common.rc where prep_fail is defined, so spell out the
commands and do what was intended.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20180326090350.30014-1-famz@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-04-09 15:13:02 +08:00
Philippe Mathieu-Daudé
0c153b4c6f docker: dump 'config.log' if ./configure fails
Suggested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20180315142713.30960-1-f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-04-09 15:13:02 +08:00
Marc-André Lureau
02f769b7ee tests: make docker-test-debug@fedora run sanitizers
Since --enable-debug no longer enable sanitizers, we need explicit
--enable-sanitizers.

llvm package is required for llvm-symbolizer, to get symbols in
backtraces.

Add make V=1 to get details about failing tests.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180312120849.20073-1-marcandre.lureau@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-03-13 01:19:56 +08:00
Paolo Bonzini
5b9b49d7bd docker: change Fedora base image to fedora:27
Using "fedora:latest" makes behavior different depending on when you
actually pulled the image from the docker repository.  In my case,
the supposedly "latest" image was a Fedora 25 download from 8 months
ago, and the new "test-debug" test was failing.

Use "27" to improve reproducibility and make it clear when the image
is obsolete.

Cc: Fam Zheng <famz@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <1515755504-21341-1-git-send-email-pbonzini@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-02-08 09:21:34 +08:00
Daniel P. Berrange
356dc290f0 docker: change Fedora images to run with python3
Fedora has switched to Python 3 by default, so it makes sense to use that
for testing QEMU builds, so we get testing of Python 3 compatibility.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <20180116134217.8725-15-berrange@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-02-05 19:53:55 -02:00
Marc-André Lureau
11545663d1 tests/docker: add test-debug
Add a new test with --enable-debug using clang/asan/ubsan, remove
--enable-debug from test-clang & test-mingw.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180104160523.22995-7-marcandre.lureau@redhat.com>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-01-12 13:22:02 +01:00
Marc-André Lureau
c08d08b27c tests/docker: add some sanitizers to fedora dockerfile
Build fedora image with ASAN/UBSan support.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180104160523.22995-6-marcandre.lureau@redhat.com>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-01-12 13:22:02 +01:00
Fam Zheng
dbd73b569b Revert "docker: Enable features explicitly in test-full"
This reverts commit 5e8a7fe673.

It's hard to get all images to have all these packages, the usual
"FEATURES" and "require" mechanism doesn't scale with so many features.
With that change, the test basically only works in ubuntu.

Until a better way comes up, leave the feature enabling to ./configure
detection.

But don't remove the "-e" removal.

Reported-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20171018082002.9406-1-famz@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-12-20 22:01:23 +08:00
Philippe Mathieu-Daudé
6423795efc docker: correctly escape $BACKEND in the help output
In Makefiles the $ must be escaped as $$ in shell uses.

Since 8a2390a4f4:

 $ make docker
     [...]
     NETWORK=1            Enable virtual network interface with default backend.
     NETWORK=ACKEND     Enable virtual network interface with ACKEND.

Once escaped:

 $ make docker
     [...]
     NETWORK=1            Enable virtual network interface with default backend.
     NETWORK=$BACKEND     Enable virtual network interface with $BACKEND.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-Id: <20171108024719.8389-1-f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-11-08 10:59:42 +08:00
Fam Zheng
c1958e9d54 docker: Improved image checksum
When a base image locally defined by QEMU, such as in the debian images,
is updated, the dockerfile checksum mechanism in docker.py still skips
updating the derived image, because it only looks at the literal content
of the dockerfile, without considering changes to the base image.

For example we have a recent fix e58c1f9b35 that fixed
debian-win64-cross by updating its base image, debian8-mxe, but due to
above "feature" of docker.py the image in question is automatically NOT
rebuilt unless you add NOCACHE=1. It is noticed on Shippable:

https://app.shippable.com/github/qemu/qemu/runs/541/2/console

because after the fix is merged, the error still occurs, and the log
shows the container image is, as explained above, not updated.

This is because at the time docker.py was written, there wasn't any
dependencies between QEMU's docker images.

Now improve this to preprocess any "FROM qemu:*" directives in the
dockerfiles while doing checksum, and inline the base image's dockerfile
content, recursively. This ensures any changes on the depended _QEMU_
images are taken into account.

This means for external images that we expect to retrieve from docker
registries, we still do it as before. It is not perfect, because
registry images can get updated too. Technically we could substitute the
image name with its hex ID as obtained with $(docker images $IMAGE
--format="{{.Id}}"), but --format is not supported by RHEL 7, so leave
it for now.

Reported-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20171103131229.4737-1-famz@redhat.com>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-11-08 10:59:21 +08:00
Philippe Mathieu-Daudé
e58c1f9b35 docker: add python stdlib dependency (required by keycodemapdb)
Since 927128222b QEMU depends of keycodemapdb, which uses the python 'csv'
module from stdlib to parse keymaps.csv.

Without this package the build fails:

    GEN     ui/input-keymap-linux-to-qcode.c
  Traceback (most recent call last):
    File "ui/keycodemapdb/tools/keymap-gen", line 15, in <module>
      import csv
  ImportError: No module named csv
    GEN     ui/input-keymap-qcode-to-qnum.c
  Traceback (most recent call last):
    File "ui/keycodemapdb/tools/keymap-gen", line 15, in <module>
      import csv
  ImportError: No module named csv
  [...]
    CC      ui/input-keymap.o
  ui/input-keymap.c:8:44: fatal error: ui/input-keymap-linux-to-qcode.c: No such file or directory
  make: *** [ui/input-keymap.o] Error 1

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-11-02 11:46:41 +00:00
Fam Zheng
6a2e11974f docker: Fix PATH for ccache
Before bcd7f06f57 we source /etc/profile
so the PATH included the right paths to ccache binaries. Now we need to
update $PATH explicitly from run script.

Keep the old /usr/lib around just so that in the future, ccache from 32
bit images will just work.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20171018073841.30062-1-famz@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-10-20 14:28:50 +08:00
Greg Kurz
1dc1700a60 docker: fix out-of-tree 'make docker-test-build@debian-powerpc-cross'
Without this patch, it fails with:

IOError: [Errno 2] No such file or directory:
 'tests/docker/dockerfiles/debian-apt-fake.sh'

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <150841324224.31292.4012464539983692364.stgit@bahia.lan>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-10-20 14:28:50 +08:00
Paolo Bonzini
bc739b8e7d docker: allow running from srcdir != builddir build
The new script uses "git submodule", which is picky about being invoked
from the top of the git checkout.  Invoke the script from $(SRC_PATH)
to avoid git's wrath.

Fixes: b7f404201e
Cc: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Message-Id: <1508331989-142364-1-git-send-email-pbonzini@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-10-20 14:28:50 +08:00
Peter Xu
08c3f6d57f docker: cleanup temp directory after test
There are temp directories named "docker-src.*" after doing docker
tests.  I don't see much point in keeping that (it only contains the
qemu.tar which is exactly current tree, and the copied "run" file).
Let's remove it after test finished.

CC: "Alex Bennée" <alex.bennee@linaro.org>
CC: Fam Zheng <famz@redhat.com>
CC: "Philippe Mathieu-Daudé" <f4bug@amsat.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20171017071247.32355-1-peterx@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-10-20 14:28:50 +08:00
Fam Zheng
4b2c6bc563 docker: Don't allocate tty unless DEBUG=1
The existence of tty in the container seems to urge gcc into colorizing
the errors, but the escape chars will clutter the report once turned
into email replies on patchew. Move -t to debug mode.

Reported-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20171013011954.9975-1-famz@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-10-20 14:28:50 +08:00
Fam Zheng
36ac78e65a docker: Don't mount ccache db if NOUSER=1
With NOUSER=1 the container runs code as root, which may create
privileged files that will not be be accssible next time. Skip ccache
dir mount in this case.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170925075458.18047-1-famz@redhat.com>
Acked-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
2017-09-29 11:14:15 +08:00
Fam Zheng
12b25a7d96 docker: test-block: Don't continue if build fails
Report error and exit upon compiling error, otherwise the iotests output
will be pure noise.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170926110134.2786-1-famz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-09-29 11:14:15 +08:00
Alex Bennée
bcd7f06f57 tests/docker/run: don't source /etc/profile
The usual behaviour of /etc/profile is to set the default PATH for
users. This runs into problems when we have updated PATH in our
dockerfile e.g. to access a cross-compiler in a non-standard
location. It shouldn't be needed anyway as we inherit the env from the
image when it was setup.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
CC: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20170926133622.14991-1-alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-09-29 11:14:15 +08:00
Fam Zheng
299d296ea9 docker: Fix test-mingw
Feature "dtc" is explicitly required by test-mingw, but is not detected
by the run script since we switched to archive-source.sh in b7f404201e.
Since it isn't available in the Fedora image which runs this test on
patchew, the way we get dtc is still from submodule.

archive-source.sh takes care of bundling the submodule files already, so
what we need to do is just checking if files are there. Makefile is
chosen because it is one that is unlikely to get renamed in the future.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170925082913.22089-1-famz@redhat.com>
Reviewed-by: Alistair Francis <alistair.francis@xilinx.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-09-29 11:14:14 +08:00
Paolo Bonzini
6283847857 docker: add installation to build tests
Basic test that "make install" works; this requires msgfmt so add
gettext to the packages.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <1506095371-23160-1-git-send-email-pbonzini@redhat.com>
[Rebase to master. - Fam]
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-09-29 11:14:14 +08:00
Fam Zheng
4f6afe41f2 docker: Drop 'set -e' from run script
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-09-22 14:51:43 +08:00
Fam Zheng
b7f404201e docker: Use archive-source.py
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
2017-09-22 14:51:43 +08:00
Fam Zheng
f300ca63c7 docker: Add test-block
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170905025614.579-6-famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Based-on: 20170905021201.25684-1-famz@redhat.com
2017-09-22 10:20:34 +08:00
Fam Zheng
18d4e35f93 docker: Add nettle-devel to fedora image
The LUKS cases in qemu-iotests requires this.

Reviewed-by: Kashyap Chamarthy <kchamart@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170905025614.579-5-famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Based-on: 20170905021201.25684-1-famz@redhat.com
2017-09-22 10:20:34 +08:00
Fam Zheng
4470749186 docker: Use unconfined security profile
Some by default blocked syscalls are required to run tests for example
userfaultfd.

Reviewed-by: Kashyap Chamarthy <kchamart@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170905025614.579-4-famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Based-on: 20170905021201.25684-1-famz@redhat.com
2017-09-22 10:20:34 +08:00
Fam Zheng
82659e844a docker: Add test_fail and prep_fail
They both print a message and exit, but with different status code so
distinguish real test errors from env preparation failures.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170905025614.579-3-famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Based-on: 20170905021201.25684-1-famz@redhat.com
2017-09-22 10:20:34 +08:00
Fam Zheng
d8a2f5116d docker: Fix return code of build_qemu()
Without "set -e", the "&&" makes sure that the return code reflects the
result status, and that make only runs if configure succeeds.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170905025614.579-2-famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Based-on: 20170905021201.25684-1-famz@redhat.com
2017-09-22 10:20:34 +08:00
Fam Zheng
05790dafef tests/docker: Clean up paths
The 'run' script already creats src, build and install directories under
$TEST_DIR, use it in common.rc.

Also the tests always run from $QEMU_SRC/tests/docker, so use a relative
$CMD string.

Message-Id: <20170817035721.11064-1-famz@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-09-22 10:20:34 +08:00
Fam Zheng
5e8a7fe673 docker: Enable features explicitly in test-full
Also avoid "set -e".

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170907141245.31946-3-famz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2017-09-22 10:20:34 +08:00
Fam Zheng
7fc581c295 docker: Update ubuntu image
Base on the newer ubuntu-lts (16.06) and include more packages for
better build coverage.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170907141245.31946-2-famz@redhat.com>
2017-09-22 10:20:34 +08:00
Alex Bennée
3f2ff267af docker: reduce noise when building travis.docker
Set the DEBIAN_FRONTEND and locale env vars to stop apt complaining so
much as we build the image.

Suggested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20170725133425.436-7-alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-09-22 10:20:34 +08:00
Alex Bennée
9b4154a570 docker: don't install device-tree-compiler build-deps in travis.docker
Installing the device-tree-compiler build-deps is a little extreme. We
only actually need the binary so include it with the other packages.

Suggested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20170725133425.436-6-alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-09-22 10:20:34 +08:00
Alex Bennée
6fe3ae3f19 docker: docker.py make --no-cache skip checksum test
If you invoke with NOCACHE=1 we pass --no-cache in the argv to
docker.py but may still not force a rebuild if the dockerfile checksum
hasn't changed. By testing for its presence we can force builds
without having to manually remove the docker image.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20170725133425.436-5-alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-09-22 10:20:34 +08:00
Alex Bennée
1fddbf7c5e docker: ensure NOUSER for travis images
While adding the current user is a useful default behaviour for
creating new images it is not appropriate for Travis which already has
a default user.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20170725133425.436-2-alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-09-22 10:20:34 +08:00
Paolo Bonzini
7437866bfc docker: fix creation of archives
The pixman submodule does not exist anymore, and its removal broke
docker-based tests.  Fix it.

Cc: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-09-19 16:20:49 +02:00
Stefan Hajnoczi
c2d3189667 docker.py: Python 2.6 argparse compatibility
Add the scripts/ directory to sys.path so Python 2.6 will be able to
import argparse.

Cc: Fam Zheng <famz@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: John Snow <jsnow@redhat.com>
Acked-by: Fam Zheng <famz@redhat.com>
Message-id: 20170825155732.15665-3-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2017-08-30 12:02:11 +01:00
Philippe Mathieu-Daudé
a8132a2f28 docker: add centos7 image
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20170728233316.13352-5-f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-08-15 15:16:45 +08:00
Philippe Mathieu-Daudé
06f3c7b852 docker: install more packages on CentOS to extend code coverage
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20170728233316.13352-4-f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-08-15 15:16:45 +08:00
Philippe Mathieu-Daudé
18a2b7affc docker: add Xen libs to centos6 image
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20170728233316.13352-3-f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-08-15 15:16:45 +08:00
Philippe Mathieu-Daudé
6aef2ad70b docker: use one package per line in CentOS config
This ease rebase/cherry-pick, also it is faster to visually find if a package
is here.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20170728233316.13352-2-f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-08-15 15:16:45 +08:00
Philippe Mathieu-Daudé
118d1becc0 docker: install clang since Shippable setup_ve() verify it is available
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:58:36 +01:00
Philippe Mathieu-Daudé
3e11974988 docker: warn users to use newer debian8/debian9 base image
to stay backward incompatible.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:58:36 +01:00
Philippe Mathieu-Daudé
f847325106 docker: add debian Ports base image
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:58:36 +01:00
Philippe Mathieu-Daudé
982e758af7 docker: add MXE (M cross environment) base image for MinGW-w64
see http://mxe.cc/

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:58:36 +01:00
Philippe Mathieu-Daudé
fd7b284d79 docker: add debian/mips64el image
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:58:36 +01:00
Philippe Mathieu-Daudé
4319db7b13 docker: add debian/mips[eb] images
change image mips little -> big endian

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:58:35 +01:00
Alex Bennée
16dcece4d5 docker: add debian/powerpc based on Jessie
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
[AJB: split from merged patch]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:58:35 +01:00
Philippe Mathieu-Daudé
bec45514e5 docker: add 'apt-fake' script which generate fake debian packages
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:58:35 +01:00
Philippe Mathieu-Daudé
a3cf6b1961 docker: add qemu:debian-jessie based on outdated jessie release
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:58:35 +01:00
Philippe Mathieu-Daudé
c705b8617b docker: enable nettle to extend code coverage on arm64
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Philippe Mathieu-Daudé
6f8bbb374b docker: enable gcrypt to extend code coverage on amd64
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Philippe Mathieu-Daudé
0cf01f5b4a docker: enable netmap to extend code coverage on amd64
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Philippe Mathieu-Daudé
b2e7717425 docker: enable virgl to extend code coverage on amd64
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: mv, comments in dockerfile]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Philippe Mathieu-Daudé
15288046e0 docker: add debian/amd64 based on Stretch
By itself this doesn't add much to our coverage. However later patches
will extend this image to include more bleeding edge libraries which
are not yet widely available in distros.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: extend commit msg]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Philippe Mathieu-Daudé
2b1c10bdcf docker: add debian/ppc64el based on Stretch
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Philippe Mathieu-Daudé
31e205d1bf docker: add debian/armel based on Stretch
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Philippe Mathieu-Daudé
aaa6ccc9ff docker: optimize debian9 base image
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Philippe Mathieu-Daudé
2a444f55eb docker: remove packages now dependent of qemu in Stretch
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Philippe Mathieu-Daudé
7d2bcf2363 docker: debian/s390x no more in unstable, now available in Stretch
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Philippe Mathieu-Daudé
94afe49eab docker: add common packages to debian base
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Philippe Mathieu-Daudé
24044d4825 docker: rename debian stable -> 9 (Stretch)
We'll also want to support some older Debian combinations for
architectures that didn't make the Debian 9 cut.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: extend commit msg]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Alex Bennée
ed7023b05b docker: add debug tools to travis.docker
When a test fails/hangs you don't want the hassle of getting the debug
tools installed. Lets install them on our image by default so we can
debug when we need to.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:23 +01:00
Alex Bennée
dca34e8c4e docker: include python-yaml in travis.docker
Although the upstream Travis images don't need this library our
"travis-lite" scripts are written in python. This allows us to do:

  make docker-travis@travis J=10

and approximate a travis run on their default image.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:13 +01:00
Paolo Bonzini
6ca9f7fedb docker: allow customizing Travis global_env variables
This is useful so that we can do builds at higher than -j3 when running
travis.py locally.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 09:39:19 +01:00
Fam Zheng
0b95ff72cb docker.py: Improve subprocess exit code handling
A few error handlings are missing because we ignore the subprocess exit
code, for example "docker build" errors are currently ignored.

Introduce _do_check() aside the existing _do() method and use it in a
few places.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170712075528.22770-3-famz@redhat.com>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-07-17 11:34:20 +08:00
Fam Zheng
58bf7b6d8c docker.py: Drop infile parameter
The **kwargs can do this just well.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170712075528.22770-2-famz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-07-17 11:34:20 +08:00
Daniel P. Berrange
8a2390a4f4 docker: Don't enable networking as a side-effect of DEBUG=1
When trying to debug problems with tests it is natural to set
DEBUG=1 when starting the docker environment. Unfortunately
this has a side-effect of enabling an eth0 network interface
in the container, which changes the operating environment of
the test suite. IOW tests with fail may suddenly start
working again if DEBUG=1 is set, due to changed network setup.

Add a separate NETWORK variable to allow enablement of
networking separately from DEBUG=1. This can be used in two
ways. To enable the default docker network backend

  make docker-test-build@fedora NETWORK=1

while to enable a specific network backend, eg join the network
associated with the container 'wibble':

  make docker-test-build@fedora NETWORK=container:wibble

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <20170713144352.2212-1-berrange@redhat.com>
[Drop the superfluous second $(subst ...). - Fam]
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-07-17 11:34:20 +08:00
Philippe Mathieu-Daudé
2e1d6bdcce docker: add mipsel build target
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
[AJB: remove apt-fake kludge]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-06-21 15:02:43 +01:00
Philippe Mathieu-Daudé
c9c06eb832 docker: add extra libs to s390x target to extend codebase coverage
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-06-21 15:01:48 +01:00
Philippe Mathieu-Daudé
905bf0ee8a docker: add extra libs to arm64 target to extend codebase coverage
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-06-21 15:01:48 +01:00
Philippe Mathieu-Daudé
32809e7f7b docker: add extra libs to armhf target to extend codebase coverage
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-06-21 15:01:48 +01:00
Philippe Mathieu-Daudé
8a98bfc6e3 docker: use eatmydata in debian arm64 image
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-06-21 15:01:48 +01:00
Philippe Mathieu-Daudé
8a48be0e87 docker: use eatmydata in debian armhf image
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-06-21 15:01:48 +01:00
Philippe Mathieu-Daudé
96e659d006 docker: use eatmydata, install common build packages in base image
The common build packages are: build-essential clang git bison flex

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
[AJB: fixups following stretch update]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-06-21 15:01:48 +01:00
Philippe Mathieu-Daudé
cf80eb8d09 docker: use better regex to generate deb-src entries
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
[AJB: fixed up following dropping emdebian]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-06-21 15:01:48 +01:00
Philippe Mathieu-Daudé
2c1c31ed55 docker: install ca-certificates package in base image
Resolve SSL verification issue at shippable container's git_sync stage:

shippable logs:
--------------
git_sync
- ssh-agent bash -c 'ssh-add /tmp/ssh/01_deploy; git clone https://github.com/philmd/qemu.git /root/src/github.com/philmd/qemu'
Identity added: /tmp/ssh/01_deploy (rsa w/o comment)
Cloning into '/root/src/github.com/philmd/qemu'...
fatal: unable to access 'https://github.com/philmd/qemu.git/': Problem with the SSL CA cert (path? access rights?)
retrying 1 of 3 times...

Suggested-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
[AJB: fixed re-base conflict following stretch updates]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-06-21 15:01:48 +01:00
Philippe Mathieu-Daudé
438d116872 docker: rebuild image if 'extra files' checksum does not match
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-06-21 15:01:48 +01:00
Philippe Mathieu-Daudé
4c84f662c2 docker: add --include-files argument to 'build' command
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-06-21 15:01:48 +01:00
Philippe Mathieu-Daudé
2499ee9fad docker: let _copy_with_mkdir() sub_path argument be optional
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-06-21 15:01:48 +01:00