Because Travis doesn't allow us to keep files produced during tests
(such as log files), let's print the complete job log to the "console"
in case of job failures.
This is a debugging aid, and given that there's been some timeouts
happening on some tests, we absolutely needs the logs to have a proper
action.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20190607152223.9467-2-crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
This reverts commit 6d7a134da4.
We'll have a better fix, that will show the full avocado job log
only if a test case failed.
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
We document this on our wiki and we might as well catch it in our CI
rather than waiting for it to be picked up on merge:
https://wiki.qemu.org/Testing#clang_UBSan
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
The toolchain PPA has it so we might as well use it. We currently have
to add:
-Wno-error=stringop-truncation
as there are still strncpy operations in the tree operating on things
that haven't been annotated with QEMU_NONSTRING.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
We are going to enable the qemu-iotests during "make check" again,
and for running the iotests, we need bash and gnu-sed.
Reviewed-by: Li-Wen Hsu <lwhsu@freebsd.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Message-Id: <20190502084506.8009-5-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta
board and verify the serial is working. One extra command added to
the QEMU command line is '-vga std', because the kernel used is
known to crash without it.
If alpha is a target being built, "make check-acceptance" will
automatically include this test by the use of the "arch:alpha" tags.
Alternatively, this test can be run using:
$ avocado run -t arch:alpha tests/acceptance
$ avocado run -t machine:clipper tests/acceptance
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Caio Carrara <ccarrara@redhat.com>
Message-Id: <20190312171824.5134-21-crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Just like the previous tests, boots a Linux kernel on a s390x target
using the s390-ccw-virtio machine.
Because it's not possible to have multiple VT220 consoles,
'-nodefaults' is used, so that the one set with set_console() works
correctly.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Caio Carrara <ccarrara@redhat.com>
Message-Id: <20190312171824.5134-20-crosa@redhat.com>
[ehabkost: Updated kernel URL to point to fedoraproject.org]
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Just like the previous tests, boots a Linux kernel on an arm target
using the virt machine.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Caio Carrara <ccarrara@redhat.com>
Message-Id: <20190312171824.5134-19-crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Just like the previous tests, boots a Linux kernel on a aarch64 target
using the virt machine.
One special option added is the CPU type, given that the kernel
selected fails to boot on the virt machine's default CPU (cortex-a15).
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Caio Carrara <ccarrara@redhat.com>
Message-Id: <20190312171824.5134-18-crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta
board and verify the serial is working.
If mips64el is a target being built, "make check-acceptance" will
automatically include this test by the use of the "arch:mips64el"
tags.
Alternatively, this test can be run using:
$ avocado run -t arch:mips64el tests/acceptance
$ avocado run -t machine:malta tests/acceptance
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <20190312171824.5134-15-crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta
board and verify the serial is working. Also, it relies on the serial
device set by the machine itself.
If mips is a target being built, "make check-acceptance" will
automatically include this test by the use of the "arch:mips" tags.
Alternatively, this test can be run using:
$ avocado run -t arch:mips tests/acceptance
$ avocado run -t machine:malta tests/acceptance
$ avocado run -t endian:big tests/acceptance
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190312171824.5134-14-crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
The current version of the "check-acceptance" target will only show
one line for execution of all tests. That's probably OK if the tests
to be run are quick enough and they're always the same.
But, there's already one test alone that takes on average ~5 seconds
to run, we intend to adapt the list of tests to match the user's build
environment (among other choices).
Because of that, let's present the default Avocado UI by default.
Users can always choose a different output by setting the AVOCADO_SHOW
variable.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Caio Carrara <ccarrara@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Message-Id: <20190312171824.5134-2-crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
This build keeps timing out on Travis and it's unlikely including the
additional guest front-ends will catch any failures in the fallback
code.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This is essentially a softmmu tweak so don't bother building
linux-user builds as well.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
We define a new class of targets (MAIN_SOFTMMU_TARGETS) to cover the
major architectures. We either just build those or use the new
target-list-exclude mechanism to remove them from the list. This will
hopefully stop some of the longer builds hitting the Travis timeout
limit.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
While used by TCG it is not explicitly part of TCG and the tests can
be run standalone in a minimal build.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
The --enable-modules build is consistently tripping the time limit so
reduce our target list to the "major" architectures.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
We will be moving all builds out of tree eventually but for now we
need to for building the docs as sphinx requires an out-of-tree build.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Travis enforce the use of the git protocol v2 on their images,
but the 'xcode10' image doesn't handle this correctly, resulting
in the brew packages installation failing:
$ git config protocol.version
2
$ rvm $brew_ruby do brew bundle --verbose --global
/usr/local/bin/brew tap homebrew/bundle
==> Tapping homebrew/bundle
Cloning into '/usr/local/Homebrew/Library/Taps/homebrew/homebrew-bundle'...
fatal: unknown value for config 'protocol.version': 2
Error: Failure while executing; `git clone https://github.com/Homebrew/homebrew-bundle /usr/local/Homebrew/Library/Taps/homebrew/homebrew-bundle --depth=1` exited with 128.
Error: Failure while executing; `/usr/local/bin/brew tap homebrew/bundle` exited with 1.
The newer 'xcode10.2' beta [*] image doesn't have this limitation.
This image comes with the following brew packages pre-installed,
which extend the current code coverage:
- libffi
- libpng
- libtasn1
- gnutls
- jpeg
- nettle
[*] https://blog.travis-ci.com/2019-02-12-xcode-10-2-beta-2-is-now-available
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190220193541.24419-1-philmd@redhat.com>
[AJB: re-enabled MacOS build first]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Commit 315d318452 turned --disable-uuid into a warning only; remove
the check from Travis.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190215094502.32149-2-dgilbert@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
We've had the build break with replication disabled, so lets
test that case in travis.
Suggsted-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190215094502.32149-1-dgilbert@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
The builds are reaching the magic 50 minute limit with regularity so
lets split them up. Rather than doing a full debug build on both just
enable debug tcg for linux-user.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
It fails to install homebrew. Unfortunately we cannot mark
it as an expected failure because Travis does not match
allow_failures rows against include rows (only against the
main test matrix, which we do not use at all), so just disable
it.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20190220105131.23479-1-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
The alternate coroutine builds are really only of interest to people
running KVM (although I think you could use them for TCG if you really
tried). As they tend to run long lets kill two birds with one stone
and fold the --disable-tcg build into them.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Re-building the tools and documents by default is a little wasteful as
they are not really affected by the main build options. Split tools
and documents into their own task with a minimal softmmu and
linux-user target list just to check they don't interact badly.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
The default package set installed on macOS builders from Travis already
includes libffi and gettext as shown by log messages:
Skipping install of libffi formula. It is already up-to-date.
Using libffi
Skipping install of gettext formula. It is already up-to-date.
Using gettext
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
For the two Python jobs, which seem to have the goal of making sure
QEMU builds successfully on the 3.0-3.6 spectrum of Python 3 versions,
the specified version is only applicable if a Python virtual
environment is used. To do that, it's necessary to define the
(primary?) language of the job to be Python.
Also, Travis doesn't have a 3.0 Python installation available for the
chosen distro, 3.4 being the lower version available.
Reference: https://docs.travis-ci.com/user/languages/python/#specifying-python-versions
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20181109150710.31085-4-crosa@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
[ehabkost: Now 3.4 is the lowest Python version available]
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
fixup! Travis CI: make specified Python versions usable on jobs
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Travis is slowly catching up. Move to Xenial based images for our
current builds. These are now all proper VMs.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
The global defaults request "trusty" and "gcc", so matrix entries do not
need to repeat this.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Travis sometimes fails a build because it produces no console output for
over 10 minutes. If this is due to a genuine hang, it would be useful to
have used verbose test output to see where it failed. If this is just
due to tests being very slow, having verbose output might allow the
build to succeed.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
The Travis container based envs are deprecated:
https://docs.travis-ci.com/user/reference/trusty/
"Container-based infrastructure is currently being deprecated.
Please remove any sudo: false keys in your .travis.yml file
to use the default fully-virtualized Linux infrastructure
instead."
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
One of the matrix entries redefines the script command in order to add
the ${MAKEFLAGS} variable. Ideally ${MAKEFLAGS} would be referenced by
the definition of the ${TEST_CMD} env variable, but this isn't possible
in travis. ${MAKEFLAGS} exists to eliminate duplication of flags in
every "make" command, but this cure causes a worse problem, namely the
reduplication of the "script" command. It is simpler to just insert "-j3"
directly into any "make" command.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Rather than poking homebrew manually we can specify the packages
needed via the homebrew addon. These are only installed on MacOS based
builds.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The configure script & Makefile are already capable of figuring out
which git submodules are required for a given build platform, and
cloning them at the right time.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Matrix entries are defining env variables using two different syntax
styles:
- env: FOO=bar
WIZZ=bang
and
- env:
- FOO=bar
- WIZZ=bang
Switch everything to use the latter style as the more normal indentation
approach.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
The current build matrix is constructed from entries listed under the
environment variable config section, as well as the general purpose
build matrix section. Move everything under the general purpose section
so it is clear at a glance what is in the matrix.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Improve the readability of the travis config by adding two blank lines
between each major section and matrix entry.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
We have reached the point where the MacOSX build was regularly timing
out. So as before I've reduced the target list to "major"
architectures to try and bring the build time down. I've added an
additional MacOSX build with the latest XCode with a minimal list of
"most likely" targets on MacOS.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
This enables the execution of the acceptance tests on Travis.
Because the Travis environment is based on Ubuntu Trusty, it requires
the python3-pip and python3.4-venv packages.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Caio Carrara <ccarrara@redhat.com>
Message-Id: <20181018153134.8493-4-crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
This gives a more useful summary, sorted by descending % coverage,
after the tests have run. The final numbers will give an idea if our
coverage is getting better or worse.
To keep the width sane we need to post process the file that the old
gcovr tool generates. This is done with a mix of sed, awk and column
in the scripts/coverage-summary.sh script.
As quite a lot of lines don't get covered at all we filter out all the
0% lines. If the file doesn't appear it is not being exercised.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Force one config to build 'out-of-tree' (object files and executables
are created in a tree outside the project source code).
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Builds only require:
- dtc
- keycodemapdb
- capstone
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: drop wget cache]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
These builds are reaching regular timeouts and probably don't need to
be so widely exercised. ftrace and ust in particular are used in
conjunction with whole system profiling which makes most sense with
KVM setups, hence the native softmmu target.
We also expand simple to cover the multiple log backends while
restricting its scope to user-mode testing only.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
This build is regularly timing out and even switching off linux-user
wasn't enough. Instead explicitly choose a target list of broadly the
"major" architectures. This is enough to check the gprof build
machinery works without worrying about the actual coverage results.
I did try various YAML constructs for specifying CONFIG with
continuation but couldn't get any of them to work hence the very long
line.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
When configure fails in CI systems we must be able to see the contents
of the config.log file to diagnose the root cause.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Fam Zheng <famz@redhat.com>
[AJB: used Eric's suggested {} form]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
