Clean up the heading levels to use === --- ~~~ ^^^ '''. Reorganize the
outline for the Avocado part, and always include headings for the
class names.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Add the possibility of running all the tests from a single file, or
multiple files, running a single test within a file or multiple tests
within multiple files using `make check-acceptance` and the
AVOCADO_TESTS environment variable.
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210923161141.232208-4-willianr@redhat.com>
Add instructions to the Acceptance tests section about running a
single test file or a test within the test file.
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210923161141.232208-3-willianr@redhat.com>
Although it is possible to run a specific test using the avocado
command-line, a user may want to use a specific tag while running the
``make check-acceptance`` during the development or debugging.
This allows using the AVOCADO_TAGS environment variable where the user
takes total control of which tests should run based on the tags defined.
This also makes the check-acceptance command flexible to restrict tests
based on tags while running on CI.
e.g.:
AVOCADO_TAGS="foo bar baz" make check-acceptance
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210923161141.232208-2-willianr@redhat.com>
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210809090114.64834-17-eesposit@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210809090114.64834-15-eesposit@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210809090114.64834-10-eesposit@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Introduce the "Debugging a test case" section, in preparation
to the additional flags that will be added in the next patches.
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210809090114.64834-5-eesposit@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
In rST markup, single backticks `like this` represent "interpreted
text", which can be handled as a bunch of different things if tagged
with a specific "role":
https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#interpreted-text
(the most common one for us is "reference to a URL, which gets
hyperlinked").
The default "role" if none is specified is "title_reference",
intended for references to book or article titles, and it renders
into the HTML as <cite>...</cite> (usually comes out as italics).
Fix various places in the devel section of the manual which were
using single backticks when double backticks (for literal text)
were intended.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210726142338.31872-6-peter.maydell@linaro.org
This introduces a new feature to the functional tests: automatic setting of
the '-cpu VALUE' option to the created vm if the test is tagged with
'cpu:VALUE'. The 'cpu' property is made available to the test object as well.
For example, for a simple test as:
def test(self):
"""
🥑 tags=cpu:host
"""
self.assertEqual(self.cpu, "host")
self.vm.launch()
The resulting QEMU evocation will be like:
qemu-system-x86_64 -display none -vga none \
-chardev socket,id=mon,path=/var/tmp/avo_qemu_sock_pdgzbgd_/qemu-1135557-monitor.sock \
-mon chardev=mon,mode=control -cpu host
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210430133414.39905-2-wainersm@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
The tests based on the LinuxTest class give the test writer a ready to
use guest operating system, currently pinned to Fedora 31.
With this change, it's now possible to choose different distros and
versions, similar to how other tags and parameter can be set for the
target arch, accelerator, etc.
One of the reasons for this work, is that some development features
depend on updates on the guest side. For instance the tests on
virtiofs_submounts.py, require newer kernels, and may benefit from
running, say on Fedora 34, without the need for a custom kernel.
Please notice that the pre-caching of the Fedora 31 images done during
the early stages of `make check-acceptance` (before the tests are
actually executed) are not expanded here to cover every new image
added. But, the tests will download other needed images (and cache
them) during the first execution.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210414221457.1653745-4-crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
With the recent move of the unit tests to tests/unit directory some
instructions under the "Unit tests" section became imprecise, which
are fixed by this change.
Fixes: da668aa15b ("tests: Move unit tests into a separate directory")
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210318174407.2299930-1-wainersm@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
For completeness reference the check-tcg tests in the container
preamble text.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Message-Id: <20210222101455.12640-7-alex.bennee@linaro.org>
This section has grown a little stale so clean-up the language and
examples for current usage:
- refer to containers at the top
- mention podman can also be used
- add podman prerequisites section
- move to using "docker-help" for online help
- mention the registry and it's purpose
- don't refer to out-of-date min-glib image
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210222101455.12640-6-alex.bennee@linaro.org>
Expand on the usage of containers for building tests and why we have
some that are not used to build QEMU itself.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Message-Id: <20210222101455.12640-5-alex.bennee@linaro.org>
The travis container that we have no longer matches what travis
currently uses. As all x86 jobs are being moved to GitLab CI too,
there is no compelling reason to update the travis container. It
is simpler to just remove it.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210209135011.1224992-2-berrange@redhat.com>
Message-Id: <20210211122750.22645-9-alex.bennee@linaro.org>
The "🥑 enable" is not necessary and was removed in 9531d26c,
so let's remove from the docs.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210203172357.1422425-4-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
- Fix crash on write to read-only devices
- iotests: Rewrite 'check' in Python, get rid of 'groups' and allow
non-numeric test case names
-----BEGIN PGP SIGNATURE-----
iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmARxVARHGt3b2xmQHJl
ZGhhdC5jb20ACgkQfwmycsiPL9ZxrA//TAulJrmJ0T+h/782HRzA8St+Yuzkbdhh
96xuxqG+flhiRi1gq2LOiOUnIzYGgG0x2bI1INZku3AsqNp2BG02DtX95jdxpPkg
8Yr/PGAZ1Z+/fdajphcAVujR8A9tDrfj4wNXHOfN5cPoIXobNQGwMHX4+fDWJ3yf
sbeihc70JmjQ3V6Psau2YAVLCESRLfPkG571YYiNu1DmBVI5n0xgMUD4Oc6hzIPR
igJ6K5EO+uacrY1b6j5WsTIn7+mmUD8EC6njNZrYcF3aq8/SsCIaLqV/+6LN589e
KdvysjU3EtRbeln2Zk83m4YTL48Cpz967aCbfD908lRmsq0tu+X0LrAQoa9Y7+0x
DHS7jkUAG2LOppVrSFKUSt6QxOz08z8qqCBZutdMEBTJtfiq04I/EJhdv4HqLjD8
4dE9PQITAILO2NzfbWS+vVhHOgoyIuSu4QtC2Nw6ztUiGrtqkbRRkfPLmEv3vrxn
daA/yHgyJFwyKPtiubKqk20LZb/5txJKXCPpo18MVIs9r70NcqfRT6J1XVqApFQA
aXPhEsBNZ/p0gCBiej98ukCb15rOqlSk7iVlSV3kIPtp/Nk5kDqC3XV48Uy+GtXW
shSM4QZxoE6CoKPGkB5hWyMAAvYwyJRRJAPy7w/Rcqd3vTsfgwX7Ad0bBVQe0kpo
+pMOKs0/Yk8=
=gFsg
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging
Block layer patches:
- Fix crash on write to read-only devices
- iotests: Rewrite 'check' in Python, get rid of 'groups' and allow
non-numeric test case names
# gpg: Signature made Wed 27 Jan 2021 19:56:00 GMT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6
* remotes/kevin/tags/for-upstream:
iotests: rename and move 169 and 199 tests
iotests: rewrite check into python
iotests: add testrunner.py
iotests: add testenv.py
iotests: add findtests.py
iotests: 146: drop extra whitespaces from .out file
virtio-scsi-test: Test writing to scsi-cd device
block: Separate blk_is_writable() and blk_supports_write_perm()
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Add python script with new logic of searching for tests:
Current ./check behavior:
- tests are named [0-9][0-9][0-9]
- tests must be registered in group file (even if test doesn't belong
to any group, like 142)
Behavior of findtests.py:
- group file is dropped
- tests are all files in tests/ subdirectory (except for .out files),
so it's not needed more to "register the test", just create it with
appropriate name in tests/ subdirectory. Old names like
[0-9][0-9][0-9] (in root iotests directory) are supported too, but
not recommended for new tests
- groups are parsed from '# group: ' line inside test files
- optional file group.local may be used to define some additional
groups for downstreams
- 'disabled' group is used to temporary disable tests. So instead of
commenting tests in old 'group' file you now can add them to
disabled group with help of 'group.local' file
- selecting test ranges like 5-15 are not supported more
(to support restarting failed ./check command from the middle of the
process, new argument is added: --start-from)
Benefits:
- no rebase conflicts in group file on patch porting from branch to
branch
- no conflicts in upstream, when different series want to occupy same
test number
- meaningful names for test files
For example, with digital number, when some person wants to add some
test about block-stream, he most probably will just create a new
test. But if there would be test-block-stream test already, he will
at first look at it and may be just add a test-case into it.
And anyway meaningful names are better.
This commit doesn't update check behavior (which will be done in
further commit), still, the documentation changed like new behavior is
already here. Let's live with this small inconsistency for the
following few commits, until final change.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210125185056.129513-3-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Documented under the "Acceptance tests using the Avocado Framework"
section in testing.rst how environment variables are used to skip tests.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210115210022.417996-1-wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Avocado documentation referred returns 404 error.
Update the broken links.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201010080741.2932406-1-philmd@redhat.com>
The qtest and libqtest doc comments will be parsed to generate
API documentation, so move QTest documentation to its own
document where the API and format documentation and will be
included.
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201005205228.697463-2-ehabkost@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
In two different places, we are not making a cross-reference to some
resource correctly.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20201009161558.107041-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
I found that there are many spelling errors in the comments of qemu,
so I used the spellcheck tool to check the spelling errors
and finally found some spelling errors in the docs folder.
Signed-off-by: zhaolichang <zhaolichang@huawei.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200917075029.313-4-zhaolichang@huawei.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
The binaries move to the root directory, e.g. qemu-system-i386 or
qemu-arm. This requires changes to qtests, CI, etc.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Adds TSan details to testing.rst.
This includes background and reference details on TSan,
and details on how to build and test with TSan
both with and without docker.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Emilio G. Cota <cota@braap.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200609200738.445-13-robert.foley@linaro.org>
Message-Id: <20200612190237.30436-16-alex.bennee@linaro.org>
The qtests have recently been moved to a separate subdirectory, so
the paths that are mentioned in the documentation have to be adjusted
accordingly. And some of the iotests are now always run as part of
"make check", so this information has to be adjusted here, too.
Message-Id: <20200122134511.23806-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
By default VM build test use qemu-img from system's PATH to
create the image disk. Due the lack of qemu-img on the system
or the desire to simply use a version built with QEMU, it would
be nice to allow one to set its path. So this patch makes that
possible by reading the path to qemu-img from QEMU_IMG if set,
otherwise it fallback to default behavior.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20191114134246.12073-2-wainersm@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
The same way the arch tag is being used as a fallback for the arch
parameter, let's do the same for QEMU's machine and avoid some boiler
plate code.
This is now possible because, since Avocado 72.0, it's possible to use
tags with names that match the machine types on QEMU.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20191104151323.9883-4-crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
It was pointed out we haven't documented the check-tcg part of the
build system. Attempt to rectify that now.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
We already have 'make check-help', use the 'make vm-help' form
to display helps about VM testing. Keep the old target to not
bother old customs.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Message-Id: <20190531064341.29730-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
A test can, optionally, be tagged for one or many architectures. If a
test has been tagged for a single architecture, there's a high chance
that the test won't run on other architectures. This changes the
default order of choosing a default target architecture to use based
on the 'arch' tag value first.
The precedence order is for choosing a QEMU binary to use for a test
is now:
* qemu_bin parameter
* arch parameter
* arch tag value (for example, x86_64 if "🥑 tags=arch:x86_64
is used)
This means that if one runs:
$ avocado run -p qemu_bin=/usr/bin/qemu-system-x86_64 test.py
No arch parameter or tag will influence the selection of the QEMU
target binary. If one runs:
$ avocado run -p arch=ppc64 test.py
The target binary selection mechanism will attempt to find a binary
such as "ppc64-softmmu/qemu-system-ppc64". And finally, if one runs
a test that is tagged (in its docstring) with "arch:aarch64":
$ avocado run aarch64.py
The target binary selection mechanism will attempt to find a binary
such as "aarch64-softmmu/qemu-system-aarch64".
At this time, no provision is made to cancel the execution of tests if
the arch parameter given (manually) does not match the test "arch"
tag, but it may be a useful default behavior to be added in the
future.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Message-Id: <20190312171824.5134-7-crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
It's useful to define the architecture that should be used in
situations such as:
* the intended target of the QEMU binary to be used on tests
* the architecture of code to be run within the QEMU binary, such
as a kernel image or a full blown guest OS image
This commit introduces both a test parameter and a test instance
attribute, that will contain such a value.
Now, when the "arch" test parameter is given, it will influence the
selection of the default QEMU binary, if one is not given explicitly
by means of the "qemu_img" parameter.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Message-Id: <20190312171824.5134-5-crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
The "this directory" reference is misleading and confusing, it's a
leftover from when this text was proposed in a README file inside
the "tests/acceptance/avocado_qemu" directory.
When that text was moved to the top level docs directory, the
reference was not updated.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Caio Carrara <ccarrara@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Message-Id: <20190312171824.5134-4-crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
This change adds the possibility to write acceptance tests with multi
virtual machine support. It's done keeping the virtual machines objects
stored in a test attribute (dictionary). This dictionary shouldn't be
accessed directly but through the new method added `get_vm`. This new
method accept a list of args (that will be added as virtual machine
arguments) and an optional name argument. The name is the key that
identify a single virtual machine along the test machines available. If
a name without a machine is informed a new machine will be instantiated.
The current usage of vm in tests will not be broken by this change since
it keeps a property called vm in the base test class. This property only
calls the new method `get_vm` with default parameters (no args and
'default' as machine name).
Signed-off-by: Caio Carrara <ccarrara@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20190212193855.13223-2-ccarrara@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
The Avocado test runner attemps to find its INSTRUMENTED (that is,
Python based tests) in a manner that is as safe as possible to the
user. Different from plain Python unittest, it won't load or
execute test code on an operation such as:
$ avocado list tests/acceptance/
Before version 68.0, the logic implemented to identify INSTRUMENTED
tests would require either the "🥑 enable" or "🥑
recursive" statement as a flag for tests that would not inherit
directly from "avocado.Test". This is not necessary anymore,
and because of that the boiler plate statements can now be removed.
Reference: https://avocado-framework.readthedocs.io/en/68.0/release_notes/68_0.html#users-test-writers
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Caio Carrara <ccarrara@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20190218173723.26120-1-crosa@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
The option -G of usermod command will remove user from other groups
not listed, i.e.: $USER will belong only to group 'docker' after
following the documentation as is.
From usermod(8) manual page:
If the user is currently a member of a group which is not listed,
the user will be removed from the group. This behaviour can be
changed via the -a option, which appends the user to the current
supplementary group list.
This patch improves the situation by adding the -a option to the
usermod command, which will just append user to the supplementary
group list.
Cc: qemu-trivial@nongnu.org
Signed-off-by: Murilo Opsfelder Araujo <muriloo@linux.ibm.com>
Message-Id: <20190207184346.6840-1-muriloo@linux.ibm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
The acceptance (aka functional, aka Avocado-based) tests are
Python files located in "tests/acceptance" that need to be run
with the Avocado libs and test runner.
Let's provide a convenient way for QEMU developers to run them,
by making use of the tests-venv with the required setup.
Also, while the Avocado test runner will take care of creating a
location to save test results to, it was understood that it's better
if the results are kept within the build tree.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Caio Carrara <ccarrara@redhat.com>
Message-Id: <20181018153134.8493-3-crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
The line immediate following a ".. code::" block is considered
to contains arguments to the "code directive". The lack of a
new line gives me during at parse time:
testing.rst:63: (ERROR/3) Error in "code" directive:
maximum 1 argument(s) allowed, 3 supplied.
.. code::
make check-unit V=1
testing.rst:120: (ERROR/3) Error in "code" directive:
maximum 1 argument(s) allowed, 3 supplied.
.. code::
make check-qtest V=1
Let's add the missing newlines, both for consistency and to
avoid the parsing errors.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-Id: <20181004161852.11673-6-crosa@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Commit 31d2dda ("build-system: remove per-test GCOV reporting", 2018-06-20)
removed users of the variables, since those uses can be replaced by a simple
overall report produced by gcovr. However, the variables were never removed.
Do it now.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
[thuth: Fixed up contextual conflicts with the patch from Eric]
Signed-off-by: Thomas Huth <thuth@redhat.com>
Invoking 'make vm-build-freebsd' and friends with V=1 should
propagate that verbosity setting down into the build run
inside the VM. Make sure we do that. This brings it into
line with how the container tests handle V=1.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20180803085230.30574-4-peter.maydell@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This will build a coverage report under the current directory in
reports/coverage. At the users option a report can be generated by
directly invoking something like:
make foo/bar/coverage-report.html
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
This can be used to remove any stale coverage data before any
particular test run. This is useful for analysing individual tests.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>---
