Commit Graph

7629 Commits

Author SHA1 Message Date
Alex Bennée
81c4edc39e tests/tcg: add a multiarch signals test to stress test signal delivery
This adds a simple signal test that combines the POSIX timer_create
with signal delivery across multiple threads. The aim is to provide a
bit more of a stress test to flush out signal handling issues for
easily than the occasional random crash we sometimes see in linux-test
or threadcount.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20210527160319.19834-2-alex.bennee@linaro.org>
2021-06-07 14:42:42 +01:00
Kit Westneat
6b731a96aa tests/qtest/virtio-scsi-test: add unmap large LBA with 4k blocks test
Add test for issue #345

Signed-off-by: Kit Westneat <kit.westneat@gmail.com>
Message-Id: <20210603142022.676395-1-kit.westneat@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-06-04 13:47:08 +02:00
Alex Bennée
90d004524e tests/fp: Enable more tests
Fix the trivial typo in extF80_lt_quiet, and re-enable
all of the floatx80 tests that are now fixed.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <87bl9iyahr.fsf@linaro.org>
[rth: Squash the fix for lt_quiet, and enable that too.]
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-03 14:09:03 -07:00
Richard Henderson
2fa3546c8f softfloat: Move floatN_log2 to softfloat-parts.c.inc
Rename to parts$N_log2.  Though this is partly a ruse, since I do not
believe the code will succeed for float128 without work.  Which is ok
for now, because we do not need this for more than float32 and float64.

Since berkeley-testfloat-3 doesn't support log2, compare float64_log2
vs the system log2.  Fix the errors for inputs near 1.0:

test: 3ff00000000000b0  +0x1.00000000000b0p+0
  sf: 3d2fa00000000000  +0x1.fa00000000000p-45
libm: 3d2fbd422b1bd36f  +0x1.fbd422b1bd36fp-45
Error in fraction: 32170028290927 ulp

test: 3feec24f6770b100  +0x1.ec24f6770b100p-1
  sf: bfad3740d13c9ec0  -0x1.d3740d13c9ec0p-5
libm: bfad3740d13c9e98  -0x1.d3740d13c9e98p-5
Error in fraction: 40 ulp

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-03 14:09:02 -07:00
Richard Henderson
7ccae4ce7e tests/fp/fp-test: Reverse order of floatx80 precision tests
Many qemu softfloat will check floatx80_rounding_precision
even when berkeley testfloat will not.  So begin with
floatx80_precision_x, so that's the one we use
when !FUNC_EFF_ROUNDINGPRECISION.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-03 14:09:02 -07:00
Richard Henderson
8da5f1dbb0 softfloat: Introduce Floatx80RoundPrec
Use an enumeration instead of raw 32/64/80 values.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-03 14:04:02 -07:00
Peter Maydell
1c86188589 tests/unit/test-vmstate: Assert that dup() and mkstemp() succeed
Coverity complains that we don't check for failures from dup()
and mkstemp(); add asserts that these syscalls succeeded.

Fixes: Coverity CID 1432516, 1432574
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210525134458.6675-7-peter.maydell@linaro.org
2021-06-03 16:43:27 +01:00
Peter Maydell
d2304612b5 tests/qtest/tpm-tests: Remove unnecessary NULL checks
Coverity points out that in tpm_test_swtpm_migration_test() we
assume that src_tpm_addr and dst_tpm_addr are non-NULL (we
pass them to tpm_util_migration_start_qemu() which will
unconditionally dereference them) but then later explicitly
check them for NULL. Remove the pointless checks.

Fixes: Coverity CID 1432367, 1432359

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Message-id: 20210525134458.6675-6-peter.maydell@linaro.org
2021-06-03 16:43:27 +01:00
Peter Maydell
909e4a0826 tests/qtest/pflash-cfi02-test: Avoid potential integer overflow
Coverity points out that we calculate a 64-bit value using 32-bit
arithmetic; add the cast to force the multiply to be done as 64-bits.
(The overflow will never happen with the current test data.)

Fixes: Coverity CID 1432320
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Message-id: 20210525134458.6675-5-peter.maydell@linaro.org
2021-06-03 16:43:27 +01:00
Peter Maydell
2c398ee5e3 tests/qtest/hd-geo-test: Fix checks on mkstemp() return value
Coverity notices that the checks against mkstemp() failing in
create_qcow2_with_mbr() are wrong: mkstemp returns -1 on failure but
the check is just "g_assert(fd)".  Fix to use "g_assert(fd >= 0)",
matching the correct check in create_test_img().

Fixes: Coverity CID 1432274
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Message-id: 20210525134458.6675-4-peter.maydell@linaro.org
2021-06-03 16:43:27 +01:00
Peter Maydell
380822edb3 tests/qtest/e1000e-test: Check qemu_recv() succeeded
The e1000e_send_verify() test calls qemu_recv() but doesn't
check that the call succeeded, which annoys Coverity. Add
an explicit test check for the length of the data.

(This is a test check, not a "we assume this syscall always
succeeds", so we use g_assert_cmpint() rather than g_assert().)

Fixes: Coverity CID 1432324
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Message-id: 20210525134458.6675-3-peter.maydell@linaro.org
2021-06-03 16:43:27 +01:00
Peter Maydell
3c65e439b2 tests/qtest/bios-tables-test: Check for dup2() failure
Coverity notes that we don't check for dup2() failing.  Add some
assertions so that if it does ever happen we get some indication.
(This is similar to how we handle other "don't expect this syscall to
fail" checks in this test code.)

Fixes: Coverity CID 1432346
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Message-id: 20210525134458.6675-2-peter.maydell@linaro.org
2021-06-03 16:43:27 +01:00
Peter Maydell
8e6dad2028 Block layer patches
- NBD server: Fix crashes related to switching between AioContexts
 - file-posix: Workaround for discard/write_zeroes on buggy filesystems
 - Follow-up fixes for the reopen vs. permission changes
 - quorum: Fix error handling for flush
 - block-copy: Refactor copy_range handling
 - docs: Describe how to use 'null-co' block driver
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmC3iy8RHGt3b2xmQHJl
 ZGhhdC5jb20ACgkQfwmycsiPL9ZP0hAAuh07CFWLzHCcRC7PBzekSPfzRYYBLDSW
 EObJ1Ov4mvz8UZoP6BDJ5QVzLPhel6hXkxTd83B1D7t/Dq+yJYR0z8Kv3USpaVJ4
 2U26SsoGQM8BmtVDL1Q8tQ5eDWQ4ykxNx6F2/lKBe1EH1lfaun04Xj1rNh7jpilo
 nNmKMDDI1UOkH0lKDR3tqfEV0XQE7o+ZKfPlIbvYMjXk9ZPKUHfjNPGVdCLQVnqH
 VJI01hF7eEx1ykSMdlC+TzNoVGG+mCBokGuW0JlUvOpX6FcGnAlxXQx8u53c1I8O
 lggZV8b2IbrNBUVwXQHLLrXxjOo+u54Ct9y/gXUTAj8qai+9jVRp60Y1pnCyeIeu
 DzFx10xwy04PGleb7AAZ4dT8du2+PTkuyQ9KmlvQ2U4IUcgW124CrDeO7XYr1aif
 hCTJPeEDrC9YNU6AQ8rLXrYUtkumSm2zUzU5nZ//i5WH41475/vsmgP5A+Jr457A
 Xu0yiI2Gqkr9CNsP9ZzMkNj03oIBhPFuGxiwibLQsy/6UVnaDYS0+rQ8FXYnF5+K
 iEpgXe3vKTWxM097kzJMBTDVRMXRa75NtK7KWXMDgVpHTbcv1t1otsn+6dfv+B55
 ULJM1ETsyYS0T6BqNglvdytsraSt7JgSF+ZLHbYk3KVDshwnq/0ksgSqHNNA14ca
 kYTzHhMgo5w=
 =gSq/
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

- NBD server: Fix crashes related to switching between AioContexts
- file-posix: Workaround for discard/write_zeroes on buggy filesystems
- Follow-up fixes for the reopen vs. permission changes
- quorum: Fix error handling for flush
- block-copy: Refactor copy_range handling
- docs: Describe how to use 'null-co' block driver

# gpg: Signature made Wed 02 Jun 2021 14:44:15 BST
# gpg:                using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg:                issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
  docs/secure-coding-practices: Describe how to use 'null-co' block driver
  block-copy: refactor copy_range handling
  block-copy: fix block_copy_task_entry() progress update
  nbd/server: Use drained block ops to quiesce the server
  block-backend: add drained_poll
  block: improve permission conflict error message
  block: simplify bdrv_child_user_desc()
  block/vvfat: inherit child_vvfat_qcow from child_of_bds
  block: improve bdrv_child_get_parent_desc()
  block-backend: improve blk_root_get_parent_desc()
  block: document child argument of bdrv_attach_child_common()
  block/file-posix: Try other fallbacks after invalid FALLOC_FL_ZERO_RANGE
  block/file-posix: Fix problem with fallocate(PUNCH_HOLE) on GPFS
  block: drop BlockBackendRootState::read_only
  block: drop BlockDriverState::read_only
  block: consistently use bdrv_is_read_only()
  block/vvfat: fix vvfat_child_perm crash
  block/vvfat: child_vvfat_qcow: add .get_parent_aio_context, fix crash
  qemu-io-cmds: assert that we don't have .perm requested in no-blk case
  block/quorum: Provide .bdrv_co_flush instead of .bdrv_co_flush_to_disk

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-06-02 19:34:03 +01:00
Peter Maydell
8c345b3e6a * Update the references to some doc files (use *.rst instead of *.txt)
* Bump minimum versions of some requirements after removing CentOS 7 support
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmC3L1IRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbU8wA//aRAIn0Qa3xoZdsT434P99GHauCZ5ePq3
 meY4co69c+TfkQZ/b0xlyvT3+7bd9Ni92CQL7n/LtX5bs3pIhRjHNIsm50e4x6Da
 0jb422cKWfRIltXlCfdL/dnFRtJluH83M0seRGMvmhveuWPZ19oSIasEiQjeg//e
 KMNK5z4HEnJ1czb3Bf8p38bmmY/O/QEAA5wAqk7iJkJHz6T/GlqImLYYwpFPlHj6
 JQttm0aWsHrsqEXxnuV0/DT1yHyXDB6S4iuAvABZWhv/M/nCaXo0ib0gW5NPtRPo
 Yf3HO163F9/fewJCc1AUsBe1C/2UwmSWRhEtxpr9uuW2Mv9qEl3hkJwd4k6sEOvh
 U4i+GONC4eElPcmECKUfHA9EP+7faDs6xnM6Ev/PIEp+cPJ2QRfklZv4qpMUWYtb
 3KkADchOyVZAsdB8cGmnznDEVmno1Dt0adVUq8CF6uW6MwD3pb4838arrfwIfwOp
 g4yTI1AQQykkxxOaR7GSNoxWRti7TH4fzLhx/xXDd9TKIOWuOepiyuhB7+Q48WAJ
 6EW/JUIzOe7k3GsI0iBsk+y67ED2tpATiRKWWw4QS1BwIhNEMUqgZkucFfKE6Ze1
 B1Xw+Di+1CzrLHV6lCjtdLVmqWqcGcO43HqhSK4sodE87BDGGwy9UxE6oBVt2C1x
 HPQ6/omDlFk=
 =RAkI
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/thuth-gitlab/tags/pull-request-2021-06-02' into staging

* Update the references to some doc files (use *.rst instead of *.txt)
* Bump minimum versions of some requirements after removing CentOS 7 support

# gpg: Signature made Wed 02 Jun 2021 08:12:18 BST
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/thuth-gitlab/tags/pull-request-2021-06-02:
  configure: bump min required CLang to 6.0 / XCode 10.0
  configure: bump min required GCC to 7.5.0
  configure: bump min required glib version to 2.56
  tests/docker: drop CentOS 7 container
  tests/vm: convert centos VM recipe to CentOS 8
  crypto: drop used conditional check
  crypto: bump min gnutls to 3.5.18, dropping RHEL-7 support
  crypto: bump min gcrypt to 1.8.0, dropping RHEL-7 support
  crypto: drop back compatibility typedefs for nettle
  crypto: bump min nettle to 3.4, dropping RHEL-7 support
  patchew: move quick build job from CentOS 7 to CentOS 8 container
  block/ssh: Bump minimum libssh version to 0.8.7
  docs: fix references to docs/devel/s390-dasd-ipl.rst
  docs: fix references to docs/specs/tpm.rst
  docs: fix references to docs/devel/build-system.rst
  docs: fix references to docs/devel/atomics.rst
  docs: fix references to docs/devel/tracing.rst

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-06-02 17:08:11 +01:00
Vladimir Sementsov-Ogievskiy
30ebb9aa92 block: improve permission conflict error message
Now permissions are updated as follows:
 1. do graph modifications ignoring permissions
 2. do permission update

 (of course, we rollback [1] if [2] fails)

So, on stage [2] we can't say which users are "old" and which are
"new" and exist only since [1]. And current error message is a bit
outdated. Let's improve it, to make everything clean.

While being here, add also a comment and some good assertions.

iotests 283, 307, qsd-jobs outputs are updated.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210601075218.79249-7-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-06-02 14:23:20 +02:00
Vladimir Sementsov-Ogievskiy
2c0a3acb95 block: improve bdrv_child_get_parent_desc()
We have different types of parents: block nodes, block backends and
jobs. So, it makes sense to specify type together with name.

Next, this handler us used to compose an error message about permission
conflict. And permission conflict occurs in a specific place of block
graph. We shouldn't report name of parent device (as it refers another
place in block graph), but exactly and only the name of the node. So,
use bdrv_get_node_name() directly.

iotest 283 output is updated.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-Id: <20210601075218.79249-4-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-06-02 14:23:20 +02:00
Vladimir Sementsov-Ogievskiy
fd240a184b block-backend: improve blk_root_get_parent_desc()
We have different types of parents: block nodes, block backends and
jobs. So, it makes sense to specify type together with name.

While being here also use g_autofree.

iotest 307 output is updated.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-Id: <20210601075218.79249-3-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-06-02 14:23:20 +02:00
Vladimir Sementsov-Ogievskiy
975da07374 block: drop BlockDriverState::read_only
This variable is just a cache for !(bs->open_flags & BDRV_O_RDWR),
which we have to synchronize everywhere. Let's just drop it and
consistently use bdrv_is_read_only().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210527154056.70294-3-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-06-02 14:23:20 +02:00
Daniel P. Berrangé
6f0748a048 tests/docker: drop CentOS 7 container
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210514120415.1368922-10-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-06-02 09:11:32 +02:00
Daniel P. Berrangé
fb1fa97c69 tests/vm: convert centos VM recipe to CentOS 8
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210514120415.1368922-9-berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-06-02 09:11:05 +02:00
Stefano Garzarella
551bdfa079 docs: fix references to docs/devel/build-system.rst
Commit a14f0bf165 ("docs: convert build system documentation to rST")
converted docs/devel/build-system.txt to docs/devel/build-system.rst.

We still have several references to the old file, so let's fix them
with the following command:

  sed -i s/build-system.txt/build-system.rst/ \
      $(git grep -l docs/devel/build-system.txt)

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
Message-Id: <20210517151702.109066-4-sgarzare@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-06-02 06:51:09 +02:00
John Snow
6b9c277797 gitlab: add python linters to CI
Add a Python container that has just enough juice for us to run the
Python code quality analysis tools. Base this container on Fedora,
because Fedora has very convenient packaging for testing multiple Python
versions.

We need python3, pip (for pulling packages), pipenv and virtualenv for
creating virtual environments, and tox for running tests. make is needed
for running 'make check-tox' and 'make venv-check' targets. Python3.10
is needed explicitly because the tox package only pulls in 3.6-3.9, but
we wish to test the forthcoming release of Python as well to help
predict any problems. Lastly, we need gcc to compile PyPI packages that
may not have a binary distribution available.

Add two tests:

check-python-pipenv uses pipenv to test a frozen, very explicit set of
packages against our minimum supported python version, Python 3.6. This
test is not allowed to fail. The dependencies this test uses do not
change unless python/Pipfile.lock is changed.

check-python-tox uses tox to install the latest versions of required
python dependencies against a wide array of Python versions from 3.6 to
3.9, even including the yet-to-be-released Python 3.10. This test is
allowed to fail with a warning.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-32-jsnow@redhat.com
[Fix rebase conflict over .gitlab-ci.yml --js]
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:21 -04:00
John Snow
beb6b57b3b python: create qemu packages
move python/qemu/*.py to python/qemu/[machine, qmp, utils]/*.py and
update import directives across the tree.

This is done to create a PEP420 namespace package, in which we may
create subpackages. To do this, the namespace directory ("qemu") should
not have any modules in it. Those files will go into new 'machine',
'qmp' and 'utils' subpackages instead.

Implement machine/__init__.py making the top-level classes and functions
from its various modules available directly inside the package. Change
qmp.py to qmp/__init__.py similarly, such that all of the useful QMP
library classes are available directly from "qemu.qmp" instead of
"qemu.qmp.qmp".

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-10-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:21 -04:00
John Snow
7f0a143b0c iotests/297: add --namespace-packages to mypy arguments
mypy is kind of weird about how it handles imports. For legacy reasons,
it won't load PEP 420 namespaces, because of logic implemented prior to
that becoming a standard.

So, if you plan on using any, you have to pass
--namespace-packages. Alright, fine.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:21 -04:00
Willian Rampazzo
41787552de acceptance tests: bump Avocado version to 88.1
Besides some internal changes, new features, and bug fixes, on the QEMU side,
this version fixes the following message seen when running the acceptance
tests: "Error running method "pre_tests" of plugin "fetchasset": 'bytes'
object has no attribute 'encode'".

The release notes are available at
https://avocado-framework.readthedocs.io/en/latest/releases/88_0.html.

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210520204747.210764-2-willianr@redhat.com>
Acked-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:21 -04:00
Cleber Rosa
d214740c99 tests/acceptance/virtiofs_submounts.py: fix setup of SSH pubkey
The public key argument should be a path to a file, and not the
public key data.

Reported-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210412044644.55083-12-crosa@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:21 -04:00
Cleber Rosa
fd1ce58d90 Acceptance Tests: introduce CPU hotplug test
Even though there are qtest based tests for hotplugging CPUs (from
which this test took some inspiration from), this one adds checks
from a Linux guest point of view.

It should also serve as an example for tests that follow a similar
pattern and need to interact with QEMU (via qmp) and with the Linux
guest via SSH.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Message-Id: <20210412044644.55083-11-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:21 -04:00
Cleber Rosa
a273387aec tests/acceptance/virtiofs_submounts.py: remove launch_vm()
The LinuxTest class' launch_and_wait() method now behaves the same way
as this test's custom launch_vm(), so let's just use the upper layer
(common) method.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210412044644.55083-9-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Cleber Rosa
c6620c443d Acceptance Tests: set up SSH connection by default after boot for LinuxTest
The LinuxTest specifically targets users that need to interact with Linux
guests.  So, it makes sense to give a connection by default, and avoid
requiring it as boiler-plate code.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210412044644.55083-8-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Cleber Rosa
d8c6a89968 Acceptance Tests: make username/password configurable
This makes the username/password used for authentication configurable,
because some guest operating systems may have restrictions on accounts
to be used for logins, and it just makes it better documented.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210412044644.55083-7-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Cleber Rosa
54914114af Acceptance Tests: add port redirection for ssh by default
For users of the LinuxTest class, let's set up the VM with the port
redirection for SSH, instead of requiring each test to set the same
arguments.

It also sets the network device, by default, to virtio-net.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210412044644.55083-6-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Cleber Rosa
7edee7ad94 Acceptance Tests: move useful ssh methods to base class
Both the virtiofs submounts and the linux ssh mips malta tests
contains useful methods related to ssh that deserve to be made
available to other tests.  Let's move them to an auxiliary, mix-in
class that will be used on the base LinuxTest class.

The method that helps with setting up an ssh connection will now
support both key and password based authentication, defaulting to key
based.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210412044644.55083-5-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Cleber Rosa
976218cbe7 Python: add utility function for retrieving port redirection
Slightly different versions for the same utility code are currently
present on different locations.  This unifies them all, giving
preference to the version from virtiofs_submounts.py, because of the
last tweaks added to it.

While at it, this adds a "qemu.utils" module to host the utility
function and a test.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210412044644.55083-4-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
[Squashed in below fix. --js]
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210601154546.130870-2-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Cleber Rosa
c028691e65 tests/acceptance/virtiofs_submounts.py: evaluate string not length
If the vmlinuz variable is set to anything that evaluates to True,
then the respective arguments should be set.  If the variable contains
an empty string, than it will evaluate to False, and the extra
arguments will not be set.

This keeps the same logic, but improves readability a bit.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210412044644.55083-3-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Cleber Rosa
f084e148aa tests/acceptance/virtiofs_submounts.py: add missing accel tag
The tag is useful to select tests that depend/use a particular
feature.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Message-Id: <20210412044644.55083-2-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Cleber Rosa
2ca6e26cea Python: expose QEMUMachine's temporary directory
Each instance of qemu.machine.QEMUMachine currently has a "test
directory", which may not have any relation to a "test", and it's
really a temporary directory.

Users instantiating the QEMUMachine class will be able to set the
location of the directory that will *contain* the QEMUMachine unique
temporary directory, so that parameter name has been changed from
test_dir to base_temp_dir.

A property has been added to allow users to access it without using
private attributes, and with that, the directory is created on first
use of the property.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210211220146.2525771-3-crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Peter Maydell
8385235ba9 * submodule cleanups (Philippe, myself)
* tiny step towards a usable preconfig mode (myself)
 * Kconfig and LOCK_GUARD cleanups (philippe)
 * new x86 CPUID feature (Yang Zhong)
 * "-object qtest" support (myself)
 * Dirty ring support for KVM (Peter)
 * Fixes for 6.0 command line parsing breakage (myself)
 * Fix for macOS 11.3 SDK (Katsuhiro)
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmCuRAQUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroOL6Qf/bUjQNAUc2QQJya1lu8TEf1o4vjkK
 C3EzFPVAj+m2O3OZOGEHcTh8+lDSzBeE2gB3bt4AD+KvFbQGXhLM3gMu/Ztymv8m
 3rVEe/NxNyq/CgC307GIwF3in7rEzjH0+WHaOuoU340e3Po1FA7s20VnMysVxxng
 4Pf4m4Y0k0eq022HgqZ/r/kbnINxDHagmzuyiFARkt8ooiuj4NyOMW7UKMk3fBvY
 MLMPsBe3imWmVnkOF0n/qJ+Svbtx15iLgGIIggshy3rmPereUpIQYaJ9FS6jcXO2
 YHuYDc2aGelMU84r+x+9UQra6auzJfc4UbylOsGjopCeFG2aU8rLMphvpw==
 =UQwU
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

* submodule cleanups (Philippe, myself)
* tiny step towards a usable preconfig mode (myself)
* Kconfig and LOCK_GUARD cleanups (philippe)
* new x86 CPUID feature (Yang Zhong)
* "-object qtest" support (myself)
* Dirty ring support for KVM (Peter)
* Fixes for 6.0 command line parsing breakage (myself)
* Fix for macOS 11.3 SDK (Katsuhiro)

# gpg: Signature made Wed 26 May 2021 13:50:12 BST
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream: (28 commits)
  gitlab-ci: use --meson=git for CFI jobs
  hw/scsi: Fix sector translation bug in scsi_unmap_complete_noio
  configure: Avoid error messages about missing *-config-*.h files
  doc: Add notes about -mon option mode=control argument.
  qemu-config: load modules when instantiating option groups
  vl: allow not specifying size in -m when using -M memory-backend
  replication: move include out of root directory
  remove qemu-options* from root directory
  meson: Set implicit_include_directories to false
  tests/qtest/fuzz: Fix build failure
  KVM: Dirty ring support
  KVM: Disable manual dirty log when dirty ring enabled
  KVM: Add dirty-ring-size property
  KVM: Cache kvm slot dirty bitmap size
  KVM: Simplify dirty log sync in kvm_set_phys_mem
  KVM: Provide helper to sync dirty bitmap from slot to ramblock
  KVM: Provide helper to get kvm dirty log
  KVM: Create the KVMSlot dirty bitmap on flag changes
  KVM: Use a big lock to replace per-kml slots_lock
  memory: Introduce log_sync_global() to memory listener
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-05-26 21:05:36 +01:00
Paolo Bonzini
b02629550d replication: move include out of root directory
The replication.h file is included from migration/colo.c and tests/unit/test-replication.c,
so it should be in include/.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-26 14:49:46 +02:00
Philippe Mathieu-Daudé
aaad026480 tests/qtest/fuzz: Fix build failure
On Fedora 32, using clang (version 10.0.1-3.fc32) we get:

  tests/qtest/fuzz/fuzz.c:237:5: error: implicit declaration of function 'qemu_init' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
      qemu_init(result.we_wordc, result.we_wordv, NULL);
      ^

qemu_init() is declared in "sysemu/sysemu.h", include this
header to fix.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210513162008.3922223-1-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-26 14:49:46 +02:00
Mahmoud Mandour
a6851b49e3 plugins/syscall: Added a table-like summary output
Added a table-like output which contains the total number of calls
for each used syscall along with the number of errors that occurred.

Per-call tracing is still available through supplying the argument
``print`` to the plugin.

Signed-off-by: Mahmoud Mandour <ma.mandourr@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210519032409.3041-1-ma.mandourr@gmail.com>
Message-Id: <20210520174303.12310-9-alex.bennee@linaro.org>
2021-05-25 16:52:50 +01:00
Peter Maydell
371ebfe286 target/xtensa updates for v6.1:
- don't generate extra EXCP_DEBUG on exception
 - fix l32ex access ring
 - clean up unaligned access
 -----BEGIN PGP SIGNATURE-----
 
 iQJHBAABCgAxFiEEK2eFS5jlMn3N6xfYUfnMkfg/oEQFAmCnvMITHGpjbXZia2Jj
 QGdtYWlsLmNvbQAKCRBR+cyR+D+gRHCVEACR5nXm21Y7Jc9R+Pkpe5rd8xyw5J/m
 2PKM8uAH6XSo/DGanuzXUW345NmVbpL0DWSmWI6WQ7LfXpqgkO/UM76DeZGrWlTB
 lqQXTbVBv+crkYmxFlnYAaoVYEmSZb900RaKyFiWG95edjV/YoPnq6qzT8Xy3N6K
 jYRGh+cWnzxTLCsscEK8DySA79yox+Kw+49gmHddXdjhwYDs0ekT3cGhQyK8vlTU
 Zpu7tVwQvE3SxwSgNDrg5eG12dLTfnfLrE/wFAyQdlDUTNxjpF36mvWKUm9BmnNT
 L8AedMMbdqH2+0gcFfetoVOcSmquDT4axiQq/eCaudBqUzhBncaDEHeXZVpMoADI
 +Je5awz9pr0JStPvMANfpczRmf3WrD/de9olwRYfSZmaZ0O1w0RV8SAa4/9YPpHl
 H0B5bNN9fl69nGdM6Nm+ERp2evrk2qnqH9/TXJC/waU7QphbkGC2MO5BmKjRNyWj
 7hxWTAW1R5/BJLgFIEkuVkVV8G6a+Jz17olhGKiXgWTJORBQE9ACWmOK/A17Nh09
 LUX0nTA5eVWUgV78P599naT0Xx6SLK9QHUw6tZs3US3hrZDQvjmrrxFsf1b6qaF/
 1D0XRQlqxdEZvdgvqJcW0AlMz2eFx1/vjgHPzVbHK7L4sA6XK2UFskuttgk2g9m9
 8Ro/rH/zK6Qgmw==
 =fLOY
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/xtensa/tags/20210521-xtensa' into staging

target/xtensa updates for v6.1:

- don't generate extra EXCP_DEBUG on exception
- fix l32ex access ring
- clean up unaligned access

# gpg: Signature made Fri 21 May 2021 14:59:30 BST
# gpg:                using RSA key 2B67854B98E5327DCDEB17D851F9CC91F83FA044
# gpg:                issuer "jcmvbkbc@gmail.com"
# gpg: Good signature from "Max Filippov <filippov@cadence.com>" [unknown]
# gpg:                 aka "Max Filippov <max.filippov@cogentembedded.com>" [full]
# gpg:                 aka "Max Filippov <jcmvbkbc@gmail.com>" [full]
# Primary key fingerprint: 2B67 854B 98E5 327D CDEB  17D8 51F9 CC91 F83F A044

* remotes/xtensa/tags/20210521-xtensa:
  target/xtensa: clean up unaligned access
  target/xtensa: fix access ring in l32ex
  target/xtensa: don't generate extra EXCP_DEBUG on exception

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-05-24 12:00:33 +01:00
Max Filippov
583e6a5f55 target/xtensa: clean up unaligned access
Xtensa cores may or may not have hardware support for unaligned memory
access. Remove TARGET_ALIGNED_ONLY=y from all xtensa configurations and
pass MO_ALIGN in memory access flags for all operations that would raise
an exception.
Simplify use of gen_load_store_alignment by passing access size and
alignment requirements in single parameter.
Drop condition from xtensa_cpu_do_unaligned_access and replace it with
assertion.
Add a test.

Suggested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
2021-05-20 13:02:58 -07:00
Peter Maydell
0b5acf89c1 QAPI patches patches for 2021-05-20
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmCme90SHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZTnIcP/1XUG8dn8jI57s1D4Dq9XUgyFYHAK7oZ
 gNwY9uzlNWxJLpVQthSuOlUS/9f50/xc7wEoRpuAYR8v+480bpu77FEy6NAD+KH3
 yO1iSlHZtivSzNvpLWxj5vGUZE4SOWRyUiEBrBXcwhZ2YCz/FsxtGLK5heCubPQw
 QFGg70FaFrblZZp6RCUp3O/OLNG93DptrhM7Mcr6XeUOyU884pgSZwodjyoYE8KZ
 bwiYgppiiydeFexZgyyJ7+IWREAlb/2bOCIgS3bziaQBJHJPAkteRRCt1BKCv4F0
 q4blIiF8TsNQ/oqVM0KaX8WsbU2F3Ci4+RWyzJCNHi4Ickf9tcNBJ/RMRiUIOk9U
 A6zIGhm/L6g8h7ia1avfgqjGmoZIA6lUn1GEhudoQcgagM3hPbrFdZoPLDtfi639
 bher5gTPHyVb2B/xpqKEoek2bRKtpIGPdHzoCsvttQuUh787dM7QbPOPJZ8tHrCu
 uLmRIXg64BZZ7kzSxmrYiN2Z2ptY9+xxmBOds2cm3jO6DjCEC9LEe583Gcn+yf5N
 MKo72SEv9/ctyXs8CvNtBDjzabltMg7qwJCqmna9r6PdGh2rR6jVMzUWXsZDsc2Z
 lCrn7srlG4TkeTSr0o7pmqdWjWgr7ryQlbF2Fp2EpZQBh4KyBY21EHs5mOJBp8Mp
 aSEgdeyO+l9+
 =GxcF
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2021-05-20' into staging

QAPI patches patches for 2021-05-20

# gpg: Signature made Thu 20 May 2021 16:10:21 BST
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2021-05-20:
  qapi/parser: add docstrings
  qapi/parser: allow 'ch' variable name
  qapi/parser: Remove superfluous list comprehension
  qapi/parser: add type hint annotations
  qapi/parser: Rework _check_pragma_list_of_str as a TypeGuard
  qapi/parser: Fix token membership tests when token can be None
  qapi: add must_match helper
  qapi/parser: Use @staticmethod where appropriate
  qapi/parser: assert object keys are strings
  qapi/parser: enforce all top-level expressions must be dict in _parse()
  qapi/parser: Assert lexer value is a string
  qapi/parser: factor parsing routine into method
  qapi/source: Remove line number from QAPISourceInfo initializer
  qapi: Add test for nonexistent schema file
  qapi/parser: Don't try to handle file errors

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-05-20 20:17:55 +01:00
Ilya Leoshkevich
f66487756b tests/tcg/x86_64: add vsyscall smoke test
Having a small test will prevent trivial regressions in the future.

Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20210519045738.1335210-3-iii@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2021-05-20 14:19:30 +02:00
John Snow
c256263f3d qapi/parser: Fix token membership tests when token can be None
When the token can be None (EOF), we can't use 'x in "abc"' style
membership tests to group types of tokens together, because 'None in
"abc"' is a TypeError.

Easy enough to fix. (Use a tuple: It's neither a static typing error nor
a runtime error to check for None in Tuple[str, ...])

Add tests to prevent a regression. (Note: they cannot be added prior to
this fix, as the unhandled stack trace will not match test output in the
CI system.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
9cd0205d55 qapi/parser: enforce all top-level expressions must be dict in _parse()
Instead of using get_expr nested=False, allow get_expr to always return
any expression. In exchange, add a new error message to the top-level
parser that explains the semantic error: Top-level expressions must
always be JSON objects.

This helps mypy understand the rest of this function which assumes that
get_expr did indeed return a dict.

The exception type changes from QAPIParseError to QAPISemError as a
result, and the error message in two tests now changes.

Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210519183951.3946870-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
334c3cd58a qapi: Add test for nonexistent schema file
This tests the error-return pathway introduced in the previous commit.
(Thanks to Paolo for the help with the Meson magic.)

Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210519183951.3946870-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:27 +02:00
John Snow
3404e57410 qapi/parser: Don't try to handle file errors
Fixes: f5d4361cda
Fixes: 52a474180a
Fixes: 46f49468c6

Remove the try/except block that handles file-opening errors in
QAPISchemaParser.__init__() and add one each to
QAPISchemaParser._include() and QAPISchema.__init__() respectively.

This simultaneously fixes the typing of info.fname (f5d4361cda), A
static typing violation in test-qapi (46f49468c6), and a regression of
an error message (52a474180a).

The short-ish version of what motivates this patch is:

- It's hard to write a good error message in the init method,
  because we need to determine the context of our caller to do so.
  It's easier to just let the caller write the message.
- We don't want to allow QAPISourceInfo(None, None, None) to exist. The
  typing introduced by commit f5d4361cda types the 'fname' field as
  (non-optional) str, which was premature until the removal of this
  construct.
- Errors made using such an object are currently incorrect (since
  52a474180a)
- It's not technically a semantic error if we cannot open the schema.
- There are various typing constraints that make mixing these two cases
  undesirable for a single special case.
- test-qapi's code handling an fname of 'None' is now dead, drop it.
  Additionally, Not all QAPIError objects have an 'info' field (since
  46f49468), so deleting this stanza corrects a typing oversight in
  test-qapi introduced by that commit.

Other considerations:

- open() is moved to a 'with' block to ensure file pointers are
  cleaned up deterministically.
- Python 3.3 deprecated IOError and made it a synonym for OSError.
  Avoid the misleading perception these exception handlers are
  narrower than they really are.

The long version:

The error message here is incorrect (since commit 52a474180a):

> python3 qapi-gen.py 'fake.json'
qapi-gen.py: qapi-gen.py: can't read schema file 'fake.json': No such file or directory

In pursuing it, we find that QAPISourceInfo has a special accommodation
for when there's no filename. Meanwhile, the intent when QAPISourceInfo
was typed (f5d4361cda) was non-optional 'str'. This usage was
overlooked.

To remove this, I'd want to avoid having a "fake" QAPISourceInfo
object. I also don't want to explicitly begin accommodating
QAPISourceInfo itself being None, because we actually want to eventually
prove that this can never happen -- We don't want to confuse "The file
isn't open yet" with "This error stems from a definition that wasn't
defined in any file".

(An earlier series tried to create a dummy info object, but it was tough
to prove in review that it worked correctly without creating new
regressions. This patch avoids that distraction. We would like to first
prove that we never raise QAPISemError for any built-in object before we
add "special" info objects. We aren't ready to do that yet.)

So, which way out of the labyrinth?

Here's one way: Don't try to handle errors at a level with "mixed"
semantic contexts; i.e. don't mix inclusion errors (should report a
source line where the include was triggered) and command line errors
(where we specified a file we couldn't read).

Remove the error handling from the initializer of the parser. Pythonic!
Now it's the caller's job to figure out what to do about it. Handle the
error in QAPISchemaParser._include() instead, where we can write a
targeted error message where we are guaranteed to have an 'info' context
to report with.

The root level error can similarly move to QAPISchema.__init__(), where
we know we'll never have an info context to report with, so we use a
more abstract error type.

Now the error looks sensible again:

> python3 qapi-gen.py 'fake.json'
qapi-gen.py: can't read schema file 'fake.json': No such file or directory

With these error cases separated, QAPISourceInfo can be solidified as
never having placeholder arguments that violate our desired types. Clean
up test-qapi along similar lines.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:27 +02:00
Peter Maydell
fea2ad71c3 testing and plugin updates:
- various fixes for binfmt_misc docker images
   - add hexagon check-tcg support docker image
   - add tricore check-tcg support
   - refactor ppc docker images
   - add missing ppc64le tests
   - don't use host_cc for test fallback
   - check-tcg configure.sh tweaks for cross compile/clang
   - fix some memory leaks in plugins
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmCjfMEACgkQ+9DbCVqe
 KkQ+aQf/QX5prjX9IW64GzwmZLdgufrc9fwdbrhyNTglcS0gN/hti6is3gBhe9dU
 KynWvmjvnosDT1EI53ciVkSRrwDLptpyq3bz140RjmSmPFQvUP16yF3KTThBxXuO
 1BZk7iq5jzv3jVHD4j4zQPyMRatyBBToKtIboSZntljq6WTomv8gWu8+vdlWGzeG
 0ubW1k9YHZNnQ0Et5xjqQBgBbjoOQvbkLYwNyyDjkOem8p4oKFIyTcRP9PYSJ9t3
 cq314LuFJVjW1TMywvgHOlZEwzqRqOtdzW1RC1EbVqYR+lifAgYRI/kTEwL3Ab7j
 Z+hybdzlfJdUPNiJ5Q4rjDpuKI+1Nw==
 =FYwg
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-and-plugin-updates-180521-2' into staging

testing and plugin updates:

  - various fixes for binfmt_misc docker images
  - add hexagon check-tcg support docker image
  - add tricore check-tcg support
  - refactor ppc docker images
  - add missing ppc64le tests
  - don't use host_cc for test fallback
  - check-tcg configure.sh tweaks for cross compile/clang
  - fix some memory leaks in plugins

# gpg: Signature made Tue 18 May 2021 09:37:21 BST
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-and-plugin-updates-180521-2: (29 commits)
  configure: use cc, not host_cc to set cross_cc for build arch
  tests/tcg: don't allow clang as a cross compiler
  tests/tcg: fix missing return
  tests/tcg/ppc64le: tests for brh/brw/brd
  tests/docker: gcc-10 based images for ppc64{,le} tests
  tests/tcg/tricore: Add muls test
  tests/tcg/tricore: Add msub test
  tests/tcg/tricore: Add madd test
  tests/tcg/tricore: Add ftoi test
  tests/tcg/tricore: Add fmul test
  tests/tcg/tricore: Add fadd test
  tests/tcg/tricore: Add dvstep test
  tests/tcg/tricore: Add clz test
  tests/tcg/tricore: Add bmerge test
  tests/tcg/tricore: Add macros to create tests and first test 'abs'
  configure: Emit HOST_CC to config-host.mak
  tests/tcg/tricore: Add build infrastructure
  hw/tricore: Add testdevice for tests in tests/tcg/
  tests/tcg: Run timeout cmds using --foreground
  tests/tcg: Add docker_as and docker_ld cmds
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-05-20 10:00:58 +01:00
Peter Maydell
d874bc0816 Block layer patches
- vhost-user-blk: Fix error handling during initialisation
 - Add test cases for the vhost-user-blk export
 - Fix leaked Transaction objects
 - qcow2: Expose dirty bit in 'qemu-img info'
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmCjnaoRHGt3b2xmQHJl
 ZGhhdC5jb20ACgkQfwmycsiPL9ZRDRAAw814/9O2E5dXDK3dZfqHqxWBdvNuPEuj
 LTUSGpuF+cAPAMJhzZm5Hy8n8G4KGwzpqt/vgBSnBAeAjHPBORGu4gEpr0G/GiHk
 OltElQ2GBBtHdixVhhimk7XoaE90Rmonh3uvolzdI8Ej8a2uvVYJcZhOD4JNZiOV
 HweG5SWcL3mvF4S8m0SOTFdlzeyA8NloaDduILiX+GiwiHystWw7e9bMMyP9Mejx
 95Qip9Huco3KFpYSpnGCvhTJ4jZwuQdqKs2d0dJBtIiU5PeRBY4mw4XT8xh6K3SA
 d/DBIDJ0iEQiVT/a2eNVtY/MsOklJYqnGVUWs18piOu1d/r6zQ2whLtIx/H6z3eD
 PLk1/mKNYcHDM6sTdBpliRsBDRJ7yeeaSqUGY3JPbyIhc7A/gqgfkXiRfEvN9lHF
 O/zerX4tgg7HRlqsyEyT937wiT7I8HHhbS0JtC0c5uxwmk4J0L+PUCnFptUtmZim
 iJTO5h90PKldnzKz0VNXgvrvCFTgmyR/aq89VZfafTE4sNSlZOTbasPADcPUUr/V
 Ju5J1r3J3lctjBGAReiTRxeTHyYHP2BEfzBqt3Orf86qrsNuWY3SXbzAEbcT1Pyu
 O6rIOF6B8DWEN5R2krPC/aw4/lXDST1FdVHibrmbUyQaayJrEWwvudqgTzCcfPNr
 c96LQH3gkTY=
 =Xe4O
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

- vhost-user-blk: Fix error handling during initialisation
- Add test cases for the vhost-user-blk export
- Fix leaked Transaction objects
- qcow2: Expose dirty bit in 'qemu-img info'

# gpg: Signature made Tue 18 May 2021 11:57:46 BST
# gpg:                using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg:                issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
  vhost-user-blk: Check that num-queues is supported by backend
  virtio: Fail if iommu_platform is requested, but unsupported
  vhost-user-blk: Get more feature flags from vhost device
  vhost-user-blk: Improve error reporting in realize
  vhost-user-blk: Don't reconnect during initialisation
  vhost-user-blk: Make sure to set Error on realize failure
  vhost-user-blk-test: test discard/write zeroes invalid inputs
  tests/qtest: add multi-queue test case to vhost-user-blk-test
  test: new qTest case to test the vhost-user-blk-server
  block/export: improve vu_blk_sect_range_ok()
  block: Fix Transaction leak in bdrv_reopen_multiple()
  block: Fix Transaction leak in bdrv_root_attach_child()
  qcow2: set bdi->is_dirty

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-05-19 16:10:35 +01:00