Commit Graph

1598 Commits

Author SHA1 Message Date
Kevin Wolf
1de6b45fb5 block: bdrv_reopen() with backing file in different AioContext
This patch allows bdrv_reopen() (and therefore the x-blockdev-reopen QMP
command) to attach a node as the new backing file even if the node is in
a different AioContext than the parent if one of both nodes can be moved
to the AioContext of the other node.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Tested-by: Peter Krempa <pkrempa@redhat.com>
Message-Id: <20200306141413.30705-3-kwolf@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-03-06 17:34:09 +01:00
Kevin Wolf
97518e11c3 iotests: Refactor blockdev-reopen test for iothreads
We'll want to test more than one successful case in the future, so
prepare the test for that by a refactoring that runs each scenario in a
separate VM.

test_iothreads_switch_{backing,overlay} currently produce errors, but
these are cases that should actually work, by switching either the
backing file node or the overlay node to the AioContext of the other
node.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Tested-by: Peter Krempa <pkrempa@redhat.com>
Message-Id: <20200306141413.30705-2-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-03-06 17:34:01 +01:00
Max Reitz
81311255f2 iotests/026: Test EIO on allocation in a data-file
Test what happens when writing data to an external data file, where the
write requires an L2 entry to be allocated, but the data write fails.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200225143130.111267-4-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-03-06 17:15:37 +01:00
Max Reitz
31ab00f374 iotests/026: Test EIO on preallocated zero cluster
Test what happens when writing data to a preallocated zero cluster, but
the data write fails.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200225143130.111267-3-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-03-06 17:15:37 +01:00
Alex Bennée
002375895c tests/iotests: be a little more forgiving on the size test
At least on ZFS this was failing as 512 was less than or equal to 512.
I suspect the reason is additional compression done by ZFS and however
qemu-img gets the actual size.

Loosen the criteria to make sure after is not bigger than before and
also dump the values in the report.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200225124710.14152-9-alex.bennee@linaro.org>
2020-02-25 20:20:09 +00:00
Max Reitz
dff8d44c96 iotests: Test snapshot -l field separation
Add a test that all fields in "qemu-img snapshot -l"s output are
separated by spaces.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200117105859.241818-3-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[mreitz: Renamed test from 284 to 286]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20 16:43:42 +01:00
Max Reitz
f535cc90b5 iotests: Test convert -n -B to backing-less target
This must not crash.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200121155915.98232-3-mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20 16:43:42 +01:00
Max Reitz
4dddeac115 iotests: Add test for image creation fallback
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200122164532.178040-6-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
[mreitz: Added a note that NBD does not support resizing, which is why
         the second case is expected to fail]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20 16:43:42 +01:00
Max Reitz
b7e9eae98c iotests/279: Fix for non-qcow2 formats
First, driver=qcow2 will not work so well with non-qcow2 formats (and
this test claims to support qcow, qed, and vmdk).

Second, vmdk will always report the backing file format to be vmdk.
Filter that out so the output looks like for all other formats.

Third, the flat vmdk subformats do not support backing files, so they
will not work with this test.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20191219144243.1763246-1-mreitz@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20 16:43:42 +01:00
Daniel P. Berrangé
087ab8e775 block: always fill entire LUKS header space with zeros
When initializing the LUKS header the size with default encryption
parameters will currently be 2068480 bytes. This is rounded up to
a multiple of the cluster size, 2081792, with 64k sectors. If the
end of the header is not the same as the end of the cluster we fill
the extra space with zeros. This was forgetting that not even the
space allocated for the header will be fully initialized, as we
only write key material for the first key slot. The space left
for the other 7 slots is never written to.

An optimization to the ref count checking code:

  commit a5fff8d4b4 (refs/bisect/bad)
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   Wed Feb 27 16:14:30 2019 +0300

    qcow2-refcount: avoid eating RAM

made the assumption that every cluster which was allocated would
have at least some data written to it. This was violated by way
the LUKS header is only partially written, with much space simply
reserved for future use.

Depending on the cluster size this problem was masked by the
logic which wrote zeros between the end of the LUKS header and
the end of the cluster.

$ qemu-img create --object secret,id=cluster_encrypt0,data=123456 \
   -f qcow2 -o cluster_size=2k,encrypt.iter-time=1,\
               encrypt.format=luks,encrypt.key-secret=cluster_encrypt0 \
               cluster_size_check.qcow2 100M
  Formatting 'cluster_size_check.qcow2', fmt=qcow2 size=104857600
    encrypt.format=luks encrypt.key-secret=cluster_encrypt0
    encrypt.iter-time=1 cluster_size=2048 lazy_refcounts=off refcount_bits=16

$ qemu-img check --object secret,id=cluster_encrypt0,data=redhat \
    'json:{"driver": "qcow2", "encrypt.format": "luks", \
           "encrypt.key-secret": "cluster_encrypt0", \
           "file.driver": "file", "file.filename": "cluster_size_check.qcow2"}'
ERROR: counting reference for region exceeding the end of the file by one cluster or more: offset 0x2000 size 0x1f9000
Leaked cluster 4 refcount=1 reference=0
...snip...
Leaked cluster 130 refcount=1 reference=0

1 errors were found on the image.
Data may be corrupted, or further writes to the image may corrupt it.

127 leaked clusters were found on the image.
This means waste of disk space, but no harm to data.
Image end offset: 268288

The problem only exists when the disk image is entirely empty. Writing
data to the disk image payload will solve the problem by causing the
end of the file to be extended further.

The change fixes it by ensuring that the entire allocated LUKS header
region is fully initialized with zeros. The qemu-img check will still
fail for any pre-existing disk images created prior to this change,
unless at least 1 byte of the payload is written to.

Fully writing zeros to the entire LUKS header is a good idea regardless
as it ensures that space has been allocated on the host filesystem (or
whatever block storage backend is used).

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200207135520.2669430-1-berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20 16:43:42 +01:00
Max Reitz
b0c4cf21b0 iotests/147: Fix drive parameters
8dff69b94 added an aio parameter to the drive parameter but forgot to
add a comma before, thus breaking the test.  Fix it again.

Fixes: 8dff69b941
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200206130812.612960-1-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20 16:43:42 +01:00
Thomas Huth
6756696e38 iotests: Remove the superfluous 2nd check for the availability of quorum
Commit d9df28e7b0 ("iotests: check whitelisted formats") added the
modern @iotests.skip_if_unsupported() to the functions in this test,
so we don't need the old explicit test here anymore.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200129141751.32652-1-thuth@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Tested-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20 16:43:42 +01:00
Max Reitz
c45a88f442 iotests: Check that @replaces can replace filters
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200218103454.296704-20-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 14:52:16 +01:00
Max Reitz
a1da187860 iotests: Add tests for invalid Quorum @replaces
Add two tests to see that you cannot replace a Quorum child with the
mirror job while the child is in use by a different parent.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200218103454.296704-19-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 14:52:16 +01:00
Max Reitz
89e2194524 iotests: Use self.image_len in TestRepairQuorum
041's TestRepairQuorum has its own image_len, no need to refer to
TestSingleDrive.  (This patch allows commenting out TestSingleDrive to
speed up 041 during test testing.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200218103454.296704-18-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 11:55:41 +01:00
Max Reitz
c351afd6f3 iotests: Resolve TODOs in 041
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200218103454.296704-17-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 11:55:41 +01:00
Max Reitz
5d016a69e3 iotests/041: Drop superfluous shutdowns
All tearDowns in 041 shutdown the VM.  Thus, test cases do not need to
do it themselves (unless they need the VM to be down for some
post-operation check).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200218103454.296704-16-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 11:55:41 +01:00
Max Reitz
6a3d0f1e3f iotests: Add VM.assert_block_path()
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200218103454.296704-15-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 11:55:41 +01:00
Max Reitz
6644d0e619 iotests: Use complete_and_wait() in 155
This way, we get to see errors during the completion phase.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200218103454.296704-14-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 11:55:40 +01:00
Max Reitz
f718ca147d iotests: Let 041 use -blockdev for quorum children
Using -drive with default options means that a virtio-blk drive will be
created that has write access to the to-be quorum children.  Quorum
should have exclusive write access to them, so we should use -blockdev
instead.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200218103454.296704-5-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 11:55:39 +01:00
Kevin Wolf
d439848941 iotests: Test error handling policies with block-commit
This tests both read failure (from the top node) and write failure (to
the base node) for on-error=report/stop/ignore.

As block-commit actually starts two different types of block jobs
(mirror.c for committing the active later, commit.c for intermediate
layers), all tests are run for both cases.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200214200812.28180-8-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 10:53:56 +01:00
Kevin Wolf
a0cf8daf77 iotests: Test copy offloading with external data file
This adds a test for 'qemu-img convert' with copy offloading where the
target image has an external data file. If the test hosts supports it,
it tests both the case where copy offloading is supported and the case
where it isn't (otherwise we just test unsupported twice).

More specifically, the case with unsupported copy offloading tests
qcow2_alloc_cluster_abort() with external data files.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200211094900.17315-4-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 10:53:56 +01:00
Peter Maydell
73d336510c - Python 3 cleanups:
. Remove text about Python 2 in qemu-deprecated (Thomas)
   . Remove shebang header (Paolo, Philippe)
   . scripts/checkpatch.pl now allows Python 3 interpreter (Philippe)
   . Explicit usage of Python 3 interpreter in scripts (Philippe)
   . Fix Python scripts permissions (Paolo, Philippe)
   . Drop 'from __future__ import print_function' (Paolo)
   . Specify minimum python requirements in ReadTheDocs configuration (Alex)
 - Test UNIX/EXEC transports with migration (Oksana)
 - Added extract_from_rpm helper, improved extract_from_deb (Liam)
 - Allow to use other serial consoles than default one (Philippe)
 - Various improvements in QEMUMonitorProtocol (Wainer)
 - Fix kvm_available() on ppc64le (Wainer)
 -----BEGIN PGP SIGNATURE-----
 
 iQIyBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAl49e+QACgkQ4+MsLN6t
 wN7E6A/3dzCLP0QDoT/C2rC1uu2yXBpXsSUSGeRCA+NLGEOpkyWiEK/vCQp/WxGO
 YU3ToO5NIGMxjVGhy14D1U4NrmiUhxmkGlJQJ60WKAO/oJBElHz8aR2keGlbkBp3
 mh/Amyw+ubM3aBYcAmBbgn0y7yr+lV62mD1JmffQebwIMsT+6aL4XhZ6Lf8PAEkY
 0pCK30O0j8Mwk8ppwKjoRT0WhhcKInmaLElbcOstoqVjQerGV7ho7VjlS9v2BPeg
 uQbDbv5lask7XVoJzortQ9nP2Tq3R+T7hNc0RPa0/tuqZ/6ucq6eYbFgbr/S7r3/
 Lem3ByA1EPlNzjq1HZoU3zx0/Ydq6qPyiudO9fLOCDIGVG6ZKXmKVq71lWHotcAw
 Oir+RPSVhPmOekhOPXAEi3mwAqRX3iSUGSsm+aLgq9I1DJfdE8dG2n5ZKVSWClmF
 xwb8Dhlz1T5SIlLderfVFryLzV2/LoHQOpr37phw6Ec5gox6UR0i4ozKEaY1zmZV
 THvweFraoBUGwIN7ULfP8jxs8hjQY9HIxhYjNvPxqAQekH6hO97qkjbs44ujvNrS
 D/yL4Gk4XgDjGP37aeBoY0zCUMM3cpWZdVfycDiXe6r949ifOoF0gZSny7ryRTGh
 s1RvQAM9VaYcGTERkLLCC+QMuz24nN/cEg9H3SFlfHBGVBxaBw==
 =8xBQ
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/python-next-20200207' into staging

- Python 3 cleanups:
  . Remove text about Python 2 in qemu-deprecated (Thomas)
  . Remove shebang header (Paolo, Philippe)
  . scripts/checkpatch.pl now allows Python 3 interpreter (Philippe)
  . Explicit usage of Python 3 interpreter in scripts (Philippe)
  . Fix Python scripts permissions (Paolo, Philippe)
  . Drop 'from __future__ import print_function' (Paolo)
  . Specify minimum python requirements in ReadTheDocs configuration (Alex)
- Test UNIX/EXEC transports with migration (Oksana)
- Added extract_from_rpm helper, improved extract_from_deb (Liam)
- Allow to use other serial consoles than default one (Philippe)
- Various improvements in QEMUMonitorProtocol (Wainer)
- Fix kvm_available() on ppc64le (Wainer)

# gpg: Signature made Fri 07 Feb 2020 15:01:56 GMT
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/python-next-20200207: (46 commits)
  .readthedocs.yml: specify some minimum python requirements
  drop "from __future__ import print_function"
  make all Python scripts executable
  scripts/signrom: remove Python 2 support, add shebang
  tests/qemu-iotests/check: Only check for Python 3 interpreter
  scripts: Explicit usage of Python 3 (scripts without __main__)
  tests/qemu-iotests: Explicit usage of Python3 (scripts without __main__)
  tests/vm: Remove shebang header
  tests/acceptance: Remove shebang header
  scripts/tracetool: Remove shebang header
  scripts/minikconf: Explicit usage of Python 3
  scripts: Explicit usage of Python 3 (scripts with __main__)
  tests: Explicit usage of Python 3
  tests/qemu-iotests: Explicit usage of Python 3 (scripts with __main__)
  tests/qemu-iotests/check: Allow use of python3 interpreter
  scripts/checkpatch.pl: Only allow Python 3 interpreter
  tests/acceptance/migration: Default to -nodefaults
  tests/acceptance/migration: Add the 'migration' tag
  tests/acceptance/migration: Test EXEC transport when migrating
  tests/acceptance/migration: Test UNIX transport when migrating
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-10 12:07:26 +00:00
Paolo Bonzini
423edd9a31 drop "from __future__ import print_function"
This is only needed for Python 2, which we do not support anymore.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200204160604.19883-1-pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:15:16 +01:00
Paolo Bonzini
6dd360ce3f make all Python scripts executable
Scripts that have a Python shebang are meant to be executed directly from the
shell; give them 755 permissions.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200204160237.16889-1-pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:15:16 +01:00
Philippe Mathieu-Daudé
351aa2706e tests/qemu-iotests/check: Only check for Python 3 interpreter
All the iotests Python scripts have been converted to search for
the Python 3 interpreter. Update the ./check script accordingly.

Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-13-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:15:16 +01:00
Philippe Mathieu-Daudé
7c47752608 tests/qemu-iotests: Explicit usage of Python3 (scripts without __main__)
Use the program search path to find the Python 3 interpreter.

Patch created mechanically by running:

  $ sed -i "s,^#\!/usr/bin/\(env\ \)\?python$,#\!/usr/bin/env python3," \
      $(git grep -lF '#!/usr/bin/env python' \
      | xargs grep -L 'if __name__.*__main__')

Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-11-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:15:16 +01:00
Philippe Mathieu-Daudé
903cb1bf39 tests/qemu-iotests: Explicit usage of Python 3 (scripts with __main__)
Use the program search path to find the Python 3 interpreter.

Patch created mechanically by running:

  $ sed -i "s,^#\!/usr/bin/\(env\ \)\?python$,#\!/usr/bin/env python3," \
       $(git grep -l 'if __name__.*__main__')

Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-4-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé
15b015690b tests/qemu-iotests/check: Allow use of python3 interpreter
As we want to enforce a unique and explicit Python 3 interpreter,
we need let this script handle 'python3' too.

Suggested-by: Kevin Wolf <kwolf@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-3-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Peter Maydell
5b7686f3fa nbd patches for 2020-02-06
- Allow setting NBD description from QMP for parity with qemu-nbd
 - Remove deprecated 'qemu-nbd --partition'
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAl48C6YACgkQp6FrSiUn
 Q2qZQwf/Q6oUzYisDoMT58eRI1DPVzwBSUXPnKf6yikDyEcQr/KByB9OJGhm5ang
 YsT52OpgZTzMXSsGL9RqitvgZtp4vwuDcYjoNafUkNXYHgno6ZojGyVBbypgb4VA
 FiTxoSH0CAD/m/PwVg+k/q0vxca3sFL/l759O55RaNbM7RVuuC6FoNpNvds2q4ch
 l71r5WKEwWBBTy94L4KU/+9RR5CrKmVBF8CqPCP0xE5fm/gYo5j+LFsWgUxYU8Bn
 5hjYboZN1FQJHQve7ErErR7OAf7trHEVVtkuNKMQnu9uU9OgqwS3ReQ2S3c11dsL
 sa0mRL1dPTw55HzRYCV6m0f5TRezuQ==
 =29Vs
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2020-02-06' into staging

nbd patches for 2020-02-06

- Allow setting NBD description from QMP for parity with qemu-nbd
- Remove deprecated 'qemu-nbd --partition'

# gpg: Signature made Thu 06 Feb 2020 12:50:46 GMT
# gpg:                using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg:                 aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2020-02-06:
  qemu-nbd: Removed deprecated --partition option
  docs: Fix typo in qemu-nbd -P replacement
  nbd: Allow description when creating NBD blockdev

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-06 18:59:12 +00:00
Vladimir Sementsov-Ogievskiy
a541fcc27c iotests: add test for backup-top failure on permission activation
This test checks that bug is really fixed by previous commit.

Cc: qemu-stable@nongnu.org # v4.2.0
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20200121142802.21467-3-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Thomas Huth
ce95a15e42 iotests: Enable more tests in the 'auto' group to improve test coverage
According to Kevin, tests 030, 040 and 041 are among the most valuable
tests that we have, so we should always run them if possible, even if
they take a little bit longer.

According to Max, it would be good to have a test for iothreads and
migration. 127 and 256 seem to be good candidates for iothreads. For
migration, let's enable 181 and 203 (which also tests iothreads).
(091 would be a good candidate for migration, too, but Alex Bennée
reported that this test fails on ZFS file systems, so it can't be
included yet)

Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20200121095205.26323-7-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Thomas Huth
cd2058289b iotests: Skip Python-based tests if QEMU does not support virtio-blk
We are going to enable some of the python-based tests in the "auto" group,
and these tests require virtio-blk to work properly. Running iotests
without virtio-blk likely does not make too much sense anyway, so instead
of adding a check for the availability of virtio-blk to each and every
test (which does not sound very appealing), let's rather add a check for
this a central spot in the "check" script instead (so that it is still
possible to run "make check" for qemu-system-tricore for example).

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20200121095205.26323-6-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Thomas Huth
9bdabfbe72 iotests: Check for the availability of the required devices in 267 and 127
We are going to enable 127 in the "auto" group, but it only works if
virtio-scsi and scsi-hd are available - which is not the case with
QEMU binaries like qemu-system-tricore for example, so we need a
proper check for the availability of these devices here.

A very similar problem exists in iotest 267 - it has been added to
the "auto" group already, but requires virtio-blk and thus currently
fails with qemu-system-tricore for example. Let's also add aproper
check there.

Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20200121095205.26323-5-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Thomas Huth
30ad36f55f iotests: Test 183 does not work on macOS and OpenBSD
In the long run, we might want to add test 183 to the "auto" group
(but it still fails occasionally, so we cannot do that yet). However,
when running 183 in Cirrus-CI on macOS, or with our vm-build-openbsd
target, it currently always fails with an "Timeout waiting for return
on handle 0" error.

Let's mark it as supported only on systems where the test is working
most of the time (i.e. Linux, FreeBSD and NetBSD).

Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20200121095205.26323-4-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Thomas Huth
877d18f2aa iotests: Test 041 only works on certain systems
041 works fine on Linux, FreeBSD, NetBSD and OpenBSD, but fails on macOS.
Let's mark it as only supported on the systems where we know that it is
working fine.

Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20200121095205.26323-3-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
John Snow
72b2903056 iotests: remove 'linux' from default supported platforms
verify_platform will check an explicit whitelist and blacklist instead.
The default will now be assumed to be allowed to run anywhere.

For tests that do not specify their platforms explicitly, this has the effect of
enabling these tests on non-linux platforms. For tests that always specified
linux explicitly, there is no change.

For Python tests on FreeBSD at least; only seven python tests fail:
045 147 149 169 194 199 211

045 and 149 appear to be misconfigurations,
147 and 194 are the AF_UNIX path too long error,
169 and 199 are bitmap migration bugs, and
211 is a bug that shows up on Linux platforms, too.

This is at least good evidence that these tests are not Linux-only. If
they aren't suitable for other platforms, they should be disabled on a
per-platform basis as appropriate.

Therefore, let's switch these on and deal with the failures.

Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20200121095205.26323-2-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Eric Blake
deb6ccb077 nbd: Allow description when creating NBD blockdev
Allow blockdevs to match the feature already present in qemu-nbd -D.
Enhance iotest 223 to cover it.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20191114024635.11363-5-eblake@redhat.com>
2020-02-05 17:22:13 -06:00
Aarushi Mehta
8dff69b941 tests/qemu-iotests: use AIOMODE with various tests
Signed-off-by: Aarushi Mehta <mehta.aaru20@gmail.com>
Acked-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200120141858.587874-16-stefanha@redhat.com
Message-Id: <20200120141858.587874-16-stefanha@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-01-30 21:01:40 +00:00
Aarushi Mehta
7156ca4888 tests/qemu-iotests: enable testing with aio options
Signed-off-by: Aarushi Mehta <mehta.aaru20@gmail.com>
Acked-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200120141858.587874-15-stefanha@redhat.com
Message-Id: <20200120141858.587874-15-stefanha@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-01-30 21:01:40 +00:00
Sergio Lopez
9b8c59e761 iotests: Test handling of AioContexts with some blockdev actions
Includes the following tests:

 - Adding a dirty bitmap.
   * RHBZ: 1782175

 - Starting a drive-mirror to an NBD-backed target.
   * RHBZ: 1746217, 1773517

 - Aborting an external snapshot transaction.
   * RHBZ: 1779036

 - Aborting a blockdev backup transaction.
   * RHBZ: 1782111

For each one of them, a VM with a number of disks running in an
IOThread AioContext is used.

Signed-off-by: Sergio Lopez <slp@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-01-27 17:19:53 +01:00
Sergio Lopez
2288ccfac9 blockdev: unify qmp_drive_backup and drive-backup transaction paths
Issuing a drive-backup from qmp_drive_backup takes a slightly
different path than when it's issued from a transaction. In the code,
this is manifested as some redundancy between do_drive_backup() and
drive_backup_prepare().

This change unifies both paths, merging do_drive_backup() and
drive_backup_prepare(), and changing qmp_drive_backup() to create a
transaction instead of calling do_backup_common() direcly.

As a side-effect, now qmp_drive_backup() is executed inside a drained
section, as it happens when creating a drive-backup transaction. This
change is visible from the user's perspective, as the job gets paused
and immediately resumed before starting the actual work.

Also fix tests 141, 185 and 219 to cope with the extra
JOB_STATUS_CHANGE lines.

Signed-off-by: Sergio Lopez <slp@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-01-27 17:19:53 +01:00
Thomas Huth
9442bebe6e iotests: Add more "skip_if_unsupported" statements to the python tests
The python code already contains a possibility to skip tests if the
corresponding driver is not available in the qemu binary - use it
in more spots to avoid that the tests are failing if the driver has
been disabled.

While we're at it, we can now also remove some of the old checks that
were using iotests.supports_quorum() - and which were apparently not
working as expected since the tests aborted instead of being skipped
when "quorum" was missing in the QEMU binary.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-01-27 17:19:53 +01:00
Max Reitz
8da7969bd7 iotests.py: Let wait_migration wait even more
The "migration completed" event may be sent (on the source, to be
specific) before the migration is actually completed, so the VM runstate
will still be "finish-migrate" instead of "postmigrate".  So ask the
users of VM.wait_migration() to specify the final runstate they desire
and then poll the VM until it has reached that state.  (This should be
over very quickly, so busy polling is fine.)

Without this patch, I see intermittent failures in the new iotest 280
under high system load.  I have not yet seen such failures with other
iotests that use VM.wait_migration() and query-status afterwards, but
maybe they just occur even more rarely, or it is because they also wait
on the destination VM to be running.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-01-27 17:19:53 +01:00
Philippe Mathieu-Daudé
75ab574b4c tests/qemu-iotests: Update tests to recent desugarized -accel option
Commit 6f6e1698a6 desugarized "-machine accel=" to a list
of "-accel" options. Since now "-machine accel" and "-accel"
became incompatible, update the iotests to the new format.

Error reported here:
https://gitlab.com/qemu-project/qemu/-/jobs/385801004#L3400

Reported-by: GitLab CI
Fixes: 6f6e1698a6 (vl: configure accelerators from -accel options)
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20200106130951.29873-1-philmd@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 14:14:41 +01:00
Andrey Shinkevich
19959445f5 tests/qemu-iotests: add case to write compressed data of multiple clusters
Add the case to the iotest #214 that checks possibility of writing
compressed data of more than one cluster size. The test case involves
the compress filter driver showing a sample usage of that.

Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 1575288906-551879-4-git-send-email-andrey.shinkevich@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
1b35b85abb iotests: Allow check -o data_file
The problem with allowing the data_file option is that you want to use a
different data file per image used in the test.  Therefore, we need to
allow patterns like -o data_file='$TEST_IMG.data_file'.

Then, we need to filter it out from qemu-img map, qemu-img create, and
remove the data file in _rm_test_img.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-23-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
3be2024aef iotests: Disable data_file where it cannot be used
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191107163708.833192-22-mreitz@redhat.com
[mreitz: Also disable 273]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
39d2c7dc8f iotests: Make 198 work with data_file
We do not care about the json:{} filenames here, so we can just filter
them out and thus make the test work both with and without external data
files.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-21-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
e66566e6a7 iotests: Make 137 work with data_file
When using an external data file, there are no refcounts for data
clusters.  We thus have to adjust the corruption test in this patch to
not be based around a data cluster allocation, but the L2 table
allocation (L2 tables are still refcounted with external data files).

Furthermore, we should not print qcow2.py's list of incompatible
features because it differs depending on whether there is an external
data file or not.

With those two changes, the test will work both with and without
external data files (once that options works with the iotests at all).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-20-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00