Commit Graph

5291 Commits

Author SHA1 Message Date
Max Reitz 325dd915b2 iotests: Disable broken streamOptimized tests
streamOptimized does not support writes that do not span exactly one
cluster.  Furthermore, it cannot rewrite already allocated clusters.
As such, many iotests do not work with it.  Disable them.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190815153638.4600-6-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-09-03 14:55:35 +02:00
Max Reitz 12b7cbcabc iotests: Keep testing broken relative extent paths
We had a test for a case where relative extent paths did not work, but
unfortunately we just fixed the underlying problem, so it works now.
This patch adds a new test case that still fails.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190815153638.4600-4-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-09-03 14:55:35 +02:00
Max Reitz cdc0dd2586 vmdk: Use bdrv_dirname() for relative extent paths
This makes iotest 033 pass with e.g. subformat=monolithicFlat.  It also
turns a former error in 059 into success.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190815153638.4600-3-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-09-03 14:55:35 +02:00
Max Reitz f158ffdba6 iotests: Fix _filter_img_create()
fe646693ac changed qemu-img create's output so that it no longer prints
single quotes around parameter values.  The subformat and adapter_type
filters in _filter_img_create() have never been adapted to that change.

Fixes: fe646693ac
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190815153638.4600-2-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-09-03 14:55:35 +02:00
Nir Soffer 7e3dc2ba9a iotests: Test allocate_first_block() with O_DIRECT
Using block_resize we can test allocate_first_block() with file
descriptor opened with O_DIRECT, ensuring that it works for any size
larger than 4096 bytes.

Testing smaller sizes is tricky as the result depends on the filesystem
used for testing. For example on NFS any size will work since O_DIRECT
does not require any alignment.

Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190827010528.8818-3-nsoffer@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-09-03 14:55:35 +02:00
Nir Soffer 3a20013fbb block: posix: Always allocate the first block
When creating an image with preallocation "off" or "falloc", the first
block of the image is typically not allocated. When using Gluster
storage backed by XFS filesystem, reading this block using direct I/O
succeeds regardless of request length, fooling alignment detection.

In this case we fallback to a safe value (4096) instead of the optimal
value (512), which may lead to unneeded data copying when aligning
requests.  Allocating the first block avoids the fallback.

Since we allocate the first block even with preallocation=off, we no
longer create images with zero disk size:

    $ ./qemu-img create -f raw test.raw 1g
    Formatting 'test.raw', fmt=raw size=1073741824

    $ ls -lhs test.raw
    4.0K -rw-r--r--. 1 nsoffer nsoffer 1.0G Aug 16 23:48 test.raw

And converting the image requires additional cluster:

    $ ./qemu-img measure -f raw -O qcow2 test.raw
    required size: 458752
    fully allocated size: 1074135040

When using format like vmdk with multiple files per image, we allocate
one block per file:

    $ ./qemu-img create -f vmdk -o subformat=twoGbMaxExtentFlat test.vmdk 4g
    Formatting 'test.vmdk', fmt=vmdk size=4294967296 compat6=off hwversion=undefined subformat=twoGbMaxExtentFlat

    $ ls -lhs test*.vmdk
    4.0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f001.vmdk
    4.0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f002.vmdk
    4.0K -rw-r--r--. 1 nsoffer nsoffer  353 Aug 27 03:23 test.vmdk

I did quick performance test for copying disks with qemu-img convert to
new raw target image to Gluster storage with sector size of 512 bytes:

    for i in $(seq 10); do
        rm -f dst.raw
        sleep 10
        time ./qemu-img convert -f raw -O raw -t none -T none src.raw dst.raw
    done

Here is a table comparing the total time spent:

Type    Before(s)   After(s)    Diff(%)
---------------------------------------
real      530.028    469.123      -11.4
user       17.204     10.768      -37.4
sys        17.881      7.011      -60.7

We can see very clear improvement in CPU usage.

Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Message-id: 20190827010528.8818-2-nsoffer@redhat.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-09-03 14:55:35 +02:00
Marc-André Lureau d39b626343 tests: fix modules-test with no default machine
Fixes: eb062cfa73 ("tests: add module loading test")
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20190827140241.20818-2-marcandre.lureau@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-08-27 17:15:04 +01:00
Marc-André Lureau f54d432e9a build-sys: build ui-spice-app as a module
This reverts commit 45db1ac157 ("modules-test: ui-spice-app is not
built as module") and fixes commit d8aec9d9f1 ("display: add -display
spice-app launching a Spice client").

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20190827140241.20818-1-marcandre.lureau@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-08-27 17:15:04 +01:00
Paolo Bonzini 45db1ac157 modules-test: ui-spice-app is not built as module
$(call land, $(CONFIG_SPICE), $(CONFIG_GIO)) will never return "m" so
ui-spice-app is always linked into QEMU.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Miroslav Rezanina <mrezanin@redhat.com>
Message-id: 1566495734-23297-2-git-send-email-pbonzini@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-08-23 14:07:00 +01:00
Paolo Bonzini 0712c1b15b modules-test: fix const cast
Add a missing cast; this fixes a build failure with --enable-modules.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Miroslav Rezanina <mrezanin@redhat.com>
Message-id: 1566495734-23297-1-git-send-email-pbonzini@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-08-23 13:36:26 +01:00
Marc-André Lureau a4eb74a66a test: skip tests if socket_check_protocol_support() failed
Skip the tests if socket_check_protocol_support() failed, but do run
g_test_run() to keep TAP harness happy.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-08-22 17:30:25 +04:00
Marc-André Lureau e7b6ba4186 test-char: skip tcp tests if ipv4 check failed
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-08-22 17:30:25 +04:00
Marc-André Lureau 71714178fa tests: specify the address family when checking bind
getaddrinfo() may succeed with PF_UNSPEC, but fail when more specific.

(this allows to skip some tests that would fail under podman)

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-08-22 17:30:25 +04:00
Marc-André Lureau 05af039d1e tests/docker: add podman support
Allow to specify the container engine to run with ENGINE variable.

By default, ENGINE=auto and will select either podman or docker.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-08-22 17:30:25 +04:00
Marc-André Lureau 9459f75413 docker.py: add podman support
Add a --engine option to select either docker, podman or auto.

Among other advantages, podman allows to run rootless & daemonless
containers, fortunately sharing compatible CLI with docker.

With current podman, we have to use a uidmap trick in order to be able
to rw-share the ccache directory with the container user.

With a user 1000, the default mapping is:                                                                                                                                                                         1000 (host) -> 0 (container).
So write access to /var/tmp/ccache ends will end with permission
denied error.

With "--uidmap 1000:0:1 --uidmap 0:1:1000", the mapping is:
1000 (host) -> 0 (container, 1st namespace) -> 1000 (container, 2nd namespace).
(the rest is mumbo jumbo to avoid holes in the range of UIDs)

A future podman version may have an option such as --userns-keep-uid.
Thanks to Debarshi Ray <rishi@redhat.com> for the help!

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-08-22 17:30:25 +04:00
Marc-André Lureau 2461d80e6c docker.py: add --run-as-current-user
(podman will need further tweaks)

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-08-22 17:30:25 +04:00
Peter Maydell 8109234808 * Longstanding chardev race condition fix (Berto)
* Cleanups and tests from the Meson POC (Marc-André, myself)
 * Coalesced range cleanup (Peter)
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQEcBAABAgAGBQJdXX8PAAoJEL/70l94x66DkR4H/1Phk6zuTQfHISBXGACjDkFS
 kbVq2WE6gQXwqmbf/04mM/DZzsi9bCsHFlGhf7mFQoq9g2ywm7kgtjF7kiRewlqh
 YZYSLLEoEVKpZRxC8kQbbbZ/tfYDM7ejnKMPFK6ibNiiN7acuKJuY8z18OLg4PEA
 V5LfuaGPjsx98NUAuU6bHRs5ZDCL25DzBQ9pdcE5wkJ2pVJXpP6UBafN55l8Clzq
 PzuuNnP18QyuqbdRVm5YrY+0nUeDCdlU2clCC3/H9rS57Ts/z3gocs71VT8GKqIa
 ppNRg3wVADXtfql3MkpnmvZefx+QK6DguAAiKR4CtPax7Xx6WoA4DwtMKfsLHsY=
 =RwuS
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* Longstanding chardev race condition fix (Berto)
* Cleanups and tests from the Meson POC (Marc-André, myself)
* Coalesced range cleanup (Peter)

# gpg: Signature made Wed 21 Aug 2019 18:27:43 BST
# gpg:                using RSA key BFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream:
  char-socket: Lock tcp_chr_disconnect() and socket_reconnect_timeout()
  main-loop: Fix GSource leak in qio_task_thread_worker()
  memory: Fix up memory_region_{add|del}_coalescing
  memory: Remove has_coalesced_range counter
  memory: Split zones when do coalesced_io_del()
  memory: Refactor memory_region_clear_coalescing
  minikconf: don't print CONFIG_FOO=n lines
  configure: remove AUTOCONF_HOST
  tests: add module loading test
  module: return success on module load
  module: use g_hash_table_add()
  configure: define CONFIG_TOOLS here
  qemu-ga: clean up TOOLS variable

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-08-22 13:13:35 +01:00
Peter Maydell fe066b4848 Various trivial fixes
-----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAwFiEEzS913cjjpNwuT1Fz8ww4vT8vvjwFAl1dKK8SHGxhdXJlbnRA
 dml2aWVyLmV1AAoJEPMMOL0/L748WhUP93YZ60SfAFhFLkAexO3fJSMzzb02Zvy3
 6Q0UyR56xigESPhlfD3rPtPdMpQRnROtIsS1qWcd6x/ejKINh08xyx1HkWLfMKOn
 F1Cc43AdkSqsuEbIfYVJbL/Yz7H6SnD7B9ghV6B2S+nDnDXGMzObAb6QXUtlnMV+
 4as6pawKOLZiP0zFb+98m1as1gjjuQA/3jrQjWPfF3pzYAaBQPh/pHgDhUSVWjsi
 G7KWu4iljKio15WhhRlzKylhSD8Z4u1OD5x/3pbPnborVnOzvryWQ1hiRQ1feB/h
 NUaWs3A2KcgM54LIZN0dC4APGdPlLohwNhakaZR8EnJD9eik2kRO3JBmzorkSHua
 y0UBpNkjeIR/Cv4ayZv6NtrNFmuEJRRcusPBHg7sg0IBUtd4YdXrfE5d2tcYS3Hu
 WoZaYLME1FmbuByJFs+cQhMw88ISYvGD5PkzfkZuQC9/nM/zd6pJyRCAhgchIkfh
 G5iCtOMK8gs5xZjr22pOx/XYLFRzFsOIsGx3kcHH3vSJVS5K3QYFt2xH47YmQZHk
 1k3wQfc8ePFPFpR2rnWqg+iYbPgl8FpOsEfz4fuPRwBeRGFRRzfkpXLQ2DJww59/
 8pd2UAWgdwrvtimmykfEmTu+LUuZKwNFJNH3KfNputbBqjnv1KkbmsPqnElj3deo
 a7LJ72Z1k4Q=
 =dzAT
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-pull-request' into staging

Various trivial fixes

# gpg: Signature made Wed 21 Aug 2019 12:19:11 BST
# gpg:                using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg:                issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/trivial-branch-pull-request:
  hw/display: Compile various display devices as common object
  hw/display/sm501: Remove unused include
  spapr_events: Rewrite a fall through comment
  vl: Rewrite a fall through comment
  target/ppc: Rewrite a fall through comment
  hw/ipmi: Rewrite a fall through comment
  hw/dma/omap_dma: Move switch 'fall through' comment to correct place
  json: Move switch 'fall through' comment to correct place
  hw/net/e1000: Fix erroneous comment
  .gitignore: ignore some vhost-user* related files
  configure: fix sdl detection using sdl2-config
  configure: remove obsoleted $sparc_cpu variable
  misc: fix naming scheme of compatiblity arrays
  test: Use g_strndup instead of plain strndup

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-08-21 16:59:22 +01:00
Marc-André Lureau eb062cfa73 tests: add module loading test
This test will simply check that modules can be loaded, and no symbols
are missing.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-08-21 16:29:57 +02:00
tony.nguyen@bt.com 348fbd5816 test: Use g_strndup instead of plain strndup
Due to memory management rules. See HACKING.

Signed-off-by: Tony Nguyen <tony.nguyen@bt.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>

Message-Id: <dce313b46d294ada8826d34609a3447e@tpw09926dag18e.domain1.systemhost.net>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-08-21 10:27:13 +02:00
Wei Yang a060297822 test-bitmap: test set 1 bit case for bitmap_set
All current bitmap_set test cases set range across word, while the
handle of a range within one word is different from that.

Add case to set 1 bit as a represent for set range within one word.

Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-08-20 20:00:51 +02:00
Pavel Dovgalyuk dcb1578069 util/qemu-timer: refactor deadline calculation for external timers
icount-based record/replay uses qemu_clock_deadline_ns_all to measure
the period until vCPU may be interrupted.
This function takes in account the virtual timers, because they belong
to the virtual devices that may generate interrupt request or affect
the virtual machine state.
However, there are a subset of virtual timers, that are marked with
'external' flag. These do not change the virtual machine state and
only based on virtual clock. Calculating the deadling using the external
timers breaks the determinism, because they do not belong to the replayed
part of the virtual machine.
This patch fixes the deadline calculation for this case by adding
new parameter for skipping the external timers when it is needed.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>

--

v2 changes:
 - added new parameter for timer attribute mask
Message-Id: <156404426682.18669.17014100602930969222.stgit@pasha-Precision-3630-Tower>

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-08-20 17:26:21 +02:00
Andrey Shinkevich f673174e3f tests: Fix uninitialized byte in test_visitor_in_fuzz
One byte in the local buffer stays uninitialized, at least with the
first iteration, because of the double decrement in the
test_visitor_in_fuzz(). This is what Valgrind does not like and not
critical for the test itself. So, reduce the number of the memory
issues reports.

Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Message-Id: <1564502498-805893-3-git-send-email-andrey.shinkevich@virtuozzo.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-08-20 17:26:19 +02:00
Andrey Shinkevich ef0aa6af47 test-throttle: Fix uninitialized use of burst_length
ThrottleState::cfg of the static variable 'ts' is reassigned with the
local one in the do_test_accounting() and then is passed to the
throttle_account() with uninitialized member LeakyBucket::burst_length.

Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Message-Id: <1564502498-805893-2-git-send-email-andrey.shinkevich@virtuozzo.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-08-20 17:26:19 +02:00
Peter Maydell bbd69d36d1 MIPS queue for August 20th, 2019
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQEcBAABAgAGBQJdWuVWAAoJENSXKoln91plH+UH/j2N0RdS/pLbJwW7JnmqDvDF
 SKYZbK5i0KBzSMYMyiNimh+j7gQOfuPqbLJM/Y+FVPasJDfqqEsTdpHNc/HabbO2
 fJNuviWT5LgiJ4E8K/y4RUa60uOdQFfaepukFFsGC1TanlDqGid0qRU2KXZwU1sQ
 BV4LyM2FHsDG9AqPKfMiH012YsFQN5Qizu5He6JZxoW5tmqR3Mp7wIYJj6nqEEts
 +zCGkFJAAYh8ZhkiRuYu0FwGjfjl3AGNKnjlmqDWsz/gjE19BHT9PDg9z5pWvOAH
 IRfcRk9HH+GWUMXDgYti50i0/vILfU4O8nYwcC5FN2bHB3To/sCEfW6A/XiiFM8=
 =eB6a
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-aug-20-2019' into staging

MIPS queue for August 20th, 2019

# gpg: Signature made Mon 19 Aug 2019 19:07:18 BST
# gpg:                using RSA key D4972A8967F75A65
# gpg: Good signature from "Aleksandar Markovic <amarkovic@wavecomp.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 8526 FBF1 5DA3 811F 4A01  DD75 D497 2A89 67F7 5A65

* remotes/amarkovic/tags/mips-queue-aug-20-2019:
  target/mips: tests/tcg: Fix target configurations for MSA tests
  target/mips: tests/tcg: Add optional printing of more detailed failure info
  target/mips: Style improvements in mips_mipssim.c
  target/mips: Style improvements in mips_malta.c
  target/mips: Style improvements in mips_int.c
  target/mips: Style improvements in mips_fulong2e.c
  target/mips: Style improvements in cps.c
  target/mips: Style improvements in translate.c
  target/mips: Style improvements in machine.c
  target/mips: Style improvements in cpu.c
  target/mips: Style improvements in cp0_timer.c

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-08-20 13:40:48 +01:00
Peter Maydell 156d320349 Block patches:
- preallocation=falloc/full support for LUKS
 - Various minor fixes
 -----BEGIN PGP SIGNATURE-----
 
 iQFGBAABCAAwFiEEkb62CjDbPohX0Rgp9AfbAGHVz0AFAl1awg0SHG1yZWl0ekBy
 ZWRoYXQuY29tAAoJEPQH2wBh1c9Au70H/jTfnZ5HGCnuMG9Qfz592b3OJzLxAUxy
 zRYZeYeQv2iBAJ8D3guJb4TIsvs9FgiZq6W+a+uXPZUfV2cPjHEcl2votnKuyWsx
 udzR5wyGEbZjeMIv5lhzWMf3oqWin5MVDFQ2xUSjt9rfNDiHH4pMHuLO6AGkVaRS
 MEOEs1/GEZ1mXPckG00YBJXMgmsk+7xWZ5cMjUI8Hsa1/nC2GLckh2j5y2rt3ppc
 dt2he6/etnK0d8Yt8yZXyNl8xSW95f0Z+a1RefgGX5n3kCV6K/9hkRIhCTdunzDd
 hULNummeZghjbn0HLyFjEnJlzSglWYprUsQj1HaXB4/LwMUwj3yNxHk=
 =dDid
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-08-19' into staging

Block patches:
- preallocation=falloc/full support for LUKS
- Various minor fixes

# gpg: Signature made Mon 19 Aug 2019 16:36:45 BST
# gpg:                using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg:                issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2019-08-19:
  doc: Preallocation does not require writing zeroes
  iotests: Fix 141 when run with qed
  vpc: Do not return RAW from block_status
  vmdk: Make block_status recurse for flat extents
  vdi: Make block_status recurse for fixed images
  iotests: Full mirror to existing non-zero image
  iotests: Test convert -n to pre-filled image
  iotests: Convert to preallocated encrypted qcow2
  vhdx: Fix .bdrv_has_zero_init()
  vdi: Fix .bdrv_has_zero_init()
  qcow2: Fix .bdrv_has_zero_init()
  block: Use bdrv_has_zero_init_truncate()
  block: Implement .bdrv_has_zero_init_truncate()
  block: Add bdrv_has_zero_init_truncate()
  mirror: Fix bdrv_has_zero_init() use
  qemu-img: Fix bdrv_has_zero_init() use in convert
  LUKS: support preallocation

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-08-20 10:27:24 +01:00
Aleksandar Markovic 6eed53f71b target/mips: tests/tcg: Fix target configurations for MSA tests
At this moment, the only MIPS CPUs that are emulated in QEMU and
support MSA extension are R5600 (mips32r5), and I6400/I6500 (mips64r6).
Therefore, mips32r5 and mips64r6 are the only ISAs that could support
MSA in QEMU. This means mips32r6 currently do not make much sense, and
mips32r5 support for MSA tests is needed, which is done by this patch.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1566216496-17375-38-git-send-email-aleksandar.markovic@rt-rk.com>
2019-08-19 19:53:37 +02:00
Aleksandar Markovic d7059fd737 target/mips: tests/tcg: Add optional printing of more detailed failure info
There is a need for printing input and output data for failure cases,
for debugging purpose. This is achieved by this patch, and only if a
preprocessor constant is manually set to 1. (Assumption is that the
need for such printout is relatively rare.)

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1566216496-17375-37-git-send-email-aleksandar.markovic@rt-rk.com>
2019-08-19 19:53:37 +02:00
Max Reitz 9c46f4a06d iotests: Fix 141 when run with qed
69f47505ee has changed qcow2 in such a way that the commit job run in
test 141 (and 144[1]) returns before it emits the READY event.  However,
141 also runs with qed, where the order is still the other way around.
Just filter out the {"return": {}} so the test passes for qed again.

[1] 144 only runs with qcow2, so it is fine as it is.

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Fixes: 69f47505ee
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190809185253.17535-1-mreitz@redhat.com
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-08-19 17:13:26 +02:00
Max Reitz 9463ee1f5f iotests: Full mirror to existing non-zero image
The result of a sync=full mirror should always be the equal to the
input.  Therefore, existing images should be treated as potentially
non-zero and thus should be explicitly initialized to be zero
beforehand.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190724171239.8764-12-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-08-19 17:13:26 +02:00
Max Reitz c2acc95bef iotests: Test convert -n to pre-filled image
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190724171239.8764-11-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-08-19 17:13:26 +02:00
Max Reitz 5a840549ee iotests: Convert to preallocated encrypted qcow2
Add a test case for converting an empty image (which only returns zeroes
when read) to a preallocated encrypted qcow2 image.
qcow2_has_zero_init() should return 0 then, thus forcing qemu-img
convert to create zero clusters.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Acked-by: Stefano Garzarella <sgarzare@redhat.com>
Tested-by: Stefano Garzarella <sgarzare@redhat.com>
Message-id: 20190724171239.8764-10-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-08-19 17:13:26 +02:00
Max Reitz cdf3bc934a mirror: Fix bdrv_has_zero_init() use
bdrv_has_zero_init() only has meaning for newly created images or image
areas.  If the mirror job itself did not create the image, it cannot
rely on bdrv_has_zero_init()'s result to carry any meaning.

This is the case for drive-mirror with mode=existing and always for
blockdev-mirror.

Note that we only have to zero-initialize the target with sync=full,
because other modes actually do not promise that the target will contain
the same data as the source after the job -- sync=top only promises to
copy anything allocated in the top layer, and sync=none will only copy
new I/O.  (Which is how mirror has always handled it.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190724171239.8764-3-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-08-19 17:13:26 +02:00
Richard Henderson 94597b6146 decodetree: Allow !function with no input bits
Call this form a "parameter", returning a value extracted
from the DisasContext.

Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-08-19 08:03:41 -07:00
Peter Maydell 3fbd3405d2 - Run the iotest during "make check"
-----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAl1XvtURHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbUqqRAAqfjPRPtXSZxaxl63O5EADugLY72s+04i
 Zc5MC4Ivwj0x1WA6JIG77kz6xmjIOizRIkF1jkyEkG+AgIrjw3rdYhCD4Iav0n/v
 nqltkOaf1FzdQYCHUTn0WUYn7Df2bSkjSTPhnqbCaGq5WjXzgzi9jhCFlZpo374J
 9yWk74nt3QlOUjLw6+cm0HxEf9IlRQdPwJNYJwsrYHgspJwcwAYJ7xaL34huoxkJ
 10fA9q6QK1bh67nZpAJOte3wQ8r35cUT4ZaIiyO0MFMrEiLp4/1gKYpkZwWq0+iV
 25rWVjogzRjp+LejMAltY9MmUekCl5ZzVVuhdt2jGPbNanzdHxHydYFUELP6WmrM
 zAyYQkDvG7JiY2F0M9rKJZOnpO2pYF2hxc/nqD04qF3HD0zG1eoIg056UlKcc5b/
 kIgR2srlj0aHWhGJ2/DV3w5ZowjJGKBAYYxQdEJmiuLpGdOimSoWXbjacDEd4mUf
 DevXv6k9cAvexZxU4cOUzpip4U3MGC0rJ1BNIgTs6eIeKq3geROTpuHjJFSBHZiP
 H/MqtoT4xTXJYomc9MiZG90fG9KyywlEF6e0GjVIcadJEmFIbJ+DfrAknVRjuaij
 ThMSIuvMEpXFhyghlApURePNi8W3FIIYHISw0JE/u7+4/7L/iYDToYeM/o563+8O
 zbj0n9fSewI=
 =oVZx
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2019-08-17' into staging

- Run the iotest during "make check"

# gpg: Signature made Sat 17 Aug 2019 09:46:13 BST
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2019-08-17:
  gitlab-ci: Remove qcow2 tests that are handled by "make check" already
  tests: Run the iotests during "make check" again
  block: fix NetBSD qemu-iotests failure

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-08-19 14:14:09 +01:00
Thomas Huth bdd95e4784 tests: Run the iotests during "make check" again
People often forget to run the iotests before submitting patches or pull
requests - this is likely due to the fact that we do not run the tests
during our mandatory "make check" tests yet. Now that we've got a proper
"auto" group of iotests that should be fine to run in every environment,
we can enable the iotests during "make check" again by running the "auto"
tests by default from the check-block.sh script.

Some cases still need to be checked first, though: iotests need bash and
GNU sed (otherwise they fail), and if gprof is enabled, it spoils the
output of some test cases causing them to fail. So if we detect that one
of the required programs is missing or that gprof is enabled, we still
have to skip the iotests to avoid failures.

And finally, since we are using check-block.sh now again, this patch also
removes the qemu-iotests-quick.sh script since we do not need that anymore
(and having two shell wrapper scripts around the block tests seems rather
confusing than helpful).

Message-Id: <20190717111947.30356-4-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
[AJB: -makecheck to check-block.sh, move check-block to start and gate it]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-08-17 09:03:35 +02:00
Vladimir Sementsov-Ogievskiy a5f8a60b3e tests/test-hbitmap: test next_zero and _next_dirty_area after truncate
Test that hbitmap_next_zero and hbitmap_next_dirty_area can find things
after old bitmap end.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20190805164652.42409-1-vsementsov@virtuozzo.com
Tested-by: John Snow <jsnow@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2019-08-16 18:29:43 -04:00
John Snow bd5ceebf6d iotests/257: test traditional sync modes
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190716000117.25219-12-jsnow@redhat.com
[Edit 'Bitmap' --> 'bitmap' in 257.out --js]
Signed-off-by: John Snow <jsnow@redhat.com>
2019-08-16 18:29:43 -04:00
John Snow 0fff1f1371 block/backup: improve sync=bitmap work estimates
When making backups based on bitmaps, the work estimate can be more
accurate. Update iotests to reflect the new strategy.

TOP work estimates are broken, but do not get worse with this commit.
That issue is addressed in the following commits instead.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190716000117.25219-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2019-08-16 16:28:03 -04:00
John Snow 352092d382 iotests/257: test API failures
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190716000117.25219-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2019-08-16 16:28:03 -04:00
John Snow 0af2a09c6b iotests/257: Refactor backup helpers
This test needs support for non-bitmap backups and missing or
unspecified bitmap sync modes, so rewrite the helpers to be a little
more generic.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190716000117.25219-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2019-08-16 16:28:03 -04:00
John Snow 32afa5a1d4 iotests/257: add EmulatedBitmap class
Represent a bitmap with an object that we can mark and clear bits in.
This makes it easier to manage partial writes when we don't write a
full group's worth of patterns before an error.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190716000117.25219-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2019-08-16 16:28:03 -04:00
John Snow b0a32bef7b iotests/257: add Pattern class
Just kidding, this is easier to manage with a full class instead of a
namedtuple.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190716000117.25219-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2019-08-16 16:28:03 -04:00
Vladimir Sementsov-Ogievskiy 3f7b2fa8cd iotests: test bitmap moving inside 254
Test persistent bitmap copying with and without removal of original
bitmap.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190708220502.12977-4-jsnow@redhat.com
[Edited comment "bitmap1" --> "bitmap2" as per review. --js]
Signed-off-by: John Snow <jsnow@redhat.com>
2019-08-16 16:28:03 -04:00
John Snow dfdc48d5e2 iotests: add test 257 for bitmap-mode backups
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190709232550.10724-18-jsnow@redhat.com
[Removed 'auto' group, as per new testing config guidelines --js]
Signed-off-by: John Snow <jsnow@redhat.com>
2019-08-16 16:28:02 -04:00
John Snow f357576fa9 iotests: Add virtio-scsi device helper
Seems that it comes up enough.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190709232550.10724-17-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2019-08-16 16:28:02 -04:00
John Snow de263986b5 iotests: teach FilePath to produce multiple paths
Use "FilePaths" instead of "FilePath" to request multiple files be
cleaned up after we leave that object's scope.

This is not crucial; but it saves a little typing.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190709232550.10724-16-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2019-08-16 16:28:02 -04:00
John Snow d443b74b3d iotests: teach run_job to cancel pending jobs
run_job can cancel pending jobs to simulate failure. This lets us use
the pending callback to issue test commands while the job is open, but
then still have the job fail in the end.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190709232550.10724-15-jsnow@redhat.com
[Maintainer edit: Merge conflict resolution in run_job]
Signed-off-by: John Snow <jsnow@redhat.com>
2019-08-16 16:28:02 -04:00
John Snow 456a2d5ac7 iotests: add testing shim for script-style python tests
Because the new-style python tests don't use the iotests.main() test
launcher, we don't turn on the debugger logging for these scripts
when invoked via ./check -d.

Refactor the launcher shim into new and old style shims so that they
share environmental configuration.

Two cleanup notes: debug was not actually used as a global, and there
was no reason to create a class in an inner scope just to achieve
default variables; we can simply create an instance of the runner with
the values we want instead.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190709232550.10724-14-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2019-08-16 16:28:02 -04:00
Peter Maydell e018ccb3fb Block layer patches:
- file-posix: Fix O_DIRECT alignment detection
 - Fixes for concurrent block jobs
 - block-backend: Queue requests while drained (fix IDE vs. job crashes)
 - qemu-img convert: Deprecate using -n and -o together
 - iotests: Migration tests with filter nodes
 - iotests: More media change tests
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJdVnduAAoJEH8JsnLIjy/W0IgQAKft/M3aDgt0sbTzQh8vdy6A
 yAfTnnSL4Z56+8qAsqhEnplC3rZxvTkg9AGOoNYHOZKl3FgRH9r8g9/Enemh4fWu
 MH52hiRf2ytlFVurIQal3aj9O+i0YTnzuvYbysvkH4ID5zbv2QnwdagtEcBxbbYL
 NZTMZBynDzp4rKIZ7p6T/kkaklLHh4vZrjW+Mzm3LQx9JJr8TwVNqqetSfc4VKIJ
 ByaNbbihDUVjQyIaJ24DXXJdzonGrrtSbSZycturc5FzXymzSRgrXZCeSKCs8X+i
 fjwMXH5v4/UfK511ILsXiumeuxBfD2Ck4sAblFxVo06oMPRNmsAKdRLeDByE7IC1
 lWep/pB3y/au9CW2/pkWJOiaz5s5iuv2fFYidKUJ0KQ1dD7G8M9rzkQlV3FUmTZO
 jBKSxHEffXsYl0ojn0vGmZEd7FAPi3fsZibGGws1dVgxlWI93aUJsjCq0E+lHIRD
 hEmQcjqZZa4taKpj0Y3Me05GkL7tH6RYA153jDNb8rPdzriGRCLZSObEISrOJf8H
 Mh0gTLi8KJNh6bULd12Ake1tKn7ZeTXpHH+gadz9OU7eIModh1qYTSHPlhy5oAv0
 Hm9BikNlS1Hzw+a+EbLcOW7TrsteNeGr7r8T6QKPMq1sfsYcp3svbC2c+zVlQ6Ll
 mLoTssksXOkgBevVqSiS
 =T7L5
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- file-posix: Fix O_DIRECT alignment detection
- Fixes for concurrent block jobs
- block-backend: Queue requests while drained (fix IDE vs. job crashes)
- qemu-img convert: Deprecate using -n and -o together
- iotests: Migration tests with filter nodes
- iotests: More media change tests

# gpg: Signature made Fri 16 Aug 2019 10:29:18 BST
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
  file-posix: Handle undetectable alignment
  qemu-img convert: Deprecate using -n and -o together
  block-backend: Queue requests while drained
  mirror: Keep mirror_top_bs drained after dropping permissions
  block: Remove blk_pread_unthrottled()
  iotests: Add test for concurrent stream/commit
  tests: Test mid-drain bdrv_replace_child_noperm()
  tests: Test polling in bdrv_drop_intermediate()
  block: Reduce (un)drains when replacing a child
  block: Keep subtree drained in drop_intermediate
  block: Simplify bdrv_filter_default_perms()
  iotests: Test migration with all kinds of filter nodes
  iotests: Move migration helpers to iotests.py
  iotests/118: Add -blockdev based tests
  iotests/118: Create test classes dynamically
  iotests/118: Test media change for scsi-cd

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-08-16 16:43:46 +01:00