OpenE2K
/
qemu-e2k
13
4
Fork 0
Code Issues 4 Pull Requests Projects 2 Releases Activity
qemu-e2k/tests/qemu-iotests/154

458 lines
19 KiB
Plaintext
Raw Permalink Normal View History

qemu-iotests: Improve portability by searching bash in the $PATH Bash is not always installed as /bin/bash. In particular on OpenBSD, the package installs it in /usr/local/bin. Use the 'env' shebang to search bash in the $PATH. Patch created mechanically by running: $ git grep -lE '#! ?/bin/bash' -- tests/qemu-iotests \ | while read f; do \ sed -i 's|^#!.\?/bin/bash$|#!/usr/bin/env bash|' $f; \ done Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-07 15:58:38 +01:00
#!/usr/bin/env bash
iotests: define group in each iotest We are going to drop group file. Define group in tests as a preparatory step. The patch is generated by cd tests/qemu-iotests grep '^[0-9]\{3\} ' group | while read line; do file=$(awk '{print $1}' <<< "$line"); groups=$(sed -e 's/^... //' <<< "$line"); awk "NR==2{print \"# group: $groups\"}1" $file > tmp; cat tmp > $file; done Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20210116134424.82867-7-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2021-01-16 14:44:19 +01:00
# group: rw auto backing quick
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
#
block: Switch bdrv_write_zeroes() to byte interface Rename to bdrv_pwrite_zeroes() to let the compiler ensure we cater to the updated semantics. Do the same for bdrv_co_write_zeroes(). Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2016-06-01 23:10:04 +02:00
# qcow2 specific bdrv_pwrite_zeroes tests with backing files (complements 034)
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
#
qcow2: Optimize write zero of unaligned tail cluster We've already improved discards to operate efficiently on the tail of an unaligned qcow2 image; it's time to make a similar improvement to write zeroes. The special case is only valid at the tail cluster of a file, where we must recognize that any sectors beyond the image end would implicitly read as zero, and therefore should not penalize our logic for widening a partial cluster into writing the whole cluster as zero. However, note that for now, the special case of end-of-file is only recognized if there is no backing file, or if the backing file has the same length; that's because when the backing file is shorter than the active layer, we don't have code in place to recognize that reads of a sector unallocated at the top and beyond the backing end-of-file are implicitly zero. It's not much of a real loss, because most people don't use images that aren't cluster-aligned, or where the active layer is a different size than the backing layer (especially where the difference falls within a single cluster). Update test 154 to cover the new scenarios, using two images of intentionally differing length. While at it, fix the test to gracefully skip when run as ./check -qcow2 -o compat=0.10 154 since the older format lacks zero clusters already required earlier in the test. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170507000552.20847-11-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-05-07 02:05:50 +02:00
# Copyright (C) 2016-2017 Red Hat, Inc.
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# creator
owner=kwolf@redhat.com
seq=`basename $0`
echo "QA output created by $seq"
status=1 # failure is the default!
_cleanup()
{
_cleanup_test_img
}
trap "_cleanup; exit \$status" 0 1 2 3 15
# get standard environment, filters and checks
. ./common.rc
. ./common.filter
_supported_fmt qcow2
iotests: Enable fuse for many tests Many tests (that do not support generic protocols) can run just fine with FUSE-exported images, so allow them to. Note that this is no attempt at being definitely complete. There are some tests that might be modified to run on FUSE, but this patch still skips them. This patch only tries to pick the rather low-hanging fruits. Note that 221 and 250 only pass when .lseek is correctly implemented, which is only possible with a libfuse that is 3.8 or newer. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20201027190600.192171-20-mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-10-27 20:05:59 +01:00
_supported_proto file fuse
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
_supported_os Linux
CLUSTER_SIZE=4k
qcow2: Optimize write zero of unaligned tail cluster We've already improved discards to operate efficiently on the tail of an unaligned qcow2 image; it's time to make a similar improvement to write zeroes. The special case is only valid at the tail cluster of a file, where we must recognize that any sectors beyond the image end would implicitly read as zero, and therefore should not penalize our logic for widening a partial cluster into writing the whole cluster as zero. However, note that for now, the special case of end-of-file is only recognized if there is no backing file, or if the backing file has the same length; that's because when the backing file is shorter than the active layer, we don't have code in place to recognize that reads of a sector unallocated at the top and beyond the backing end-of-file are implicitly zero. It's not much of a real loss, because most people don't use images that aren't cluster-aligned, or where the active layer is a different size than the backing layer (especially where the difference falls within a single cluster). Update test 154 to cover the new scenarios, using two images of intentionally differing length. While at it, fix the test to gracefully skip when run as ./check -qcow2 -o compat=0.10 154 since the older format lacks zero clusters already required earlier in the test. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170507000552.20847-11-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-05-07 02:05:50 +02:00
size=$((128 * 1024 * 1024))
# This test requires zero clusters, added in v3 images
_unsupported_imgopts compat=0.10
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
echo
echo == backing file contains zeros ==
CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
# Make sure that the whole cluster is allocated even for partial write_zeroes
# when the backing file contains zeros
# X = non-zero data sector in backing file
# - = sector unallocated in whole backing chain
# 0 = sector touched by write_zeroes request
# 1. Tail unaligned: 00 00 -- --
# 2. Head unaligned: -- -- 00 00
# 3. Both unaligned: -- 00 00 --
# 4. Both, 2 clusters: -- -- -- 00 | 00 -- -- --
$QEMU_IO -c "write -z 0 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "write -z 10k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "write -z 17k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "write -z 27k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
echo
echo == backing file contains non-zero data before write_zeroes ==
CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
# Single cluster; non-zero data at the cluster start
# ... | XX -- 00 -- | ...
$QEMU_IO -c "write -P 0x11 32k 1k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 34k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 32k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 33k 3k" "$TEST_IMG" | _filter_qemu_io
# Single cluster; non-zero data exists, but not at the cluster start
# ... | -- XX 00 -- | ...
$QEMU_IO -c "write -P 0x11 65k 1k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 66k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 65k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 64k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 66k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
echo
echo == backing file contains non-zero data after write_zeroes ==
CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
# Single cluster; non-zero data directly after request
# ... | -- 00 XX -- | ...
$QEMU_IO -c "write -P 0x11 34k 1k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 33k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 32k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 34k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 35k 1k" "$TEST_IMG" | _filter_qemu_io
# Single cluster; non-zero data exists, but not directly after request
# ... | -- 00 -- XX | ...
$QEMU_IO -c "write -P 0x11 43k 1k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 41k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 43k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 40k 3k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
qemu-iotests: Test one more spot for optimizing write_zeroes Add another test to 154, showing that we currently allocate a data cluster in the top layer if any sector of the backing file was allocated. The next patch will optimize this case. Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2016-05-26 05:48:48 +02:00
echo
echo == write_zeroes covers non-zero data ==
CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT
qemu-iotests: Test one more spot for optimizing write_zeroes Add another test to 154, showing that we currently allocate a data cluster in the top layer if any sector of the backing file was allocated. The next patch will optimize this case. Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2016-05-26 05:48:48 +02:00
# non-zero data at front of request
# Backing file: -- XX -- --
# Active layer: -- 00 00 --
$QEMU_IO -c "write -P 0x11 5k 1k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 5k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 4k 4k" "$TEST_IMG" | _filter_qemu_io
# non-zero data at end of request
# Backing file: -- -- XX --
# Active layer: -- 00 00 --
$QEMU_IO -c "write -P 0x11 14k 1k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 13k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 12k 4k" "$TEST_IMG" | _filter_qemu_io
# non-zero data matches size of request
# Backing file: -- XX XX --
# Active layer: -- 00 00 --
$QEMU_IO -c "write -P 0x11 21k 2k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 21k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 20k 4k" "$TEST_IMG" | _filter_qemu_io
# non-zero data smaller than request
# Backing file: -- -X X- --
# Active layer: -- 00 00 --
$QEMU_IO -c "write -P 0x11 30208 1k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 29k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 28k 4k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
echo
echo == spanning two clusters, non-zero before request ==
CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
# Two clusters; non-zero data before request:
# 1. At cluster start: 32k: XX -- -- 00 | 00 -- -- --
# 2. Between unallocated space: 48k: -- XX -- 00 | 00 -- -- --
# 3. Directly before request: 64k: -- -- XX 00 | 00 -- -- --
$QEMU_IO -c "write -P 0x11 32k 1k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 35k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 32k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 33k 7k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "write -P 0x11 49k 1k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 51k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 48k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 49k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 50k 6k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "write -P 0x11 66k 1k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 67k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 64k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 66k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 67k 5k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
echo
echo == spanning two clusters, non-zero after request ==
CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
# Two clusters; non-zero data after request:
# 1. Directly after request: 32k: -- -- -- 00 | 00 XX -- --
# 2. Between unallocated space: 48k: -- -- -- 00 | 00 -- XX --
# 3. At cluster end: 64k: -- -- -- 00 | 00 -- -- XX
$QEMU_IO -c "write -P 0x11 37k 1k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 35k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 32k 5k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 37k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 38k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "write -P 0x11 54k 1k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 51k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 48k 6k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 54k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 55k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "write -P 0x11 71k 1k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 67k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 64k 7k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 71k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
echo
echo == spanning two clusters, partially overwriting backing file ==
CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
# Backing file: -- -- XX XX | XX XX -- --
# Active layer: -- -- XX 00 | 00 XX -- --
$QEMU_IO -c "write -P 0x11 2k 4k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 3k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 0k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 2k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 3k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 5k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 6k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
echo
echo == spanning multiple clusters, non-zero in first cluster ==
CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
# Backing file: 64k: XX XX -- -- | -- -- -- -- | -- -- -- --
# Active layer: 64k: XX XX 00 00 | 00 00 00 00 | 00 -- -- --
$QEMU_IO -c "write -P 0x11 64k 2k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 66k 7k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 64k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 66k 10k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
echo
echo == spanning multiple clusters, non-zero in intermediate cluster ==
CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
# Backing file: 64k: -- -- -- -- | -- XX XX -- | -- -- -- --
# Active layer: 64k: -- -- 00 00 | 00 00 00 00 | 00 -- -- --
$QEMU_IO -c "write -P 0x11 69k 2k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 66k 7k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 64k 12k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
echo
echo == spanning multiple clusters, non-zero in final cluster ==
CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
# Backing file: 64k: -- -- -- -- | -- -- -- -- | -- -- XX XX
# Active layer: 64k: -- -- 00 00 | 00 00 00 00 | 00 -- XX XX
$QEMU_IO -c "write -P 0x11 74k 2k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 66k 7k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 64k 10k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 74k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
echo
echo == spanning multiple clusters, partially overwriting backing file ==
CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $size
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
# Backing file: 64k: -- XX XX XX | XX XX XX XX | XX XX XX --
# Active layer: 64k: -- XX 00 00 | 00 00 00 00 | 00 XX XX --
$QEMU_IO -c "write -P 0x11 65k 10k" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z 66k 7k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 64k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 65k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 66k 7k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0x11 73k 2k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 75k 1k" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
qcow2: Optimize write zero of unaligned tail cluster We've already improved discards to operate efficiently on the tail of an unaligned qcow2 image; it's time to make a similar improvement to write zeroes. The special case is only valid at the tail cluster of a file, where we must recognize that any sectors beyond the image end would implicitly read as zero, and therefore should not penalize our logic for widening a partial cluster into writing the whole cluster as zero. However, note that for now, the special case of end-of-file is only recognized if there is no backing file, or if the backing file has the same length; that's because when the backing file is shorter than the active layer, we don't have code in place to recognize that reads of a sector unallocated at the top and beyond the backing end-of-file are implicitly zero. It's not much of a real loss, because most people don't use images that aren't cluster-aligned, or where the active layer is a different size than the backing layer (especially where the difference falls within a single cluster). Update test 154 to cover the new scenarios, using two images of intentionally differing length. While at it, fix the test to gracefully skip when run as ./check -qcow2 -o compat=0.10 154 since the older format lacks zero clusters already required earlier in the test. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170507000552.20847-11-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-05-07 02:05:50 +02:00
echo
echo == unaligned image tail cluster, no allocation needed ==
# With no backing file, write to all or part of unallocated partial cluster
# will mark the cluster as zero, but does not allocate.
# Re-create the image each time to get back to unallocated clusters.
# Write at the front: sector-wise, the request is: 128m... | 00 -- -- --
_make_test_img $((size + 2048))
$QEMU_IO -c "write -z $size 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "alloc $size 2048" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
# Write at the back: sector-wise, the request is: 128m... | -- -- -- 00
_make_test_img $((size + 2048))
$QEMU_IO -c "write -z $((size + 1536)) 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "alloc $size 2048" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
# Write at middle: sector-wise, the request is: 128m... | -- 00 00 --
_make_test_img $((size + 2048))
$QEMU_IO -c "write -z $((size + 512)) 1024" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "alloc $size 2048" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
# Write entire cluster: sector-wise, the request is: 128m... | 00 00 00 00
_make_test_img $((size + 2048))
$QEMU_IO -c "write -z $size 2048" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "alloc $size 2048" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
# Repeat with backing file holding unallocated cluster.
CLUSTER_SIZE=2048 TEST_IMG="$TEST_IMG.base" _make_test_img $((size + 1024))
# Write at the front: sector-wise, the request is:
# backing: 128m... | -- --
# active: 128m... | 00 -- -- --
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT $((size + 2048))
qcow2: Optimize write zero of unaligned tail cluster We've already improved discards to operate efficiently on the tail of an unaligned qcow2 image; it's time to make a similar improvement to write zeroes. The special case is only valid at the tail cluster of a file, where we must recognize that any sectors beyond the image end would implicitly read as zero, and therefore should not penalize our logic for widening a partial cluster into writing the whole cluster as zero. However, note that for now, the special case of end-of-file is only recognized if there is no backing file, or if the backing file has the same length; that's because when the backing file is shorter than the active layer, we don't have code in place to recognize that reads of a sector unallocated at the top and beyond the backing end-of-file are implicitly zero. It's not much of a real loss, because most people don't use images that aren't cluster-aligned, or where the active layer is a different size than the backing layer (especially where the difference falls within a single cluster). Update test 154 to cover the new scenarios, using two images of intentionally differing length. While at it, fix the test to gracefully skip when run as ./check -qcow2 -o compat=0.10 154 since the older format lacks zero clusters already required earlier in the test. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170507000552.20847-11-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-05-07 02:05:50 +02:00
$QEMU_IO -c "write -z $size 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "alloc $size 2048" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
# Write at the back: sector-wise, the request is:
# backing: 128m... | -- --
# active: 128m... | -- -- -- 00
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT $((size + 2048))
qcow2: Optimize write zero of unaligned tail cluster We've already improved discards to operate efficiently on the tail of an unaligned qcow2 image; it's time to make a similar improvement to write zeroes. The special case is only valid at the tail cluster of a file, where we must recognize that any sectors beyond the image end would implicitly read as zero, and therefore should not penalize our logic for widening a partial cluster into writing the whole cluster as zero. However, note that for now, the special case of end-of-file is only recognized if there is no backing file, or if the backing file has the same length; that's because when the backing file is shorter than the active layer, we don't have code in place to recognize that reads of a sector unallocated at the top and beyond the backing end-of-file are implicitly zero. It's not much of a real loss, because most people don't use images that aren't cluster-aligned, or where the active layer is a different size than the backing layer (especially where the difference falls within a single cluster). Update test 154 to cover the new scenarios, using two images of intentionally differing length. While at it, fix the test to gracefully skip when run as ./check -qcow2 -o compat=0.10 154 since the older format lacks zero clusters already required earlier in the test. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170507000552.20847-11-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-05-07 02:05:50 +02:00
$QEMU_IO -c "write -z $((size + 1536)) 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "alloc $size 2048" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
# Write at middle: sector-wise, the request is:
# backing: 128m... | -- --
# active: 128m... | -- 00 00 --
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT $((size + 2048))
qcow2: Optimize write zero of unaligned tail cluster We've already improved discards to operate efficiently on the tail of an unaligned qcow2 image; it's time to make a similar improvement to write zeroes. The special case is only valid at the tail cluster of a file, where we must recognize that any sectors beyond the image end would implicitly read as zero, and therefore should not penalize our logic for widening a partial cluster into writing the whole cluster as zero. However, note that for now, the special case of end-of-file is only recognized if there is no backing file, or if the backing file has the same length; that's because when the backing file is shorter than the active layer, we don't have code in place to recognize that reads of a sector unallocated at the top and beyond the backing end-of-file are implicitly zero. It's not much of a real loss, because most people don't use images that aren't cluster-aligned, or where the active layer is a different size than the backing layer (especially where the difference falls within a single cluster). Update test 154 to cover the new scenarios, using two images of intentionally differing length. While at it, fix the test to gracefully skip when run as ./check -qcow2 -o compat=0.10 154 since the older format lacks zero clusters already required earlier in the test. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170507000552.20847-11-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-05-07 02:05:50 +02:00
$QEMU_IO -c "write -z $((size + 512)) 1024" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "alloc $size 2048" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
# Write entire cluster: sector-wise, the request is:
# backing: 128m... | -- --
# active: 128m... | 00 00 00 00
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT $((size + 2048))
qcow2: Optimize write zero of unaligned tail cluster We've already improved discards to operate efficiently on the tail of an unaligned qcow2 image; it's time to make a similar improvement to write zeroes. The special case is only valid at the tail cluster of a file, where we must recognize that any sectors beyond the image end would implicitly read as zero, and therefore should not penalize our logic for widening a partial cluster into writing the whole cluster as zero. However, note that for now, the special case of end-of-file is only recognized if there is no backing file, or if the backing file has the same length; that's because when the backing file is shorter than the active layer, we don't have code in place to recognize that reads of a sector unallocated at the top and beyond the backing end-of-file are implicitly zero. It's not much of a real loss, because most people don't use images that aren't cluster-aligned, or where the active layer is a different size than the backing layer (especially where the difference falls within a single cluster). Update test 154 to cover the new scenarios, using two images of intentionally differing length. While at it, fix the test to gracefully skip when run as ./check -qcow2 -o compat=0.10 154 since the older format lacks zero clusters already required earlier in the test. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170507000552.20847-11-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-05-07 02:05:50 +02:00
$QEMU_IO -c "write -z $size 2048" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "alloc $size 2048" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
# Repeat with backing file holding zero'd cluster
$QEMU_IO -c "write -z $size 512" "$TEST_IMG.base" | _filter_qemu_io
# Write at the front: sector-wise, the request is:
# backing: 128m... | 00 00
# active: 128m... | 00 -- -- --
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT $((size + 2048))
qcow2: Optimize write zero of unaligned tail cluster We've already improved discards to operate efficiently on the tail of an unaligned qcow2 image; it's time to make a similar improvement to write zeroes. The special case is only valid at the tail cluster of a file, where we must recognize that any sectors beyond the image end would implicitly read as zero, and therefore should not penalize our logic for widening a partial cluster into writing the whole cluster as zero. However, note that for now, the special case of end-of-file is only recognized if there is no backing file, or if the backing file has the same length; that's because when the backing file is shorter than the active layer, we don't have code in place to recognize that reads of a sector unallocated at the top and beyond the backing end-of-file are implicitly zero. It's not much of a real loss, because most people don't use images that aren't cluster-aligned, or where the active layer is a different size than the backing layer (especially where the difference falls within a single cluster). Update test 154 to cover the new scenarios, using two images of intentionally differing length. While at it, fix the test to gracefully skip when run as ./check -qcow2 -o compat=0.10 154 since the older format lacks zero clusters already required earlier in the test. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170507000552.20847-11-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-05-07 02:05:50 +02:00
$QEMU_IO -c "write -z $size 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "alloc $size 2048" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
# Write at the back: sector-wise, the request is:
# backing: 128m... | 00 00
# active: 128m... | -- -- -- 00
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT $((size + 2048))
qcow2: Optimize write zero of unaligned tail cluster We've already improved discards to operate efficiently on the tail of an unaligned qcow2 image; it's time to make a similar improvement to write zeroes. The special case is only valid at the tail cluster of a file, where we must recognize that any sectors beyond the image end would implicitly read as zero, and therefore should not penalize our logic for widening a partial cluster into writing the whole cluster as zero. However, note that for now, the special case of end-of-file is only recognized if there is no backing file, or if the backing file has the same length; that's because when the backing file is shorter than the active layer, we don't have code in place to recognize that reads of a sector unallocated at the top and beyond the backing end-of-file are implicitly zero. It's not much of a real loss, because most people don't use images that aren't cluster-aligned, or where the active layer is a different size than the backing layer (especially where the difference falls within a single cluster). Update test 154 to cover the new scenarios, using two images of intentionally differing length. While at it, fix the test to gracefully skip when run as ./check -qcow2 -o compat=0.10 154 since the older format lacks zero clusters already required earlier in the test. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170507000552.20847-11-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-05-07 02:05:50 +02:00
$QEMU_IO -c "write -z $((size + 1536)) 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "alloc $size 2048" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
# Write at middle: sector-wise, the request is:
# backing: 128m... | 00 00
# active: 128m... | -- 00 00 --
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT $((size + 2048))
qcow2: Optimize write zero of unaligned tail cluster We've already improved discards to operate efficiently on the tail of an unaligned qcow2 image; it's time to make a similar improvement to write zeroes. The special case is only valid at the tail cluster of a file, where we must recognize that any sectors beyond the image end would implicitly read as zero, and therefore should not penalize our logic for widening a partial cluster into writing the whole cluster as zero. However, note that for now, the special case of end-of-file is only recognized if there is no backing file, or if the backing file has the same length; that's because when the backing file is shorter than the active layer, we don't have code in place to recognize that reads of a sector unallocated at the top and beyond the backing end-of-file are implicitly zero. It's not much of a real loss, because most people don't use images that aren't cluster-aligned, or where the active layer is a different size than the backing layer (especially where the difference falls within a single cluster). Update test 154 to cover the new scenarios, using two images of intentionally differing length. While at it, fix the test to gracefully skip when run as ./check -qcow2 -o compat=0.10 154 since the older format lacks zero clusters already required earlier in the test. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170507000552.20847-11-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-05-07 02:05:50 +02:00
$QEMU_IO -c "write -z $((size + 512)) 1024" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "alloc $size 2048" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
# Write entire cluster: sector-wise, the request is:
# backing: 128m... | 00 00
# active: 128m... | 00 00 00 00
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT $((size + 2048))
qcow2: Optimize write zero of unaligned tail cluster We've already improved discards to operate efficiently on the tail of an unaligned qcow2 image; it's time to make a similar improvement to write zeroes. The special case is only valid at the tail cluster of a file, where we must recognize that any sectors beyond the image end would implicitly read as zero, and therefore should not penalize our logic for widening a partial cluster into writing the whole cluster as zero. However, note that for now, the special case of end-of-file is only recognized if there is no backing file, or if the backing file has the same length; that's because when the backing file is shorter than the active layer, we don't have code in place to recognize that reads of a sector unallocated at the top and beyond the backing end-of-file are implicitly zero. It's not much of a real loss, because most people don't use images that aren't cluster-aligned, or where the active layer is a different size than the backing layer (especially where the difference falls within a single cluster). Update test 154 to cover the new scenarios, using two images of intentionally differing length. While at it, fix the test to gracefully skip when run as ./check -qcow2 -o compat=0.10 154 since the older format lacks zero clusters already required earlier in the test. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170507000552.20847-11-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-05-07 02:05:50 +02:00
$QEMU_IO -c "write -z $size 2048" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "alloc $size 2048" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
# A preallocated cluster maintains its allocation, whether it stays as
# data due to a partial write:
# Convert 128m... | XX XX => ... | XX 00
_make_test_img $((size + 1024))
$QEMU_IO -c "write -P 1 $((size)) 1024" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "write -z $((size + 512)) 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 1 $((size)) 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 $((size + 512)) 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "alloc $size 1024" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
# or because it is the entire cluster and can use the zero flag:
# Convert 128m... | XX XX => ... | 00 00
$QEMU_IO -c "write -z $((size)) 1024" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "alloc $size 1024" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 $size 1024" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
echo
echo == unaligned image tail cluster, allocation required ==
# Write beyond backing file must COW
# Backing file: 128m... | XX --
# Active layer: 128m... | -- -- 00 --
CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $((size + 1024))
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT $((size + 2048))
qcow2: Optimize write zero of unaligned tail cluster We've already improved discards to operate efficiently on the tail of an unaligned qcow2 image; it's time to make a similar improvement to write zeroes. The special case is only valid at the tail cluster of a file, where we must recognize that any sectors beyond the image end would implicitly read as zero, and therefore should not penalize our logic for widening a partial cluster into writing the whole cluster as zero. However, note that for now, the special case of end-of-file is only recognized if there is no backing file, or if the backing file has the same length; that's because when the backing file is shorter than the active layer, we don't have code in place to recognize that reads of a sector unallocated at the top and beyond the backing end-of-file are implicitly zero. It's not much of a real loss, because most people don't use images that aren't cluster-aligned, or where the active layer is a different size than the backing layer (especially where the difference falls within a single cluster). Update test 154 to cover the new scenarios, using two images of intentionally differing length. While at it, fix the test to gracefully skip when run as ./check -qcow2 -o compat=0.10 154 since the older format lacks zero clusters already required earlier in the test. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170507000552.20847-11-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-05-07 02:05:50 +02:00
$QEMU_IO -c "write -P 1 $((size)) 512" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z $((size + 1024)) 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 1 $((size)) 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 $((size + 512)) 1536" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
# Writes at boundaries of (partial) cluster must not lose mid-cluster data
# Backing file: 128m: ... | -- XX
# Active layer: 128m: ... | 00 -- -- 00
CLUSTER_SIZE=512 TEST_IMG="$TEST_IMG.base" _make_test_img $((size + 1024))
iotests: Specify explicit backing format where sensible There are many existing qcow2 images that specify a backing file but no format. This has been the source of CVEs in the past, but has become more prominent of a problem now that libvirt has switched to -blockdev. With older -drive, at least the probing was always done by qemu (so the only risk of a changed format between successive boots of a guest was if qemu was upgraded and probed differently). But with newer -blockdev, libvirt must specify a format; if libvirt guesses raw where the image was formatted, this results in data corruption visible to the guest; conversely, if libvirt guesses qcow2 where qemu was using raw, this can result in potential security holes, so modern libvirt instead refuses to use images without explicit backing format. The change in libvirt to reject images without explicit backing format has pointed out that a number of tools have been far too reliant on probing in the past. It's time to set a better example in our own iotests of properly setting this parameter. iotest calls to create, rebase, and convert are all impacted to some degree. It's a bit annoying that we are inconsistent on command line - while all of those accept -o backing_file=...,backing_fmt=..., the shortcuts are different: create and rebase have -b and -F, while convert has -B but no -F. (amend has no shortcuts, but the previous patch just deprecated the use of amend to change backing chains). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-9-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 22:39:52 +02:00
_make_test_img -b "$TEST_IMG.base" -F $IMGFMT $((size + 2048))
qcow2: Optimize write zero of unaligned tail cluster We've already improved discards to operate efficiently on the tail of an unaligned qcow2 image; it's time to make a similar improvement to write zeroes. The special case is only valid at the tail cluster of a file, where we must recognize that any sectors beyond the image end would implicitly read as zero, and therefore should not penalize our logic for widening a partial cluster into writing the whole cluster as zero. However, note that for now, the special case of end-of-file is only recognized if there is no backing file, or if the backing file has the same length; that's because when the backing file is shorter than the active layer, we don't have code in place to recognize that reads of a sector unallocated at the top and beyond the backing end-of-file are implicitly zero. It's not much of a real loss, because most people don't use images that aren't cluster-aligned, or where the active layer is a different size than the backing layer (especially where the difference falls within a single cluster). Update test 154 to cover the new scenarios, using two images of intentionally differing length. While at it, fix the test to gracefully skip when run as ./check -qcow2 -o compat=0.10 154 since the older format lacks zero clusters already required earlier in the test. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20170507000552.20847-11-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-05-07 02:05:50 +02:00
$QEMU_IO -c "write -P 1 $((size + 512)) 512" "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -z $((size)) 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 $((size)) 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 1 $((size + 512)) 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 $((size + 1024)) 1024" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "write -z $((size + 1536)) 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 $((size)) 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 1 $((size + 512)) 512" "$TEST_IMG" | _filter_qemu_io
$QEMU_IO -c "read -P 0 $((size + 1024)) 1024" "$TEST_IMG" | _filter_qemu_io
$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
qemu-iotests: Some more write_zeroes tests This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons: We check the allocation status after write_zeroes with 'qemu-img map' and the optimised behaviour that produces zero clusters is only implemented in qcow2; second, the map command returns offsets that are qcow2 specific; and finally, we also use 512 byte clusters which aren't supported for formats like qed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-17 18:37:39 +02:00
# success, all done
echo "*** done"
rm -f $seq.full
status=0