2021-07-09 17:39:49 +02:00
|
|
|
QA output created by qemu-img-bitmaps
|
2020-05-21 21:21:37 +02:00
|
|
|
|
|
|
|
|
=== Initial image setup ===
|
|
|
|
|
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=10485760
|
|
|
|
|
wrote 1048576/1048576 bytes at offset 3145728
|
|
|
|
|
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=10485760 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
|
|
|
|
|
wrote 1048576/1048576 bytes at offset 0
|
|
|
|
|
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
wrote 1048576/1048576 bytes at offset 3145728
|
|
|
|
|
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
wrote 1048576/1048576 bytes at offset 1048576
|
|
|
|
|
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
wrote 1048576/1048576 bytes at offset 2097152
|
|
|
|
|
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
|
|
|
|
|
=== Bitmap preservation not possible to non-qcow2 ===
|
|
|
|
|
|
|
|
|
|
qemu-img: Format driver 'raw' does not support bitmaps
|
|
|
|
|
|
|
|
|
|
=== Convert with bitmap preservation ===
|
|
|
|
|
|
|
|
|
|
image: TEST_DIR/t.IMGFMT
|
|
|
|
|
file format: IMGFMT
|
|
|
|
|
virtual size: 10 MiB (10485760 bytes)
|
2020-06-08 21:56:29 +02:00
|
|
|
cluster_size: 65536
|
2020-05-21 21:21:37 +02:00
|
|
|
Format specific information:
|
|
|
|
|
bitmaps:
|
|
|
|
|
[0]:
|
|
|
|
|
flags:
|
|
|
|
|
name: b1
|
|
|
|
|
granularity: 524288
|
|
|
|
|
[1]:
|
|
|
|
|
flags:
|
|
|
|
|
[0]: auto
|
|
|
|
|
name: b2
|
|
|
|
|
granularity: 65536
|
|
|
|
|
corrupt: false
|
|
|
|
|
image: TEST_DIR/t.IMGFMT
|
|
|
|
|
file format: IMGFMT
|
|
|
|
|
virtual size: 10 MiB (10485760 bytes)
|
2020-06-08 21:56:29 +02:00
|
|
|
cluster_size: 65536
|
2020-05-21 21:21:37 +02:00
|
|
|
Format specific information:
|
|
|
|
|
bitmaps:
|
|
|
|
|
[0]:
|
|
|
|
|
flags:
|
|
|
|
|
name: b1
|
|
|
|
|
granularity: 524288
|
|
|
|
|
[1]:
|
|
|
|
|
flags:
|
|
|
|
|
[0]: auto
|
|
|
|
|
name: b2
|
|
|
|
|
granularity: 65536
|
|
|
|
|
[2]:
|
|
|
|
|
flags:
|
|
|
|
|
name: b0
|
|
|
|
|
granularity: 65536
|
|
|
|
|
corrupt: false
|
|
|
|
|
|
qemu-img: Support bitmap --merge into backing image
If you have the chain 'base.qcow2 <- top.qcow2' and want to merge a
bitmap from top into base, qemu-img was failing with:
qemu-img: Could not open 'top.qcow2': Could not open backing file: Failed to get shared "write" lock
Is another process using the image [base.qcow2]?
The easiest fix is to not open the entire backing chain of either
image (source or destination); after all, the point of 'qemu-img
bitmap' is solely to manipulate bitmaps directly within a single qcow2
image, and this is made more precise if we don't pay attention to
other images in the chain that may happen to have a bitmap by the same
name.
However, note that on a case-by-case analysis, there _are_ times where
we treat it as a feature that we can access a bitmap from a backing
layer in association with an overlay BDS. A demonstration of this is
using NBD to expose both an overlay BDS (for constant contents) and a
bitmap (for learning which blocks are interesting) during an
incremental backup:
Base <- Active <- Temporary
\--block job ->/
where Temporary is being fed by a backup 'sync=none' job. When
exposing Temporary over NBD, referring to a bitmap that lives only in
Active is less effort than having to copy a bitmap into Temporary [1].
So the testsuite additions in this patch check both where bitmaps get
allocated (the qemu-img info output), and that qemu-nbd is indeed able
to access a bitmap inherited from the backing chain since it is a
different use case than 'qemu-img bitmap'.
[1] Full disclosure: prior to the recent commit 374eedd1c4 and
friends, we were NOT able to see bitmaps through filters, which meant
that we actually did not have nice clean semantics for uniformly being
able to pick up bitmaps from anywhere in the backing chain (seen as a
change in behavior between qemu 4.1 and 4.2 at commit 00e30f05de, when
block-copy swapped from a one-off to a filter). Which means libvirt
was already coded to copy bitmaps around for the sake of older qemu,
even though modern qemu no longer needs it. Oh well.
Fixes: http://bugzilla.redhat.com/1877209
Reported-by: Eyal Shenitzky <eshenitz@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200914191009.644842-1-eblake@redhat.com>
[eblake: more commit message tweaks, per Max Reitz review]
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2020-09-14 21:10:09 +02:00
|
|
|
=== Merge from top layer into backing image ===
|
|
|
|
|
|
|
|
|
|
image: TEST_DIR/t.IMGFMT
|
|
|
|
|
file format: IMGFMT
|
|
|
|
|
virtual size: 10 MiB (10485760 bytes)
|
|
|
|
|
cluster_size: 65536
|
|
|
|
|
backing file: TEST_DIR/t.IMGFMT.base
|
|
|
|
|
backing file format: IMGFMT
|
|
|
|
|
Format specific information:
|
|
|
|
|
bitmaps:
|
|
|
|
|
[0]:
|
|
|
|
|
flags:
|
|
|
|
|
name: b1
|
|
|
|
|
granularity: 524288
|
|
|
|
|
[1]:
|
|
|
|
|
flags:
|
|
|
|
|
[0]: auto
|
|
|
|
|
name: b2
|
|
|
|
|
granularity: 65536
|
|
|
|
|
[2]:
|
|
|
|
|
flags:
|
|
|
|
|
name: b0
|
|
|
|
|
granularity: 65536
|
|
|
|
|
corrupt: false
|
|
|
|
|
|
|
|
|
|
image: TEST_DIR/t.IMGFMT.base
|
|
|
|
|
file format: IMGFMT
|
|
|
|
|
virtual size: 10 MiB (10485760 bytes)
|
|
|
|
|
cluster_size: 65536
|
|
|
|
|
Format specific information:
|
|
|
|
|
bitmaps:
|
|
|
|
|
[0]:
|
|
|
|
|
flags:
|
|
|
|
|
[0]: auto
|
|
|
|
|
name: b0
|
|
|
|
|
granularity: 65536
|
|
|
|
|
[1]:
|
|
|
|
|
flags:
|
|
|
|
|
[0]: auto
|
|
|
|
|
name: b3
|
|
|
|
|
granularity: 65536
|
|
|
|
|
corrupt: false
|
|
|
|
|
|
2020-05-21 21:21:37 +02:00
|
|
|
=== Check bitmap contents ===
|
|
|
|
|
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 21:06:55 +02:00
|
|
|
[{ "start": 0, "length": 3145728, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET},
|
|
|
|
|
{ "start": 3145728, "length": 1048576, "depth": 0, "present": false, "zero": false, "data": false},
|
|
|
|
|
{ "start": 4194304, "length": 6291456, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}]
|
|
|
|
|
[{ "start": 0, "length": 1048576, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET},
|
|
|
|
|
{ "start": 1048576, "length": 1048576, "depth": 0, "present": false, "zero": false, "data": false},
|
|
|
|
|
{ "start": 2097152, "length": 8388608, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}]
|
|
|
|
|
[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET},
|
|
|
|
|
{ "start": 2097152, "length": 1048576, "depth": 0, "present": false, "zero": false, "data": false},
|
|
|
|
|
{ "start": 3145728, "length": 7340032, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}]
|
|
|
|
|
[{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET},
|
|
|
|
|
{ "start": 2097152, "length": 1048576, "depth": 0, "present": false, "zero": false, "data": false},
|
|
|
|
|
{ "start": 3145728, "length": 7340032, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}]
|
2021-07-09 17:39:49 +02:00
|
|
|
|
|
|
|
|
=== Check handling of inconsistent bitmap ===
|
|
|
|
|
|
|
|
|
|
image: TEST_DIR/t.IMGFMT
|
|
|
|
|
file format: IMGFMT
|
|
|
|
|
virtual size: 10 MiB (10485760 bytes)
|
|
|
|
|
cluster_size: 65536
|
|
|
|
|
backing file: TEST_DIR/t.IMGFMT.base
|
|
|
|
|
backing file format: IMGFMT
|
|
|
|
|
Format specific information:
|
|
|
|
|
bitmaps:
|
|
|
|
|
[0]:
|
|
|
|
|
flags:
|
|
|
|
|
[0]: in-use
|
|
|
|
|
[1]: auto
|
|
|
|
|
name: b2
|
|
|
|
|
granularity: 65536
|
|
|
|
|
[1]:
|
|
|
|
|
flags:
|
|
|
|
|
[0]: in-use
|
|
|
|
|
name: b0
|
|
|
|
|
granularity: 65536
|
|
|
|
|
[2]:
|
|
|
|
|
flags:
|
|
|
|
|
[0]: auto
|
|
|
|
|
name: b4
|
|
|
|
|
granularity: 65536
|
|
|
|
|
corrupt: false
|
|
|
|
|
|
|
|
|
|
qemu-img: Failed to populate bitmap b0: Bitmap 'b0' is inconsistent and cannot be used
|
|
|
|
|
Try block-dirty-bitmap-remove to delete this bitmap from disk
|
|
|
|
|
image: TEST_DIR/t.IMGFMT.copy
|
|
|
|
|
file format: IMGFMT
|
|
|
|
|
virtual size: 10 MiB (10485760 bytes)
|
|
|
|
|
cluster_size: 65536
|
|
|
|
|
Format specific information:
|
|
|
|
|
bitmaps:
|
|
|
|
|
[0]:
|
|
|
|
|
flags:
|
|
|
|
|
name: b0
|
|
|
|
|
granularity: 65536
|
|
|
|
|
[1]:
|
|
|
|
|
flags:
|
|
|
|
|
[0]: auto
|
|
|
|
|
name: b4
|
|
|
|
|
granularity: 65536
|
|
|
|
|
corrupt: false
|
2020-05-21 21:21:37 +02:00
|
|
|
*** done
|