qemu-e2k/block
Vladimir Sementsov-Ogievskiy af5bcd775f block: copy-before-write: realize snapshot-access API
Current scheme of image fleecing looks like this:

[guest]                    [NBD export]
  |                              |
  |root                          | root
  v                              v
[copy-before-write] -----> [temp.qcow2]
  |                 target  |
  |file                     |backing
  v                         |
[active disk] <-------------+

 - On guest writes copy-before-write filter copies old data from active
   disk to temp.qcow2. So fleecing client (NBD export) when reads
   changed regions from temp.qcow2 image and unchanged from active disk
   through backing link.

This patch makes possible new image fleecing scheme:

[guest]                   [NBD export]
   |                            |
   | root                       | root
   v                 file       v
[copy-before-write]<------[snapshot-access]
   |           |
   | file      | target
   v           v
[active-disk] [temp.img]

 - copy-before-write does CBW operations and also provides
   snapshot-access API. The API may be accessed through
   snapshot-access driver.

Benefits of new scheme:

1. Access control: if remote client try to read data that not covered
   by original dirty bitmap used on copy-before-write open, client gets
   -EACCES.

2. Discard support: if remote client do DISCARD, this additionally to
   discarding data in temp.img informs block-copy process to not copy
   these clusters. Next read from discarded area will return -EACCES.
   This is significant thing: when fleecing user reads data that was
   not yet copied to temp.img, we can avoid copying it on further guest
   write.

3. Synchronisation between client reads and block-copy write is more
   efficient. In old scheme we just rely on BDRV_REQ_SERIALISING flag
   used for writes to temp.qcow2. New scheme is less blocking:
     - fleecing reads are never blocked: if data region is untouched or
       in-flight, we just read from active-disk, otherwise we read from
       temp.img
     - writes to temp.img are not blocked by fleecing reads
     - still, guest writes of-course are blocked by in-flight fleecing
       reads, that currently read from active-disk - it's the minimum
       necessary blocking

4. Temporary image may be of any format, as we don't rely on backing
   feature.

5. Permission relation are simplified. With old scheme we have to share
   write permission on target child of copy-before-write, otherwise
   backing link conflicts with copy-before-write file child write
   permissions. With new scheme we don't have backing link, and
   copy-before-write node may have unshared access to temporary node.
   (Not realized in this commit, will be in future).

6. Having control on fleecing reads we'll be able to implement
   alternative behavior on failed copy-before-write operations.
   Currently we just break guest request (that's a historical behavior
   of backup). But in some scenarios it's a bad behavior: better
   is to drop the backup as failed but don't break guest request.
   With new scheme we can simply unset some bits in a bitmap on CBW
   failure and further fleecing reads will -EACCES, or something like
   this. (Not implemented in this commit, will be in future)
   Additional application for this is implementing timeout for CBW
   operations.

Iotest 257 output is updated, as two more bitmaps now live in
copy-before-write filter.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20220303194349.2304213-13-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-03-07 09:33:31 +01:00
..
export block/export/fuse.c: allow writable exports to take RESIZE permission 2022-03-04 18:18:25 +01:00
monitor block/dirty-bitmap: bdrv_merge_dirty_bitmap(): add return value 2022-03-07 09:33:30 +01:00
accounting.c block/accounting: Use lock guard macros 2020-12-11 17:52:39 +01:00
aio_task.c block/aio_task: assert `max_busy_tasks` is greater than 0 2021-10-05 18:56:41 +02:00
amend.c block/amend: Keep strong reference to BDS 2022-03-04 18:18:26 +01:00
backup.c assertions for block_int global state API 2022-03-04 18:18:25 +01:00
blkdebug.c block: use int64_t instead of int in driver discard handlers 2021-09-29 13:46:32 -05:00
blklogwrites.c block: use int64_t instead of int in driver discard handlers 2021-09-29 13:46:32 -05:00
blkreplay.c block: use int64_t instead of int in driver discard handlers 2021-09-29 13:46:32 -05:00
blkverify.c block: use int64_t instead of uint64_t in driver write handlers 2021-09-29 13:46:31 -05:00
block-backend.c block/coroutines: I/O and "I/O or GS" API 2022-03-04 18:18:25 +01:00
block-copy.c block: intoduce reqlist 2022-03-07 09:33:30 +01:00
block-gen.h scripts: add block-coroutine-wrapper.py 2020-10-05 10:59:06 +01:00
bochs.c block: use int64_t instead of uint64_t in driver read handlers 2021-09-29 13:46:31 -05:00
cloop.c block: use int64_t instead of uint64_t in driver read handlers 2021-09-29 13:46:31 -05:00
commit.c assertions for block_int global state API 2022-03-04 18:18:25 +01:00
copy-before-write.c block: copy-before-write: realize snapshot-access API 2022-03-07 09:33:31 +01:00
copy-before-write.h block/copy-before-write.h: global state API + assertions 2022-03-04 18:18:25 +01:00
copy-on-read.c block: use int64_t instead of int in driver discard handlers 2021-09-29 13:46:32 -05:00
copy-on-read.h copy-on-read: add filter drop function 2021-01-26 11:26:54 +01:00
coroutines.h block/coroutines: I/O and "I/O or GS" API 2022-03-04 18:18:25 +01:00
create.c block_int-common.h: assertions in the callers of BlockDriver function pointers 2022-03-04 18:18:25 +01:00
crypto.c crypto: distinguish between main loop and I/O in block_crypto_amend_options_generic_luks 2022-03-04 18:14:40 +01:00
crypto.h nomaintainer: Fix Lesser GPL version number 2020-11-15 17:04:40 +01:00
curl.c block/curl.c: Check error return from curl_easy_setopt() 2022-03-07 09:19:20 +01:00
dirty-bitmap.c block/dirty-bitmap: introduce bdrv_dirty_bitmap_status() 2022-03-07 09:33:30 +01:00
dmg-bz2.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
dmg-lzfse.c block: Remove unused include 2020-11-09 15:44:21 +01:00
dmg.c block: use int64_t instead of uint64_t in driver read handlers 2021-09-29 13:46:31 -05:00
dmg.h Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
file-posix.c block/file-posix: Simplify the XFS_IOC_DIOINFO handling 2022-01-12 14:09:04 +01:00
file-win32.c block: use int64_t instead of uint64_t in driver write handlers 2021-09-29 13:46:31 -05:00
filter-compress.c block: use int64_t instead of int in driver discard handlers 2021-09-29 13:46:32 -05:00
gluster.c block: use int64_t instead of int in driver discard handlers 2021-09-29 13:46:32 -05:00
io.c block/io: introduce block driver snapshot-access API 2022-03-07 09:33:31 +01:00
io_uring.c aio-posix: split poll check from ready handler 2022-01-12 17:09:39 +00:00
iscsi-opts.c modules: add block module annotations 2021-07-09 18:20:27 +02:00
iscsi.c aio-posix: split poll check from ready handler 2022-01-12 17:09:39 +00:00
linux-aio.c aio-posix: split poll check from ready handler 2022-01-12 17:09:39 +00:00
meson.build block: introduce snapshot-access block driver 2022-03-07 09:33:31 +01:00
mirror.c assertions for block_int global state API 2022-03-04 18:18:25 +01:00
nbd.c block/coroutines: I/O and "I/O or GS" API 2022-03-04 18:18:25 +01:00
nfs.c aio-posix: split poll check from ready handler 2022-01-12 17:09:39 +00:00
null.c block: use int64_t instead of uint64_t in driver write handlers 2021-09-29 13:46:31 -05:00
nvme.c aio-posix: split poll check from ready handler 2022-01-12 17:09:39 +00:00
parallels-ext.c parallels: support bitmap extension for read-only mode 2021-03-08 14:56:55 +01:00
parallels.c block: introduce bdrv_activate 2022-03-04 18:14:40 +01:00
parallels.h parallels: support bitmap extension for read-only mode 2021-03-08 14:56:55 +01:00
preallocate.c block: fix preallocate filter: don't do unaligned preallocate requests 2022-03-07 09:19:20 +01:00
progress_meter.c progressmeter: protect with a mutex 2021-06-25 14:24:24 +03:00
qapi-sysemu.c block: Move system emulator QMP commands to block/qapi-sysemu.c 2020-03-06 17:15:38 +01:00
qapi.c block: use GDateTime for formatting timestamp when dumping snapshot info 2021-06-14 13:28:50 +01:00
qcow.c block: use int64_t instead of uint64_t in driver write handlers 2021-09-29 13:46:31 -05:00
qcow2-bitmap.c nbd patches for 2021-03-09 2021-03-11 13:57:08 +00:00
qcow2-cache.c core: replace getpagesize() with qemu_real_host_page_size 2019-10-26 15:38:06 +02:00
qcow2-cluster.c qcow2: Silence clang -m32 compiler warning 2021-10-15 15:39:38 -05:00
qcow2-refcount.c qcow2-refcount: check_refblocks(): add separate message for reserved 2021-09-15 18:42:38 +02:00
qcow2-snapshot.c block: consistently use bdrv_is_read_only() 2021-06-02 14:23:20 +02:00
qcow2-threads.c qcow2: add zstd cluster compression 2020-05-13 14:20:31 +02:00
qcow2.c qcow2: simple case support for downgrading of qcow2 images with zstd 2022-02-01 10:51:39 +01:00
qcow2.h qcow2-refcount: check_refblocks(): add separate message for reserved 2021-09-15 18:42:38 +02:00
qed-check.c block/qed: add missed coroutine_fn markers 2019-04-30 15:29:00 +02:00
qed-cluster.c qed: protect table cache with CoMutex 2017-07-17 11:34:11 +08:00
qed-l2-cache.c qed: protect table cache with CoMutex 2017-07-17 11:34:11 +08:00
qed-table.c block/qed: add missed coroutine_fn markers 2019-04-30 15:29:00 +02:00
qed.c block: use int64_t instead of int in driver write_zeroes handlers 2021-09-29 13:46:32 -05:00
qed.h qed: Simplify backing reads 2020-07-06 10:34:14 +02:00
quorum.c block: use int64_t instead of int in driver write_zeroes handlers 2021-09-29 13:46:32 -05:00
raw-format.c block: use int64_t instead of int in driver discard handlers 2021-09-29 13:46:32 -05:00
rbd.c block/rbd: workaround for ceph issue #53784 2022-02-01 15:16:32 +01:00
replication.c job: @force parameter for job_cancel_sync() 2021-10-07 10:42:09 +02:00
reqlist.c block/reqlist: add reqlist_wait_all() 2022-03-07 09:33:30 +01:00
snapshot-access.c block: introduce snapshot-access block driver 2022-03-07 09:33:31 +01:00
snapshot.c include/block/snapshot: global state API + assertions 2022-03-04 18:18:25 +01:00
ssh.c block: print the server key type and fingerprint on failure 2022-02-16 14:34:16 +00:00
stream.c assertions for block_int global state API 2022-03-04 18:18:25 +01:00
throttle-groups.c block/throttle-groups: throttle_group_co_io_limits_intercept(): 64bit bytes 2021-02-03 08:14:00 -06:00
throttle.c block: use int64_t instead of int in driver discard handlers 2021-09-29 13:46:32 -05:00
trace-events block/nvme: Display CQ/SQ pointer in nvme_free_queue_pair() 2021-11-02 15:49:12 +01:00
trace.h trace: switch position of headers to what Meson requires 2020-08-21 06:18:24 -04:00
vdi.c block: use int64_t instead of uint64_t in driver write handlers 2021-09-29 13:46:31 -05:00
vhdx-endian.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
vhdx-log.c block: consistently use bdrv_is_read_only() 2021-06-02 14:23:20 +02:00
vhdx.c block/vhdx: Support vhdx image only with 512 bytes logical sector size 2020-09-15 11:05:13 +02:00
vhdx.h block/vhdx: Use IEC binary prefixes for size constants 2019-04-30 15:29:00 +02:00
vmdk.c vmdk: allow specification of tools version 2021-11-02 12:47:51 +01:00
vpc.c block/vpc: Add a sanity check that fixed-size images have the right type 2021-11-02 12:47:51 +01:00
vvfat.c vvfat: Fix vvfat_write() for writes before the root directory 2022-01-14 12:03:16 +01:00
win32-aio.c aio-posix: split poll check from ready handler 2022-01-12 17:09:39 +00:00
write-threshold.c write-threshold: deal with includes 2021-05-14 16:14:10 +02:00