2013-06-24 17:13:11 +02:00
|
|
|
/*
|
|
|
|
* QEMU backup
|
|
|
|
*
|
|
|
|
* Copyright (C) 2013 Proxmox Server Solutions
|
block/backup: use backup-top instead of write notifiers
Drop write notifiers and use filter node instead.
= Changes =
1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.
2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.
3. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.
4. Block-copy is now using BdrvChildren instead of BlockBackends
5. As backup-top owns these children, we also move block-copy state
into backup-top's ownership.
= Iotest changes =
56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.
Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.
141: Output changed: prepatch, "Node is in use" comes from bdrv_has_blk
check inside qmp_blockdev_del. But we've dropped block-copy blk
objects, so no more blk objects on source bs (job blk is on backup-top
filter bs). New message is from op-blocker, which is the next check in
qmp_blockdev_add.
257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191001131409.14202-6-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-01 15:14:09 +02:00
|
|
|
* Copyright (c) 2019 Virtuozzo International GmbH.
|
2013-06-24 17:13:11 +02:00
|
|
|
*
|
|
|
|
* Authors:
|
|
|
|
* Dietmar Maurer (dietmar@proxmox.com)
|
|
|
|
*
|
|
|
|
* This work is licensed under the terms of the GNU GPL, version 2 or later.
|
|
|
|
* See the COPYING file in the top-level directory.
|
|
|
|
*
|
|
|
|
*/
|
|
|
|
|
2016-01-18 19:01:42 +01:00
|
|
|
#include "qemu/osdep.h"
|
2013-06-24 17:13:11 +02:00
|
|
|
|
|
|
|
#include "trace.h"
|
|
|
|
#include "block/block.h"
|
|
|
|
#include "block/block_int.h"
|
2016-10-27 18:07:00 +02:00
|
|
|
#include "block/blockjob_int.h"
|
2016-07-27 09:01:43 +02:00
|
|
|
#include "block/block_backup.h"
|
2019-09-20 16:20:48 +02:00
|
|
|
#include "block/block-copy.h"
|
include/qemu/osdep.h: Don't include qapi/error.h
Commit 57cb38b included qapi/error.h into qemu/osdep.h to get the
Error typedef. Since then, we've moved to include qemu/osdep.h
everywhere. Its file comment explains: "To avoid getting into
possible circular include dependencies, this file should not include
any other QEMU headers, with the exceptions of config-host.h,
compiler.h, os-posix.h and os-win32.h, all of which are doing a
similar job to this file and are under similar constraints."
qapi/error.h doesn't do a similar job, and it doesn't adhere to
similar constraints: it includes qapi-types.h. That's in excess of
100KiB of crap most .c files don't actually need.
Add the typedef to qemu/typedefs.h, and include that instead of
qapi/error.h. Include qapi/error.h in .c files that need it and don't
get it now. Include qapi-types.h in qom/object.h for uint16List.
Update scripts/clean-includes accordingly. Update it further to match
reality: replace config.h by config-target.h, add sysemu/os-posix.h,
sysemu/os-win32.h. Update the list of includes in the qemu/osdep.h
comment quoted above similarly.
This reduces the number of objects depending on qapi/error.h from "all
of them" to less than a third. Unfortunately, the number depending on
qapi-types.h shrinks only a little. More work is needed for that one.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
[Fix compilation without the spice devel packages. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-03-14 09:01:28 +01:00
|
|
|
#include "qapi/error.h"
|
2015-03-17 17:22:46 +01:00
|
|
|
#include "qapi/qmp/qerror.h"
|
2016-03-20 18:16:19 +01:00
|
|
|
#include "qemu/cutils.h"
|
2015-10-19 17:53:22 +02:00
|
|
|
#include "sysemu/block-backend.h"
|
2016-03-08 05:44:52 +01:00
|
|
|
#include "qemu/bitmap.h"
|
2017-02-28 20:33:40 +01:00
|
|
|
#include "qemu/error-report.h"
|
2013-06-24 17:13:11 +02:00
|
|
|
|
block/backup: use backup-top instead of write notifiers
Drop write notifiers and use filter node instead.
= Changes =
1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.
2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.
3. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.
4. Block-copy is now using BdrvChildren instead of BlockBackends
5. As backup-top owns these children, we also move block-copy state
into backup-top's ownership.
= Iotest changes =
56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.
Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.
141: Output changed: prepatch, "Node is in use" comes from bdrv_has_blk
check inside qmp_blockdev_del. But we've dropped block-copy blk
objects, so no more blk objects on source bs (job blk is on backup-top
filter bs). New message is from op-blocker, which is the next check in
qmp_blockdev_add.
257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191001131409.14202-6-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-01 15:14:09 +02:00
|
|
|
#include "block/backup-top.h"
|
|
|
|
|
2016-02-25 21:58:29 +01:00
|
|
|
#define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16)
|
2013-06-24 17:13:11 +02:00
|
|
|
|
|
|
|
typedef struct BackupBlockJob {
|
|
|
|
BlockJob common;
|
block/backup: use backup-top instead of write notifiers
Drop write notifiers and use filter node instead.
= Changes =
1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.
2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.
3. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.
4. Block-copy is now using BdrvChildren instead of BlockBackends
5. As backup-top owns these children, we also move block-copy state
into backup-top's ownership.
= Iotest changes =
56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.
Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.
141: Output changed: prepatch, "Node is in use" comes from bdrv_has_blk
check inside qmp_blockdev_del. But we've dropped block-copy blk
objects, so no more blk objects on source bs (job blk is on backup-top
filter bs). New message is from op-blocker, which is the next check in
qmp_blockdev_add.
257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191001131409.14202-6-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-01 15:14:09 +02:00
|
|
|
BlockDriverState *backup_top;
|
2019-09-20 16:20:46 +02:00
|
|
|
BlockDriverState *source_bs;
|
2021-02-05 17:37:19 +01:00
|
|
|
BlockDriverState *target_bs;
|
2019-07-29 22:35:53 +02:00
|
|
|
|
2015-04-18 01:49:58 +02:00
|
|
|
BdrvDirtyBitmap *sync_bitmap;
|
2019-07-29 22:35:53 +02:00
|
|
|
|
2013-07-26 20:39:04 +02:00
|
|
|
MirrorSyncMode sync_mode;
|
2019-07-29 22:35:52 +02:00
|
|
|
BitmapSyncMode bitmap_mode;
|
2013-06-24 17:13:11 +02:00
|
|
|
BlockdevOnError on_source_error;
|
|
|
|
BlockdevOnError on_target_error;
|
2018-01-18 18:08:22 +01:00
|
|
|
uint64_t len;
|
2016-02-25 21:58:29 +01:00
|
|
|
int64_t cluster_size;
|
qapi: backup: add perf.use-copy-range parameter
Experiments show, that copy_range is not always making things faster.
So, to make experimentation simpler, let's add a parameter. Some more
perf parameters will be added soon, so here is a new struct.
For now, add new backup qmp parameter with x- prefix for the following
reasons:
- We are going to add more performance parameters, some will be
related to the whole block-copy process, some only to background
copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other
block jobs, which will need performance options as well.. And it
should be the same structure or at least somehow related.
So, there are too much unclean things about how the interface and now
we need the new options mostly for testing. Let's keep them
experimental for a while.
In do_backup_common() new x-perf parameter handled in a way to
make further options addition simpler.
We add use-copy-range with default=true, and we'll change the default
in further patch, after moving backup to use block-copy.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-2-vsementsov@virtuozzo.com>
[mreitz: s/5\.2/6.0/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-16 22:46:43 +01:00
|
|
|
BackupPerf perf;
|
2017-10-12 15:53:10 +02:00
|
|
|
|
2019-09-20 16:20:46 +02:00
|
|
|
BlockCopyState *bcs;
|
2021-01-16 22:46:59 +01:00
|
|
|
|
|
|
|
bool wait;
|
|
|
|
BlockCopyCallState *bg_bcs_call;
|
2013-06-24 17:13:11 +02:00
|
|
|
} BackupBlockJob;
|
|
|
|
|
2018-01-19 15:54:40 +01:00
|
|
|
static const BlockJobDriver backup_job_driver;
|
|
|
|
|
2015-11-06 00:13:10 +01:00
|
|
|
static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret)
|
|
|
|
{
|
|
|
|
BdrvDirtyBitmap *bm;
|
2019-07-29 22:35:53 +02:00
|
|
|
bool sync = (((ret == 0) || (job->bitmap_mode == BITMAP_SYNC_MODE_ALWAYS)) \
|
|
|
|
&& (job->bitmap_mode != BITMAP_SYNC_MODE_NEVER));
|
2015-11-06 00:13:10 +01:00
|
|
|
|
2019-07-29 22:35:53 +02:00
|
|
|
if (sync) {
|
2019-07-29 22:35:53 +02:00
|
|
|
/*
|
2019-07-29 22:35:53 +02:00
|
|
|
* We succeeded, or we always intended to sync the bitmap.
|
|
|
|
* Delete this bitmap and install the child.
|
2019-07-29 22:35:53 +02:00
|
|
|
*/
|
2019-09-16 16:19:09 +02:00
|
|
|
bm = bdrv_dirty_bitmap_abdicate(job->sync_bitmap, NULL);
|
2019-07-29 22:35:53 +02:00
|
|
|
} else {
|
|
|
|
/*
|
|
|
|
* We failed, or we never intended to sync the bitmap anyway.
|
|
|
|
* Merge the successor back into the parent, keeping all data.
|
|
|
|
*/
|
2019-09-16 16:19:09 +02:00
|
|
|
bm = bdrv_reclaim_dirty_bitmap(job->sync_bitmap, NULL);
|
2019-07-29 22:35:53 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
assert(bm);
|
|
|
|
|
|
|
|
if (ret < 0 && job->bitmap_mode == BITMAP_SYNC_MODE_ALWAYS) {
|
|
|
|
/* If we failed and synced, merge in the bits we didn't copy: */
|
2020-03-11 11:30:04 +01:00
|
|
|
bdrv_dirty_bitmap_merge_internal(bm, block_copy_dirty_bitmap(job->bcs),
|
2019-07-29 22:35:53 +02:00
|
|
|
NULL, true);
|
2015-11-06 00:13:10 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-04-19 17:30:16 +02:00
|
|
|
static void backup_commit(Job *job)
|
2015-11-06 00:13:16 +01:00
|
|
|
{
|
2018-04-19 17:30:16 +02:00
|
|
|
BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
|
2015-11-06 00:13:16 +01:00
|
|
|
if (s->sync_bitmap) {
|
|
|
|
backup_cleanup_sync_bitmap(s, 0);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-04-19 17:30:16 +02:00
|
|
|
static void backup_abort(Job *job)
|
2015-11-06 00:13:16 +01:00
|
|
|
{
|
2018-04-19 17:30:16 +02:00
|
|
|
BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
|
2015-11-06 00:13:16 +01:00
|
|
|
if (s->sync_bitmap) {
|
|
|
|
backup_cleanup_sync_bitmap(s, -1);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-04-19 17:30:16 +02:00
|
|
|
static void backup_clean(Job *job)
|
blockjob: add .clean property
Cleaning up after we have deferred to the main thread but before the
transaction has converged can be dangerous and result in deadlocks
if the job cleanup invokes any BH polling loops.
A job may attempt to begin cleaning up, but may induce another job to
enter its cleanup routine. The second job, part of our same transaction,
will block waiting for the first job to finish, so neither job may now
make progress.
To rectify this, allow jobs to register a cleanup operation that will
always run regardless of if the job was in a transaction or not, and
if the transaction job group completed successfully or not.
Move sensitive cleanup to this callback instead which is guaranteed to
be run only after the transaction has converged, which removes sensitive
timing constraints from said cleanup.
Furthermore, in future patches these cleanup operations will be performed
regardless of whether or not we actually started the job. Therefore,
cleanup callbacks should essentially confine themselves to undoing create
operations, e.g. setup actions taken in what is now backup_start.
Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 1478587839-9834-3-git-send-email-jsnow@redhat.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
2016-11-08 07:50:35 +01:00
|
|
|
{
|
2018-04-19 17:30:16 +02:00
|
|
|
BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
|
2021-02-19 16:33:46 +01:00
|
|
|
block_job_remove_all_bdrv(&s->common);
|
block/backup: use backup-top instead of write notifiers
Drop write notifiers and use filter node instead.
= Changes =
1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.
2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.
3. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.
4. Block-copy is now using BdrvChildren instead of BlockBackends
5. As backup-top owns these children, we also move block-copy state
into backup-top's ownership.
= Iotest changes =
56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.
Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.
141: Output changed: prepatch, "Node is in use" comes from bdrv_has_blk
check inside qmp_blockdev_del. But we've dropped block-copy blk
objects, so no more blk objects on source bs (job blk is on backup-top
filter bs). New message is from op-blocker, which is the next check in
qmp_blockdev_add.
257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191001131409.14202-6-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-01 15:14:09 +02:00
|
|
|
bdrv_backup_top_drop(s->backup_top);
|
blockjob: add .clean property
Cleaning up after we have deferred to the main thread but before the
transaction has converged can be dangerous and result in deadlocks
if the job cleanup invokes any BH polling loops.
A job may attempt to begin cleaning up, but may induce another job to
enter its cleanup routine. The second job, part of our same transaction,
will block waiting for the first job to finish, so neither job may now
make progress.
To rectify this, allow jobs to register a cleanup operation that will
always run regardless of if the job was in a transaction or not, and
if the transaction job group completed successfully or not.
Move sensitive cleanup to this callback instead which is guaranteed to
be run only after the transaction has converged, which removes sensitive
timing constraints from said cleanup.
Furthermore, in future patches these cleanup operations will be performed
regardless of whether or not we actually started the job. Therefore,
cleanup callbacks should essentially confine themselves to undoing create
operations, e.g. setup actions taken in what is now backup_start.
Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 1478587839-9834-3-git-send-email-jsnow@redhat.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
2016-11-08 07:50:35 +01:00
|
|
|
}
|
|
|
|
|
2016-07-27 09:01:43 +02:00
|
|
|
void backup_do_checkpoint(BlockJob *job, Error **errp)
|
|
|
|
{
|
|
|
|
BackupBlockJob *backup_job = container_of(job, BackupBlockJob, common);
|
|
|
|
|
2018-01-19 15:54:40 +01:00
|
|
|
assert(block_job_driver(job) == &backup_job_driver);
|
2016-07-27 09:01:43 +02:00
|
|
|
|
|
|
|
if (backup_job->sync_mode != MIRROR_SYNC_MODE_NONE) {
|
|
|
|
error_setg(errp, "The backup job only supports block checkpoint in"
|
|
|
|
" sync=none mode");
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
2020-03-11 11:30:04 +01:00
|
|
|
bdrv_set_dirty_bitmap(block_copy_dirty_bitmap(backup_job->bcs), 0,
|
|
|
|
backup_job->len);
|
2016-07-27 09:01:43 +02:00
|
|
|
}
|
|
|
|
|
2013-06-24 17:13:11 +02:00
|
|
|
static BlockErrorAction backup_error_action(BackupBlockJob *job,
|
|
|
|
bool read, int error)
|
|
|
|
{
|
|
|
|
if (read) {
|
2016-04-18 11:36:38 +02:00
|
|
|
return block_job_error_action(&job->common, job->on_source_error,
|
|
|
|
true, error);
|
2013-06-24 17:13:11 +02:00
|
|
|
} else {
|
2016-04-18 11:36:38 +02:00
|
|
|
return block_job_error_action(&job->common, job->on_target_error,
|
|
|
|
false, error);
|
2013-06-24 17:13:11 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-01-16 22:46:59 +01:00
|
|
|
static void coroutine_fn backup_block_copy_callback(void *opaque)
|
2015-04-18 01:49:58 +02:00
|
|
|
{
|
2021-01-16 22:46:59 +01:00
|
|
|
BackupBlockJob *s = opaque;
|
2015-04-18 01:49:58 +02:00
|
|
|
|
2021-01-16 22:46:59 +01:00
|
|
|
if (s->wait) {
|
|
|
|
s->wait = false;
|
|
|
|
aio_co_wake(s->common.job.co);
|
|
|
|
} else {
|
|
|
|
job_enter(&s->common.job);
|
2015-04-18 01:49:58 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-04-29 11:08:41 +02:00
|
|
|
static int coroutine_fn backup_loop(BackupBlockJob *job)
|
2015-04-18 01:49:58 +02:00
|
|
|
{
|
2021-01-16 22:46:59 +01:00
|
|
|
BlockCopyCallState *s = NULL;
|
2019-07-29 22:35:53 +02:00
|
|
|
int ret = 0;
|
2021-01-16 22:46:59 +01:00
|
|
|
bool error_is_read;
|
|
|
|
BlockErrorAction act;
|
|
|
|
|
|
|
|
while (true) { /* retry loop */
|
|
|
|
job->bg_bcs_call = s = block_copy_async(job->bcs, 0,
|
|
|
|
QEMU_ALIGN_UP(job->len, job->cluster_size),
|
|
|
|
job->perf.max_workers, job->perf.max_chunk,
|
|
|
|
backup_block_copy_callback, job);
|
|
|
|
|
|
|
|
while (!block_copy_call_finished(s) &&
|
|
|
|
!job_is_cancelled(&job->common.job))
|
|
|
|
{
|
|
|
|
job_yield(&job->common.job);
|
|
|
|
}
|
2015-04-18 01:49:58 +02:00
|
|
|
|
2021-01-16 22:46:59 +01:00
|
|
|
if (!block_copy_call_finished(s)) {
|
|
|
|
assert(job_is_cancelled(&job->common.job));
|
|
|
|
/*
|
|
|
|
* Note that we can't use job_yield() here, as it doesn't work for
|
|
|
|
* cancelled job.
|
|
|
|
*/
|
|
|
|
block_copy_call_cancel(s);
|
|
|
|
job->wait = true;
|
|
|
|
qemu_coroutine_yield();
|
|
|
|
assert(block_copy_call_finished(s));
|
|
|
|
ret = 0;
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (job_is_cancelled(&job->common.job) ||
|
|
|
|
block_copy_call_succeeded(s))
|
|
|
|
{
|
|
|
|
ret = 0;
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (block_copy_call_cancelled(s)) {
|
|
|
|
/*
|
|
|
|
* Job is not cancelled but only block-copy call. This is possible
|
|
|
|
* after job pause. Now the pause is finished, start new block-copy
|
|
|
|
* iteration.
|
|
|
|
*/
|
|
|
|
block_copy_call_free(s);
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* The only remaining case is failed block-copy call. */
|
|
|
|
assert(block_copy_call_failed(s));
|
|
|
|
|
|
|
|
ret = block_copy_call_status(s, &error_is_read);
|
|
|
|
act = backup_error_action(job, error_is_read, -ret);
|
|
|
|
switch (act) {
|
|
|
|
case BLOCK_ERROR_ACTION_REPORT:
|
|
|
|
goto out;
|
|
|
|
case BLOCK_ERROR_ACTION_STOP:
|
|
|
|
/*
|
|
|
|
* Go to pause prior to starting new block-copy call on the next
|
|
|
|
* iteration.
|
|
|
|
*/
|
|
|
|
job_pause_point(&job->common.job);
|
|
|
|
break;
|
|
|
|
case BLOCK_ERROR_ACTION_IGNORE:
|
|
|
|
/* Proceed to new block-copy call to retry. */
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
abort();
|
|
|
|
}
|
|
|
|
|
|
|
|
block_copy_call_free(s);
|
2015-04-18 01:49:58 +02:00
|
|
|
}
|
|
|
|
|
2021-01-16 22:46:59 +01:00
|
|
|
out:
|
|
|
|
block_copy_call_free(s);
|
|
|
|
job->bg_bcs_call = NULL;
|
2019-07-29 22:35:53 +02:00
|
|
|
return ret;
|
2015-04-18 01:49:58 +02:00
|
|
|
}
|
|
|
|
|
2020-03-11 11:30:04 +01:00
|
|
|
static void backup_init_bcs_bitmap(BackupBlockJob *job)
|
2017-10-12 15:53:11 +02:00
|
|
|
{
|
2019-07-29 22:35:55 +02:00
|
|
|
bool ret;
|
|
|
|
uint64_t estimate;
|
2020-03-11 11:30:04 +01:00
|
|
|
BdrvDirtyBitmap *bcs_bitmap = block_copy_dirty_bitmap(job->bcs);
|
2019-07-29 22:35:55 +02:00
|
|
|
|
|
|
|
if (job->sync_mode == MIRROR_SYNC_MODE_BITMAP) {
|
2020-03-11 11:30:04 +01:00
|
|
|
ret = bdrv_dirty_bitmap_merge_internal(bcs_bitmap, job->sync_bitmap,
|
2019-07-29 22:35:55 +02:00
|
|
|
NULL, true);
|
|
|
|
assert(ret);
|
|
|
|
} else {
|
2019-07-29 22:35:55 +02:00
|
|
|
if (job->sync_mode == MIRROR_SYNC_MODE_TOP) {
|
|
|
|
/*
|
|
|
|
* We can't hog the coroutine to initialize this thoroughly.
|
|
|
|
* Set a flag and resume work when we are able to yield safely.
|
|
|
|
*/
|
2020-03-11 11:30:04 +01:00
|
|
|
block_copy_set_skip_unallocated(job->bcs, true);
|
2019-07-29 22:35:55 +02:00
|
|
|
}
|
2020-03-11 11:30:04 +01:00
|
|
|
bdrv_set_dirty_bitmap(bcs_bitmap, 0, job->len);
|
2019-07-29 22:35:55 +02:00
|
|
|
}
|
2017-10-12 15:53:11 +02:00
|
|
|
|
2020-03-11 11:30:04 +01:00
|
|
|
estimate = bdrv_get_dirty_count(bcs_bitmap);
|
2019-07-29 22:35:55 +02:00
|
|
|
job_progress_set_remaining(&job->common.job, estimate);
|
2017-10-12 15:53:11 +02:00
|
|
|
}
|
|
|
|
|
2018-08-30 03:57:32 +02:00
|
|
|
static int coroutine_fn backup_run(Job *job, Error **errp)
|
2013-06-24 17:13:11 +02:00
|
|
|
{
|
2018-08-30 03:57:32 +02:00
|
|
|
BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
|
2021-01-16 22:46:58 +01:00
|
|
|
int ret;
|
2013-06-24 17:13:11 +02:00
|
|
|
|
2020-03-11 11:30:04 +01:00
|
|
|
backup_init_bcs_bitmap(s);
|
2017-10-12 15:53:11 +02:00
|
|
|
|
2019-07-29 22:35:55 +02:00
|
|
|
if (s->sync_mode == MIRROR_SYNC_MODE_TOP) {
|
|
|
|
int64_t offset = 0;
|
|
|
|
int64_t count;
|
|
|
|
|
|
|
|
for (offset = 0; offset < s->len; ) {
|
2021-01-16 22:46:59 +01:00
|
|
|
if (job_is_cancelled(job)) {
|
|
|
|
return -ECANCELED;
|
|
|
|
}
|
|
|
|
|
|
|
|
job_pause_point(job);
|
|
|
|
|
|
|
|
if (job_is_cancelled(job)) {
|
2021-01-16 22:46:58 +01:00
|
|
|
return -ECANCELED;
|
2019-07-29 22:35:55 +02:00
|
|
|
}
|
|
|
|
|
2019-09-20 16:20:46 +02:00
|
|
|
ret = block_copy_reset_unallocated(s->bcs, offset, &count);
|
2019-07-29 22:35:55 +02:00
|
|
|
if (ret < 0) {
|
2021-01-16 22:46:58 +01:00
|
|
|
return ret;
|
2019-07-29 22:35:55 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
offset += count;
|
|
|
|
}
|
2020-03-11 11:30:04 +01:00
|
|
|
block_copy_set_skip_unallocated(s->bcs, false);
|
2019-07-29 22:35:55 +02:00
|
|
|
}
|
|
|
|
|
2018-08-30 03:57:32 +02:00
|
|
|
if (s->sync_mode == MIRROR_SYNC_MODE_NONE) {
|
2019-09-20 16:20:47 +02:00
|
|
|
/*
|
2020-03-11 11:30:04 +01:00
|
|
|
* All bits are set in bcs bitmap to allow any cluster to be copied.
|
2019-09-20 16:20:47 +02:00
|
|
|
* This does not actually require them to be copied.
|
|
|
|
*/
|
2018-08-30 03:57:32 +02:00
|
|
|
while (!job_is_cancelled(job)) {
|
2019-09-20 16:20:47 +02:00
|
|
|
/*
|
|
|
|
* Yield until the job is cancelled. We just let our before_write
|
|
|
|
* notify callback service CoW requests.
|
|
|
|
*/
|
2018-08-30 03:57:32 +02:00
|
|
|
job_yield(job);
|
2013-06-24 17:13:11 +02:00
|
|
|
}
|
2013-07-26 20:39:04 +02:00
|
|
|
} else {
|
2021-01-16 22:46:58 +01:00
|
|
|
return backup_loop(s);
|
2013-06-24 17:13:11 +02:00
|
|
|
}
|
|
|
|
|
2021-01-16 22:46:58 +01:00
|
|
|
return 0;
|
2013-06-24 17:13:11 +02:00
|
|
|
}
|
|
|
|
|
2021-01-16 22:46:59 +01:00
|
|
|
static void coroutine_fn backup_pause(Job *job)
|
|
|
|
{
|
|
|
|
BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
|
|
|
|
|
|
|
|
if (s->bg_bcs_call && !block_copy_call_finished(s->bg_bcs_call)) {
|
|
|
|
block_copy_call_cancel(s->bg_bcs_call);
|
|
|
|
s->wait = true;
|
|
|
|
qemu_coroutine_yield();
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
static void coroutine_fn backup_set_speed(BlockJob *job, int64_t speed)
|
|
|
|
{
|
|
|
|
BackupBlockJob *s = container_of(job, BackupBlockJob, common);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* block_job_set_speed() is called first from block_job_create(), when we
|
|
|
|
* don't yet have s->bcs.
|
|
|
|
*/
|
|
|
|
if (s->bcs) {
|
|
|
|
block_copy_set_speed(s->bcs, speed);
|
|
|
|
if (s->bg_bcs_call) {
|
|
|
|
block_copy_kick(s->bg_bcs_call);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-04-21 09:58:58 +02:00
|
|
|
static void backup_cancel(Job *job, bool force)
|
2021-02-05 17:37:19 +01:00
|
|
|
{
|
|
|
|
BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
|
|
|
|
|
|
|
|
bdrv_cancel_in_flight(s->target_bs);
|
|
|
|
}
|
|
|
|
|
2016-11-08 07:50:36 +01:00
|
|
|
static const BlockJobDriver backup_job_driver = {
|
2018-04-12 17:29:59 +02:00
|
|
|
.job_driver = {
|
|
|
|
.instance_size = sizeof(BackupBlockJob),
|
2018-04-12 17:57:08 +02:00
|
|
|
.job_type = JOB_TYPE_BACKUP,
|
2018-04-13 18:50:05 +02:00
|
|
|
.free = block_job_free,
|
2018-04-18 17:10:26 +02:00
|
|
|
.user_resume = block_job_user_resume,
|
2018-08-30 03:57:26 +02:00
|
|
|
.run = backup_run,
|
2018-04-19 17:30:16 +02:00
|
|
|
.commit = backup_commit,
|
|
|
|
.abort = backup_abort,
|
|
|
|
.clean = backup_clean,
|
2021-01-16 22:46:59 +01:00
|
|
|
.pause = backup_pause,
|
2021-02-05 17:37:19 +01:00
|
|
|
.cancel = backup_cancel,
|
2021-01-16 22:46:59 +01:00
|
|
|
},
|
|
|
|
.set_speed = backup_set_speed,
|
2016-11-08 07:50:36 +01:00
|
|
|
};
|
|
|
|
|
2019-04-29 11:08:42 +02:00
|
|
|
static int64_t backup_calculate_cluster_size(BlockDriverState *target,
|
|
|
|
Error **errp)
|
|
|
|
{
|
|
|
|
int ret;
|
|
|
|
BlockDriverInfo bdi;
|
2019-06-12 17:46:45 +02:00
|
|
|
bool target_does_cow = bdrv_backing_chain_next(target);
|
2019-04-29 11:08:42 +02:00
|
|
|
|
|
|
|
/*
|
|
|
|
* If there is no backing file on the target, we cannot rely on COW if our
|
|
|
|
* backup cluster size is smaller than the target cluster size. Even for
|
|
|
|
* targets with a backing file, try to avoid COW if possible.
|
|
|
|
*/
|
|
|
|
ret = bdrv_get_info(target, &bdi);
|
2019-06-12 17:46:45 +02:00
|
|
|
if (ret == -ENOTSUP && !target_does_cow) {
|
2019-04-29 11:08:42 +02:00
|
|
|
/* Cluster size is not defined */
|
|
|
|
warn_report("The target block device doesn't provide "
|
|
|
|
"information about the block size and it doesn't have a "
|
|
|
|
"backing file. The default block size of %u bytes is "
|
|
|
|
"used. If the actual block size of the target exceeds "
|
|
|
|
"this default, the backup may be unusable",
|
|
|
|
BACKUP_CLUSTER_SIZE_DEFAULT);
|
|
|
|
return BACKUP_CLUSTER_SIZE_DEFAULT;
|
2019-06-12 17:46:45 +02:00
|
|
|
} else if (ret < 0 && !target_does_cow) {
|
2019-04-29 11:08:42 +02:00
|
|
|
error_setg_errno(errp, -ret,
|
|
|
|
"Couldn't determine the cluster size of the target image, "
|
|
|
|
"which has no backing file");
|
|
|
|
error_append_hint(errp,
|
|
|
|
"Aborting, since this may create an unusable destination image\n");
|
|
|
|
return ret;
|
2019-06-12 17:46:45 +02:00
|
|
|
} else if (ret < 0 && target_does_cow) {
|
2019-04-29 11:08:42 +02:00
|
|
|
/* Not fatal; just trudge on ahead. */
|
|
|
|
return BACKUP_CLUSTER_SIZE_DEFAULT;
|
|
|
|
}
|
|
|
|
|
|
|
|
return MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster_size);
|
|
|
|
}
|
|
|
|
|
2016-11-08 07:50:38 +01:00
|
|
|
BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
|
2016-07-05 16:28:58 +02:00
|
|
|
BlockDriverState *target, int64_t speed,
|
|
|
|
MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap,
|
2019-07-29 22:35:52 +02:00
|
|
|
BitmapSyncMode bitmap_mode,
|
2016-07-22 10:17:52 +02:00
|
|
|
bool compress,
|
block/backup: use backup-top instead of write notifiers
Drop write notifiers and use filter node instead.
= Changes =
1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.
2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.
3. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.
4. Block-copy is now using BdrvChildren instead of BlockBackends
5. As backup-top owns these children, we also move block-copy state
into backup-top's ownership.
= Iotest changes =
56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.
Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.
141: Output changed: prepatch, "Node is in use" comes from bdrv_has_blk
check inside qmp_blockdev_del. But we've dropped block-copy blk
objects, so no more blk objects on source bs (job blk is on backup-top
filter bs). New message is from op-blocker, which is the next check in
qmp_blockdev_add.
257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191001131409.14202-6-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-01 15:14:09 +02:00
|
|
|
const char *filter_node_name,
|
qapi: backup: add perf.use-copy-range parameter
Experiments show, that copy_range is not always making things faster.
So, to make experimentation simpler, let's add a parameter. Some more
perf parameters will be added soon, so here is a new struct.
For now, add new backup qmp parameter with x- prefix for the following
reasons:
- We are going to add more performance parameters, some will be
related to the whole block-copy process, some only to background
copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other
block jobs, which will need performance options as well.. And it
should be the same structure or at least somehow related.
So, there are too much unclean things about how the interface and now
we need the new options mostly for testing. Let's keep them
experimental for a while.
In do_backup_common() new x-perf parameter handled in a way to
make further options addition simpler.
We add use-copy-range with default=true, and we'll change the default
in further patch, after moving backup to use block-copy.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-2-vsementsov@virtuozzo.com>
[mreitz: s/5\.2/6.0/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-16 22:46:43 +01:00
|
|
|
BackupPerf *perf,
|
2013-06-24 17:13:11 +02:00
|
|
|
BlockdevOnError on_source_error,
|
|
|
|
BlockdevOnError on_target_error,
|
2016-10-27 18:06:57 +02:00
|
|
|
int creation_flags,
|
2014-10-07 13:59:15 +02:00
|
|
|
BlockCompletionFunc *cb, void *opaque,
|
2018-04-19 16:09:52 +02:00
|
|
|
JobTxn *txn, Error **errp)
|
2013-06-24 17:13:11 +02:00
|
|
|
{
|
2020-04-30 16:27:54 +02:00
|
|
|
int64_t len, target_len;
|
2016-04-14 12:59:55 +02:00
|
|
|
BackupBlockJob *job = NULL;
|
2019-04-29 11:08:42 +02:00
|
|
|
int64_t cluster_size;
|
2019-09-20 16:20:46 +02:00
|
|
|
BdrvRequestFlags write_flags;
|
block/backup: use backup-top instead of write notifiers
Drop write notifiers and use filter node instead.
= Changes =
1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.
2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.
3. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.
4. Block-copy is now using BdrvChildren instead of BlockBackends
5. As backup-top owns these children, we also move block-copy state
into backup-top's ownership.
= Iotest changes =
56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.
Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.
141: Output changed: prepatch, "Node is in use" comes from bdrv_has_blk
check inside qmp_blockdev_del. But we've dropped block-copy blk
objects, so no more blk objects on source bs (job blk is on backup-top
filter bs). New message is from op-blocker, which is the next check in
qmp_blockdev_add.
257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191001131409.14202-6-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-01 15:14:09 +02:00
|
|
|
BlockDriverState *backup_top = NULL;
|
|
|
|
BlockCopyState *bcs = NULL;
|
2013-06-24 17:13:11 +02:00
|
|
|
|
|
|
|
assert(bs);
|
|
|
|
assert(target);
|
|
|
|
|
2019-07-29 22:35:55 +02:00
|
|
|
/* QMP interface protects us from these cases */
|
|
|
|
assert(sync_mode != MIRROR_SYNC_MODE_INCREMENTAL);
|
|
|
|
assert(sync_bitmap || sync_mode != MIRROR_SYNC_MODE_BITMAP);
|
|
|
|
|
2014-12-18 11:37:05 +01:00
|
|
|
if (bs == target) {
|
|
|
|
error_setg(errp, "Source and target cannot be the same");
|
2016-11-08 07:50:38 +01:00
|
|
|
return NULL;
|
2014-12-18 11:37:05 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
if (!bdrv_is_inserted(bs)) {
|
|
|
|
error_setg(errp, "Device is not inserted: %s",
|
|
|
|
bdrv_get_device_name(bs));
|
2016-11-08 07:50:38 +01:00
|
|
|
return NULL;
|
2014-12-18 11:37:05 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
if (!bdrv_is_inserted(target)) {
|
|
|
|
error_setg(errp, "Device is not inserted: %s",
|
|
|
|
bdrv_get_device_name(target));
|
2016-11-08 07:50:38 +01:00
|
|
|
return NULL;
|
2014-12-18 11:37:05 +01:00
|
|
|
}
|
|
|
|
|
2019-06-12 17:46:45 +02:00
|
|
|
if (compress && !bdrv_supports_compressed_writes(target)) {
|
2016-07-22 10:17:52 +02:00
|
|
|
error_setg(errp, "Compression is not supported for this drive %s",
|
|
|
|
bdrv_get_device_name(target));
|
2016-11-08 07:50:38 +01:00
|
|
|
return NULL;
|
2016-07-22 10:17:52 +02:00
|
|
|
}
|
|
|
|
|
2014-12-18 11:37:05 +01:00
|
|
|
if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
|
2016-11-08 07:50:38 +01:00
|
|
|
return NULL;
|
2014-12-18 11:37:05 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
if (bdrv_op_is_blocked(target, BLOCK_OP_TYPE_BACKUP_TARGET, errp)) {
|
2016-11-08 07:50:38 +01:00
|
|
|
return NULL;
|
2014-12-18 11:37:05 +01:00
|
|
|
}
|
|
|
|
|
2021-01-16 22:46:52 +01:00
|
|
|
cluster_size = backup_calculate_cluster_size(target, errp);
|
|
|
|
if (cluster_size < 0) {
|
|
|
|
goto error;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (perf->max_workers < 1) {
|
|
|
|
error_setg(errp, "max-workers must be greater than zero");
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (perf->max_chunk < 0) {
|
|
|
|
error_setg(errp, "max-chunk must be zero (which means no limit) or "
|
|
|
|
"positive");
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (perf->max_chunk && perf->max_chunk < cluster_size) {
|
|
|
|
error_setg(errp, "Required max-chunk (%" PRIi64 ") is less than backup "
|
|
|
|
"cluster size (%" PRIi64 ")", perf->max_chunk, cluster_size);
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2019-07-29 22:35:55 +02:00
|
|
|
if (sync_bitmap) {
|
2019-07-29 22:35:54 +02:00
|
|
|
/* If we need to write to this bitmap, check that we can: */
|
|
|
|
if (bitmap_mode != BITMAP_SYNC_MODE_NEVER &&
|
|
|
|
bdrv_dirty_bitmap_check(sync_bitmap, BDRV_BITMAP_DEFAULT, errp)) {
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
2015-04-18 01:49:58 +02:00
|
|
|
/* Create a new bitmap, and freeze/disable this one. */
|
2019-09-16 16:19:09 +02:00
|
|
|
if (bdrv_dirty_bitmap_create_successor(sync_bitmap, errp) < 0) {
|
2016-11-08 07:50:38 +01:00
|
|
|
return NULL;
|
2015-04-18 01:49:58 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2013-06-24 17:13:11 +02:00
|
|
|
len = bdrv_getlength(bs);
|
|
|
|
if (len < 0) {
|
2020-04-30 16:27:53 +02:00
|
|
|
error_setg_errno(errp, -len, "Unable to get length for '%s'",
|
|
|
|
bdrv_get_device_or_node_name(bs));
|
2015-04-18 01:49:58 +02:00
|
|
|
goto error;
|
2013-06-24 17:13:11 +02:00
|
|
|
}
|
|
|
|
|
2020-04-30 16:27:54 +02:00
|
|
|
target_len = bdrv_getlength(target);
|
|
|
|
if (target_len < 0) {
|
|
|
|
error_setg_errno(errp, -target_len, "Unable to get length for '%s'",
|
|
|
|
bdrv_get_device_or_node_name(bs));
|
|
|
|
goto error;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (target_len != len) {
|
|
|
|
error_setg(errp, "Source and target image have different sizes");
|
|
|
|
goto error;
|
|
|
|
}
|
|
|
|
|
2019-07-30 18:32:51 +02:00
|
|
|
/*
|
2019-09-20 16:20:45 +02:00
|
|
|
* If source is in backing chain of target assume that target is going to be
|
|
|
|
* used for "image fleecing", i.e. it should represent a kind of snapshot of
|
|
|
|
* source at backup-start point in time. And target is going to be read by
|
|
|
|
* somebody (for example, used as NBD export) during backup job.
|
|
|
|
*
|
|
|
|
* In this case, we need to add BDRV_REQ_SERIALISING write flag to avoid
|
|
|
|
* intersection of backup writes and third party reads from target,
|
|
|
|
* otherwise reading from target we may occasionally read already updated by
|
|
|
|
* guest data.
|
|
|
|
*
|
|
|
|
* For more information see commit f8d59dfb40bb and test
|
|
|
|
* tests/qemu-iotests/222
|
2019-07-30 18:32:51 +02:00
|
|
|
*/
|
2019-09-20 16:20:46 +02:00
|
|
|
write_flags = (bdrv_chain_contains(target, bs) ? BDRV_REQ_SERIALISING : 0) |
|
|
|
|
(compress ? BDRV_REQ_WRITE_COMPRESSED : 0),
|
|
|
|
|
block/backup: use backup-top instead of write notifiers
Drop write notifiers and use filter node instead.
= Changes =
1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.
2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.
3. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.
4. Block-copy is now using BdrvChildren instead of BlockBackends
5. As backup-top owns these children, we also move block-copy state
into backup-top's ownership.
= Iotest changes =
56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.
Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.
141: Output changed: prepatch, "Node is in use" comes from bdrv_has_blk
check inside qmp_blockdev_del. But we've dropped block-copy blk
objects, so no more blk objects on source bs (job blk is on backup-top
filter bs). New message is from op-blocker, which is the next check in
qmp_blockdev_add.
257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191001131409.14202-6-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-01 15:14:09 +02:00
|
|
|
backup_top = bdrv_backup_top_append(bs, target, filter_node_name,
|
qapi: backup: add perf.use-copy-range parameter
Experiments show, that copy_range is not always making things faster.
So, to make experimentation simpler, let's add a parameter. Some more
perf parameters will be added soon, so here is a new struct.
For now, add new backup qmp parameter with x- prefix for the following
reasons:
- We are going to add more performance parameters, some will be
related to the whole block-copy process, some only to background
copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other
block jobs, which will need performance options as well.. And it
should be the same structure or at least somehow related.
So, there are too much unclean things about how the interface and now
we need the new options mostly for testing. Let's keep them
experimental for a while.
In do_backup_common() new x-perf parameter handled in a way to
make further options addition simpler.
We add use-copy-range with default=true, and we'll change the default
in further patch, after moving backup to use block-copy.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-2-vsementsov@virtuozzo.com>
[mreitz: s/5\.2/6.0/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-16 22:46:43 +01:00
|
|
|
cluster_size, perf,
|
|
|
|
write_flags, &bcs, errp);
|
block/backup: use backup-top instead of write notifiers
Drop write notifiers and use filter node instead.
= Changes =
1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.
2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.
3. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.
4. Block-copy is now using BdrvChildren instead of BlockBackends
5. As backup-top owns these children, we also move block-copy state
into backup-top's ownership.
= Iotest changes =
56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.
Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.
141: Output changed: prepatch, "Node is in use" comes from bdrv_has_blk
check inside qmp_blockdev_del. But we've dropped block-copy blk
objects, so no more blk objects on source bs (job blk is on backup-top
filter bs). New message is from op-blocker, which is the next check in
qmp_blockdev_add.
257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191001131409.14202-6-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-01 15:14:09 +02:00
|
|
|
if (!backup_top) {
|
|
|
|
goto error;
|
|
|
|
}
|
|
|
|
|
2019-10-01 15:14:06 +02:00
|
|
|
/* job->len is fixed, so we can't allow resize */
|
block/backup: use backup-top instead of write notifiers
Drop write notifiers and use filter node instead.
= Changes =
1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.
2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.
3. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.
4. Block-copy is now using BdrvChildren instead of BlockBackends
5. As backup-top owns these children, we also move block-copy state
into backup-top's ownership.
= Iotest changes =
56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.
Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.
141: Output changed: prepatch, "Node is in use" comes from bdrv_has_blk
check inside qmp_blockdev_del. But we've dropped block-copy blk
objects, so no more blk objects on source bs (job blk is on backup-top
filter bs). New message is from op-blocker, which is the next check in
qmp_blockdev_add.
257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191001131409.14202-6-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-01 15:14:09 +02:00
|
|
|
job = block_job_create(job_id, &backup_job_driver, txn, backup_top,
|
|
|
|
0, BLK_PERM_ALL,
|
2019-10-01 15:14:06 +02:00
|
|
|
speed, creation_flags, cb, opaque, errp);
|
|
|
|
if (!job) {
|
|
|
|
goto error;
|
|
|
|
}
|
|
|
|
|
block/backup: use backup-top instead of write notifiers
Drop write notifiers and use filter node instead.
= Changes =
1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.
2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.
3. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.
4. Block-copy is now using BdrvChildren instead of BlockBackends
5. As backup-top owns these children, we also move block-copy state
into backup-top's ownership.
= Iotest changes =
56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.
Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.
141: Output changed: prepatch, "Node is in use" comes from bdrv_has_blk
check inside qmp_blockdev_del. But we've dropped block-copy blk
objects, so no more blk objects on source bs (job blk is on backup-top
filter bs). New message is from op-blocker, which is the next check in
qmp_blockdev_add.
257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191001131409.14202-6-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-01 15:14:09 +02:00
|
|
|
job->backup_top = backup_top;
|
2019-10-01 15:14:06 +02:00
|
|
|
job->source_bs = bs;
|
2021-02-05 17:37:19 +01:00
|
|
|
job->target_bs = target;
|
2019-10-01 15:14:06 +02:00
|
|
|
job->on_source_error = on_source_error;
|
|
|
|
job->on_target_error = on_target_error;
|
|
|
|
job->sync_mode = sync_mode;
|
|
|
|
job->sync_bitmap = sync_bitmap;
|
|
|
|
job->bitmap_mode = bitmap_mode;
|
block/backup: use backup-top instead of write notifiers
Drop write notifiers and use filter node instead.
= Changes =
1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.
2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.
3. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.
4. Block-copy is now using BdrvChildren instead of BlockBackends
5. As backup-top owns these children, we also move block-copy state
into backup-top's ownership.
= Iotest changes =
56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.
Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.
141: Output changed: prepatch, "Node is in use" comes from bdrv_has_blk
check inside qmp_blockdev_del. But we've dropped block-copy blk
objects, so no more blk objects on source bs (job blk is on backup-top
filter bs). New message is from op-blocker, which is the next check in
qmp_blockdev_add.
257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191001131409.14202-6-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-01 15:14:09 +02:00
|
|
|
job->bcs = bcs;
|
2019-04-29 11:08:42 +02:00
|
|
|
job->cluster_size = cluster_size;
|
2019-10-01 15:14:06 +02:00
|
|
|
job->len = len;
|
qapi: backup: add perf.use-copy-range parameter
Experiments show, that copy_range is not always making things faster.
So, to make experimentation simpler, let's add a parameter. Some more
perf parameters will be added soon, so here is a new struct.
For now, add new backup qmp parameter with x- prefix for the following
reasons:
- We are going to add more performance parameters, some will be
related to the whole block-copy process, some only to background
copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other
block jobs, which will need performance options as well.. And it
should be the same structure or at least somehow related.
So, there are too much unclean things about how the interface and now
we need the new options mostly for testing. Let's keep them
experimental for a while.
In do_backup_common() new x-perf parameter handled in a way to
make further options addition simpler.
We add use-copy-range with default=true, and we'll change the default
in further patch, after moving backup to use block-copy.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-2-vsementsov@virtuozzo.com>
[mreitz: s/5\.2/6.0/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-16 22:46:43 +01:00
|
|
|
job->perf = *perf;
|
block/backup: avoid copying less than full target clusters
During incremental backups, if the target has a cluster size that is
larger than the backup cluster size and we are backing up to a target
that cannot (for whichever reason) pull clusters up from a backing image,
we may inadvertantly create unusable incremental backup images.
For example:
If the bitmap tracks changes at a 64KB granularity and we transmit 64KB
of data at a time but the target uses a 128KB cluster size, it is
possible that only half of a target cluster will be recognized as dirty
by the backup block job. When the cluster is allocated on the target
image but only half populated with data, we lose the ability to
distinguish between zero padding and uninitialized data.
This does not happen if the target image has a backing file that points
to the last known good backup.
Even if we have a backing file, though, it's likely going to be faster
to just buffer the redundant data ourselves from the live image than
fetching it from the backing file, so let's just always round up to the
target granularity.
The same logic applies to backup modes top, none, and full. Copying
fractional clusters without the guarantee of COW is dangerous, but even
if we can rely on COW, it's likely better to just re-copy the data.
Reported-by: Fam Zheng <famz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-id: 1456433911-24718-3-git-send-email-jsnow@redhat.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
2016-02-25 21:58:30 +01:00
|
|
|
|
block/block-copy: fix progress calculation
Assume we have two regions, A and B, and region B is in-flight now,
region A is not yet touched, but it is unallocated and should be
skipped.
Correspondingly, as progress we have
total = A + B
current = 0
If we reset unallocated region A and call progress_reset_callback,
it will calculate 0 bytes dirty in the bitmap and call
job_progress_set_remaining, which will set
total = current + 0 = 0 + 0 = 0
So, B bytes are actually removed from total accounting. When job
finishes we'll have
total = 0
current = B
, which doesn't sound good.
This is because we didn't considered in-flight bytes, actually when
calculating remaining, we should have set (in_flight + dirty_bytes)
as remaining, not only dirty_bytes.
To fix it, let's refactor progress calculation, moving it to block-copy
itself instead of fixing callback. And, of course, track in_flight
bytes count.
We still have to keep one callback, to maintain backup job bytes_read
calculation, but it will go on soon, when we turn the whole backup
process into one block_copy call.
Cc: qemu-stable@nongnu.org
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Message-Id: <20200311103004.7649-3-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-03-11 11:29:57 +01:00
|
|
|
block_copy_set_progress_meter(bcs, &job->common.job.progress);
|
2021-01-16 22:46:59 +01:00
|
|
|
block_copy_set_speed(bcs, speed);
|
2019-10-01 15:14:07 +02:00
|
|
|
|
block/backup: use backup-top instead of write notifiers
Drop write notifiers and use filter node instead.
= Changes =
1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.
2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.
3. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.
4. Block-copy is now using BdrvChildren instead of BlockBackends
5. As backup-top owns these children, we also move block-copy state
into backup-top's ownership.
= Iotest changes =
56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.
Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.
141: Output changed: prepatch, "Node is in use" comes from bdrv_has_blk
check inside qmp_blockdev_del. But we've dropped block-copy blk
objects, so no more blk objects on source bs (job blk is on backup-top
filter bs). New message is from op-blocker, which is the next check in
qmp_blockdev_add.
257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191001131409.14202-6-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-01 15:14:09 +02:00
|
|
|
/* Required permissions are already taken by backup-top target */
|
2017-01-17 11:56:42 +01:00
|
|
|
block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL,
|
|
|
|
&error_abort);
|
2016-11-08 07:50:38 +01:00
|
|
|
|
|
|
|
return &job->common;
|
2015-04-18 01:49:58 +02:00
|
|
|
|
|
|
|
error:
|
|
|
|
if (sync_bitmap) {
|
2019-09-16 16:19:09 +02:00
|
|
|
bdrv_reclaim_dirty_bitmap(sync_bitmap, NULL);
|
2015-04-18 01:49:58 +02:00
|
|
|
}
|
2019-10-17 16:21:22 +02:00
|
|
|
if (backup_top) {
|
block/backup: use backup-top instead of write notifiers
Drop write notifiers and use filter node instead.
= Changes =
1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.
2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.
3. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.
4. Block-copy is now using BdrvChildren instead of BlockBackends
5. As backup-top owns these children, we also move block-copy state
into backup-top's ownership.
= Iotest changes =
56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.
Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.
141: Output changed: prepatch, "Node is in use" comes from bdrv_has_blk
check inside qmp_blockdev_del. But we've dropped block-copy blk
objects, so no more blk objects on source bs (job blk is on backup-top
filter bs). New message is from op-blocker, which is the next check in
qmp_blockdev_add.
257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191001131409.14202-6-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-01 15:14:09 +02:00
|
|
|
bdrv_backup_top_drop(backup_top);
|
2016-04-14 12:59:55 +02:00
|
|
|
}
|
2016-11-08 07:50:38 +01:00
|
|
|
|
|
|
|
return NULL;
|
2013-06-24 17:13:11 +02:00
|
|
|
}
|