Switching to the target of the migration is done mostly asynchronously,
and reported to management via the BLOCK_JOB_COMPLETED event; the only
synchronous phase is opening the backing files. bdrv_open_backing_file
can always be done, even for migration of the full image (aka sync:
'full'). In this case, qmp_drive_mirror will create the target disk
with no backing file at all, and bdrv_open_backing_file will be a no-op.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This adds the monitor commands that start the mirroring job.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This patch adds the implementation of a new job that mirrors a disk to
a new image while letting the guest continue using the old image.
The target is treated as a "black box" and data is copied from the
source to the target in the background. This can be used for several
purposes, including storage migration, continuous replication, and
observation of the guest I/O in an external program. It is also a
first step in replacing the inefficient block migration code that is
part of QEMU.
The job is possibly never-ending, but it is logically structured into
two phases: 1) copy all data as fast as possible until the target
first gets in sync with the source; 2) keep target in sync and
ensure that reopening to the target gets a correct (full) copy
of the source data.
The second phase is indicated by the progress in "info block-jobs"
reporting the current offset to be equal to the length of the file.
When the job is cancelled in the second phase, QEMU will run the
job until the source is clean and quiescent, then it will report
successful completion of the job.
In other words, the BLOCK_JOB_CANCELLED event means that the target
may _not_ be consistent with a past state of the source; the
BLOCK_JOB_COMPLETED event means that the target is consistent with
a past state of the source. (Note that it could already happen
that management lost the race against QEMU and got a completion
event instead of cancellation).
It is not yet possible to complete the job and switch over to the target
disk. The next patches will fix this and add many refinements to the
basic idea introduced here. These include improved error management,
some tunable knobs and performance optimizations.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Even for jobs that need to be manually completed, management may want
to take care itself of the completion, not requiring the user to issue
a command to terminate the job. In this case we want to avoid that
they poll us continuously, waiting for completion to become available.
Thus, add a new event that signals the phase switch and the availability
of the block-job-complete command.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
While streaming can be dropped as soon as it progressed through the whole
image, mirroring needs to be completed manually for two reasons: 1) so that
management knows exactly when the VM switches to the target; 2) because
for other use cases such as replication, we may leave the operation running
for the whole life of the virtual machine.
Add a new block job command that manually completes background operations.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Assert that write_compressed is never used with the dirty bitmap.
Setting the bits early is wrong, because a coroutine might concurrently
examine them and copy incomplete data from the source.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Mirroring runs without the backing file so that it can be copied outside
QEMU. However, we need to add it at the time the job is completed and
QEMU switches to the target. Factor out the common bits of opening an
image and completing a mirroring operation.
The new function does not assume that the file is closed immediately after
it returns failure, so it keeps the BDRV_O_NO_BACKING flag up-to-date.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
qmp_query_blockstat cannot have errors, remove the Error argument and
create a new public function bdrv_query_stats out of it.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This option can be used for passing file descriptors on the
command line. It mirrors the existing add-fd QMP command which
allows an fd to be passed to QEMU via SCM_RIGHTS and added to an
fd set.
This can be combined with commands such as -drive to link file
descriptors in an fd set to a drive:
qemu-kvm -add-fd fd=3,set=2,opaque="rdwr:/path/to/file"
-add-fd fd=4,set=2,opaque="rdonly:/path/to/file"
-drive file=/dev/fdset/2,index=0,media=disk
This example adds dups of fds 3 and 4, and the accompanying opaque
strings to the fd set with ID=2. qemu_open() already knows how
to handle a filename of this format. qemu_open() searches the
corresponding fd set for an fd and when it finds a match, QEMU
goes on to use a dup of that fd just like it would have used an
fd that it opened itself.
Signed-off-by: Corey Bryant <coreyb@linux.vnet.ibm.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
If an fd is added to an fd set via the command line, and it is not
referenced by another command line option (ie. -drive), then clean
it up after QEMU initialization is complete.
Signed-off-by: Corey Bryant <coreyb@linux.vnet.ibm.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
qmp_add_fd() gets an fd that was received over a socket with
SCM_RIGHTS and adds it to an fd set. This patch adds support
that will enable adding an fd that was inherited on the
command line to an fd set.
Note: All of the code added to monitor_fdset_add_fd(), with the
exception of the error path for non-valid fdset-id, is code motion
from qmp_add_fd().
Signed-off-by: Corey Bryant <coreyb@linux.vnet.ibm.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
The first call to add an fd to an fd set was previously not
allowed to choose the fd set ID. The ID was generated as
the first available and ensuing calls could add more fds by
specifying the fd set ID. This change allows users to
choose the fd set ID on the first call.
Signed-off-by: Corey Bryant <coreyb@linux.vnet.ibm.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kashyap Chamarthy <kashyap.cv@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This new test verifies that qemu-img info --backing-chain safely aborts
when an image file has a backing file infinite loop.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
The previous block commit used absolute filenames for all block-commit
images and commands; this adds relative filenames for the same tests.
Signed-off-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This simplifies some code and error checking, and also fixes a bug.
bdrv_find_backing_image() should only be passed absolute filenames,
or filenames relative to the chain. In the QMP message handler for
block commit, when looking up the base do so from the determined top
image, so we know it is reachable from top.
Some of the error messages put out by block-commit have changed
slightly, which causes 2 tests cases for block-commit to fail.
This patch updates the test cases to look for the correct error
output.
Signed-off-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Currently, bdrv_find_backing_image compares bs->backing_file with
what is passed in as a backing_file name. Mismatches may occur,
however, when bs->backing_file and backing_file are not both
absolute or relative.
Use path_combine() to make sure any relative backing filenames are
relative to the current image filename being searched, and then use
realpath() to make all comparisons based on absolute filenames.
If either backing_file or bs->backing_file is determine to be a
protocol, then no filename normalization is performed.
This also changes bdrv_find_backing_image to no longer be recursive,
but iterative.
Signed-off-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This patch allows an empty filename to be passed as the new base image name
for qemu-img rebase to mean base the image on no backing file (i.e.
independent of any backing file). According to Eric Blake, qemu-img rebase
already supports this when '-u' is used; this adds support when -u is not
used.
Signed-off-by: Alex Bligh <alex@alex.org.uk>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
In QEMUMonitorProtocol, commit e9d17b6 removed the __sockfile creation
from __negotiate_capabilities(), which breaks _accept(). This causes
failures in qemu-io python based tests (i.e. tests 030 and 040).
This patch creates the sockfile in __accept() as well.
Signed-off-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Luiz Capitulino <lcapitulino@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Adding an NBD server inside QEMU is trivial, since all the logic is
in nbd.c and can be shared easily between qemu-nbd and QEMU itself.
The main difference is that qemu-nbd serves a single unnamed export,
while QEMU serves named exports.
Acked-by: Luiz Capitulino <lcapitulino@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
The first user of close notifiers will be the embedded NBD server.
It would be possible to use them to do some of the ad hoc processing
(e.g. for block jobs and I/O limits) that is currently done by
bdrv_close.
Acked-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
There is no reason in principle to skip job cancellation and draining
of pending I/O when there is no medium in the disk. Do these unconditionally,
which also prepares the code for the next patch.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
These are QAPI-friendly versions of the qemu-sockets functions. They
support IP sockets, Unix sockets, and named file descriptors, using a
QAPI union to dispatch to the correct function.
Reviewed-by: Luiz Capitulino <lcapitulino@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
We need them because qemu-sockets will soon be using SocketAddress.
Acked-by: Luiz Capitulino <lcapitulino@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
target_phys_addr_t is unwieldly, violates the C standard (_t suffixes are
reserved) and its purpose doesn't match the name (most target_phys_addr_t
addresses are not target specific). Replace it with a finger-friendly,
standards conformant hwaddr.
Outstanding patchsets can be fixed up with the command
git rebase -i --exec 'find -name "*.[ch]"
| xargs s/target_phys_addr_t/hwaddr/g' origin
Signed-off-by: Avi Kivity <avi@redhat.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
We now always return "nice" error messages in errp when we goto fail.
Drop the default error message.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Before:
$ qemu-system-x86_64 -monitor unix:/vvv,server=off
connect(unix:/vvv): No such file or directory
chardev: opening backend "socket" failed
After:
$ x86_64-softmmu/qemu-system-x86_64 -monitor unix:/vvv,server=off
qemu-system-x86_64: -monitor unix:/vvv,server=off: Failed to connect to socket: No such file or directory
chardev: opening backend "socket" failed
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
perror and fprintf can be removed because all clients can now consume
Errors properly. However, we'll need to change the non-blocking connect
handlers to take an Error, in order to improve error handling for
migration with the TCP protocol.
This is a minor degradation in error reporting for outgoing migration.
However, until 1.2 this case just failed without even attempting to
connect, so it is still an improvement as far as overall QoI is
concerned.
Reviewed-by: Luiz Capitulino <lcapitulino@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Among others, before:
$ qemu-system-x86_64 -chardev socket,port=12345,id=char
inet_connect: host and/or port not specified
chardev: opening backend "socket" failed
After:
$ x86_64-softmmu/qemu-system-x86_64 -chardev socket,port=12345,id=char
qemu-system-x86_64: -chardev socket,port=12345,id=char: host and/or port not specified
chardev: opening backend "socket" failed
perror and fprintf can be removed because all clients can now
consume Errors properly.
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Before:
$ qemu-system-x86_64 -vnc foo.bar:12345
getaddrinfo(foo.bar,18245): Name or service not known
Failed to start VNC server on `foo.bar:12345'
$ qemu-system-x86_64 -vnc localhost:12345,reverse=on
inet_connect_opts: connect(ipv4,yakj.usersys.redhat.com,127.0.0.1,12345): Connection refused
Failed to start VNC server on `localhost:12345,reverse=on'
After:
$ x86_64-softmmu/qemu-system-x86_64 -vnc foo.bar:12345
Failed to start VNC server on `foo.bar:12345': address resolution failed for foo.bar:18245: Name or service not known
$ x86_64-softmmu/qemu-system-x86_64 -vnc localhost:12345,reverse=on
Failed to start VNC server on `localhost:12345,reverse=on': Failed to connect to socket: Connection refused
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Before:
$ qemu-system-x86_64 nbd:localhost:12345
inet_connect_opts: connect(ipv4,yakj.usersys.redhat.com,127.0.0.1,12345): Connection refused
qemu-system-x86_64: could not open disk image nbd:localhost:12345: Connection refused
After:
$ x86_64-softmmu/qemu-system-x86_64 nbd:localhost:12345
qemu-system-x86_64: Failed to connect to socket: Connection refused
qemu-system-x86_64: could not open disk image nbd:localhost:12345: Connection refused
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
And remove the superfluous integer return value.
Reviewed-by: Luiz Capitulino <lcapitulino@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini
Corey Bryant
Luiz Capitulino
Kashyap Chamarthy
Stefan Hajnoczi
Jeff Cody
Alex Bligh
Kevin Wolf
Avi Kivity