The macOS jobs in our CI recently started failing, complaining that
the distutils module is not available anymore. And indeed, according to
https://peps.python.org/pep-0632/ it's been deprecated since a while
and now likely got removed in recent Python versions.
Fortunately, we only use it for a version check via LooseVersion here
which we don't really need anymore - according to Repology.org, these
are the versions of sphinx-rtd-theme that are currently used by the
various distros:
centos_stream_8: 0.3.1
centos_stream_9: 0.5.1
fedora_38: 1.1.1
fedora_39: 1.2.2
freebsd: 1.0.0
haikuports_master: 1.2.1
openbsd: 1.2.2
opensuse_leap_15_5: 0.5.1
pkgsrc_current: 2.0.0
debian_11: 0.5.1
debian_12: 1.2.0
ubuntu_20_04: 0.4.3
ubuntu_22_04: 1.0.0
ubuntu_24_04: 2.0.0
So except for CentOS 8, all distros are using a newer version of
sphinx-rtd-theme, and for CentOS 8 we don't support compiling with
the Sphinx of the distro anymore anyway, since it's based on the
Python 3.6 interpreter there. For compiling on CentOS 8, you have
to use the alternative Python 3.8 interpreter which comes without
Sphinx, so that needs the Sphinx installed via pip in the venv
instead, and that is using a newer version, too, according to our
pythondeps.toml file.
Thus we can simply drop the version check now to get rid of the
distutils dependency here.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Message-id: 20240304130403.129543-1-thuth@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Simplify the exec migration code by using list utility functions.
As a side effect, this also fixes a minor memory leak. On function return,
"g_auto(GStrv) argv" frees argv and each element, which is wrong, because
the function does not own the individual elements. To compensate, the code
uses g_steal_pointer which NULLs argv and prevents the destructor from
running, but argv is leaked.
Fixes: cbab4face5 ("migration: convert exec backend ...")
Signed-off-by: Steve Sistare <steven.sistare@oracle.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Message-ID: <20240227153321.467343-4-armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Avoid "JSON" when talking about the QAPI schema syntax. Capitalize
QEMU. Don't claim all HMP commands live in monitor/hmp-cmds.c (this
was never true). Fix punctuation and drop inappropriate "the" here
and there.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227115617.237875-3-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
The tutorial doesn't match reality since at least 2013. Repairing it
involves fixing the following issues:
* Update for commit 6d32717155 (aio / timers: Remove alarm timers):
replace the broken examples. Instead of having one for returning a
struct and another for returning a list of structs, do just one for
the latter. This resolves the FIXME added in commit
e218052f92 (aio / timers: De-document -clock) back in 2014.
* Update for commit 895a2a80e0 (qapi: Use 'struct' instead of 'type'
in schema).
* Update for commit 3313b6124b (qapi: add qapi2texi script): add
required documentation to the schema snippets, and drop section
"Command Documentation".
* Update for commit a3c45b3e62 (qapi: New special feature flag
"unstable"): supply the required feature, deemphasize the x- prefix.
* Update for commit dd98234c05 (qapi: introduce x-query-roms QMP
command): rephrase from "add new command" to "examine existing
command".
* Update for commit 9492718b7c (qapi misc: Elide redundant has_FOO in
generated C): hello-world's message argument no longer comes with a
has_message, add a second argument that does.
* Update for moved and renamed files.
While there, update QMP version output to current output.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227115617.237875-2-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Whitespace tidied up, typo fixed]
Documentation claims the command can "return NULL". "NULL" doesn't
exist in JSON. "null" does, but the command returns lists, and null
isn't. Correct documentation to "return an empty list".
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-13-armbru@redhat.com>
"Returns:" sections of guest-fsfreeze-freeze and
guest-fsfreeze-freeze-list describe both command behavior and success
response. Move behavior out, so "Returns:" is only about success
response.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-12-armbru@redhat.com>
Drop "on success" where it is redundant with "Returns:".
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-11-armbru@redhat.com>
Phrase it like "If <condition>, <error>", like we do elsewhere.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-8-armbru@redhat.com>
Drop list markup, since there's just one item left.
Drop "on success" where it is redundant with "Returns:".
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-7-armbru@redhat.com>
We use section "Returns" for documenting both success and error
response of commands.
I intend to generate better command success response documentation.
Easier when "Returns" documents just he success response.
Create new section tag "Errors". The next two commits will move error
response documentation from "Returns" sections to "Errors" sections.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-4-armbru@redhat.com>
Change "'Returns:' is only valid for commands" to "'Returns' section
is only valid for commands".
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-3-armbru@redhat.com>
This is chiefly to make code that looks up these sections easier to
read.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-2-armbru@redhat.com>
Add two documentations for mapped-ram migration on two spots that may not
be extremely clear.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240301091524.39900-1-peterx@redhat.com
Cc: Prasad Pandit <ppandit@redhat.com>
[peterx: fix two English errors per Prasad]
Signed-off-by: Peter Xu <peterx@redhat.com>
Add the missing 64-bit hppa firmware blob so that it gets installed.
Signed-off-by: Helge Deller <deller@gmx.de>
Fixes: 7c0dfcf939 ("target/hppa: Update SeaBIOS-hppa to version 16")
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
When calculating the IOR for the exception handlers, the current
unwind_breg value is needed on 64-bit hppa machines.
Restore that value by calling cpu_restore_state() earlier, which in turn
calls hppa_restore_state_to_opc() which restores the unwind_breg for the
current instruction.
Signed-off-by: Helge Deller <deller@gmx.de>
Fixes: 3824e0d643 ("target/hppa: Export function hppa_set_ior_and_isr()")
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Unaligned 64-bit accesses were found in Linux to clobber carry bits,
resulting in bad results if an arithmetic operation involving a
carry bit was executed after an unaligned 64-bit operation.
hppa 2.0 defines additional carry bits in PSW register bits 32..39.
When restoring PSW after executing an unaligned instruction trap, those
bits were not cleared and ended up to be active all the time. Since there
are no bits other than the upper carry bits needed in the upper 32 bit of
env->psw and since those are stored in env->psw_cb, just clear the entire
upper 32 bit when storing psw to solve the problem unconditionally.
Fixes: 931adff314 ("target/hppa: Update cpu_hppa_get/put_psw for hppa64")
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Charlie Jenkins <charlie@rivosinc.com>
Cc: Helge Deller <deller@gmx.de>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Helge Deller <deller@gmx.de>
Add a regression test for a recently fixed issue, where shmat()
desynced the guest and the host view of the address space and caused
open("/proc/self/maps") to SEGV.
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <jwyuvao4apydvykmsnvacwshdgy3ixv7qvkh4dbxm3jkwgnttw@k4wpaayou7oq>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* Fix GIC settings when using --without-default-devices
* Fix USB PCAP streams on Windows
* Remove temporary files from test-util-sockets
* Fix TLS io channels sending too much data to the backend
-----BEGIN PGP SIGNATURE-----
iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmXhjBcRHHRodXRoQHJl
ZGhhdC5jb20ACgkQLtnXdP5wLbWzZw/+OTSsKg6JTX0z3fjL6If+Ns/EnFiniHHI
y1jR7fPub3ybFokgHIWrRVG+9ol+1tJuPlO5Bsx1SKFzNC8++l1iyK7E25xnmp2P
Ry/Z0ZSWp3JWCtLm6km5pKl3wWI9p0XwfUOJbCWYM5dOsd2a4C4DSvjcwNXR7pTB
4AxXFj5G5UwXuffgPnBzeerm5baNweSqa/uczUY6Od+iJwmHpaSk0SJ4NIlIYvnS
z3BH81trQAoKYZyywp0aF+jH9w3YNwD1XDp2Bcf3qSWDdokN55V13yyiSllTfUwX
Lq3fotoLqFJL0A4m8a3TdCKMP8ReoJATSriPHJmGMgqWGTnvtRBIsx1pmMpYRB6U
EltaKW0WXeQAad0ZYGl4odx7kQUQjdc8HTQRTevQD9g3XpZZN0o9MfTPdM2eR5h6
xaS8UEc61YJAcmD27Ir92JGGGMLBTYz4OGLx3Rj7l7S759GQ67qih9TGcE9nqSj+
+wAHfiOjUZnSWnp7Hq4M/TIjNh0BuyBW4oRBASaikjjQ82Hih5bPTdRu1J8lYuXC
PhN8dLwdk0NMrnnHuFDKq/sXjDeXF3Hg0AmtVvOGiB2z2OeVSPtMxl/YgniGVTbj
CHXDqDrBAa7i6WxAo+TtUgE20iVz5uYzGqHmxTNpq6FzIRb1uvHksbjFlin79Pkw
eF0Dj80zxAY=
=9JAI
-----END PGP SIGNATURE-----
Merge tag 'pull-request-2024-03-01' of https://gitlab.com/thuth/qemu into staging
* Fix some bugs in the vring setup of libqos
* Fix GIC settings when using --without-default-devices
* Fix USB PCAP streams on Windows
* Remove temporary files from test-util-sockets
* Fix TLS io channels sending too much data to the backend
# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmXhjBcRHHRodXRoQHJl
# ZGhhdC5jb20ACgkQLtnXdP5wLbWzZw/+OTSsKg6JTX0z3fjL6If+Ns/EnFiniHHI
# y1jR7fPub3ybFokgHIWrRVG+9ol+1tJuPlO5Bsx1SKFzNC8++l1iyK7E25xnmp2P
# Ry/Z0ZSWp3JWCtLm6km5pKl3wWI9p0XwfUOJbCWYM5dOsd2a4C4DSvjcwNXR7pTB
# 4AxXFj5G5UwXuffgPnBzeerm5baNweSqa/uczUY6Od+iJwmHpaSk0SJ4NIlIYvnS
# z3BH81trQAoKYZyywp0aF+jH9w3YNwD1XDp2Bcf3qSWDdokN55V13yyiSllTfUwX
# Lq3fotoLqFJL0A4m8a3TdCKMP8ReoJATSriPHJmGMgqWGTnvtRBIsx1pmMpYRB6U
# EltaKW0WXeQAad0ZYGl4odx7kQUQjdc8HTQRTevQD9g3XpZZN0o9MfTPdM2eR5h6
# xaS8UEc61YJAcmD27Ir92JGGGMLBTYz4OGLx3Rj7l7S759GQ67qih9TGcE9nqSj+
# +wAHfiOjUZnSWnp7Hq4M/TIjNh0BuyBW4oRBASaikjjQ82Hih5bPTdRu1J8lYuXC
# PhN8dLwdk0NMrnnHuFDKq/sXjDeXF3Hg0AmtVvOGiB2z2OeVSPtMxl/YgniGVTbj
# CHXDqDrBAa7i6WxAo+TtUgE20iVz5uYzGqHmxTNpq6FzIRb1uvHksbjFlin79Pkw
# eF0Dj80zxAY=
# =9JAI
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 01 Mar 2024 08:04:39 GMT
# gpg: using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg: issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg: aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg: aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg: aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3 EAB9 2ED9 D774 FE70 2DB5
* tag 'pull-request-2024-03-01' of https://gitlab.com/thuth/qemu:
chardev/char-socket: Fix TLS io channels sending too much data to the backend
tests/unit/test-util-sockets: Remove temporary file after test
hw/usb/bus.c: PCAP adding 0xA in Windows version
hw/intc/Kconfig: Fix GIC settings when using "--without-default-devices"
libqos/virtio.c: fix 'avail_event' offset in qvring_init()
libqos/virtio.c: init all elems in qvring_indirect_desc_setup()
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
V2: fix build error on mipsel
-----BEGIN PGP SIGNATURE-----
iLMEAAEKAB0WIQS4/x2g0v3LLaCcbCxAov/yOSY+3wUCZeBrwAAKCRBAov/yOSY+
33YXA/4+A5Bpe/3+mSAWZSUlluGTqUi0ILBYRMyX1RXovMx4uCRGr7PXzAf03yKS
MZzlVzTuOK69WmTm/iTdYWOxkXisC3gzxL/wm8hP4lzh4c0dHrHRsKHqq6gR3+t2
ojdZn7TefeflnNqIhxXxgxb1OETofhBNnBJ74pvqxO7XV5SWnA==
=J2Kb
-----END PGP SIGNATURE-----
Merge tag 'pull-loongarch-20240229' of https://gitlab.com/gaosong/qemu into staging
pull-loongarch-20240229
V2: fix build error on mipsel
# -----BEGIN PGP SIGNATURE-----
#
# iLMEAAEKAB0WIQS4/x2g0v3LLaCcbCxAov/yOSY+3wUCZeBrwAAKCRBAov/yOSY+
# 33YXA/4+A5Bpe/3+mSAWZSUlluGTqUi0ILBYRMyX1RXovMx4uCRGr7PXzAf03yKS
# MZzlVzTuOK69WmTm/iTdYWOxkXisC3gzxL/wm8hP4lzh4c0dHrHRsKHqq6gR3+t2
# ojdZn7TefeflnNqIhxXxgxb1OETofhBNnBJ74pvqxO7XV5SWnA==
# =J2Kb
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 29 Feb 2024 11:34:24 GMT
# gpg: using RSA key B8FF1DA0D2FDCB2DA09C6C2C40A2FFF239263EDF
# gpg: Good signature from "Song Gao <m17746591750@163.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: B8FF 1DA0 D2FD CB2D A09C 6C2C 40A2 FFF2 3926 3EDF
* tag 'pull-loongarch-20240229' of https://gitlab.com/gaosong/qemu:
loongarch: Change the UEFI loading mode to loongarch
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
If we receive a file descriptor that points to a regular file, there's
nothing stopping us from doing multifd migration with mapped-ram to
that file.
Enable the fd: URI to work with multifd + mapped-ram.
Note that the fds passed into multifd are duplicated because we want
to avoid cross-thread effects when doing cleanup (i.e. close(fd)). The
original fd doesn't need to be duplicated because monitor_get_fd()
transfers ownership to the caller.
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Peter Xu <peterx@redhat.com>
Link: https://lore.kernel.org/r/20240229153017.2221-23-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
For the incoming mapped-ram migration we need to read the ramblock
headers, get the pages bitmap and send the host address of each
non-zero page to the multifd channel thread for writing.
Usage on HMP is:
(qemu) migrate_set_capability multifd on
(qemu) migrate_set_capability mapped-ram on
(qemu) migrate_incoming file:migfile
(the ram.h include needs to move because we've been previously relying
on it being included from migration.c. Now file.h will start including
multifd.h before migration.o is processed)
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240229153017.2221-22-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
The new mapped-ram stream format uses a file transport and puts ram
pages in the migration file at their respective offsets and can be
done in parallel by using the pwritev system call which takes iovecs
and an offset.
Add support to enabling the new format along with multifd to make use
of the threading and page handling already in place.
This requires multifd to stop sending headers and leaving the stream
format to the mapped-ram code. When it comes time to write the data, we
need to call a version of qio_channel_write that can take an offset.
Usage on HMP is:
(qemu) stop
(qemu) migrate_set_capability multifd on
(qemu) migrate_set_capability mapped-ram on
(qemu) migrate_set_parameter max-bandwidth 0
(qemu) migrate_set_parameter multifd-channels 8
(qemu) migrate file:migfile
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240229153017.2221-21-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
The mapped-ram migration can be performed live or non-live, but it is
always asynchronous, i.e. the source machine and the destination
machine are not migrating at the same time. We only need some pieces
of the multifd sync operations.
multifd_send_sync_main()
------------------------
Issued by the ram migration code on the migration thread, causes the
multifd send channels to synchronize with the migration thread and
makes the sending side emit a packet with the MULTIFD_FLUSH flag.
With mapped-ram we want to maintain the sync on the sending side
because that provides ordering between the rounds of dirty pages when
migrating live.
MULTIFD_FLUSH
-------------
On the receiving side, the presence of the MULTIFD_FLUSH flag on a
packet causes the receiving channels to start synchronizing with the
main thread.
We're not using packets with mapped-ram, so there's no MULTIFD_FLUSH
flag and therefore no channel sync on the receiving side.
multifd_recv_sync_main()
------------------------
Issued by the migration thread when the ram migration flag
RAM_SAVE_FLAG_MULTIFD_FLUSH is received, causes the migration thread
on the receiving side to start synchronizing with the recv
channels. Due to compatibility, this is also issued when
RAM_SAVE_FLAG_EOS is received.
For mapped-ram we only need to synchronize the channels at the end of
migration to avoid doing cleanup before the channels have finished
their IO.
Make sure the multifd syncs are only issued at the appropriate times.
Note that due to pre-existing backward compatibility issues, we have
the multifd_flush_after_each_section property that can cause a sync to
happen at EOS. Since the EOS flag is needed on the stream, allow
mapped-ram to just ignore it.
Also emit an error if any other unexpected flags are found on the
stream.
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Peter Xu <peterx@redhat.com>
Link: https://lore.kernel.org/r/20240229153017.2221-20-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
On the receiving side we don't need to differentiate between main
channel and threads, so whichever channel is defined first gets to be
the main one. And since there are no packets, use the atomic channel
count to index into the params array.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240229153017.2221-19-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
Allow multifd to open file-backed channels. This will be used when
enabling the mapped-ram migration stream format which expects a
seekable transport.
The QIOChannel read and write methods will use the preadv/pwritev
versions which don't update the file offset at each call so we can
reuse the fd without re-opening for every channel.
Contrary to the socket migration, the file migration doesn't need an
asynchronous channel creation process, so expose
multifd_channel_connect() and call it directly.
Note that this is just setup code and multifd cannot yet make use of
the file channels.
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Peter Xu <peterx@redhat.com>
Link: https://lore.kernel.org/r/20240229153017.2221-18-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
We'll need to access multifd_send_state->channels_created from outside
multifd.c, so introduce a helper for that.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240229153017.2221-17-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
Currently multifd does not need to have knowledge of pages on the
receiving side because all the information needed is within the
packets that come in the stream.
We're about to add support to mapped-ram migration, which cannot use
packets because it expects the ramblock section in the migration file
to contain only the guest pages data.
Add a data structure to transfer pages between the ram migration code
and the multifd receiving threads.
We don't want to reuse MultiFDPages_t for two reasons:
a) multifd threads don't really need to know about the data they're
receiving.
b) the receiving side has to be stopped to load the pages, which means
we can experiment with larger granularities than page size when
transferring data.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240229153017.2221-16-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
For the upcoming support to the new 'mapped-ram' migration stream
format, we cannot use multifd packets because each write into the
ramblock section in the migration file is expected to contain only the
guest pages. They are written at their respective offsets relative to
the ramblock section header.
There is no space for the packet information and the expected gains
from the new approach come partly from being able to write the pages
sequentially without extraneous data in between.
The new format also simply doesn't need the packets and all necessary
information can be taken from the standard migration headers with some
(future) changes to multifd code.
Use the presence of the mapped-ram capability to decide whether to
send packets.
This only moves code under multifd_use_packets(), it has no effect for
now as mapped-ram cannot yet be enabled with multifd.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240229153017.2221-15-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
Next patches will abstract the type of data being received by the
channels, so do some cleanup now to remove references to pages and
dependency on 'normal_num'.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240229153017.2221-14-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
Use a more specific name for the compression data so we can use the
generic for the multifd core code.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240229153017.2221-13-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
Add the necessary code to parse the format changes for the
'mapped-ram' capability.
One of the more notable changes in behavior is that in the
'mapped-ram' case ram pages are restored in one go rather than
constantly looping through the migration stream.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240229153017.2221-11-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
Implement the outgoing migration side for the 'mapped-ram' capability.
A bitmap is introduced to track which pages have been written in the
migration file. Pages are written at a fixed location for every
ramblock. Zero pages are ignored as they'd be zero in the destination
migration as well.
The migration stream is altered to put the dirty pages for a ramblock
after its header instead of having a sequential stream of pages that
follow the ramblock headers.
Without mapped-ram (current): With mapped-ram (new):
--------------------- --------------------------------
| ramblock 1 header | | ramblock 1 header |
--------------------- --------------------------------
| ramblock 2 header | | ramblock 1 mapped-ram header |
--------------------- --------------------------------
| ... | | padding to next 1MB boundary |
--------------------- | ... |
| ramblock n header | --------------------------------
--------------------- | ramblock 1 pages |
| RAM_SAVE_FLAG_EOS | | ... |
--------------------- --------------------------------
| stream of pages | | ramblock 2 header |
| (iter 1) | --------------------------------
| ... | | ramblock 2 mapped-ram header |
--------------------- --------------------------------
| RAM_SAVE_FLAG_EOS | | padding to next 1MB boundary |
--------------------- | ... |
| stream of pages | --------------------------------
| (iter 2) | | ramblock 2 pages |
| ... | | ... |
--------------------- --------------------------------
| ... | | ... |
--------------------- --------------------------------
| RAM_SAVE_FLAG_EOS |
--------------------------------
| ... |
--------------------------------
where:
- ramblock header: the generic information for a ramblock, such as
idstr, used_len, etc.
- ramblock mapped-ram header: the new information added by this
feature: bitmap of pages written, bitmap size and offset of pages
in the migration file.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240229153017.2221-10-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
The mapped-ram migration format needs a channel that supports seeking
to be able to write each page to an arbitrary offset in the migration
stream.
Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240229153017.2221-9-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
Add a new migration capability 'mapped-ram'.
The core of the feature is to ensure that RAM pages are mapped
directly to offsets in the resulting migration file instead of being
streamed at arbitrary points.
The reasons why we'd want such behavior are:
- The resulting file will have a bounded size, since pages which are
dirtied multiple times will always go to a fixed location in the
file, rather than constantly being added to a sequential
stream. This eliminates cases where a VM with, say, 1G of RAM can
result in a migration file that's 10s of GBs, provided that the
workload constantly redirties memory.
- It paves the way to implement O_DIRECT-enabled save/restore of the
migration stream as the pages are ensured to be written at aligned
offsets.
- It allows the usage of multifd so we can write RAM pages to the
migration file in parallel.
For now, enabling the capability has no effect. The next couple of
patches implement the core functionality.
Acked-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240229153017.2221-8-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
Add utility methods that will be needed when implementing 'mapped-ram'
migration capability.
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Link: https://lore.kernel.org/r/20240229153017.2221-7-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
Make sure the data is flushed to disk before closing file
channels. This is to ensure data is on disk and not lost in the event
of a host crash.
This is currently being implemented to affect the migration code when
migrating to a file, but all QIOChannelFile users should benefit from
the change.
Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Acked-by: "Daniel P. Berrangé" <berrange@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240229153017.2221-6-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
The upcoming 'mapped-ram' feature will require qemu to write data to
(and restore from) specific offsets of the migration file.
Add a minimal implementation of pwritev/preadv and expose them via the
io_pwritev and io_preadv interfaces.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240229153017.2221-5-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
