Commit Graph

93206 Commits

Author SHA1 Message Date
Peter Xu cfd66f30fb migration: Simplify unqueue_page()
This patch simplifies unqueue_page() on both sides of it (itself, and caller).

Firstly, due to the fact that right after unqueue_page() returned true, we'll
definitely send a huge page (see ram_save_huge_page() call - it will _never_
exit before finish sending that huge page), so unqueue_page() does not need to
jump in small page size if huge page is enabled on the ramblock.  IOW, it's
destined that only the 1st 4K page will be valid, when unqueue the 2nd+ time
we'll notice the whole huge page has already been sent anyway.  Switching to
operating on huge page reduces a lot of the loops of redundant unqueue_page().

Meanwhile, drop the dirty check.  It's not helpful to call test_bit() every
time to jump over clean pages, as ram_save_host_page() has already done so,
while in a faster way (see commit ba1b7c812c ("migration/ram: Optimize
ram_save_host_page()", 2021-05-13)).  So that's not necessary too.

Drop the two tracepoints along the way - based on above analysis it's very
possible that no one is really using it..

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Peter Xu a1fe28df75 migration: Add postcopy_has_request()
Add a helper to detect whether postcopy has pending request.

Since at it, cleanup the code a bit, e.g. in unqueue_page() we shouldn't need
to check it again on queue empty because we're the only one (besides cleanup
code, which should never run during this process) that will take a request off
the list, so the request list can only grow but not shrink under the hood.

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Peter Xu 2d1c37c603 migration: Enable UFFD_FEATURE_THREAD_ID even without blocktime feat
This patch allows us to read the tid even without blocktime feature enabled.
It's useful when tracing postcopy fault thread on faulted pages to show thread
id too with the address.

Remove the comments - they're merely not helpful at all.

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Peter Xu 258f5c9825 migration: No off-by-one for pss->page update in host page size
We used to do off-by-one fixup for pss->page when finished one host huge page
transfer.  That seems to be unnecesary at all.  Drop it.

Cc: Keqian Zhu <zhukeqian1@huawei.com>
Cc: Kunkun Jiang <jiangkunkun@huawei.com>
Cc: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
David Edmondson ae68066880 migration: Tally pre-copy, downtime and post-copy bytes independently
Provide information on the number of bytes copied in the pre-copy,
downtime and post-copy phases of migration.

Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
David Edmondson 4c2d0f6dca migration: Introduce ram_transferred_add()
Replace direct manipulation of ram_counters.transferred with a
function.

Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Philippe Mathieu-Daudé 9e7d1223ac migration: Don't return for postcopy_send_discard_bm_ram()
postcopy_send_discard_bm_ram() always return zero. Since it can't
fail, simplify and do not return anything.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: David Edmondson <david.edmondson@oracle.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Peter Xu 739fcc1b0e migration: Drop return code for disgard ram process
It will just never fail.  Drop those return values where they're constantly
zeros.

A tiny touch-up on the tracepoint so trace_ram_postcopy_send_discard_bitmap()
is called after the logic itself (which sounds more reasonable).

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Peter Xu f30c2e5ba8 migration: Do chunk page in postcopy_each_ram_send_discard()
Right now we loop ramblocks for twice, the 1st time chunk the dirty bits with
huge page information; the 2nd time we send the discard ranges.  That's not
necessary - we can do them in a single loop.

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Peter Xu e3fbf76021 migration: Drop postcopy_chunk_hostpages()
This function calls three functions:

  - postcopy_discard_send_init(ms, block->idstr);
  - postcopy_chunk_hostpages_pass(ms, block);
  - postcopy_discard_send_finish(ms);

However only the 2nd function call is meaningful.  It's major role is to make
sure dirty bits are applied in host-page-size granule, so there will be no
partial dirty bits set for a whole host page if huge pages are used.

The 1st/3rd call are for latter when we want to send the disgard ranges.
They're mostly no-op here besides some tracepoints (which are misleading!).

Drop them, then we can directly drop postcopy_chunk_hostpages() as a whole
because we can call postcopy_chunk_hostpages_pass() directly.

There're still some nice comments above postcopy_chunk_hostpages() that explain
what it does.  Copy it over to the caller's site.

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Peter Xu dc57d6f2ec migration: Don't return for postcopy_chunk_hostpages()
It always return zero, because it just can't go wrong so far.  Simplify the
code with no functional change.

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Peter Xu 53405ffb33 migration: Drop dead code of ram_debug_dump_bitmap()
I planned to add "#ifdef DEBUG_POSTCOPY" around the function too because
otherwise it'll be compiled into qemu binary even if it'll never be used.  Then
I found that maybe it's easier to just drop it for good..

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Xu Zheng a6d1223b4a migration/ram: clean up unused comment.
Just a removal of an unused comment.
a0a8aa147a did many fixes and removed the parameter named "ms", but forget to remove the corresponding comment in function named "ram_save_host_page".

Signed-off-by: Xu Zheng <xuzheng@cmss.chinamobile.com>
Signed-off-by: Mao Zhongyi <maozhongyi@cmss.chinamobile.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
David Edmondson 9200265838 migration: Report the error returned when save_live_iterate fails
Should qemu_savevm_state_iterate() encounter a failure when calling a
particular save_live_iterate function, report the error code returned
by the function.

Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Zhang Chen 01ee5e3556 migration/migration.c: Remove the MIGRATION_STATUS_ACTIVE when migration finished
The MIGRATION_STATUS_ACTIVE indicates that migration is running.
Remove it to be handled by the default operation,
It should be part of the unknown ending states.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Zhang Chen eeeb48ee33 migration/migration.c: Avoid COLO boot in postcopy migration
COLO dose not support postcopy migration and remove the Fixme.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Zhang Chen 444252b96a migration/migration.c: Add missed default error handler for migration state
In the migration_completion() no other status is expected, for
example MIGRATION_STATUS_CANCELLING, MIGRATION_STATUS_CANCELLED, etc.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Peter Maydell 17e3134061 Remove unnecessary minimum_version_id_old fields
The migration code will not look at a VMStateDescription's
minimum_version_id_old field unless that VMSD has set the
load_state_old field to something non-NULL.  (The purpose of
minimum_version_id_old is to specify what migration version is needed
for the code in the function pointed to by load_state_old to be able
to handle it on incoming migration.)

We have exactly one VMSD which still has a load_state_old,
in the PPC CPU; every other VMSD which sets minimum_version_id_old
is doing so unnecessarily. Delete all the unnecessary ones.

Commit created with:
  sed -i '/\.minimum_version_id_old/d' $(git grep -l '\.minimum_version_id_old')
with the one legitimate use then hand-edited back in.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>

Signed-off-by: Juan Quintela <quintela@redhat.com>

---

It missed vmstate_ppc_cpu.
2022-01-28 15:38:23 +01:00
Juan Quintela 8c0ec0b2b0 multifd: Rename pages_used to normal_pages
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela faf60935df multifd: recv side only needs the RAMBlock host address
So we can remove the MultiFDPages.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela cf2d4aa8a2 multifd: Use normal pages array on the recv side
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

---

Rename num_normal_pages to total_normal_pages (peter)
2022-01-28 15:38:23 +01:00
Juan Quintela 815956f039 multifd: Use normal pages array on the send side
We are only sending normal pages through multifd channels.
Later on this series, we are going to also send zero pages.
We are going to detect if a page is zero or non zero in the multifd
channel thread, not on the main thread.

So we receive an array of pages page->offset[N]

And we will end with:

p->normal[N - zero_pages]
p->zero[zero_pages].

In this patch, we just copy all the pages in offset to normal.

for (i = 0; i < pages->num; i++) {
    p->narmal[p->normal_num] = pages->offset[i];
    p->normal_num++:
}

Later in the series this becomes:

for (i = 0; i < pages->num; i++) {
    if (buffer_is_zero(page->offset[i])) {
        p->zerol[p->zero_num] = pages->offset[i];
        p->zero_num++:
    } else {
        p->narmal[p->normal_num] = pages->offset[i];
        p->normal_num++:
    }
}

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

---

Improving comment (dave)
Renaming num_normal_pages to total_normal_pages (peter)
2022-01-28 15:38:23 +01:00
Juan Quintela c27779a215 multifd: Unfold "used" variable by its value
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela d48c3a0445 multifd: Use a single writev on the send side
Until now, we wrote the packet header with write(), and the rest of the
pages with writev().  Just increase the size of the iovec and do a
single writev().

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela 468fcb5dd0 multifd: Remove send_write() method
Everything use now iov's.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela 0a818b89eb multifd: Make zstd use iov's
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela 48a4a44c1c multifd: Make zlib use iov's
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela 226468ba3d multifd: Move iov from pages to params
This will allow us to reduce the number of system calls on the next patch.

Signed-off-by: Juan Quintela <quintela@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela fc6705229c multifd: Use proper maximum compression values
It happens that there are functions to calculate the worst possible
compression size for a packet.  Use them.

Suggested-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela 47fe16ff66 migration: Move ram_release_pages() call to save_zero_page_to_file()
We always need to call it when we find a zero page, so put it in a
single place.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela e7f2e190e5 migration: simplify do_compress_ram_page
The goto is not needed at all.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela 20d549cb0b migration: Remove masking for compression
Remove the mask in the call to ram_release_pages().  Nothing else does
it, and if the offset has that bits set, we have a lot of trouble.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2022-01-28 15:38:23 +01:00
Juan Quintela 0189c72291 migration: ram_release_pages() always receive 1 page as argument
Remove the pages argument. And s/pages/page/

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>

---

- Use 1LL instead of casts (philmd)
- Change the whole 1ULL for TARGET_PAGE_SIZE
2022-01-28 15:38:22 +01:00
Juan Quintela 05931ec561 migration: We only need last_stage in two places
We only need last_stage in two places and we are passing it all
around.  Just add a field to RAMState that passes it.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>

---

Repeat subject (philmd suggestion)
2022-01-28 15:38:22 +01:00
Juan Quintela 04e1140494 migration: All this fields are unsigned
So printing it as %d is wrong.  Notice that for the channel id, that
is an uint8_t, but I changed it anyways for consistency.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
2022-01-28 15:38:22 +01:00
Peter Maydell b367db4812 Xen patches
- bug fixes for mapcache and ioreq handling
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEE+AwAYwjiLP2KkueYDPVXL9f7Va8FAmHyt4IACgkQDPVXL9f7
 Va8SVgf+N6uOYoCVooWFk5H6MpXPRGLNw7YB1L5bR1T8DmXef+149p3NaqYuD3Kr
 Fz+6Kc1xpRivtfz7psy6ME+VVLQaCL5pYDYvoWBW/EPDJaBIGe6oednY4N0KEPZH
 4AqyuLwNZ+xPhv78YMnjmvF6foIiuO712yAwlG7oi+E+wp25SlKi6DaWHlY2U4/F
 hRC4CS3fJKUONn+s511EdSYhjZVqC0+Dp5pyAPGShyp7AMFQUV7mKl+lC2E3/gWo
 Qudq/VqX/k4dlLt1zudh/qQTUxJA+nddM1a7grvXmgZL0nngdvKyXQHTOv60HYgF
 yFC5OkHu9XohxYaY8gQYNreYiqCxeQ==
 =p6C5
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/aperard/tags/pull-xen-20220127' into staging

Xen patches

- bug fixes for mapcache and ioreq handling

# gpg: Signature made Thu 27 Jan 2022 15:17:22 GMT
# gpg:                using RSA key F80C006308E22CFD8A92E7980CF5572FD7FB55AF
# gpg: Good signature from "Anthony PERARD <anthony.perard@gmail.com>" [unknown]
# gpg:                 aka "Anthony PERARD <anthony.perard@citrix.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 5379 2F71 024C 600F 778A  7161 D8D5 7199 DF83 42C8
#      Subkey fingerprint: F80C 0063 08E2 2CFD 8A92  E798 0CF5 572F D7FB 55AF

* remotes/aperard/tags/pull-xen-20220127:
  xen-mapcache: Avoid entry->lock overflow
  xen-hvm: Allow disabling buffer_io_timer

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2022-01-28 11:05:29 +00:00
Peter Maydell cfe63e46be QAPI patches patches for 2022-01-27
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmHyqgcSHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZT3A0P/3FpDflF8zwLlKEuTaLdprUCowEjouTq
 8lIKE1La0NK+cvuH0lDMSVLitqdk57WzopLvoSU7sSUXtvb9oJuJNnBPiCXVU8Ou
 uapzUnYxSM0KF3ehRtH+eOefyQ9MWpdzfzImmR6DDFjf1Ff03g4dNW9qrNUbMl9D
 Oy0ZUMsTms3OtXqkKeg9giAbDOARYgHcHcQEkm41xp28ORMUJvORiP6HV7CTWF4Q
 M2fLNT2cOtKZcby+YBU1HUYARFZuAII5718OD4YDWcv+ADMpwp0yCl7BRtLHDDNu
 cSm+6KNF+r8SfvwyLVHIvA1W5jP32lzJ/VpgK+2N5mWw8wKAgGzvIZHL8i5rE+b+
 NHXv3bFburrBAM/nzHntbFxzLCl5ysgWvQb32Uc/sDd6KKOwHqf1NqoBsn1POn3D
 AeQCBaXGlgBpKmeHeU99GnmgV5UKputHahAQqTzslSy8OKkEK9P0GTyEwl9KUmgJ
 g8rY2fhsSP8j+dwSRIhHE6B3WDVJ+bxUMUg0Io3BZjAP7SQosoUUYZYhOubTn4B4
 6usz2+QVoJePokC7GFl+LdyO5NOIfxvuRkCH9nw3q5eU5WYskvrh/vBvQ8H9UAvP
 NBeFHdKwTm82ZGTKz18w2/9imyy5NaAZGte0S3isDOLdGdUqmDxUC0hsHuibrSq2
 hjsQSYrsGS5C
 =gEIj
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2022-01-27-v2' into staging

QAPI patches patches for 2022-01-27

# gpg: Signature made Thu 27 Jan 2022 14:19:51 GMT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2022-01-27-v2:
  qapi: generate trace events by default
  meson: document why we don't generate trace events for tests/ and qga/
  docs/qapi-code-gen: update to cover trace events code generation
  meson: generate trace events for qmp commands
  qapi/commands: Optionally generate trace for QMP commands
  qapi/commands: refactor error handling code
  qapi/gen: Add FOO.trace-events output module
  schemas: add missing vim modeline

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2022-01-27 15:45:13 +00:00
Ross Lagerwall a021a2dd8b xen-mapcache: Avoid entry->lock overflow
In some cases, a particular mapcache entry may be mapped 256 times
causing the lock field to wrap to 0. For example, this may happen when
using emulated NVME and the guest submits a large scatter-gather write.
At this point, the entry map be remapped causing QEMU to write the wrong
data or crash (since remap is not atomic).

Avoid this overflow by increasing the lock field to a uint32_t and also
detect it and abort rather than continuing regardless.

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Message-Id: <20220124104450.152481-1-ross.lagerwall@citrix.com>
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
2022-01-27 15:14:21 +00:00
Jason Andryuk 9288e803e6 xen-hvm: Allow disabling buffer_io_timer
commit f37f29d314 "xen: slightly simplify bufioreq handling" hard
coded setting req.count = 1 during initial field setup before the main
loop.  This missed a subtlety that an early exit from the loop when
there are no ioreqs to process, would have req.count == 0 for the return
value.  handle_buffered_io() would then remove state->buffered_io_timer.
Instead handle_buffered_iopage() is basically always returning true and
handle_buffered_io() always re-setting the timer.

Restore the disabling of the timer by introducing a new handled_ioreq
boolean and use as the return value.  The named variable will more
clearly show the intent of the code.

Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Message-Id: <20211210193434.75566-1-jandryuk@gmail.com>
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
2022-01-27 15:14:11 +00:00
Vladimir Sementsov-Ogievskiy 761a1a488e qapi: generate trace events by default
We don't generate trace events for tests/ and qga/ because that it is
not simple and not necessary. We have corresponding comments in both
tests/meson.build and qga/meson.build.

Still to not miss possible future qapi code generation call, and not to
forget to enable trace events generation, let's enable it by default.
So, turn option --gen-trace into opposite --no-trace-events and use new
option only in tests/ and qga/ where we already have good comments why
we don't generate trace events code.

Note that this commit enables trace-events generation for qapi-gen.py
call from tests/qapi-schema/meson.build and storage-daemon/meson.build.
Still, both are kind of noop: tests/qapi-schema/ doesn't seem to
generate any QMP command code and no .trace-events files anyway,
storage-daemon/ uses common QMP command implementations and just
generate empty .trace-events

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220126161130.3240892-8-vsementsov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 15:17:35 +01:00
Vladimir Sementsov-Ogievskiy 378dfa482d meson: document why we don't generate trace events for tests/ and qga/
Making trace generation work for tests/ and qga/ would involve some
Meson hackery to ensure we generate the trace-events files before
trace-tool uses them. Since we don't actually support tracing there
anyway, we bypass that problem.

Let's add corresponding comments.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20220126161130.3240892-7-vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Pasto fixed, commit message punctuation tidied up]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 15:17:35 +01:00
Vladimir Sementsov-Ogievskiy ff8e4827ad docs/qapi-code-gen: update to cover trace events code generation
Previous commits enabled trace events generation for most of QAPI
generated code (except for tests/ and qga/). Let's update documentation
to illustrate it.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20220126161130.3240892-6-vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 15:16:28 +01:00
Vladimir Sementsov-Ogievskiy b83a80e831 meson: generate trace events for qmp commands
1. Use --gen-trace when generate qmp commands
2. Add corresponding .trace-events files as outputs in qapi_files
   custom target
3. Define global qapi_trace_events list of .trace-events file targets,
   to fill in trace/qapi.build and to use in trace/meson.build
4. In trace/meson.build use the new array as an additional source of
   .trace_events files to be processed

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220126161130.3240892-5-vsementsov@virtuozzo.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 11:26:40 +01:00
Vladimir Sementsov-Ogievskiy bd2017bc41 qapi/commands: Optionally generate trace for QMP commands
Add trace generation disabled by default and new option --gen-trace to
enable it.  The next commit will enable it for qapi/, but not for qga/
and tests/.  Making it work for the latter two would involve some Meson
hackery to ensure we generate the trace-events files before trace-tool
uses them.  Since we don't actually support tracing there, we'll bypass
that problem.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220126161130.3240892-4-vsementsov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Superfluous #include dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 11:25:52 +01:00
Vladimir Sementsov-Ogievskiy 167d913f34 qapi/commands: refactor error handling code
Move error_propagate() to if (err) and make "if (err)" block mandatory.
This is to simplify further commit, which will bring trace events
generation for QMP commands.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220126161130.3240892-3-vsementsov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 11:25:09 +01:00
Vladimir Sementsov-Ogievskiy 4e86df1732 qapi/gen: Add FOO.trace-events output module
We are going to generate trace events for QMP commands. We should
generate both trace_*() function calls and trace-events files listing
events for trace generator.

So, add an output module FOO.trace-events for each FOO schema module.

Since we're going to add trace events only to command marshallers,
make the trace-events output optional, so we don't generate so many
useless empty files.

Currently nobody set add_trace_events to True, so new functionality is
disabled. It will be enabled for QAPISchemaGenCommandVisitor
in a further commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220126161130.3240892-2-vsementsov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 11:24:18 +01:00
Victor Toso 7e7237cd2b schemas: add missing vim modeline
Similar to f7160f3218 "schemas: Add vim modeline"

Signed-off-by: Victor Toso <victortoso@redhat.com>
Message-Id: <20211220145624.52801-1-victortoso@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2022-01-27 11:20:50 +01:00
Peter Maydell 48302d4eb6 virtiofsd: Security fix
Fixes: CVE-2022-0358
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEERfXHG0oMt/uXep+pBRYzHrxb/ecFAmHxJpQACgkQBRYzHrxb
 /ecd/xAAl4o5rqmLHIpuzUbCbfYliFkBqn9X1zsZc3VJ1T3e3Vxq36h4kUQGY63g
 4KPRghjjO3v3+YU9a3Tmlp704TaalNWhOMCTWnFT28Pet2u6492zDjGUz3h5scms
 18NZmyjmFHREumeSFM5s7rgouhhsa8dv8pqrkvIezBAmp6BQDhZThkd4dlx1OTAq
 zdiOwCmEY/1z55bPFBeU6Tz1BMGgWP+0kqoY2YLFms+kRgWrmacsNsUfdAXSuLr1
 jUosr3f96inwts9dlTVR0jdMs1/Dh0eLSuasWa1VgQmak4vYdx3ytiqjf1eaLyjc
 r41GQOvqyqRy+BqzjXLHkMOXdwMKGTtj9pNBnnKMnEcUA4Kja0d4i37Gf9T3tWT7
 WVcA1It3kdYuXPrxY1++toasCv4BId8dofJWnjgULf8gsEgKdLm/Aee9EiUYsBHv
 OPdX/iQB1B3eyc4g6vZHwnQNx7r2+yvh6eSho9NiRD57u8FZwuHWXudVTrZ9QFwZ
 AQ7p6b0qf9Un5n7fQ48t2hJKmHLKGZmiLp4zVCqSHP0DF5wImepwgOIHaqxaToN/
 iu/Y6tc9UZarNFKA3rSKSlRrerbRVPEL36+1KyYZHOq3685q+xIsK64iVD6Z0FeX
 9deb0lOCYQkG/fhetv1HTsCLlqAUK+M4Vk4g+4pd176gjOUBRhw=
 =7mdJ
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/dgilbert-gitlab/tags/pull-virtiofs-20220126' into staging

virtiofsd: Security fix

Fixes: CVE-2022-0358

# gpg: Signature made Wed 26 Jan 2022 10:46:44 GMT
# gpg:                using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert-gitlab/tags/pull-virtiofs-20220126:
  virtiofsd: Drop membership of all supplementary groups (CVE-2022-0358)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2022-01-26 10:59:50 +00:00
Vivek Goyal 449e8171f9 virtiofsd: Drop membership of all supplementary groups (CVE-2022-0358)
At the start, drop membership of all supplementary groups. This is
not required.

If we have membership of "root" supplementary group and when we switch
uid/gid using setresuid/setsgid, we still retain membership of existing
supplemntary groups. And that can allow some operations which are not
normally allowed.

For example, if root in guest creates a dir as follows.

$ mkdir -m 03777 test_dir

This sets SGID on dir as well as allows unprivileged users to write into
this dir.

And now as unprivileged user open file as follows.

$ su test
$ fd = open("test_dir/priviledge_id", O_RDWR|O_CREAT|O_EXCL, 02755);

This will create SGID set executable in test_dir/.

And that's a problem because now an unpriviliged user can execute it,
get egid=0 and get access to resources owned by "root" group. This is
privilege escalation.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2044863
Fixes: CVE-2022-0358
Reported-by: JIETAO XIAO <shawtao1125@gmail.com>
Suggested-by: Miklos Szeredi <mszeredi@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Message-Id: <YfBGoriS38eBQrAb@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  dgilbert: Fixed missing {}'s style nit
2022-01-26 10:32:05 +00:00
Peter Maydell aeb0ae95b7 Python patches
A few fixes to the Python CI tests, a few fixes to the (async) QMP
 library, and a set of patches that begin to shift us towards using the
 new qmp lib.
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+ber27ys35W+dsvQfe+BBqr8OQ4FAmHrSt4ACgkQfe+BBqr8
 OQ4BLg/5AdhcWsAtKq+xZj/nz2DNAbvXmrGJRuVln1yofrj22w2MYUTGfpQ0m8JO
 Ezu+LYtSTPQAaQw54WByaliD5h2ucYl5W8H13cDc8NPZkbsX+dD7G99u4XkSIY4I
 sSCYDMKi4j/P+4YR2MN1Iol4362fWbi3O3rsRX6PqNymIAaaklDrH3QZCGMiBMjl
 2OAcgba31uguyXnMuM3WY8XAnnRsib3wZ/a+a3WWcEsEm1HAUC0pb8VmrRzH5Rv9
 CpR3EBYoVc3of96jd/qLjucnoUL0+K2RVN2qPeie3+o7yEM2VYj5o+cG2H8pEG5p
 Fk/J7kqs0XWBOeX3A3IlGqKEXFPGDjEJZpcjwd8+IhgA2Y/MByTqRr1EvrOSO+bg
 q3njEg5DsORQS/xgZrnAidk5fdgLj7Cv39LfsxMnv77RBnlLubEAet7pT1XtprAv
 DI7STKknVpPu0VtYI8ALVjVhpeCkIt95DXACMtPZiSJ5X1NdoY5qubV1y8/vsExI
 RMDMepcS2A75Un2DA1bkStHTPN2PSUfM15fmUCebxbHp53FlJCh44gxAAsfj9j41
 xUmwSz1c81bCU4m+jsMBdNrbtkpPz/gX/3ZS8KqGoZmWN0wDkh3vEYFj5Y/310HY
 xmzug6o+tR7OD3bBGxZ73k9rn86X3+1PsYxOZjvYM0wiJIisPk4=
 =D2kj
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/jsnow-gitlab/tags/python-pull-request' into staging

Python patches

A few fixes to the Python CI tests, a few fixes to the (async) QMP
library, and a set of patches that begin to shift us towards using the
new qmp lib.

# gpg: Signature made Sat 22 Jan 2022 00:07:58 GMT
# gpg:                using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F  18F2 88A9 064D 1835 61EB
#      Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76  CBD0 7DEF 8106 AAFC 390E

* remotes/jsnow-gitlab/tags/python-pull-request:
  scripts/render-block-graph: switch to AQMP
  scripts/cpu-x86-uarch-abi: switch to AQMP
  scripts/cpu-x86-uarch-abi: fix CLI parsing
  python: move qmp-shell under the AQMP package
  python: move qmp utilities to python/qemu/utils
  python/qmp: switch qmp-shell to AQMP
  python/qmp: switch qom tools to AQMP
  python/qmp: switch qemu-ga-client to AQMP
  python/qemu-ga-client: don't use deprecated CLI syntax in usage comment
  python/aqmp: rename AQMPError to QMPError
  python/aqmp: add SocketAddrT to package root
  python/aqmp: copy type definitions from qmp
  python/aqmp: handle asyncio.TimeoutError on execute()
  python/aqmp: add __del__ method to legacy interface
  python/aqmp: fix docstring typo
  python: use avocado's "new" runner
  python: pin setuptools below v60.0.0

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2022-01-22 12:03:22 +00:00