Commit Graph

107969 Commits

Author SHA1 Message Date
Juan Quintela ebdb85f9d1 migration/rdma: Use i as for index instead of idx
Once there, all the uses are local to the for, so declare the variable
inside the for statement.

Reviewed-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231011203527.9061-12-quintela@redhat.com>
2023-10-17 09:25:13 +02:00
Juan Quintela a4832d299d migration/rdma: Check sooner if we are in postcopy for save_page()
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231011203527.9061-11-quintela@redhat.com>
2023-10-17 09:25:13 +02:00
Juan Quintela b1b3838722 migration/rdma: Remove qemu_ prefix from exported functions
Functions are long enough even without this.

Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231011203527.9061-10-quintela@redhat.com>
2023-10-17 09:25:13 +02:00
Juan Quintela 10cb3336b1 migration/rdma: Move rdma constants from qemu-file.h to rdma.h
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231011203527.9061-9-quintela@redhat.com>
2023-10-17 09:25:13 +02:00
Juan Quintela 8b670f48ed qemu-file: Remove QEMUFileHooks
The only user was rdma, and its use is gone.

Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231011203527.9061-8-quintela@redhat.com>
2023-10-17 09:25:13 +02:00
Juan Quintela e493008d50 migration/rdma: Create rdma_control_save_page()
The only user of ram_control_save_page() and save_page() hook was
rdma. Just move the function to rdma.c, rename it to
rdma_control_save_page().

Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231011203527.9061-7-quintela@redhat.com>
2023-10-17 09:25:13 +02:00
Juan Quintela a6323300e8 migration/rdma: Unfold hook_ram_load()
There is only one flag called with: RAM_CONTROL_BLOCK_REG.

Reviewed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231011203527.9061-6-quintela@redhat.com>
2023-10-17 09:25:13 +02:00
Juan Quintela f6d6c089b7 migration/rdma: Remove all uses of RAM_CONTROL_HOOK
Instead of going through ram_control_load_hook(), call
qemu_rdma_registration_handle() directly.

Reviewed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231011203527.9061-5-quintela@redhat.com>
2023-10-17 09:25:13 +02:00
Juan Quintela 5f5b8858dc migration/rdma: Unfold ram_control_after_iterate()
Once there:
- Remove unused data parameter
- unfold it in its callers
- change all callers to call qemu_rdma_registration_stop()
- We need to call QIO_CHANNEL_RDMA() after we check for migrate_rdma()

Reviewed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231011203527.9061-4-quintela@redhat.com>
2023-10-17 09:25:13 +02:00
Juan Quintela 48408174a7 migration/rdma: Unfold ram_control_before_iterate()
Once there:
- Remove unused data parameter
- unfold it in its callers.
- change all callers to call qemu_rdma_registration_start()
- We need to call QIO_CHANNEL_RDMA() after we check for migrate_rdma()

Reviewed-by: Li Zhijian <lizhijian@fujitsu.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231011203527.9061-3-quintela@redhat.com>
2023-10-17 09:25:13 +02:00
Juan Quintela 27fd25b0fb migration: Create migrate_rdma()
Helper to say if we are doing a migration over rdma.

Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231011203527.9061-2-quintela@redhat.com>
2023-10-17 09:25:13 +02:00
Juan Quintela d4f34485ca migration: Non multifd migration don't care about multifd flushes
RDMA was having trouble because
migrate_multifd_flush_after_each_section() can only be true or false,
but we don't want to send any flush when we are not in multifd
migration.

CC: Fabiano Rosas <farosas@suse.de
Fixes: 294e5a4034 ("multifd: Only flush once each full round of memory")

Reported-by: Li Zhijian <lizhijian@fujitsu.com>
Reviewed-by: Li Zhijian <lizhijian@fujitsu.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231011205548.10571-2-quintela@redhat.com>
2023-10-17 09:25:13 +02:00
Fiona Ebner 930e239d11 migration: hold the BQL during setup
This is intended to be a semantic revert of commit 9b09503752
("migration: run setup callbacks out of big lock"). There have been so
many changes since that commit (e.g. a new setup callback
dirty_bitmap_save_setup() that also needs to be adapted now), it's
easier to do the revert manually.

For snapshots, the bdrv_writev_vmstate() function is used during setup
(in QIOChannelBlock backing the QEMUFile), but not holding the BQL
while calling it could lead to an assertion failure. To understand
how, first note the following:

1. Generated coroutine wrappers for block layer functions spawn the
coroutine and use AIO_WAIT_WHILE()/aio_poll() to wait for it.
2. If the host OS switches threads at an inconvenient time, it can
happen that a bottom half scheduled for the main thread's AioContext
is executed as part of a vCPU thread's aio_poll().

An example leading to the assertion failure is as follows:

main thread:
1. A snapshot-save QMP command gets issued.
2. snapshot_save_job_bh() is scheduled.

vCPU thread:
3. aio_poll() for the main thread's AioContext is called (e.g. when
the guest writes to a pflash device, as part of blk_pwrite which is a
generated coroutine wrapper).
4. snapshot_save_job_bh() is executed as part of aio_poll().
3. qemu_savevm_state() is called.
4. qemu_mutex_unlock_iothread() is called. Now
qemu_get_current_aio_context() returns 0x0.
5. bdrv_writev_vmstate() is executed during the usual savevm setup
via qemu_fflush(). But this function is a generated coroutine wrapper,
so it uses AIO_WAIT_WHILE. There, the assertion
assert(qemu_get_current_aio_context() == qemu_get_aio_context());
will fail.

To fix it, ensure that the BQL is held during setup. While it would
only be needed for snapshots, adapting migration too avoids additional
logic for conditional locking/unlocking in the setup callbacks.
Writing the header could (in theory) also trigger qemu_fflush() and
thus bdrv_writev_vmstate(), so the locked section also covers the
qemu_savevm_state_header() call, even for migration for consistency.

The section around multifd_send_sync_main() needs to be unlocked to
avoid a deadlock. In particular, the multifd_save_setup() function calls
socket_send_channel_create() using multifd_new_send_channel_async() as a
callback and then waits for the callback to signal via the
channels_ready semaphore. The connection happens via
qio_task_run_in_thread(), but the callback is only executed via
qio_task_thread_result() which is scheduled for the main event loop.
Without unlocking the section, the main thread would never get to
process the task result and the callback meaning there would be no
signal via the channels_ready semaphore.

The comment in ram_init_bitmaps() was introduced by 4987783400
("migration: fix incorrect memory_global_dirty_log_start outside BQL")
and is removed, because it referred to the qemu_mutex_lock_iothread()
call.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231013105839.415989-1-f.ebner@proxmox.com>
2023-10-17 09:25:13 +02:00
Fabiano Rosas 3dc35470c8 tests/qtest: migration-test: Add tests for file-based migration
Add basic tests for file-based migration.

Note that we cannot use test_precopy_common because that routine
expects it to be possible to run the migration live. With the file
transport there is no live migration because we must wait for the
source to finish writing the migration data to the file before the
destination can start reading. Add a new migration function
specifically to handle the file migration.

Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20230712190742.22294-7-farosas@suse.de>
2023-10-17 09:25:06 +02:00
Fabiano Rosas d864756e87 tests/qtest/migration: Add a test for the analyze-migration script
Add a smoke test that migrates to a file and gives it to the
script. It should catch the most annoying errors such as changes in
the ram flags.

After code has been merged it becomes way harder to figure out what is
causing the script to fail, the person making the change is the most
likely to know right away what the problem is.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231009184326.15777-7-farosas@suse.de>
2023-10-17 09:14:32 +02:00
Fabiano Rosas caea03279e migration: Fix analyze-migration read operation signedness
The migration code uses unsigned values for 16, 32 and 64-bit
operations. Fix the script to do the same.

This was causing an issue when parsing the migration stream generated
on the ppc64 target because one of instance_ids was larger than the
32bit signed maximum:

Traceback (most recent call last):
  File "/home/fabiano/kvm/qemu/build/scripts/analyze-migration.py", line 658, in <module>
    dump.read(dump_memory = args.memory)
  File "/home/fabiano/kvm/qemu/build/scripts/analyze-migration.py", line 592, in read
    classdesc = self.section_classes[section_key]
KeyError: ('spapr_iommu', -2147483648)

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231009184326.15777-6-farosas@suse.de>
2023-10-17 09:14:32 +02:00
Fabiano Rosas ff40c7f0b7 migration: Fix analyze-migration.py when ignore-shared is used
The script is currently broken when the x-ignore-shared capability is
used:

Traceback (most recent call last):
  File "./scripts/analyze-migration.py", line 656, in <module>
    dump.read(dump_memory = args.memory)
  File "./scripts/analyze-migration.py", line 593, in read
    section.read()
  File "./scripts/analyze-migration.py", line 163, in read
    self.name = self.file.readstr(len = namelen)
  File "./scripts/analyze-migration.py", line 53, in readstr
    return self.readvar(len).decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x82 in position 55: invalid start byte

We're currently adding data to the middle of the ram section depending
on the presence of the capability. As a consequence, any code loading
the ram section needs to know about capabilities so it can interpret
the stream.

Skip the byte that's added when x-ignore-shared is used to fix the
script.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231009184326.15777-5-farosas@suse.de>
2023-10-17 09:14:32 +02:00
Fabiano Rosas 31499a9dc1 migration: Add capability parsing to analyze-migration.py
The script is broken when the configuration/capabilities section is
present. Add support for parsing the capabilities so we can fix it in
the next patch.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231009184326.15777-4-farosas@suse.de>
2023-10-17 09:14:32 +02:00
Fabiano Rosas c36c31c86b migration: Fix analyze-migration.py 'configuration' parsing
The 'configuration' state subsections are currently not being parsed
and the script fails when analyzing an aarch64 stream:

Traceback (most recent call last):
  File "./scripts/analyze-migration.py", line 625, in <module>
    dump.read(dump_memory = args.memory)
  File "./scripts/analyze-migration.py", line 571, in read
    raise Exception("Unknown section type: %d" % section_type)
Exception: Unknown section type: 5

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231009184326.15777-3-farosas@suse.de>
2023-10-17 09:14:32 +02:00
Nikolay Borisov 2aae1eb8da migration: Add the configuration vmstate to the json writer
Make the migration json writer part of MigrationState struct, allowing
the 'configuration' object be serialized to json.

This will facilitate the parsing of the 'configuration' object in the
next patch that fixes analyze-migration.py for arm.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231009184326.15777-2-farosas@suse.de>
2023-10-17 09:14:32 +02:00
Dmitry Frolov f75ed59f40 migration: fix RAMBlock add NULL check
qemu_ram_block_from_host() may return NULL, which will be dereferenced w/o
check. Usualy return value is checked for this function.
Found by Linux Verification Center (linuxtesting.org) with SVACE.

Signed-off-by: Dmitry Frolov <frolov@swemel.ru>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231010104851.802947-1-frolov@swemel.ru>
2023-10-17 09:14:32 +02:00
Peter Xu 8b2395970a migration: Allow user to specify available switchover bandwidth
Migration bandwidth is a very important value to live migration.  It's
because it's one of the major factors that we'll make decision on when to
switchover to destination in a precopy process.

This value is currently estimated by QEMU during the whole live migration
process by monitoring how fast we were sending the data.  This can be the
most accurate bandwidth if in the ideal world, where we're always feeding
unlimited data to the migration channel, and then it'll be limited to the
bandwidth that is available.

However in reality it may be very different, e.g., over a 10Gbps network we
can see query-migrate showing migration bandwidth of only a few tens of
MB/s just because there are plenty of other things the migration thread
might be doing.  For example, the migration thread can be busy scanning
zero pages, or it can be fetching dirty bitmap from other external dirty
sources (like vhost or KVM).  It means we may not be pushing data as much
as possible to migration channel, so the bandwidth estimated from "how many
data we sent in the channel" can be dramatically inaccurate sometimes.

With that, the decision to switchover will be affected, by assuming that we
may not be able to switchover at all with such a low bandwidth, but in
reality we can.

The migration may not even converge at all with the downtime specified,
with that wrong estimation of bandwidth, keeping iterations forever with a
low estimation of bandwidth.

The issue is QEMU itself may not be able to avoid those uncertainties on
measuing the real "available migration bandwidth".  At least not something
I can think of so far.

One way to fix this is when the user is fully aware of the available
bandwidth, then we can allow the user to help providing an accurate value.

For example, if the user has a dedicated channel of 10Gbps for migration
for this specific VM, the user can specify this bandwidth so QEMU can
always do the calculation based on this fact, trusting the user as long as
specified.  It may not be the exact bandwidth when switching over (in which
case qemu will push migration data as fast as possible), but much better
than QEMU trying to wildly guess, especially when very wrong.

A new parameter "avail-switchover-bandwidth" is introduced just for this.
So when the user specified this parameter, instead of trusting the
estimated value from QEMU itself (based on the QEMUFile send speed), it
trusts the user more by using this value to decide when to switchover,
assuming that we'll have such bandwidth available then.

Note that specifying this value will not throttle the bandwidth for
switchover yet, so QEMU will always use the full bandwidth possible for
sending switchover data, assuming that should always be the most important
way to use the network at that time.

This can resolve issues like "unconvergence migration" which is caused by
hilarious low "migration bandwidth" detected for whatever reason.

Reported-by: Zhiyi Guo <zhguo@redhat.com>
Reviewed-by: Joao Martins <joao.m.martins@oracle.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231010221922.40638-1-peterx@redhat.com>
2023-10-17 09:14:32 +02:00
Philippe Mathieu-Daudé 1a36e4c9d0 migration: Use g_autofree to simplify ram_dirty_bitmap_reload()
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231011023627.86691-1-philmd@linaro.org>
2023-10-17 09:14:32 +02:00
Wei Wang d7f5a04320 migration: refactor migration_completion
Current migration_completion function is a bit long. Refactor the long
implementation into different subfunctions:
- migration_completion_precopy: completion code related to precopy
- migration_completion_postcopy: completion code related to postcopy

Rename await_return_path_close_on_source to
close_return_path_on_source: It is renamed to match with
open_return_path_on_source.

This improves readability and is easier for future updates (e.g. add new
subfunctions when completion code related to new features are needed). No
functional changes intended.

Signed-off-by: Wei Wang <wei.w.wang@intel.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Isaku Yamahata <isaku.yamahata@intel.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20230804093053.5037-1-wei.w.wang@intel.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2023-10-17 09:14:32 +02:00
Stefan Hajnoczi 800485762e Python Pullreq
Python PR:
 
 - Use socketpair for all machine.py connections
 - Support Python 3.12
 - Switch iotests over to using raise-on-error QMP command interface
   (Thank you very much, Vladimir!)
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+ber27ys35W+dsvQfe+BBqr8OQ4FAmUpldkACgkQfe+BBqr8
 OQ4NtRAAnkEmXsECAxQ2ewvf3yK8PTFm4Oq5nqMIw+KB94ATrsGzk3z1rLvatSl3
 6VLsV2+FWoOEyKrsfu5DIfbuo4d3TZTU7N2DIZpVpvO166K+fXbzp8skAg+n3BMC
 tWkSOcnsT6+8aqyxxyASdHvbbE7pvPw8OA3oIIstsYeZ5/HHpOWXNj1kjCsnL0lW
 7y5h6UUKGmnCPdixyk042+AvKkT7GAKVjFnjUF5JHv0iR2KpQ+O9H7OEalqQT5w5
 eab4oMGuIYhzYe+MNpyybAB3Xd2pxhcppk+sl4dCE8qmMn7KRoTNw1iu+qhsNQfQ
 JILZoCPtYMhpef4X0ulH8PFBMweBptqOjo4lpz9QIdMWTf86IE0yIT9DCy3aSjpp
 ywwxhFKJS43gz4WHkEJlrY9PHwLsULaV/Cz6HKJAU6h9aFtcNdT4pkCOERnZ8X4C
 yHlNReTG5Dz1sYzKJ/k9LTjAaVDasumR8/yadaUCwalj5zexQ27qlIM6oc5wdIRQ
 up1VHi7odF5KHb6GeqdniuuEF6NBCYRAV5nz+dbd6exfKOaxYRrr48yh9SUm8QS6
 JCvMMFFAZCIrI/nkRVajbLi9L5O3fg5abtlzSzh9o4iyf8Rf/1gtKNxZRK1NZIjQ
 cTYBJXpMulNx7bM2CPNsPWGqCTAjAcu10svqTA8luGj4fqdTNyU=
 =02Bd
 -----END PGP SIGNATURE-----

Merge tag 'python-pull-request' of https://gitlab.com/jsnow/qemu into staging

Python Pullreq

Python PR:

- Use socketpair for all machine.py connections
- Support Python 3.12
- Switch iotests over to using raise-on-error QMP command interface
  (Thank you very much, Vladimir!)

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEE+ber27ys35W+dsvQfe+BBqr8OQ4FAmUpldkACgkQfe+BBqr8
# OQ4NtRAAnkEmXsECAxQ2ewvf3yK8PTFm4Oq5nqMIw+KB94ATrsGzk3z1rLvatSl3
# 6VLsV2+FWoOEyKrsfu5DIfbuo4d3TZTU7N2DIZpVpvO166K+fXbzp8skAg+n3BMC
# tWkSOcnsT6+8aqyxxyASdHvbbE7pvPw8OA3oIIstsYeZ5/HHpOWXNj1kjCsnL0lW
# 7y5h6UUKGmnCPdixyk042+AvKkT7GAKVjFnjUF5JHv0iR2KpQ+O9H7OEalqQT5w5
# eab4oMGuIYhzYe+MNpyybAB3Xd2pxhcppk+sl4dCE8qmMn7KRoTNw1iu+qhsNQfQ
# JILZoCPtYMhpef4X0ulH8PFBMweBptqOjo4lpz9QIdMWTf86IE0yIT9DCy3aSjpp
# ywwxhFKJS43gz4WHkEJlrY9PHwLsULaV/Cz6HKJAU6h9aFtcNdT4pkCOERnZ8X4C
# yHlNReTG5Dz1sYzKJ/k9LTjAaVDasumR8/yadaUCwalj5zexQ27qlIM6oc5wdIRQ
# up1VHi7odF5KHb6GeqdniuuEF6NBCYRAV5nz+dbd6exfKOaxYRrr48yh9SUm8QS6
# JCvMMFFAZCIrI/nkRVajbLi9L5O3fg5abtlzSzh9o4iyf8Rf/1gtKNxZRK1NZIjQ
# cTYBJXpMulNx7bM2CPNsPWGqCTAjAcu10svqTA8luGj4fqdTNyU=
# =02Bd
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 13 Oct 2023 15:09:13 EDT
# gpg:                using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F  18F2 88A9 064D 1835 61EB
#      Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76  CBD0 7DEF 8106 AAFC 390E

* tag 'python-pull-request' of https://gitlab.com/jsnow/qemu: (25 commits)
  python: use vm.cmd() instead of vm.qmp() where appropriate
  scripts: add python_qmp_updater.py
  tests/vm/basevm.py: use cmd() instead of qmp()
  iotests.py: pause_job(): drop return value
  iotests: drop some extra ** in qmp() call
  iotests: drop some extra semicolons
  iotests: refactor some common qmp result checks into generic pattern
  iotests: add some missed checks of qmp result
  iotests: QemuStorageDaemon: add cmd() method like in QEMUMachine.
  python/machine.py: upgrade vm.cmd() method
  python/qemu: rename command() to cmd()
  python: rename QEMUMonitorProtocol.cmd() to cmd_raw()
  scripts/cpu-x86-uarch-abi.py: use .command() instead of .cmd()
  qmp_shell.py: _fill_completion() use .command() instead of .cmd()
  python/qemu/qmp/legacy: cmd(): drop cmd_id unused argument
  Python: Enable python3.12 support
  configure: fix error message to say Python 3.8
  python/qmp: remove Server.wait_closed() call for Python 3.12
  Python/iotests: Add type hint for nbd module
  python/machine: remove unused sock_dir argument
  ...

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2023-10-16 12:37:48 -04:00
Stefan Hajnoczi 9390f0fd3e pull-loongarch-20231013
-----BEGIN PGP SIGNATURE-----
 
 iLMEAAEKAB0WIQS4/x2g0v3LLaCcbCxAov/yOSY+3wUCZSimNQAKCRBAov/yOSY+
 33XwBADF9ZKlESDBDa/huNFAKD7BsUIdglHfz9lHnLY+kQbCun4HyTLtp2IBsySu
 mZTjdfU/LnaBidFLjEnmZZMPyiI3oV1ruSzT53egSDaxrFUXGpc9oxtMNLsyfk9P
 swdngG13Fc9sWVKC7IJeYDYXgkvHY7NxsiV8U9vdqXOyw2uoHA==
 =ufPc
 -----END PGP SIGNATURE-----

Merge tag 'pull-loongarch-20231013' of https://gitlab.com/gaosong/qemu into staging

pull-loongarch-20231013

# -----BEGIN PGP SIGNATURE-----
#
# iLMEAAEKAB0WIQS4/x2g0v3LLaCcbCxAov/yOSY+3wUCZSimNQAKCRBAov/yOSY+
# 33XwBADF9ZKlESDBDa/huNFAKD7BsUIdglHfz9lHnLY+kQbCun4HyTLtp2IBsySu
# mZTjdfU/LnaBidFLjEnmZZMPyiI3oV1ruSzT53egSDaxrFUXGpc9oxtMNLsyfk9P
# swdngG13Fc9sWVKC7IJeYDYXgkvHY7NxsiV8U9vdqXOyw2uoHA==
# =ufPc
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 12 Oct 2023 22:06:45 EDT
# gpg:                using RSA key B8FF1DA0D2FDCB2DA09C6C2C40A2FFF239263EDF
# gpg: Good signature from "Song Gao <m17746591750@163.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: B8FF 1DA0 D2FD CB2D A09C  6C2C 40A2 FFF2 3926 3EDF

* tag 'pull-loongarch-20231013' of https://gitlab.com/gaosong/qemu:
  LoongArch: step down as general arch maintainer
  hw/loongarch/virt: Remove unused 'loongarch_virt_pm' region
  hw/loongarch/virt: Remove unused ISA Bus
  hw/loongarch/virt: Remove unused ISA UART
  hw/loongarch: remove global loaderparams variable
  target/loongarch: Add preldx instruction
  target/loongarch: fix ASXE flag conflict

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2023-10-16 12:37:35 -04:00
Stefan Hajnoczi 2778f754e6 hw/ufs: fixes
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEUBfYMVl8eKPZB+73EuIgTA5dtgIFAmUozswACgkQEuIgTA5d
 tgITExAAo0BSNir4I5MfeNIjZTNNdxLXDl0+92JyairB2m/gWH/02jGtrJBYp5On
 ELnixKj2Ntn9IIRr3NwQHNTnDOZHRkUBH+pRVeMbZ+IWLjEoWQdl03ge7e9sHai3
 CLXB4HPSnXddy1SmS9FEkdBWopqxKF4BLZnpAfwh/dj2fzSyDyNIMmGoRimRQhph
 9A90304ERUdpREAXncTgSdXeDZz+lScadzUJZrPPiG2ZHXL+qzDCX7ojEnNaUFxz
 W1IfriI8oeeORfCQaNEOncLKhSwE1WscGxP0vILPApKOu251tObgSbK90QlQR2qT
 BMl7k4BDfYeksXMGc0BXVFrOfv1ud86NlCE2OokK6HBZVuHio4C6TU/t65MC4Rw5
 mJ8CPgbN+7sgVmAGo0sLYzI6GiRR27VqqLh6KXVAa5c/fAdt5pHSkakwSvxiXsAl
 EqskmOY2em5O//+7CWN1CtY+I2pHyltMXAi3Cb2vjweNx88kuhmxFQWeZVI10/H3
 gNrNfu32+ihDLMqR7uQamdAZV0lnIwp97nCbf3LzpM0btjl70QvGZhsbiCDiLQrG
 mJjnaix4xDb8T21WKrI8DKcwR4rvD8hZsCUp31XJnA8HWtdPnEQldK8NEGNlU5ye
 lrGc6gxiwZLCBBIj9lwbZW3Zv9Vz9jNWISOmY+KWLCIus98DBxQ=
 =XXsQ
 -----END PGP SIGNATURE-----

Merge tag 'pull-ufs-20231013' of https://gitlab.com/jeuk20.kim/qemu into staging

hw/ufs: fixes

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCgAdFiEEUBfYMVl8eKPZB+73EuIgTA5dtgIFAmUozswACgkQEuIgTA5d
# tgITExAAo0BSNir4I5MfeNIjZTNNdxLXDl0+92JyairB2m/gWH/02jGtrJBYp5On
# ELnixKj2Ntn9IIRr3NwQHNTnDOZHRkUBH+pRVeMbZ+IWLjEoWQdl03ge7e9sHai3
# CLXB4HPSnXddy1SmS9FEkdBWopqxKF4BLZnpAfwh/dj2fzSyDyNIMmGoRimRQhph
# 9A90304ERUdpREAXncTgSdXeDZz+lScadzUJZrPPiG2ZHXL+qzDCX7ojEnNaUFxz
# W1IfriI8oeeORfCQaNEOncLKhSwE1WscGxP0vILPApKOu251tObgSbK90QlQR2qT
# BMl7k4BDfYeksXMGc0BXVFrOfv1ud86NlCE2OokK6HBZVuHio4C6TU/t65MC4Rw5
# mJ8CPgbN+7sgVmAGo0sLYzI6GiRR27VqqLh6KXVAa5c/fAdt5pHSkakwSvxiXsAl
# EqskmOY2em5O//+7CWN1CtY+I2pHyltMXAi3Cb2vjweNx88kuhmxFQWeZVI10/H3
# gNrNfu32+ihDLMqR7uQamdAZV0lnIwp97nCbf3LzpM0btjl70QvGZhsbiCDiLQrG
# mJjnaix4xDb8T21WKrI8DKcwR4rvD8hZsCUp31XJnA8HWtdPnEQldK8NEGNlU5ye
# lrGc6gxiwZLCBBIj9lwbZW3Zv9Vz9jNWISOmY+KWLCIus98DBxQ=
# =XXsQ
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 13 Oct 2023 00:59:56 EDT
# gpg:                using RSA key 5017D831597C78A3D907EEF712E2204C0E5DB602
# gpg: Good signature from "Jeuk Kim <jeuk20.kim@samsung.com>" [unknown]
# gpg:                 aka "Jeuk Kim <jeuk20.kim@gmail.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 5017 D831 597C 78A3 D907  EEF7 12E2 204C 0E5D B602

* tag 'pull-ufs-20231013' of https://gitlab.com/jeuk20.kim/qemu:
  hw/ufs: Fix incorrect register fields
  hw/ufs: Fix code coverity issues

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2023-10-16 12:37:22 -04:00
Stefan Hajnoczi 2a6299fb13 * Fix CVE-2023-1544
* Deprecate the rdma code
 * Fix flaky npcm7xx_timer test
 * i2c-echo license statement and Kconfig switch
 * Disable the failing riscv64-debian-cross CI job by default
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmUoI3IRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbXl3A//dnuUf7cXrllZc9PyTRMg99c45DRrj058
 e9iOIdQdMAxN8QQtG/6HHVGRBKP5q5LqcAHZE4DTbIxz9keyoVNQ7CiOL8Zm1GJx
 up5o/HGul8pAnlG0HrM21VJ6ZVSFW8j+50LQbxLTPOMdGAmzxBPvVIGonDOMBIJh
 GPMTFFbJcppSCY2i5ijZ1qmgWMjwErEdQ2kXxxWo5eczNUFT68A4UmQ4NzlA0DZ7
 jG19+2DzigJxnLhYXIOPGCkKjVPcue7QjB1ka35ixU6AgQcW3WNrEwqVbf+FHNyN
 U1ONWqSTkpGlxB/3a+u/HecC4qN4bCRzVdfTNOArQpgNcc+NWr+jZlAoCZ9N72aH
 jZNfW4D1TcgLh+Etj2NWRJKe/2F8N03V/9sJta0eYekzwUqu15Lc1FhhG2zJUJzO
 QZxPik6J7PkxX28AoSLjyRx80g3V06eBh2YXqIZioZnbdRQrqX1GG7YpRk7YQRCF
 o77T85R64s/z/6GSvEee2f6e6c8O/WKMhLBBoI8fCN0TZVv5VlwHLipKxRQqUZIn
 +Iu/LpKsGtmQT259D+rqkCJMzdPFcc3S8e4MwWb8KMe3/rqZuGoCO3ecXQR+qto4
 Gif0Z7T+us1hf6ExHDi+8FaRBMlbDpk1/Ebw1NvwcWqMaWipyeAY1yVmYzAMcQ9/
 dm2e2YWb4mk=
 =aPMR
 -----END PGP SIGNATURE-----

Merge tag 'pull-request-2023-10-12' of https://gitlab.com/thuth/qemu into staging

* Fix CVE-2023-1544
* Deprecate the rdma code
* Fix flaky npcm7xx_timer test
* i2c-echo license statement and Kconfig switch
* Disable the failing riscv64-debian-cross CI job by default

* tag 'pull-request-2023-10-12' of https://gitlab.com/thuth/qemu:
  gitlab-ci: Disable the riscv64-debian-cross-container by default
  MAINTAINERS: Add include/sysemu/qtest.h to the qtest section
  hw/misc/Kconfig: add switch for i2c-echo
  hw/misc/i2c-echo: add copyright/license note
  tests/qtest: Fix npcm7xx_timer-test.c flaky test
  hw/rdma: Deprecate the pvrdma device and the rdma subsystem
  hw/pvrdma: Protect against buggy or malicious guest driver

Conflicts:
  docs/about/deprecated.rst
  Context conflict between RISC-V and RDMA deprecation.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2023-10-16 12:36:55 -04:00
Stefan Hajnoczi ce2f51697b Block layer patches
- Clean up coroutine versions of bdrv_{is_allocated,block_status}*
 - Graph locking part 5 (protect children/parent links)
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmUoHL8RHGt3b2xmQHJl
 ZGhhdC5jb20ACgkQfwmycsiPL9b4uRAAjryVAaA5jXZ3mdGB80nhGtARZlIaIVO/
 tlXk065q2Cj+98f+fBPCPWvmEz28vJwBhJUsFwpHzLZrxecBpwZp0MPAkFBNkouq
 +AiO9xyTAqccEp/dnIys4Bun9Rp0Jq9lk9y29zzEmQuK5uCB56lpx2cDn/JkzSQt
 ZFtnxxTwi3MDTNvXATub8Ia/1suui0zvESS7J/NBxQNI3cFaQszp1vMwlRIoPiWo
 15YZFPZZQ2pvu6/1nL1Vl9OLbPAVcEGJpjHZv0XhudYOwRiDvjYnwfPL7BuwYEsU
 Dos4mZZd/KMU695s7OzlVYi1q4ATKUTUxyyylVhXZrFBXSE5ntnfoHTKHEruTyPb
 G31h5mribSTWjdvY5HewHbSSPjByAWsSQg9yzcHybhORiqGQCpcGQ8zuW7oNKMPV
 JicWdoRVY4U4hR0nRdDxz9zdpQ8QYok/ginBxFaOzrCfClUB7ZOBxwRMclIghuRH
 FV+ZJk0ylVOz2tbfNxUa3KhUgTPd8jgCHFI7xak5EBRtTJiJjE03Xag1Fdxy5/D5
 tRsBBW4sOFygAhjN/xyeaRv9L8rAv3x/akriFjPUbOMLkPcJpe/DTWsP8+5LaZF8
 GkQvjsg5UvmfcJ3LFtecXxfYH4UWhDmyAjF+BswiRqafDDi2CCUmdwDnzEPbwuWO
 x1y7cgxe9SE=
 =4d/s
 -----END PGP SIGNATURE-----

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Clean up coroutine versions of bdrv_{is_allocated,block_status}*
- Graph locking part 5 (protect children/parent links)

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmUoHL8RHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9b4uRAAjryVAaA5jXZ3mdGB80nhGtARZlIaIVO/
# tlXk065q2Cj+98f+fBPCPWvmEz28vJwBhJUsFwpHzLZrxecBpwZp0MPAkFBNkouq
# +AiO9xyTAqccEp/dnIys4Bun9Rp0Jq9lk9y29zzEmQuK5uCB56lpx2cDn/JkzSQt
# ZFtnxxTwi3MDTNvXATub8Ia/1suui0zvESS7J/NBxQNI3cFaQszp1vMwlRIoPiWo
# 15YZFPZZQ2pvu6/1nL1Vl9OLbPAVcEGJpjHZv0XhudYOwRiDvjYnwfPL7BuwYEsU
# Dos4mZZd/KMU695s7OzlVYi1q4ATKUTUxyyylVhXZrFBXSE5ntnfoHTKHEruTyPb
# G31h5mribSTWjdvY5HewHbSSPjByAWsSQg9yzcHybhORiqGQCpcGQ8zuW7oNKMPV
# JicWdoRVY4U4hR0nRdDxz9zdpQ8QYok/ginBxFaOzrCfClUB7ZOBxwRMclIghuRH
# FV+ZJk0ylVOz2tbfNxUa3KhUgTPd8jgCHFI7xak5EBRtTJiJjE03Xag1Fdxy5/D5
# tRsBBW4sOFygAhjN/xyeaRv9L8rAv3x/akriFjPUbOMLkPcJpe/DTWsP8+5LaZF8
# GkQvjsg5UvmfcJ3LFtecXxfYH4UWhDmyAjF+BswiRqafDDi2CCUmdwDnzEPbwuWO
# x1y7cgxe9SE=
# =4d/s
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 12 Oct 2023 12:20:15 EDT
# gpg:                using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg:                issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin: (26 commits)
  block: Add assertion for bdrv_graph_wrlock()
  block: Protect bs->children with graph_lock
  block: Protect bs->parents with graph_lock
  block: Mark bdrv_get_specific_info() and callers GRAPH_RDLOCK
  block: Mark bdrv_apply_auto_read_only() and callers GRAPH_RDLOCK
  block: Mark bdrv_op_is_blocked() and callers GRAPH_RDLOCK
  qcow2: Mark check_constraints_on_bitmap() GRAPH_RDLOCK
  qcow2: Mark qcow2_inactivate() and callers GRAPH_RDLOCK
  qcow2: Mark qcow2_signal_corruption() and callers GRAPH_RDLOCK
  block: Mark bdrv_amend_options() and callers GRAPH_RDLOCK
  block: Mark bdrv_get_parent_name() and callers GRAPH_RDLOCK
  block: Mark bdrv_primary_child() and callers GRAPH_RDLOCK
  block: Mark bdrv_refresh_filename() and callers GRAPH_RDLOCK
  block: Mark bdrv_get_xdbg_block_graph() and callers GRAPH_RDLOCK
  block: Take graph rdlock in parts of reopen
  block: Mark bdrv_snapshot_fallback() and callers GRAPH_RDLOCK
  block: Mark bdrv_parent_cb_resize() and callers GRAPH_RDLOCK
  block: Mark drain related functions GRAPH_RDLOCK
  block: Mark bdrv_first_blk() and bdrv_is_root_node() GRAPH_RDLOCK
  block: Take graph rdlock in bdrv_inactivate_all()
  ...

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2023-10-16 12:34:45 -04:00
Stefan Hajnoczi 32bfaa4ea7 -Wshadow=local patches for 2023-10-12
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmUoCNsSHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZTTocP/iQ6RggqcHrBxwZZtyydvpWCFrqfuBTk
 6GQtKGm51UcQ9kmAIsoV90pOzdUdjwrpXzKKJwsLzMcVcp1NDPsQIL54wdsRmZfH
 E9mxI7UlZf/KWzrfP1nFLcU8T5+cuXosDgjx55Y1Kq+ZRn+7x0DInBGdRryokWTG
 zcKh9T3n9KWKscLL7hvxLZS5054V9HBDYIpBBEyV2GtRrCLL0Y+9aaKkBrejHMgY
 oKrLKHz1cOGOTzQ7AbhA+Wv3eN+GYVyjnCSUXK/270jbU8Xg4m1vSbrPq2PWy5kV
 IGGKZtZsrSq0VBoTi+i9++vP5djKVUYQLqx10L+NYCp25wBnTgXKSDtdAqI68aev
 TYrOlQ1ldKXJT4ghPqoWCjRKkryV6/Gj9fHbbvsHJ7SB84VO8G/kpn5zXvN/BosG
 8vxLEL0xc1Q3Sxi91DCjVsP7UebjBt1j/JugU9zVr8OFJWriFmllYB67AOOo3gS2
 c+FNVPLle3udw5EHClMapcGSzTun4iHeEsiJMOOgGOHC09Bi+Om6LlneFWljmvQp
 a6ma+bebxCjzuO6heey2Q/1JjltR8Ex0bnbWIoNsysA6OnDtTlbxDqZEca1h6As+
 Rm9XFKf7nVQIHFKW3sjbx6MgqAL6sBakfeJah5Pj5iIKtLaZR591RyAfvfB2sBlS
 ZYtp95GIKWXZ
 =AArx
 -----END PGP SIGNATURE-----

Merge tag 'pull-shadow-2023-10-12' of https://repo.or.cz/qemu/armbru into staging

-Wshadow=local patches for 2023-10-12

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmUoCNsSHGFybWJydUBy
# ZWRoYXQuY29tAAoJEDhwtADrkYZTTocP/iQ6RggqcHrBxwZZtyydvpWCFrqfuBTk
# 6GQtKGm51UcQ9kmAIsoV90pOzdUdjwrpXzKKJwsLzMcVcp1NDPsQIL54wdsRmZfH
# E9mxI7UlZf/KWzrfP1nFLcU8T5+cuXosDgjx55Y1Kq+ZRn+7x0DInBGdRryokWTG
# zcKh9T3n9KWKscLL7hvxLZS5054V9HBDYIpBBEyV2GtRrCLL0Y+9aaKkBrejHMgY
# oKrLKHz1cOGOTzQ7AbhA+Wv3eN+GYVyjnCSUXK/270jbU8Xg4m1vSbrPq2PWy5kV
# IGGKZtZsrSq0VBoTi+i9++vP5djKVUYQLqx10L+NYCp25wBnTgXKSDtdAqI68aev
# TYrOlQ1ldKXJT4ghPqoWCjRKkryV6/Gj9fHbbvsHJ7SB84VO8G/kpn5zXvN/BosG
# 8vxLEL0xc1Q3Sxi91DCjVsP7UebjBt1j/JugU9zVr8OFJWriFmllYB67AOOo3gS2
# c+FNVPLle3udw5EHClMapcGSzTun4iHeEsiJMOOgGOHC09Bi+Om6LlneFWljmvQp
# a6ma+bebxCjzuO6heey2Q/1JjltR8Ex0bnbWIoNsysA6OnDtTlbxDqZEca1h6As+
# Rm9XFKf7nVQIHFKW3sjbx6MgqAL6sBakfeJah5Pj5iIKtLaZR591RyAfvfB2sBlS
# ZYtp95GIKWXZ
# =AArx
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 12 Oct 2023 10:55:23 EDT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* tag 'pull-shadow-2023-10-12' of https://repo.or.cz/qemu/armbru:
  target/i386: fix shadowed variable pasto
  contrib/vhost-user-gpu: Fix compiler warning when compiling with -Wshadow
  hw/virtio/virtio-gpu: Fix compiler warning when compiling with -Wshadow
  libvhost-user: Fix compiler warning with -Wshadow=local
  libvduse: Fix compiler warning with -Wshadow=local

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2023-10-16 12:34:32 -04:00
Stefan Hajnoczi bc2b89b385 Hi,
"Host Memory Backends" and "Memory devices" queue ("mem"):
 - Support memory devices with multiple memslots
 - Support memory devices that dynamically consume memslots
 - Support memory devices that can automatically decide on the number of
   memslots to use
 - virtio-mem support for exposing memory dynamically via multiple
   memslots
 - Some required cleanups/refactorings
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEG9nKrXNcTDpGDfzKTd4Q9wD/g1oFAmUn+XMRHGRhdmlkQHJl
 ZGhhdC5jb20ACgkQTd4Q9wD/g1qDHA//T01suTa+uzrcoJHoMWN11S47WnAmbuTo
 vVakucLBPMJAa9xZeCy3OavXaVGpHkw+t6g3OFknof0LfQ5/j9iE3Q1PxURN7g5j
 SJ2WJXCoceM6T4TMhPvVvgEaYjFmESqZB5FZgedMT0QRyhAxMuF9pCkWhk1O3OAV
 JqQKqLFiGcv60AEuBYGZGzgiOUv8EJ5gKwRF4VOdyHIxqZDw1aZXzlcd4TzFZBQ7
 rwW/3ef+sFmUJdmfrSrqcIlQSRrqZ2w95xATDzLTIEEUT3SWqh/E95EZWIz1M0oQ
 NgWgFiLCR1KOj7bWFhLXT7IfyLh0mEysD+P/hY6QwQ4RewWG7EW5UK+JFswssdcZ
 rEj5XpHZzev/wx7hM4bWsoQ+VIvrH7j3uYGyWkcgYRbdDEkWDv2rsT23lwGYNhht
 oBsrdEBELRw6v4C8doq/+sCmHmuxUMqTGwbArCQVnB1XnLxOEkuqlnfq5MORkzNF
 fxbIRx+LRluOllC0HVaDQd8qxRq1+UC5WIpAcDcrouy4HGgi1onWKrXpgjIAbVyH
 M6cENkK7rnRk96gpeXdmrf0h9HqRciAOY8oUsFsvLyKBOCPBWDrLyOQEY5UoSdtD
 m4QpEVgywCy2z1uU/UObeT/UxJy/9EL/Zb+DHoEK06iEhwONoUJjEBYMJD38RMkk
 mwPTB4UAk9g=
 =s69t
 -----END PGP SIGNATURE-----

Merge tag 'mem-2023-10-12' of https://github.com/davidhildenbrand/qemu into staging

Hi,

"Host Memory Backends" and "Memory devices" queue ("mem"):
- Support memory devices with multiple memslots
- Support memory devices that dynamically consume memslots
- Support memory devices that can automatically decide on the number of
  memslots to use
- virtio-mem support for exposing memory dynamically via multiple
  memslots
- Some required cleanups/refactorings

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEEG9nKrXNcTDpGDfzKTd4Q9wD/g1oFAmUn+XMRHGRhdmlkQHJl
# ZGhhdC5jb20ACgkQTd4Q9wD/g1qDHA//T01suTa+uzrcoJHoMWN11S47WnAmbuTo
# vVakucLBPMJAa9xZeCy3OavXaVGpHkw+t6g3OFknof0LfQ5/j9iE3Q1PxURN7g5j
# SJ2WJXCoceM6T4TMhPvVvgEaYjFmESqZB5FZgedMT0QRyhAxMuF9pCkWhk1O3OAV
# JqQKqLFiGcv60AEuBYGZGzgiOUv8EJ5gKwRF4VOdyHIxqZDw1aZXzlcd4TzFZBQ7
# rwW/3ef+sFmUJdmfrSrqcIlQSRrqZ2w95xATDzLTIEEUT3SWqh/E95EZWIz1M0oQ
# NgWgFiLCR1KOj7bWFhLXT7IfyLh0mEysD+P/hY6QwQ4RewWG7EW5UK+JFswssdcZ
# rEj5XpHZzev/wx7hM4bWsoQ+VIvrH7j3uYGyWkcgYRbdDEkWDv2rsT23lwGYNhht
# oBsrdEBELRw6v4C8doq/+sCmHmuxUMqTGwbArCQVnB1XnLxOEkuqlnfq5MORkzNF
# fxbIRx+LRluOllC0HVaDQd8qxRq1+UC5WIpAcDcrouy4HGgi1onWKrXpgjIAbVyH
# M6cENkK7rnRk96gpeXdmrf0h9HqRciAOY8oUsFsvLyKBOCPBWDrLyOQEY5UoSdtD
# m4QpEVgywCy2z1uU/UObeT/UxJy/9EL/Zb+DHoEK06iEhwONoUJjEBYMJD38RMkk
# mwPTB4UAk9g=
# =s69t
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 12 Oct 2023 09:49:39 EDT
# gpg:                using RSA key 1BD9CAAD735C4C3A460DFCCA4DDE10F700FF835A
# gpg:                issuer "david@redhat.com"
# gpg: Good signature from "David Hildenbrand <david@redhat.com>" [unknown]
# gpg:                 aka "David Hildenbrand <davidhildenbrand@gmail.com>" [full]
# gpg:                 aka "David Hildenbrand <hildenbr@in.tum.de>" [unknown]
# gpg: WARNING: The key's User ID is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 1BD9 CAAD 735C 4C3A 460D  FCCA 4DDE 10F7 00FF 835A

* tag 'mem-2023-10-12' of https://github.com/davidhildenbrand/qemu:
  virtio-mem: Mark memslot alias memory regions unmergeable
  memory,vhost: Allow for marking memory device memory regions unmergeable
  virtio-mem: Expose device memory dynamically via multiple memslots if enabled
  virtio-mem: Update state to match bitmap as soon as it's been migrated
  virtio-mem: Pass non-const VirtIOMEM via virtio_mem_range_cb
  memory: Clarify mapping requirements for RamDiscardManager
  memory-device,vhost: Support automatic decision on the number of memslots
  vhost: Add vhost_get_max_memslots()
  kvm: Add stub for kvm_get_max_memslots()
  memory-device,vhost: Support memory devices that dynamically consume memslots
  memory-device: Track required and actually used memslots in DeviceMemoryState
  stubs: Rename qmp_memory_device.c to memory_device.c
  memory-device: Support memory devices with multiple memslots
  vhost: Return number of free memslots
  kvm: Return number of free memslots
  softmmu/physmem: Fixup qemu_ram_block_from_host() documentation
  vhost: Remove vhost_backend_can_merge() callback
  vhost: Rework memslot filtering and fix "used_memslot" tracking

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2023-10-16 12:34:17 -04:00
Gurchetan Singh 773f61e997 docs/system: add basic virtio-gpu documentation
This adds basic documentation for virtio-gpu.

Suggested-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Signed-off-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Alyssa Ross <hi@alyssa.is>
Tested-by: Emmanouil Pitsidianakis <manos.pitsidianakis@linaro.org>
Tested-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Reviewed-by: Emmanouil Pitsidianakis <manos.pitsidianakis@linaro.org>
Reviewed-by: Antonio Caggiano <quic_acaggian@quicinc.com>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
2023-10-16 11:29:56 +04:00
Gurchetan Singh 8e7b21ca16 gfxstream + rutabaga: enable rutabaga
This change enables rutabaga to receive virtio-gpu-3d hypercalls
when it is active.

Signed-off-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Alyssa Ross <hi@alyssa.is>
Tested-by: Emmanouil Pitsidianakis <manos.pitsidianakis@linaro.org>
Tested-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Reviewed-by: Antonio Caggiano <quic_acaggian@quicinc.com>
Reviewed-by: Emmanouil Pitsidianakis <manos.pitsidianakis@linaro.org>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
2023-10-16 11:29:56 +04:00
Gurchetan Singh cd9adbefcc gfxstream + rutabaga: meson support
- Add meson detection of rutabaga_gfx
- Build virtio-gpu-rutabaga.c + associated vga/pci files when
  present

Signed-off-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Alyssa Ross <hi@alyssa.is>
Tested-by: Emmanouil Pitsidianakis <manos.pitsidianakis@linaro.org>
Tested-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Reviewed-by: Emmanouil Pitsidianakis <manos.pitsidianakis@linaro.org>
Reviewed-by: Antonio Caggiano <quic_acaggian@quicinc.com>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
2023-10-16 11:29:56 +04:00
Gurchetan Singh 1dcc6adbc1 gfxstream + rutabaga: add initial support for gfxstream
This adds initial support for gfxstream and cross-domain.  Both
features rely on virtio-gpu blob resources and context types, which
are also implemented in this patch.

gfxstream has a long and illustrious history in Android graphics
paravirtualization.  It has been powering graphics in the Android
Studio Emulator for more than a decade, which is the main developer
platform.

Originally conceived by Jesse Hall, it was first known as "EmuGL" [a].
The key design characteristic was a 1:1 threading model and
auto-generation, which fit nicely with the OpenGLES spec.  It also
allowed easy layering with ANGLE on the host, which provides the GLES
implementations on Windows or MacOS enviroments.

gfxstream has traditionally been maintained by a single engineer, and
between 2015 to 2021, the goldfish throne passed to Frank Yang.
Historians often remark this glorious reign ("pax gfxstreama" is the
academic term) was comparable to that of Augustus and both Queen
Elizabeths.  Just to name a few accomplishments in a resplendent
panoply: higher versions of GLES, address space graphics, snapshot
support and CTS compliant Vulkan [b].

One major drawback was the use of out-of-tree goldfish drivers.
Android engineers didn't know much about DRM/KMS and especially TTM so
a simple guest to host pipe was conceived.

Luckily, virtio-gpu 3D started to emerge in 2016 due to the work of
the Mesa/virglrenderer communities.  In 2018, the initial virtio-gpu
port of gfxstream was done by Cuttlefish enthusiast Alistair Delva.
It was a symbol compatible replacement of virglrenderer [c] and named
"AVDVirglrenderer".  This implementation forms the basis of the
current gfxstream host implementation still in use today.

cross-domain support follows a similar arc.  Originally conceived by
Wayland aficionado David Reveman and crosvm enjoyer Zach Reizner in
2018, it initially relied on the downstream "virtio-wl" device.

In 2020 and 2021, virtio-gpu was extended to include blob resources
and multiple timelines by yours truly, features gfxstream/cross-domain
both require to function correctly.

Right now, we stand at the precipice of a truly fantastic possibility:
the Android Emulator powered by upstream QEMU and upstream Linux
kernel.  gfxstream will then be packaged properfully, and app
developers can even fix gfxstream bugs on their own if they encounter
them.

It's been quite the ride, my friends.  Where will gfxstream head next,
nobody really knows.  I wouldn't be surprised if it's around for
another decade, maintained by a new generation of Android graphics
enthusiasts.

Technical details:
  - Very simple initial display integration: just used Pixman
  - Largely, 1:1 mapping of virtio-gpu hypercalls to rutabaga function
    calls

Next steps for Android VMs:
  - The next step would be improving display integration and UI interfaces
    with the goal of the QEMU upstream graphics being in an emulator
    release [d].

Next steps for Linux VMs for display virtualization:
  - For widespread distribution, someone needs to package Sommelier or the
    wayland-proxy-virtwl [e] ideally into Debian main. In addition, newer
    versions of the Linux kernel come with DRM_VIRTIO_GPU_KMS option,
    which allows disabling KMS hypercalls.  If anyone cares enough, it'll
    probably be possible to build a custom VM variant that uses this display
    virtualization strategy.

[a] https://android-review.googlesource.com/c/platform/development/+/34470
[b] https://android-review.googlesource.com/q/topic:%22vulkan-hostconnection-start%22
[c] https://android-review.googlesource.com/c/device/generic/goldfish-opengl/+/761927
[d] https://developer.android.com/studio/releases/emulator
[e] https://github.com/talex5/wayland-proxy-virtwl

Signed-off-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Alyssa Ross <hi@alyssa.is>
Tested-by: Emmanouil Pitsidianakis <manos.pitsidianakis@linaro.org>
Tested-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Reviewed-by: Emmanouil Pitsidianakis <manos.pitsidianakis@linaro.org>
Reviewed-by: Antonio Caggiano <quic_acaggian@quicinc.com>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
2023-10-16 11:29:56 +04:00
Gurchetan Singh ac8767c743 gfxstream + rutabaga prep: added need defintions, fields, and options
This modifies the common virtio-gpu.h file have the fields and
defintions needed by gfxstream/rutabaga, by VirtioGpuRutabaga.

Signed-off-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Alyssa Ross <hi@alyssa.is>
Tested-by: Emmanouil Pitsidianakis <manos.pitsidianakis@linaro.org>
Tested-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Reviewed-by: Emmanouil Pitsidianakis <manos.pitsidianakis@linaro.org>
Reviewed-by: Antonio Caggiano <quic_acaggian@quicinc.com>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
2023-10-16 11:29:56 +04:00
Antonio Caggiano 49a8b65932 virtio-gpu: blob prep
This adds preparatory functions needed to:

     - decode blob cmds
     - tracking iovecs

Signed-off-by: Antonio Caggiano <antonio.caggiano@collabora.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Alyssa Ross <hi@alyssa.is>
Tested-by: Emmanouil Pitsidianakis <manos.pitsidianakis@linaro.org>
Tested-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Tested-by: Huang Rui <ray.huang@amd.com>
Acked-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Emmanouil Pitsidianakis <manos.pitsidianakis@linaro.org>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
2023-10-16 11:29:56 +04:00
Gerd Hoffmann ba62dfa707 virtio-gpu: hostmem
Use VIRTIO_GPU_SHM_ID_HOST_VISIBLE as id for virtio-gpu.

Signed-off-by: Antonio Caggiano <antonio.caggiano@collabora.com>
Tested-by: Alyssa Ross <hi@alyssa.is>
Tested-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Tested-by: Huang Rui <ray.huang@amd.com>
Acked-by: Huang Rui <ray.huang@amd.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
2023-10-16 11:29:56 +04:00
Antonio Caggiano 17b98f46b9 virtio-gpu: CONTEXT_INIT feature
The feature can be enabled when a backend wants it.

Signed-off-by: Antonio Caggiano <antonio.caggiano@collabora.com>
Signed-off-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Alyssa Ross <hi@alyssa.is>
Tested-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Tested-by: Huang Rui <ray.huang@amd.com>
Acked-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
2023-10-16 11:29:56 +04:00
Dr. David Alan Gilbert 605a16a762 virtio: Add shared memory capability
Define a new capability type 'VIRTIO_PCI_CAP_SHARED_MEMORY_CFG' to allow
defining shared memory regions with sizes and offsets of 2^32 and more.
Multiple instances of the capability are allowed and distinguished
by a device-specific 'id'.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Antonio Caggiano <antonio.caggiano@collabora.com>
Signed-off-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Alyssa Ross <hi@alyssa.is>
Tested-by: Huang Rui <ray.huang@amd.com>
Tested-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Acked-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
2023-10-16 11:29:56 +04:00
Helge Deller ce32a9e991 target/hppa: Update to SeaBIOS-hppa version 10
Enhancements:
- Initial support for 64-bit CPUs with Astro/Elroy (e.g. C3700
  workstation)
- USB support (OHCI)
- better PCI support
- esp-scsi fixes from Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>

Signed-off-by: Helge Deller <deller@gmx.de>
2023-10-14 20:45:12 +02:00
Jeuk Kim ebca80bbdb hw/ufs: Fix incorrect register fields
This patch fixes invalid ufs register fields.
This fixes an issue reported by Bin Meng that
caused ufs to fail over riscv.

Fixes: bc4e68d362 ("hw/ufs: Initial commit for emulated Universal-Flash-Storage")
Signed-off-by: Jeuk Kim <jeuk20.kim@samsung.com>
Reported-by: Bin Meng <bmeng@tinylab.org>
Reviewed-by: Bin Meng <bmeng@tinylab.org>
Tested-by: Bin Meng <bmeng@tinylab.org>
2023-10-13 13:56:28 +09:00
Jeuk Kim 97970dae53 hw/ufs: Fix code coverity issues
Fixed four ufs-related coverity issues.

The coverity issues and fixes are as follows

1. CID 1519042: Security issue with the rand() function
Changed to use a fixed value (0xab) instead of rand() as
the value for testing

2. CID 1519043: Dereference after null check
Removed useless (redundant) null checks

3. CID 1519050: Out-of-bounds access issue
Fix to pass an array type variable to find_first_bit and
find_next_bit using DECLARE_BITMAP()

4. CID 1519051: Out-of-bounds read issue
Fix incorrect range check for lun

Fix coverity CID: 1519042 1519043 1519050 1519051

Signed-off-by: Jeuk Kim <jeuk20.kim@samsung.com>
2023-10-13 13:56:28 +09:00
Xiaojuan Yang 1bea6930ca LoongArch: step down as general arch maintainer
I haven't really been working on LoongArch for some time now,
so let's remove myself from this entry.

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Acked-by: Song Gao <gaosong@loongson.cn>
Message-Id: <20231012095135.1423071-1-yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
2023-10-13 10:05:32 +08:00
Song Gao 89daabe385 hw/loongarch/virt: Remove unused 'loongarch_virt_pm' region
The system test shutdown uses the 'loongarch_virt_pm' region.
We can use the write AcpiFadtData.sleep_clt register to realize the shutdown.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Message-ID: <20231012072351.1409344-1-gaosong@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
2023-10-13 10:04:52 +08:00
Philippe Mathieu-Daudé 3866e2f98e hw/loongarch/virt: Remove unused ISA Bus
The LoongArch 'virt' machine doesn't use its ISA I/O region.

If a ISA device were to be mapped there, there is no support
for ISA IRQ. Unlikely useful. Simply remove.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Song Gao <gaosong@loongson.cn>
Message-Id: <20231010135342.40219-3-philmd@linaro.org>
Signed-off-by: Song Gao <gaosong@loongson.cn>
2023-10-13 10:03:47 +08:00
Philippe Mathieu-Daudé cb041aed1b hw/loongarch/virt: Remove unused ISA UART
The LoongArch 'virt' machine doesn't use any ISA UART.
No need to build the device model, remove its Kconfig entry.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Song Gao <gaosong@loongson.cn>
Message-Id: <20231010135342.40219-2-philmd@linaro.org>
Signed-off-by: Song Gao <gaosong@loongson.cn>
2023-10-13 10:03:09 +08:00
Thomas Weißschuh 2cfdab739a hw/loongarch: remove global loaderparams variable
Passing the struct around explicitly makes the control-flow more
obvious.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Thomas Weißschuh <thomas@t-8ch.de>
Message-Id: <20231010-loongarch-loader-params-v2-1-512cc7959683@t-8ch.de>
Signed-off-by: Song Gao <gaosong@loongson.cn>
2023-10-13 10:02:11 +08:00
Song Gao 5f1a3132c6 target/loongarch: Add preldx instruction
Resolve the issue of starting the Loongnix 20.5[1] system failure.

Logs:
    Loading Linux 4.19.0-19-loongson-3 ...
    Loading initial ramdisk ...
    PROGRESS CODE: V02010004 I0
    PROGRESS CODE: V03101019 I0
    Error: unknown opcode. 90000000003a3e6c: 0x382c6d82

[1] http://pkg.loongnix.cn/loongnix/isos/Loongnix-20.5/Loongnix-20.5.cartoon.gui.loongarch64.en.qcow2

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Message-Id: <20230905123910.3052023-1-gaosong@loongson.cn>
2023-10-13 09:50:16 +08:00
Jiajie Chen 32f4916cfb target/loongarch: fix ASXE flag conflict
HW_FLAGS_EUEN_ASXE acccidentally conflicts with HW_FLAGS_CRMD_PG,
enabling LASX instructions even when CSR_EUEN.ASXE=0.

Closes: https://gitlab.com/qemu-project/qemu/-/issues/1907
Signed-off-by: Jiajie Chen <c@jia.je>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Song Gao <gaosong@loongson.cn>
Message-Id: <20230930112837.1871691-1-c@jia.je>
Signed-off-by: Song Gao <gaosong@loongson.cn>
2023-10-13 09:50:16 +08:00