Commit Graph

96836 Commits

Author SHA1 Message Date
Stefan Hajnoczi
f387cac5af linux-aio: fix unbalanced plugged counter in laio_io_unplug()
Every laio_io_plug() call has a matching laio_io_unplug() call. There is
a plugged counter that tracks the number of levels of plugging and
allows for nesting.

The plugged counter must reflect the balance between laio_io_plug() and
laio_io_unplug() calls accurately. Otherwise I/O stalls occur since
io_submit(2) calls are skipped while plugged.

Reported-by: Nikolay Tenev <nt@storpool.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Message-id: 20220609164712.1539045-2-stefanha@redhat.com
Cc: Stefano Garzarella <sgarzare@redhat.com>
Fixes: 68d7946648 ("linux-aio: add `dev_max_batch` parameter to laio_io_unplug()")
[Stefano Garzarella suggested adding a Fixes tag.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 16:43:42 +01:00
Jagannathan Raman
78e27dfa8d vfio-user: handle reset of remote device
Adds handler to reset a remote device

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 112eeadf3bc4c6cdb100bc3f9a6fcfc20b467c1b.1655151679.git.jag.raman@oracle.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 16:43:42 +01:00
Jagannathan Raman
08cf3dc611 vfio-user: handle device interrupts
Forward remote device's interrupts to the guest

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Message-id: 9523479eaafe050677f4de2af5dd0df18c27cfd9.1655151679.git.jag.raman@oracle.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 16:43:42 +01:00
Jagannathan Raman
3123f93d6b vfio-user: handle PCI BAR accesses
Determine the BARs used by the PCI device and register handlers to
manage the access to the same.

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 3373e10b5be5f42846f0632d4382466e1698c505.1655151679.git.jag.raman@oracle.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 16:43:42 +01:00
Jagannathan Raman
15ccf9bee7 vfio-user: handle DMA mappings
Define and register callbacks to manage the RAM regions used for
device DMA

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: faacbcd45c4d02c591f0dbfdc19041fbb3eae7eb.1655151679.git.jag.raman@oracle.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 16:43:42 +01:00
Jagannathan Raman
253007d147 vfio-user: IOMMU support for remote device
Assign separate address space for each device in the remote processes.

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: afe0b0a97582cdad42b5b25636a29c523265a10a.1655151679.git.jag.raman@oracle.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 16:43:42 +01:00
Jagannathan Raman
90072f29d6 vfio-user: handle PCI config space accesses
Define and register handlers for PCI config space accesses

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: be9d2ccf9b1d24e50dcd9c23404dbf284142cec7.1655151679.git.jag.raman@oracle.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 16:43:42 +01:00
Jagannathan Raman
9fb3fba149 vfio-user: run vfio-user context
Setup a handler to run vfio-user context. The context is driven by
messages to the file descriptor associated with it - get the fd for
the context and hook up the handler with it

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: e934b0090529d448b6a7972b21dfc3d7421ce494.1655151679.git.jag.raman@oracle.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 16:43:42 +01:00
Jagannathan Raman
a6e8d6d98e vfio-user: find and init PCI device
Find the PCI device with specified id. Initialize the device context
with the QEMU PCI device

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 7798dbd730099b33fdd00c4c202cfe79e5c5c151.1655151679.git.jag.raman@oracle.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 16:43:42 +01:00
Jagannathan Raman
87f7249ff6 vfio-user: instantiate vfio-user context
create a context with the vfio-user library to run a PCI device

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: a452871ac8c812ff96fc4f0ce6037f4769953fab.1655151679.git.jag.raman@oracle.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 16:43:42 +01:00
Jagannathan Raman
8f9a9259d3 vfio-user: define vfio-user-server object
Define vfio-user object which is remote process server for QEMU. Setup
object initialization functions and properties necessary to instantiate
the object

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: e45a17001e9b38f451543a664ababdf860e5f2f2.1655151679.git.jag.raman@oracle.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 16:43:42 +01:00
Jagannathan Raman
55116968de vfio-user: build library
add the libvfio-user library as a submodule. build it as a meson
subproject.

libvfio-user is distributed with BSD 3-Clause license and
json-c with MIT (Expat) license

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: c2adec87958b081d1dc8775d4aa05c897912f025.1655151679.git.jag.raman@oracle.com

[Changed submodule URL to QEMU's libvfio-user mirror on GitLab. The QEMU
project mirrors its dependencies so that it can provide full source code
even in the event that its dependencies become unavailable. Note that
the mirror repo is manually updated, so please contact me to make newer
libvfio-user commits available. If I become a bottleneck we can set up a
cronjob.

Updated scripts/meson-buildoptions.sh to match the meson_options.txt
change. Failure to do so can result in scripts/meson-buildoptions.sh
being modified by the build system later on and you end up with a dirty
working tree.
--Stefan]

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 16:42:33 +01:00
Jagannathan Raman
9b5b473eae remote/machine: add vfio-user property
Add vfio-user to x-remote machine. It is a boolean, which indicates if
the machine supports vfio-user protocol. The machine configures the bus
differently vfio-user and multiprocess protocols, so this property
informs it on how to configure the bus.

This property should be short lived. Once vfio-user fully replaces
multiprocess, this property could be removed.

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 5d51a152a419cbda35d070b8e49b772b60a7230a.1655151679.git.jag.raman@oracle.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 14:50:41 +01:00
Jagannathan Raman
661e21c48e remote/machine: add HotplugHandler for remote machine
Allow hotplugging of PCI(e) devices to remote machine

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: d1e6cfa0afb528ad343758f9b1d918be0175c5e5.1655151679.git.jag.raman@oracle.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 14:50:41 +01:00
Jagannathan Raman
217c7f01ad qdev: unplug blocker for devices
Add blocker to prevent hot-unplug of devices

TYPE_VFIO_USER_SERVER, which is introduced shortly, attaches itself to a
PCIDevice on which it depends. If the attached PCIDevice gets removed
while the server in use, it could cause it crash. To prevent this,
TYPE_VFIO_USER_SERVER adds an unplug blocker for the PCIDevice.

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: c41ef80b7cc063314d629737bed2159e5713f2e0.1655151679.git.jag.raman@oracle.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 14:50:41 +01:00
Sam Li
e2848bc574 Use io_uring_register_ring_fd() to skip fd operations
Linux recently added a new io_uring(7) optimization API that QEMU
doesn't take advantage of yet. The liburing library that QEMU uses
has added a corresponding new API calling io_uring_register_ring_fd().
When this API is called after creating the ring, the io_uring_submit()
library function passes a flag to the io_uring_enter(2) syscall
allowing it to skip the ring file descriptor fdget()/fdput()
operations. This saves some CPU cycles.

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Message-id: 20220531105011.111082-1-faithilikerun@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 14:50:41 +01:00
Vladimir Sementsov-Ogievskiy
9e4067b8a4 MAINTAINERS: update Vladimir's address and repositories
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-id: 20220526115432.138384-1-vsementsov@yandex-team.ru
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-15 14:50:41 +01:00
Paolo Bonzini
76ca98b0f8 build: include pc-bios/ part in the ROMS variable
Include the full path in TARGET_DIR, so that messages from sub-Makefiles
are clearer.  Also, prepare for possibly building firmware outside
pc-bios/ from the Makefile,

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-15 11:12:32 +02:00
Paolo Bonzini
12640f05eb meson: put cross compiler info in a separate section
While at it, remove a dead assignment and simply inline the value of the
"target" variable, which is used just once.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-15 11:12:32 +02:00
Zhenzhong Duan
766a981474 q35:Enable TSEG only when G_SMRAME and TSEG_EN both enabled
According to spec:
"TSEG Enable (T_EN): Enabling of SMRAM memory for Extended SMRAM space
only. When G_SMRAME = 1 and TSEG_EN = 1, the TSEG is enabled to appear
in the appropriate physical address space. Note that once D_LCK is set,
this bit becomes read only."

Changed to match the spec description.

Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
Message-Id: <20220615034501.2733802-1-zhenzhong.duan@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-15 11:11:37 +02:00
Alexander Bulekov
aa4f3a3b88 build: fix check for -fsanitize-coverage-allowlist
The existing check has two problems:
1. Meson uses a private directory for the get_supported_arguments check.
./instrumentation-filter does not exist in that private directory (it is
copied into the root of the build-directory).

2. fsanitize-coverage-allowlist is unused when coverage instrumentation
is not configured. No instrumentation are passed for the
get_supported_arguments check

Thus the check always fails. To work around this, change the check to an
"if cc.compiles" check and provide /dev/null, instead of the real
filter.

Meson log:
Working directory:  build/meson-private/tmpl6wld2d9
Command line:  clang-13 -m64 -mcx16
build/meson-private/tmpl6wld2d9/output.obj -c -O3 -D_FILE_OFFSET_BITS=64
-O0 -Werror=implicit-function-declaration -Werror=unknown-warning-option
-Werror=unused-command-line-argument
-Werror=ignored-optimization-argument
-fsanitize-coverage-allowlist=instrumentation-filter

Error:
error: argument unused during compilation:
'-fsanitize-coverage-allowlist=instrumentation-filter'

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20220614155415.4023833-1-alxndr@bu.edu>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-15 11:11:37 +02:00
Paolo Bonzini
39735a914d tests/vm: allow running tests in an unconfigured source tree
tests/vm/Makefile.include used to assume that it could run in an unconfigured
source tree, and Cirrus CI relies on that.  It was however broken by commit
f4c66f1705 ("tests: use tests/venv to run basevm.py-based scripts", 2022-06-06),
which co-opted the virtual environment being used by avocado tests
to also run the basevm.py tests.

For now, reintroduce the usage of qemu.qmp from the source directory, but
without the sys.path() hacks.  The CI configuration can be changed to
install the package via pip when qemu.qmp is removed from the source tree.

Cc: John Snow <jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-15 11:11:37 +02:00
Paolo Bonzini
b9eae9efae configure: cleanup -fno-pie detection
Place it only inside the 'if test "$pie" = "no"' conditional.  Since
commit 43924d1e53 ("pc-bios/optionrom: detect -fno-pie", 2022-05-12),
the PIE options are detected independently by pc-bios/optionrom/Makefile,
and the CFLAGS_NOPIE/LDFLAGS_NOPIE variables are not used anymore.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-15 11:11:37 +02:00
Paolo Bonzini
b5569e5b56 configure: update list of preserved environment variables
INSTALL and LIBTOOL are not used anymore, but OBJCFLAGS is new and
was not listed.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:30 +02:00
Paolo Bonzini
26cfd67981 virtio-mmio: cleanup reset
Make virtio_mmio_soft_reset reset the virtio device, which is performed by
both the "soft" and the "hard" reset; and then call virtio_mmio_soft_reset
from virtio_mmio_reset to emphasize that the latter is a superset of the
former.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:30 +02:00
Paolo Bonzini
9e43a83041 virtio: stop ioeventfd on reset
All calls to virtio_bus_reset are preceded by virtio_bus_stop_ioeventfd,
move the call in virtio_bus_reset: that makes sense and clarifies
that the vdc->reset function is called with ioeventfd already stopped.

Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:30 +02:00
Paolo Bonzini
a44bed2f54 virtio-mmio: stop ioeventfd on legacy reset
If the queue PFN is set to zero on a virtio-mmio device, the device is reset.
In that case however the virtio_bus_stop_ioeventfd function was not
called; add it so that the behavior is similar to when status is set to 0.

Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:30 +02:00
Paolo Bonzini
997340f3c5 s390x: simplify virtio_ccw_reset_virtio
Call virtio_bus_reset instead of virtio_reset, so that the function
need not receive the VirtIODevice.

Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:30 +02:00
Paolo Bonzini
f55ba8018c block: add more commands to preconfig mode
Of the block device commands, those that are available outside system
emulators do not require a fully constructed machine by definition.
Allow running them before machine initialization has concluded.

Of the ones that are available inside system emulation, allow querying
the PR managers, and setting up accounting and throttling.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:30 +02:00
Paolo Bonzini
39cd0c7f12 hmp: add filtering of statistics by name
Allow the user to request only a specific subset of statistics.
This can be useful when working on a feature or optimization that is
known to affect that statistic.

Example:

   (qemu) info stats vcpu halt_poll_fail_ns
   provider: kvm
       halt_poll_fail_ns (cumulative, ns): 0

In case multiple providers have the same statistic, the provider can be
specified too:

   (qemu) info stats vcpu halt_poll_fail_ns kvm
   provider: kvm
       halt_poll_fail_ns (cumulative, ns): 0

Extracted from a patch by Mark Kanda.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:30 +02:00
Paolo Bonzini
cf7405bc02 qmp: add filtering of statistics by name
Allow retrieving only a subset of statistics.  This can be useful
for example in order to plot a subset of the statistics many times
a second: KVM publishes ~40 statistics for each vCPU on x86; retrieving
and serializing all of them would be useless.

Another use will be in HMP in the following patch; implementing the
filter in the backend is easy enough that it was deemed okay to make
this a public interface.

Example:

{ "execute": "query-stats",
  "arguments": {
    "target": "vcpu",
    "vcpus": [ "/machine/unattached/device[2]",
               "/machine/unattached/device[4]" ],
    "providers": [
      { "provider": "kvm",
        "names": [ "l1d_flush", "exits" ] } } }

{ "return": {
    "vcpus": [
      { "path": "/machine/unattached/device[2]"
        "providers": [
          { "provider": "kvm",
            "stats": [ { "name": "l1d_flush", "value": 41213 },
                       { "name": "exits", "value": 74291 } ] } ] },
      { "path": "/machine/unattached/device[4]"
        "providers": [
          { "provider": "kvm",
            "stats": [ { "name": "l1d_flush", "value": 16132 },
                       { "name": "exits", "value": 57922 } ] } ] } ] } }

Extracted from a patch by Mark Kanda.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:30 +02:00
Paolo Bonzini
7716417eac hmp: add filtering of statistics by provider
Allow the user to request statistics for a single provider of interest.
Extracted from a patch by Mark Kanda.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:30 +02:00
Paolo Bonzini
068cc51d42 qmp: add filtering of statistics by provider
Allow retrieving the statistics from a specific provider only.
This can be used in the future by HMP commands such as "info
sync-profile" or "info profile".  The next patch also adds
filter-by-provider capabilities to the HMP equivalent of
query-stats, "info stats".

Example:

{ "execute": "query-stats",
  "arguments": {
    "target": "vm",
    "providers": [
      { "provider": "kvm" } ] } }

The QAPI is a bit more verbose than just a list of StatsProvider,
so that it can be subsequently extended with filtering of statistics
by name.

If a provider is specified more than once in the filter, each request
will be included separately in the output.

Extracted from a patch by Mark Kanda.

Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:30 +02:00
Mark Kanda
433815f5bd hmp: add basic "info stats" implementation
Add an HMP command to retrieve statistics collected at run-time.
The command will retrieve and print either all VM-level statistics,
or all vCPU-level statistics for the currently selected CPU.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:30 +02:00
Paolo Bonzini
cfb3448922 cutils: add functions for IEC and SI prefixes
Extract the knowledge of IEC and SI prefixes out of size_to_str and
freq_to_str, so that it can be reused when printing statistics.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:30 +02:00
Paolo Bonzini
467ef823d8 qmp: add filtering of statistics by target vCPU
Introduce a simple filtering of statistics, that allows to retrieve
statistics for a subset of the guest vCPUs.  This will be used for
example by the HMP monitor, in order to retrieve the statistics
for the currently selected CPU.

Example:
{ "execute": "query-stats",
  "arguments": {
    "target": "vcpu",
    "vcpus": [ "/machine/unattached/device[2]",
               "/machine/unattached/device[4]" ] } }

Extracted from a patch by Mark Kanda.

Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:30 +02:00
Mark Kanda
cc01a3f4ca kvm: Support for querying fd-based stats
Add support for querying fd-based KVM stats - as introduced by Linux kernel
commit:

cb082bfab59a ("KVM: stats: Add fd-based API to read binary stats data")

This allows the user to analyze the behavior of the VM without access
to debugfs.

Signed-off-by: Mark Kanda <mark.kanda@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:30 +02:00
Mark Kanda
b9f88dc071 qmp: Support for querying stats
Gathering statistics is important for development, for monitoring and
for performance measurement.  There are tools such as kvm_stat that do
this and they rely on the _user_ knowing the interesting data points
rather than the tool (which can treat them as opaque).

The commands introduced in this commit introduce QMP support for
querying stats; the goal is to take the capabilities of these tools
and making them available throughout the whole virtualization stack,
so that one can observe, monitor and measure virtual machines without
having shell access + root on the host that runs them.

query-stats returns a list of all stats per target type (only VM
and vCPU to start); future commits add extra options for specifying
stat names, vCPU qom paths, and providers.  All these are used by the
HMP command "info stats".  Because of the development usecases around
statistics, a good HMP interface is important.

query-stats-schemas returns a list of stats included in each target
type, with an option for specifying the provider.  The concepts in the
schema are based on the KVM binary stats' own introspection data, just
translated to QAPI.

There are two reasons to have a separate schema that is not tied to
the QAPI schema.  The first is the contents of the schemas: the new
introspection data provides different information than the QAPI data,
namely unit of measurement, how the numbers are gathered and change
(peak/instant/cumulative/histogram), and histogram bucket sizes.
There's really no reason to have this kind of metadata in the QAPI
introspection schema (except possibly for the unit of measure, but
there's a very weak justification).

Another reason is the dynamicity of the schema.  The QAPI introspection
data is very much static; and while QOM is somewhat more dynamic,
generally we consider that to be a bug rather than a feature these days.
On the other hand, the statistics that are exposed by QEMU might be
passed through from another source, such as KVM, and the disadvantages of
manually updating the QAPI schema for outweight the benefits from vetting
the statistics and filtering out anything that seems "too unstable".
Running old QEMU with new kernel is a supported usecase; if old QEMU
cannot expose statistics from a new kernel, or if a kernel developer
needs to change QEMU before gathering new info from the new kernel,
then that is a poor user interface.

The framework provides a method to register callbacks for these QMP
commands.  Most of the work in fact is done by the callbacks, and a
large majority of this patch is new QAPI structs and commands.

Examples (with KVM stats):

- Query all VM stats:

{ "execute": "query-stats", "arguments" : { "target": "vm" } }

{ "return": [
     { "provider": "kvm",
       "stats": [
          { "name": "max_mmu_page_hash_collisions", "value": 0 },
          { "name": "max_mmu_rmap_size", "value": 0 },
          { "name": "nx_lpage_splits", "value": 148 },
          ... ] },
     { "provider": "xyz",
       "stats": [ ... ] }
] }

- Query all vCPU stats:

{ "execute": "query-stats", "arguments" : { "target": "vcpu" } }

{ "return": [
     { "provider": "kvm",
       "qom_path": "/machine/unattached/device[0]"
       "stats": [
          { "name": "guest_mode", "value": 0 },
          { "name": "directed_yield_successful", "value": 0 },
          { "name": "directed_yield_attempted", "value": 106 },
          ... ] },
     { "provider": "kvm",
       "qom_path": "/machine/unattached/device[1]"
       "stats": [
          { "name": "guest_mode", "value": 0 },
          { "name": "directed_yield_successful", "value": 0 },
          { "name": "directed_yield_attempted", "value": 106 },
          ... ] },
] }

- Retrieve the schemas:

{ "execute": "query-stats-schemas" }

{ "return": [
    { "provider": "kvm",
      "target": "vcpu",
      "stats": [
         { "name": "guest_mode",
           "unit": "none",
           "base": 10,
           "exponent": 0,
           "type": "instant" },
        { "name": "directed_yield_successful",
           "unit": "none",
           "base": 10,
           "exponent": 0,
           "type": "cumulative" },
        ... ]
    },
    { "provider": "kvm",
      "target": "vm",
      "stats": [
        { "name": "max_mmu_page_hash_collisions",
           "unit": "none",
           "base": 10,
           "exponent": 0,
           "type": "peak" },
        ... ]
    },
    { "provider": "xyz",
      "target": "vm",
      "stats": [ ... ]
    }
] }

Signed-off-by: Mark Kanda <mark.kanda@oracle.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:29 +02:00
Warner Losh
d35020ed00 bsd-user: Implement acct and sync
Signed-off-by: Stacey Son <sson@FreeBSD.org>
Signed-off-by: Warner Losh <imp@bsdimp.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2022-06-14 08:17:44 -06:00
Warner Losh
4b795b147b bsd-user: Implement trunctate and ftruncate
Signed-off-by: Stacey Son <sson@FreeBSD.org>
Signed-off-by: Warner Losh <imp@bsdimp.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2022-06-14 08:17:44 -06:00
Warner Losh
a15699acaf bsd-user: Implement dup and dup2
Signed-off-by: Stacey Son <sson@FreeBSD.org>
Signed-off-by: Warner Losh <imp@bsdimp.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2022-06-14 08:17:44 -06:00
Warner Losh
6af8f76a9f bsd-user: Implement rmdir and undocumented __getcwd
Implemenet rmdir and __getcwd. __getcwd is the undocumented
back end to getcwd(3).

Signed-off-by: Stacey Son <sson@FreeBSD.org>
Signed-off-by: Jung-uk Kim <jkim@FreeBSD.org>
Signed-off-by: Warner Losh <imp@bsdimp.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2022-06-14 08:17:37 -06:00
Warner Losh
1ffbd5e7fe bsd-user: Implement mkdir and mkdirat
Signed-off-by: Stacey Son <sson@FreeBSD.org>
Signed-off-by: Warner Losh <imp@bsdimp.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2022-06-14 08:17:31 -06:00
Richard Henderson
8e6c70b9d4 usb: add CanoKey device, fixes for ehci + redir
ui: fixes for gtk and cocoa, rework refresh rate
 virtio-gpu: scanout flush fix
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEoDKM/7k6F6eZAf59TLbY7tPocTgFAmKoe/8ACgkQTLbY7tPo
 cTgZqw/9HD5dMjP74jwrf14dSCR6FD8PfSZU43YBZtMKMtYIzSgrG0NGmreDIhmr
 ZM+G0By+J8vFaSqDukX31077DnptyrxsANOg3zc28SfOCrI7I/mNVymd9hl+Ydpd
 A7h0DpHxs1mkpTVxGoXZoJRGXUE41rctbFVjG3CGynSG9K2vFQRsJz0jG723dg5Y
 uv+Di1WkhqNkyKNsTEGbz9LNqtdtGzvQm3COBpKoTsl4X3EXIE68Qh7i3cMTSNIw
 KKPARW3oiCOy3Fc4kQW9nSxkkHMS6NPL1uyQ52j7pXYxRdxRaREFQ9Gxst3ie9bS
 mbqSuzS2+1v0w37bq9wE0PiCkmwWnu2KWiWWkAIYlmmZTgHvgxCvPcJaeItmap27
 dsAuPUGBbhhrmUwfMgJXp/wRvoZQc2l9w9+eUklsbI+VTbr6i+r/OoLRmnDJr+K/
 yNscMU1LzoigK0NDdP+PnFl3k8pux0Awtotgfyd+UGTSW8a5L6UFAWIxcUcd0Jjv
 24jAEEc1S1ciDxJDWYn4+17KJARG7no2PRXsGXCUNaWduGEk8wPK+i6Xk82U36o7
 7j0N16RFNv1YSUaUJHgtmAMRJIQMCiB42VaYxlDfzKupvq2RgRWaWBD/HozgLhXn
 DjEX+JRAnaOYnn1NURzTNDwnhQethJRXI1ntI1U8IFLYT4baSCY=
 =L5PO
 -----END PGP SIGNATURE-----

Merge tag 'kraxel-20220614-pull-request' of git://git.kraxel.org/qemu into staging

usb: add CanoKey device, fixes for ehci + redir
ui: fixes for gtk and cocoa, rework refresh rate
virtio-gpu: scanout flush fix

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCgAdFiEEoDKM/7k6F6eZAf59TLbY7tPocTgFAmKoe/8ACgkQTLbY7tPo
# cTgZqw/9HD5dMjP74jwrf14dSCR6FD8PfSZU43YBZtMKMtYIzSgrG0NGmreDIhmr
# ZM+G0By+J8vFaSqDukX31077DnptyrxsANOg3zc28SfOCrI7I/mNVymd9hl+Ydpd
# A7h0DpHxs1mkpTVxGoXZoJRGXUE41rctbFVjG3CGynSG9K2vFQRsJz0jG723dg5Y
# uv+Di1WkhqNkyKNsTEGbz9LNqtdtGzvQm3COBpKoTsl4X3EXIE68Qh7i3cMTSNIw
# KKPARW3oiCOy3Fc4kQW9nSxkkHMS6NPL1uyQ52j7pXYxRdxRaREFQ9Gxst3ie9bS
# mbqSuzS2+1v0w37bq9wE0PiCkmwWnu2KWiWWkAIYlmmZTgHvgxCvPcJaeItmap27
# dsAuPUGBbhhrmUwfMgJXp/wRvoZQc2l9w9+eUklsbI+VTbr6i+r/OoLRmnDJr+K/
# yNscMU1LzoigK0NDdP+PnFl3k8pux0Awtotgfyd+UGTSW8a5L6UFAWIxcUcd0Jjv
# 24jAEEc1S1ciDxJDWYn4+17KJARG7no2PRXsGXCUNaWduGEk8wPK+i6Xk82U36o7
# 7j0N16RFNv1YSUaUJHgtmAMRJIQMCiB42VaYxlDfzKupvq2RgRWaWBD/HozgLhXn
# DjEX+JRAnaOYnn1NURzTNDwnhQethJRXI1ntI1U8IFLYT4baSCY=
# =L5PO
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 14 Jun 2022 05:15:59 AM PDT
# gpg:                using RSA key A0328CFFB93A17A79901FE7D4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>" [undefined]
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>" [undefined]
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: A032 8CFF B93A 17A7 9901  FE7D 4CB6 D8EE D3E8 7138

* tag 'kraxel-20220614-pull-request' of git://git.kraxel.org/qemu:
  virtio-gpu: Respect UI refresh rate for EDID
  ui: Deliver refresh rate via QemuUIInfo
  ui/console: Do not return a value with ui_info
  virtio-gpu: update done only on the scanout associated with rect
  usbredir: avoid queuing hello packet on snapshot restore
  hw/usb/hcd-ehci: fix writeback order
  MAINTAINERS: add myself as CanoKey maintainer
  docs/system/devices/usb: Add CanoKey to USB devices examples
  docs: Add CanoKey documentation
  meson: Add CanoKey
  hw/usb/canokey: Add trace events
  hw/usb: Add CanoKey Implementation
  ui/cocoa: Fix poweroff request code
  ui/gtk-gl-area: create the requested GL context version
  ui/gtk-gl-area: implement GL context destruction

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2022-06-14 06:21:46 -07:00
Akihiko Odaki
b95b56311a virtio-gpu: Respect UI refresh rate for EDID
Signed-off-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Message-Id: <20220226115516.59830-4-akihiko.odaki@gmail.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2022-06-14 10:34:37 +02:00
Akihiko Odaki
aeffd071ed ui: Deliver refresh rate via QemuUIInfo
This change adds a new member, refresh_rate to QemuUIInfo in
include/ui/console.h. It represents the refresh rate of the
physical display backend, and it is more appropriate than
GUI update interval as the refresh rate which the emulated device
reports:
- sdl may set GUI update interval shorter than the refresh rate
  of the physical display to respond to user-generated events.
- sdl and vnc aggressively changes GUI update interval, but
  a guests is typically not designed to respond to frequent
  refresh rate changes, or frequent "display mode" changes in
  general. The frequency of refresh rate changes of the physical
  display backend matches better to the guest's expectation.

QemuUIInfo also has other members representing "display mode",
which makes it suitable for refresh rate representation. It has
a throttling of update notifications, and prevents frequent changes
of the display mode.

Signed-off-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Message-Id: <20220226115516.59830-3-akihiko.odaki@gmail.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2022-06-14 10:34:37 +02:00
Akihiko Odaki
362239c05f ui/console: Do not return a value with ui_info
The returned value is not used and misleading.

Signed-off-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Message-Id: <20220226115516.59830-2-akihiko.odaki@gmail.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2022-06-14 10:34:37 +02:00
Dongwon Kim
45e64ab63d virtio-gpu: update done only on the scanout associated with rect
It only needs to update the scanouts containing the rect area
coming with the resource-flush request from the guest.

Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Vivek Kasireddy <vivek.kasireddy@intel.com>
Signed-off-by: Dongwon Kim <dongwon.kim@intel.com>
Message-Id: <20220505214030.4261-1-dongwon.kim@intel.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2022-06-14 10:34:37 +02:00
Joelle van Dyne
0631d4b448 usbredir: avoid queuing hello packet on snapshot restore
When launching QEMU with "-loadvm", usbredir_create_parser() should avoid
setting up the hello packet (just as with "-incoming". On the latest version
of libusbredir, usbredirparser_unserialize() will return error if the parser
is not "pristine."

Signed-off-by: Joelle van Dyne <j@getutm.app>
Message-Id: <20220507041850.98716-1-j@getutm.app>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2022-06-14 10:34:36 +02:00
Arnout Engelen
f471e8b060 hw/usb/hcd-ehci: fix writeback order
The 'active' bit passes control over a qTD between the guest and the
controller: set to 1 by guest to enable execution by the controller,
and the controller sets it to '0' to hand back control to the guest.

ehci_state_writeback write two dwords to main memory using DMA:
the third dword of the qTD (containing dt, total bytes to transfer,
cpage, cerr and status) and the fourth dword of the qTD (containing
the offset).

This commit makes sure the fourth dword is written before the third,
avoiding a race condition where a new offset written into the qTD
by the guest after it observed the status going to go to '0' gets
overwritten by a 'late' DMA writeback of the previous offset.

This race condition could lead to 'cpage out of range (5)' errors,
and reproduced by:

./qemu-system-x86_64 -enable-kvm -bios $SEABIOS/bios.bin -m 4096 -device usb-ehci -blockdev driver=file,read-only=on,filename=/home/aengelen/Downloads/openSUSE-Tumbleweed-DVD-i586-Snapshot20220428-Media.iso,node-name=iso -device usb-storage,drive=iso,bootindex=0 -chardev pipe,id=shell,path=/tmp/pipe -device virtio-serial -device virtconsole,chardev=shell -device virtio-rng-pci -serial mon:stdio -nographic

(press a key, select 'Installation' (2), and accept the default
values. On my machine the 'cpage out of range' is reproduced while
loading the Linux Kernel about once per 7 attempts. With the fix in
this commit it no longer fails)

This problem was previously reported as a seabios problem in
https://mail.coreboot.org/hyperkitty/list/seabios@seabios.org/thread/OUTHT5ISSQJGXPNTUPY3O5E5EPZJCHM3/
and as a nixos CI build failure in
https://github.com/NixOS/nixpkgs/issues/170803

Signed-off-by: Arnout Engelen <arnout@bzzt.net>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2022-06-14 10:34:36 +02:00