Commit Graph

81594 Commits

Author SHA1 Message Date
Paolo Bonzini
8ddf958e8d scsi/scsi-bus: scsi_device_find: don't return unrealized devices
The device core first places a device on the bus and then realizes it.
Make scsi_device_find avoid returing such devices to avoid
races in drivers that use an iothread (currently virtio-scsi)

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1812399

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200913160259.32145-7-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201006123904.610658-11-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:51 -04:00
Maxim Levitsky
a23151e8cc device-core: use atomic_set on .realized property
Some code might race with placement of new devices on a bus.
We currently first place a (unrealized) device on the bus
and then realize it.

As a workaround, users that scan the child device list, can
check the realized property to see if it is safe to access such a device.
Use an atomic write here too to aid with this.

A separate discussion is what to do with devices that are unrealized:
It looks like for this case we only call the hotplug handler's unplug
callback and its up to it to unrealize the device.
An atomic operation doesn't cause harm for this code path though.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200913160259.32145-6-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201006123904.610658-10-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:51 -04:00
Paolo Bonzini
42a90a899e scsi: switch to bus->check_address
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201006123904.610658-6-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:51 -04:00
Maxim Levitsky
2d24a64661 device-core: use RCU for list of children of a bus
This fixes the race between device emulation code that tries to find
a child device to dispatch the request to (e.g a scsi disk),
and hotplug of a new device to that bus.

Note that this doesn't convert all the readers of the list
but only these that might go over that list without BQL held.

This is a very small first step to make this code thread safe.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200913160259.32145-5-mlevitsk@redhat.com>
[Use RCU_READ_LOCK_GUARD in more places, adjust testcase now that
 the delay in DEVICE_DELETED due to RCU is more consistent. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201006123904.610658-9-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:50 -04:00
Maxim Levitsky
7bed89958b device_core: use drain_call_rcu in in qmp_device_add
Soon, a device removal might only happen on RCU callback execution.
This is okay for device-del which provides a DEVICE_DELETED event,
but not for the failure case of device-add.  To avoid changing
monitor semantics, just drain all pending RCU callbacks on error.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Suggested-by: Stefan Hajnoczi <stefanha@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200913160259.32145-4-mlevitsk@redhat.com>
[Don't use it in qmp_device_del. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:50 -04:00
Maxim Levitsky
7a8202c521 scsi/scsi_bus: switch search direction in scsi_device_find
This change will allow us to convert the bus children list to RCU,
while not changing the logic of this function

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200913160259.32145-2-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:50 -04:00
Paolo Bonzini
bb755ba47f qdev: add "check if address free" callback for buses
Check if an address is free on the bus before plugging in the
device.  This makes it possible to do the check without any
side effects, and to detect the problem early without having
to do it in the realize callback.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201006123904.610658-5-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:50 -04:00
Paolo Bonzini
d8a18da56d qemu-iotests, qtest: rewrite test 067 as a qtest
Test 067 from qemu-iotests is executing QMP commands to hotplug
and hot-unplug disks, devices and blockdevs.  Because the power
of the text-based test harness is limited, it is actually limiting
the checks that it does, for example by skipping DEVICE_DELETED
events.

tests/qtest already has a similar test, drive_del-test.c.
We can merge them, and even reuse some of the existing code in
drive_del-test.c.  This will improve the quality of the test by
covering DEVICE_DELETED events and testing multiple architectures
(therefore covering multiple PCI hotplug mechanisms as well as s390x
virtio-ccw).

The only difference is that the new test will always use null-co:// for
the medium rather than qcow2 or raw, but this should be irrelevant for
what the test is covering.  For example there are no "qemu-img check"
runs in 067 that would check that the file is properly closed.

The new tests requires PCI hot-plug support, so drive_del-test
is moved from qemu-system-ppc to qemu-system-ppc64.

Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:50 -04:00
Paolo Bonzini
9a613ddccc qtest: check that drives are really appearing and disappearing
Do not just trust the HMP commands to create and delete the drive, use
query-block to check that this is actually the case.

Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:49 -04:00
Maxim Levitsky
bb1a5b97f7 qtest: switch users back to qtest_qmp_receive
Let test use the new functionality for buffering events.
The only remaining users of qtest_qmp_receive_dict are tests
that fuzz the QMP protocol.

Tested with 'make check-qtest'.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20201006123904.610658-4-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:49 -04:00
Paolo Bonzini
c45a70d8c2 device-plug-test: use qtest_qmp to send the device_del command
Simplify the code now that events are buffered.  There is no need
anymore to separate sending the command and retrieving the response.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:49 -04:00
Maxim Levitsky
5e34005571 qtest: remove qtest_qmp_receive_success
The purpose of qtest_qmp_receive_success was mostly to process events
that arrived between the issueing of a command and the "return"
line from QMP.  This is now handled by the buffering of events
that libqtest performs automatically.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
2020-10-12 11:50:49 -04:00
Maxim Levitsky
c22045bfe6 qtest: Reintroduce qtest_qmp_receive with QMP event buffering
The new qtest_qmp_receive buffers all the received qmp events, allowing
qtest_qmp_eventwait_ref to return them.

This is intended to solve the race in regard to ordering of qmp events
vs qmp responses, as soon as the callers start using the new interface.

In addition to that, define qtest_qmp_event_ref a function which only scans
the buffer that qtest_qmp_receive stores the events to.  This is intended
for callers that are only interested in events that were received during
the last call to the qtest_qmp_receive.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20201006123904.610658-3-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:48 -04:00
Maxim Levitsky
1c3e2a38de qtest: rename qtest_qmp_receive to qtest_qmp_receive_dict
In the next patch a new version of qtest_qmp_receive will be
reintroduced that will buffer received qmp events for later
consumption in qtest_qmp_eventwait_ref

No functional change intended.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:24 -04:00
Huacai Chen
fbc5884ce2 meson.build: Re-enable KVM support for MIPS
After converting from configure to meson, KVM support is lost for MIPS,
so re-enable it in meson.build.

Fixes: fdb75aeff7 ("configure: remove target configuration")
Fixes: 8a19980e3f ("configure: move accelerator logic to meson")
Cc: aolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Message-Id: <1602059975-10115-3-git-send-email-chenhc@lemote.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:24 -04:00
Marc-André Lureau
242fb750bd build-sys: fix git version from -version
Typo introduced with the script.

Fixes: 2c273f32d3 ("meson: generate qemu-version.h")
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reported-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20200929143654.518157-1-marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:23 -04:00
Paolo Bonzini
bab88ead6f docs/devel: update instruction on how to add new unit tests
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:23 -04:00
Paolo Bonzini
da00d067ea qtest: unify extra_qtest_srcs and extra_qtest_deps
Currently the extra sources and extra dependencies of qtests are held
in two separate dictionaries.  Use the same trick as tests/meson.build
to combine them into one.  This will make it easier to update the
documentation for unit tests and qtests.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:23 -04:00
Eduardo Habkost
51c778edd3 docs/devel/qtest: Include libqtest API reference
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201005205228.697463-4-ehabkost@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:23 -04:00
Eduardo Habkost
f59c6de7f0 docs/devel/qtest: Include protocol spec in document
Include the QTest Protocol doc string in docs/devel/qtest.rst,
after converting it to use Sphinx syntax.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201005205228.697463-3-ehabkost@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:22 -04:00
Eduardo Habkost
a738a50e62 docs: Move QTest documentation to its own document
The qtest and libqtest doc comments will be parsed to generate
API documentation, so move QTest documentation to its own
document where the API and format documentation and will be
included.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201005205228.697463-2-ehabkost@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:22 -04:00
Paolo Bonzini
22fb6eb571 qom: fix objects with improper parent type
Some objects accidentally inherit ObjectClass instead of Object.
They compile silently but may crash after downcasting.

In this patch, we introduce a coccinelle script to find broken
declarations and fix them manually with proper base type.

Signed-off-by: Sergey Nizovtsev <snizovtsev@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:22 -04:00
Paolo Bonzini
d9f24bf572 exec: split out non-softmmu-specific parts
Over the years, most parts of exec.c that were not specific to softmmu
have been moved to accel/tcg; what's left is mostly the low-level part
of the memory API, which includes RAMBlock and AddressSpaceDispatch.
However exec.c also hosts 4-500 lines of code for the target specific
parts of the CPU QOM object, plus a few functions for user-mode
emulation that do not have a better place (they are not TCG-specific so
accel/tcg/user-exec.c is not a good place either).

Move these parts to a new file, so that exec.c can be moved to
softmmu/physmem.c.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:22 -04:00
Paolo Bonzini
800d4deda0 softmmu: move more files to softmmu/
Keep most softmmu_ss files into the system-emulation-specific
directory.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:21 -04:00
Philippe Mathieu-Daudé
8d0bceba24 hw/nvram: Always register FW_CFG_DATA_GENERATOR_INTERFACE
While the FW_CFG_DATA_GENERATOR_INTERFACE is only consumed
by a device only available using system-mode (fw_cfg), it is
implemented by a crypto component (tls-cipher-suites) which
is always available when crypto is used.

Commit 69699f3055 introduced the following error in the
qemu-storage-daemon binary:

  $ echo -e \
    '{"execute": "qmp_capabilities"}\r\n{"execute": "qom-list-types"}\r\n{"execute": "quit"}\r\n' \
    | storage-daemon/qemu-storage-daemon --chardev stdio,id=qmp0  --monitor qmp0
  {"QMP": {"version": {"qemu": {"micro": 50, "minor": 1, "major": 5}, "package": ""}, "capabilities": ["oob"]}}
  {"return": {}}
  missing interface 'fw_cfg-data-generator' for object 'tls-creds'
  Aborted (core dumped)

Since QOM dependencies are resolved at runtime, this issue
could not be triggered at linktime, and we don't have test
running the qemu-storage-daemon binary.

Fix by always registering the QOM interface.

Reported-by: Kevin Wolf <kwolf@redhat.com>
Fixes: 69699f3055 ("crypto/tls-cipher-suites: Produce fw_cfg consumable blob")
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201006111909.2302081-2-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:20 -04:00
Philippe Mathieu-Daudé
da33fc0987 qom: Move the creation of the library to the main meson.build
Be consistent creating all the libraries in the main meson.build file.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201006125602.2311423-10-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:20 -04:00
Philippe Mathieu-Daudé
5556789176 authz: Move the creation of the library to the main meson.build
Be consistent creating all the libraries in the main meson.build file.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201006125602.2311423-9-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:20 -04:00
Philippe Mathieu-Daudé
2389304a72 crypto: Move the creation of the library to the main meson.build
Be consistent creating all the libraries in the main meson.build file.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201006125602.2311423-8-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:20 -04:00
Philippe Mathieu-Daudé
f78536b149 io: Move the creation of the library to the main meson.build
Be consistent creating all the libraries in the main meson.build file.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201006125602.2311423-7-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:20 -04:00
Philippe Mathieu-Daudé
7e6edef3f8 migration: Move the creation of the library to the main meson.build
Be consistent creating all the libraries in the main meson.build file.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201006125602.2311423-6-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:20 -04:00
Philippe Mathieu-Daudé
c2306d713e chardev: Move the creation of the library to the main meson.build
Be consistent creating all the libraries in the main meson.build file.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201006125602.2311423-5-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:19 -04:00
Philippe Mathieu-Daudé
e28ab096bf hw/core: Move the creation of the library to the main meson.build
Be consistent creating all the libraries in the main meson.build file.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201006125602.2311423-4-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:19 -04:00
Philippe Mathieu-Daudé
7e2b888f62 meson.build: Sort sourcesets alphabetically
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201006125602.2311423-3-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:19 -04:00
Philippe Mathieu-Daudé
064f8ee7aa meson.build: Add comments to clarify code organization
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201006125602.2311423-2-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12 11:50:19 -04:00
Peter Maydell
a0bdf86687 v3 Migration+ virtiofsd pull 2020-10-12
V3
   Remove the postcopy recovery changes
 
 Migration:
   Dirtyrate measurement API cleanup
 
 Virtiofsd:
   Missing qemu_init_exec_dir call
   Support for setting the group on socket creation
   Stop a gcc warning
   Avoid tempdir in sandboxing
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEERfXHG0oMt/uXep+pBRYzHrxb/ecFAl+EQWIACgkQBRYzHrxb
 /edFdQ/+Mv2A3gHGWDeVkVmotTEBppRa22WjipSiUzdJhn25q/T6twyM7FXwZPQn
 1j8A0nmY3mogx/Z0lB1jMOEUbI7x5WedF82gdZQXVskbOtaW4bp9IktkzfNo6xUU
 A6lcG2hZeCOwnqtOkibxf1EHznO0/d3BMIGwBV773hkji0MFmyj/ekxmJIS3y6rn
 m/TEOGQeDdRp9BZcMaGEbW3IJ8+Zz6aWTOwm7/AKMkf3MZJ0ltyIhb3WUi+qsSR4
 8+bq7ayPn9sxyCReLU4AmktZ33bqkU7JLXgGahNP4ySLbKIJSMzloWroFnl6Oynx
 oWOv7PzBwA+pwOCeihoDIOthl7MIpZZg6Q8YfE6vG/7phqtOB1UW3DOWaWdTbYjE
 d3o5S1T7kjK10dLqZ5iuNnQ0iH0qwDDSHQQL8IefnyXDweUZU1JuNwHP5VT30ia5
 B4lUTYGQJ60JXnpJJKE1bUioxDzlsPKCgoFfTS9e/ULUt6RW4m6Ctmmu8aJy7mm5
 NUEWyk39oionJBZ42jBPD/w/DaZorirqIGPeZxE99shasuWazpZ2LZj0ksYV1x5f
 yGEhvXjdcKt6YwRADzno8cCisZcWhVnJhXUDBPTiBKYmFNlY2aVrB84Lixpysnp8
 Dsis6XvHb9jLceAUl0Qut6JlGzZ0NRq6bBk+YsknRJeGBdmWz8w=
 =0yqB
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20201012a' into staging

v3 Migration+ virtiofsd pull 2020-10-12

V3
  Remove the postcopy recovery changes

Migration:
  Dirtyrate measurement API cleanup

Virtiofsd:
  Missing qemu_init_exec_dir call
  Support for setting the group on socket creation
  Stop a gcc warning
  Avoid tempdir in sandboxing

# gpg: Signature made Mon 12 Oct 2020 12:43:30 BST
# gpg:                using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert/tags/pull-migration-20201012a:
  migration/dirtyrate: present dirty rate only when querying the rate has completed
  migration/dirtyrate: record start_time and calc_time while at the measuring state
  virtiofsd: avoid /proc/self/fd tempdir
  virtiofsd: Call qemu_init_exec_dir
  tools/virtiofsd: add support for --socket-group
  virtiofsd: Silence gcc warning

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-12 13:12:04 +01:00
Chuan Zheng
b1a859cfb0 migration/dirtyrate: present dirty rate only when querying the rate has completed
Make dirty_rate field optional, present dirty rate only when querying
the rate has completed.
The qmp results is shown as follow:
@unstarted:
{"return":{"status":"unstarted","start-time":0,"calc-time":0},"id":"libvirt-12"}
@measuring:
{"return":{"status":"measuring","start-time":102931,"calc-time":1},"id":"libvirt-85"}
@measured:
{"return":{"status":"measured","dirty-rate":4,"start-time":150146,"calc-time":1},"id":"libvirt-15"}

Signed-off-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: David Edmondson <david.edmondson@oracle.com>
Message-Id: <1601350938-128320-3-git-send-email-zhengchuan@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-10-12 12:39:38 +01:00
Chuan Zheng
aa84b506f7 migration/dirtyrate: record start_time and calc_time while at the measuring state
Querying could include both the start-time and the calc-time while at the measuring
state, allowing a caller to determine when they should expect to come back looking
for a result.

Signed-off-by: Chuan Zheng <zhengchuan@huawei.com>
Message-Id: <1601350938-128320-2-git-send-email-zhengchuan@huawei.com>
Reviewed-by: David Edmondson <david.edmondson@oracle.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-10-12 12:39:38 +01:00
Stefan Hajnoczi
ebf101955c virtiofsd: avoid /proc/self/fd tempdir
In order to prevent /proc/self/fd escapes a temporary directory is
created where /proc/self/fd is bind-mounted. This doesn't work on
read-only file systems.

Avoid the temporary directory by bind-mounting /proc/self/fd over /proc.
This does not affect other processes since we remounted / with MS_REC |
MS_SLAVE. /proc must exist and virtiofsd does not use it so it's safe to
do this.

Path traversal can be tested with the following function:

  static void test_proc_fd_escape(struct lo_data *lo)
  {
      int fd;
      int level = 0;
      ino_t last_ino = 0;

      fd = lo->proc_self_fd;
      for (;;) {
          struct stat st;

          if (fstat(fd, &st) != 0) {
              perror("fstat");
              return;
          }
          if (last_ino && st.st_ino == last_ino) {
              fprintf(stderr, "inode number unchanged, stopping\n");
              return;
          }
          last_ino = st.st_ino;

          fprintf(stderr, "Level %d dev %lu ino %lu\n", level,
                  (unsigned long)st.st_dev,
                  (unsigned long)last_ino);
          fd = openat(fd, "..", O_PATH | O_DIRECTORY | O_NOFOLLOW);
          level++;
      }
  }

Before and after this patch only Level 0 is displayed. Without
/proc/self/fd bind-mount protection it is possible to traverse parent
directories.

Fixes: 397ae982f4 ("virtiofsd: jail lo->proc_self_fd")
Cc: Miklos Szeredi <mszeredi@redhat.com>
Cc: Jens Freimann <jfreimann@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20201006095826.59813-1-stefanha@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Tested-by: Jens Freimann <jfreimann@redhat.com>
Reviewed-by: Jens Freimann <jfreimann@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-10-12 12:39:38 +01:00
Dr. David Alan Gilbert
ff3995e2f0 virtiofsd: Call qemu_init_exec_dir
Since fcb4f59c87 qemu_get_local_state_pathname relies on the
init_exec_dir, and virtiofsd asserts because we never set it.
Set it.

Reported-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20201002124015.44820-1-dgilbert@redhat.com>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-10-12 12:39:38 +01:00
Alex Bennée
f6698f2b03 tools/virtiofsd: add support for --socket-group
If you like running QEMU as a normal user (very common for TCG runs)
but you have to run virtiofsd as a root user you run into connection
problems. Adding support for an optional --socket-group allows the
users to keep using the command line.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

Message-Id: <20200925125147.26943-2-alex.bennee@linaro.org>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  dgilbert: Split long line
2020-10-12 12:39:38 +01:00
Dr. David Alan Gilbert
2acf4f8fdd virtiofsd: Silence gcc warning
Gcc worries fd might be used unset, in reality it's always set if
fi is set, and only used if fi is set so it's safe.  Initialise it to -1
just to keep gcc happy for now.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20200827153657.111098-2-dgilbert@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-10-12 12:39:38 +01:00
Peter Maydell
2387df497b QAPI patches patches for 2020-10-10
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAl+BgjISHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZT3zEP/iud3fGwcKlt+et2ZnKwsD2xpiT+L8CL
 r4qpKDDoqWFufNpCzLhoGNZjNJq8JlgOdjp/RufDFl+QN5aVJmogX/EKW3dan0dK
 qm67Mz7EItm3ssBUitHeipZgHTsLwtt7Bu2VWUcJ/z6UD3QJBNoJ4Bypv4stXtuG
 5OQ3Afwu1ctkmQDiRoG+pXtDVk53SGp8aRrouH4ORGasujS6vAz/gXvBbg+yuD3t
 7dXpOMtbJL3haY/P5fB3pMSwn7Kql09YPe2REcWyvz6zl+SvFcypd5j7Ag5RXj7U
 WK6vkgOgIO4YlTfrr9sHv3jQnYLT9OkcDxDiSi4Pql4EptQydyFfQMGsR3TI7yuM
 LOTLLnDTX/uxCPocVwGT9bSrrbCgUg4adc95r0RrncurJdtldtdJ6idtuB/xZQV2
 O4qTeyCZdbuMjN4GiaGmhrJy9ySWQkJTDjpeJWPrZYIAQyfUz4BBtNmfuGG+gsEK
 A5WIfPVmuI+FM3T/Ck51DXcEGgf+ezcFtzYzWDLsWGOB9m9QB4H0yAwwmqHqZBMP
 mezLuzs6A7T/38xdNGkwj4dXcUKiKz06C00Z9F6jdXS6LnonNh+PZ01Buqkhpg7F
 JQofKIsLcNLzQDyflUluBfC4wDw51BrCRs2ynHNbYLdOgBNNylCQLxxfjOAO5iDE
 LjUcXT5Dg0BU
 =/96J
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2020-10-10' into staging

QAPI patches patches for 2020-10-10

# gpg: Signature made Sat 10 Oct 2020 10:43:14 BST
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2020-10-10: (34 commits)
  qapi/visit.py: add type hint annotations
  qapi/visit.py: remove unused parameters from gen_visit_object
  qapi/visit.py: assert tag_member contains a QAPISchemaEnumType
  qapi/types.py: remove one-letter variables
  qapi/types.py: add type hint annotations
  qapi/gen.py: delint with pylint
  qapi/gen.py: update write() to be more idiomatic
  qapi/gen.py: Remove unused parameter
  qapi/gen.py: add type hint annotations
  qapi/gen: Make _is_user_module() return bool
  qapi/source.py: delint with pylint
  qapi/source.py: add type hint annotations
  qapi/commands.py: add type hint annotations
  qapi/commands.py: Don't re-bind to variable of different type
  qapi/events.py: Move comments into docstrings
  qapi/events.py: add type hint annotations
  qapi: establish mypy type-checking baseline
  qapi/common.py: move build_params into gen.py
  qapi/common.py: Convert comments into docstrings, and elaborate
  qapi/common.py: add type hint annotations
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-12 11:29:42 +01:00
Peter Maydell
48a340d9b2 Testing, gitdm and plugin fixes:
- fix acceptance regressions in MIPS and IDE
   - speed up cirrus msys2/mingw builds
   - add genisoimage to more docker images
   - slew of gitdb updates
   - fix some windows compile issues for plugins
   - add V=1 to cirrus output
   - disable rxsim in gitlab CI
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAl+AkCUACgkQ+9DbCVqe
 KkSYaQf/YnnAjygh01oUkxaXfCYuEojjYxzAPp5ZlKoejEVBh5j+Yqht5Xxsgm3V
 ODTp/VW9TKvniaaXnPcFj5msparB2fZ2qlCP9hrQQ4YiSHkoPqVjMBC1w8o/GQjq
 Divhj6ui6/C0N97KbjslgZqJXkYw9dcl1ATCo1+sNwmBzXhzOmcYqOeJCdlcAja2
 71R3R2HNSflDICk42NfGN47LcG0flhRAnDZR4FbE8xorVoqquukVVxNEJgOk5VPn
 KOSc7ieAbR+7n552r1LeVuR/QaIaqPplTIrqDVFTjhjwOwmObhwqVUlMlhtKZx5O
 tQ54PtSIyWdXY7Am2VUWace6SHGfOg==
 =ORpZ
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-various-091020-1' into staging

Testing, gitdm and plugin fixes:

  - fix acceptance regressions in MIPS and IDE
  - speed up cirrus msys2/mingw builds
  - add genisoimage to more docker images
  - slew of gitdb updates
  - fix some windows compile issues for plugins
  - add V=1 to cirrus output
  - disable rxsim in gitlab CI

# gpg: Signature made Fri 09 Oct 2020 17:30:29 BST
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-various-091020-1: (22 commits)
  tests/acceptance: disable machine_rx_gdbsim on GitLab
  cirrus: use V=1 when running tests on FreeBSD and macOS
  plugin: Fixes compiling errors on msys2/mingw
  plugins: Fixes a issue when dlsym failed, the handle not closed
  .mailmap: Fix more contributor entries
  contrib/gitdm: Add Yandex to the domain map
  contrib/gitdm: Add Yadro to the domain map
  contrib/gitdm: Add SUSE to the domain map
  contrib/gitdm: Add Nir Soffer to Red Hat domain
  contrib/gitdm: Add Qualcomm to the domain map
  contrib/gitdm: Add Nuvia to the domain map
  contrib/gitdm: Add Google to the domain map
  contrib/gitdm: Add ByteDance to the domain map
  contrib/gitdm: Add Baidu to the domain map
  contrib/gitdm: Add more individual contributors
  contrib/gitdm: Add more academic domains
  tests/docker: Add genisoimage to the docker file
  cirrus: msys2/mingw speed is up, add excluded target back
  cirrus: Fixing and speedup the msys2/mingw CI
  hw/ide: restore replay support of IDE
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-11 14:34:07 +01:00
John Snow
b4c0aa59af qapi/visit.py: add type hint annotations
Annotations do not change runtime behavior.
This commit *only* adds annotations.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201009161558.107041-37-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-10 11:37:49 +02:00
John Snow
2cc1eefb84 qapi/visit.py: remove unused parameters from gen_visit_object
And this fixes the pylint report for this file, so make sure we check
this in the future, too.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201009161558.107041-36-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-10 11:37:49 +02:00
John Snow
554df4f33f qapi/visit.py: assert tag_member contains a QAPISchemaEnumType
This is true by design, but not presently able to be expressed in the
type system. An assertion helps mypy understand our constraints.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201009161558.107041-35-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-10 11:37:49 +02:00
John Snow
dec44d3d65 qapi/types.py: remove one-letter variables
"John, if pylint told you to jump off a bridge, would you?"
Hey, if it looked like fun, I might.

Now that this file is clean, enable pylint checks on this file.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201009161558.107041-34-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-10 11:37:49 +02:00
John Snow
cd073c8fb0 qapi/types.py: add type hint annotations
Annotations do not change runtime behavior.
This commit *only* adds annotations.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201009161558.107041-33-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-10 11:37:49 +02:00
John Snow
9abddb5be4 qapi/gen.py: delint with pylint
'fp' and 'fd' are self-evident in context, add them to the list of OK
names.

_top and _bottom also need to stay standard methods because some users
override the method and need to use `self`. Tell pylint to shush.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201009161558.107041-32-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-10 11:37:49 +02:00
John Snow
cc6263c44b qapi/gen.py: update write() to be more idiomatic
Make the file handling here just a tiny bit more idiomatic.
(I realize this is heavily subjective.)

Use exist_ok=True for os.makedirs and remove the exception,
use fdopen() to wrap the file descriptor in a File-like object,
and use a context manager for managing the file pointer.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20201009161558.107041-31-jsnow@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-10 11:37:49 +02:00