Let's handle it similar to x86 ACPI PCI code and don't use a timer.
Instead, remember if an unplug request is pending and keep it pending
for eternity. (a follow up patch will process the request on
reboot).
We expect that a guest that is up and running, will process the unplug
request and trigger the unplug. This is normal operation, no timer needed.
If the guest does not react, this usually means something in the guest
is going wrong. Simply removing the device after 30 seconds does not
really sound like a good idea. It might sometimes be wanted, but I
consider this rather an "opt-in" decision as it might harm a guest not
prepared for it.
If we ever actually want a "forced/surprise removal", we will have to
implement something on top of the existing "device_del" framework. E.g.
also x86 might want to do a forced/surprise removal of PCI devices under
some conditions. "device_del X, forced=true" could be an option and will
require changes to the hotplug handler infrastructure.
This will then move the responsibility on when to do a forced removal
to a higher level. Doing a forced removal right now over-complicates
things and doesn't really seem to be required.
Let's allow to send multiple requests.
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20190130155733.32742-6-david@redhat.com>
Reviewed-by: Collin Walling <walling@linux.ibm.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
PCI on s390x is really weird and how it was modeled in QEMU might not have
been the right choice. Anyhow, right now it is the case that:
- Hotplugging a PCI device will silently create a zPCI device
(if none is provided)
- Hotunplugging a zPCI device will unplug the PCI device (if any)
- Hotunplugging a PCI device will unplug also the zPCI device
As far as I can see, we can no longer change this behavior. But we
should fix it.
Both device types are handled via a single hotplug handler call. This
is problematic for various reasons:
1. Unplugging via the zPCI device allows to unplug devices that are not
hot removable. (check performed in qdev_unplug()) - bad.
2. Hotplug handler chains are not possible for the unplug case. In the
future, the machine might want to override hotplug handlers, to
process device specific stuff and to then branch off to the actual
hotplug handler. We need separate hotplug handler calls for both the
PCI and zPCI device to make this work reliably. All other PCI
implementations are already prepared to handle this correctly, only
s390x is missing.
Therefore, introduce the unplug_request handler and properly perform
unplug checks by redirecting to the separate unplug_request handlers.
When finally unplugging, perform two separate hotplug_handler_unplug()
calls, first for the PCI device, followed by the zPCI device. This now
nicely splits unplugging paths for both devices.
The redirect part is a little hairy, as the user is allowed to trigger
unplug either via the PCI or the zPCI device. So redirect always to the
PCI unplug request handler first and remember if that check has been
performed in the zPCI device. Redirect then to the zPCI device unplug
request handler to perform the magic. Remembering that we already
checked the PCI device breaks the redirect loop.
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20190130155733.32742-5-david@redhat.com>
Reviewed-by: Collin Walling <walling@linux.ibm.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
Common function measurement block is used to report zPCI internal
counters of successful pcilg/stg/stb and rpcit instructions to
a memory location provided by the program.
This patch introduces a new ZpciFmb structure and schedules a timer
callback to copy the zPCI measures to the FMB in the guest memory
at an interval time set to 4s.
An error while attemping to update the FMB, would generate an error
event to the guest.
The pcilg/stg/stb and rpcit interception handlers increase the
related counter on a successful call.
The guest shall pass a null FMBA (FMB address) in the FIB (Function
Information Block) when it issues a Modify PCI Function Control
instruction to switch off FMB and stop the corresponding timer.
Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com>
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Message-Id: <1546969050-8884-2-git-send-email-pmorel@linux.ibm.com>
Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Collin Walling <walling@linux.ibm.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
I fail to see why this is useful as we require MSIX always and
completely fail adding a device.
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20181105110313.29312-2-david@redhat.com>
Fixes: 4f6482bfe3
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Collin Walling <walling@linux.ibm.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
The VFIO common code doesn't provide the possibility to modify a
previous mapping entry in another way than unmapping and mapping again
with new properties.
To avoid -EEXIST DMA mapping error, we introduce a GHashTable to store
S390IOTLBEntry instances in order to cache the mapped entries. When
intercepting rpcit instruction, ignore the identical mapped entries to
avoid doing map operations multiple times and do unmap and re-map
operations for the case of updating the valid entries.
Acked-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Message-Id: <20180205072258.5968-3-zyimin@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
Current s390x PCI IOMMU code is lack of flags' checking, including:
1) protection bit
2) table length
3) table offset
4) intermediate tables' invalid bit
5) format control bit
This patch introduces a new struct named S390IOTLBEntry, and makes up
these missed checkings. At the same time, inform the guest with the
corresponding error number when the check fails. Finally, in order to
get the error number, we export s390_guest_io_table_walk().
Reviewed-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Message-Id: <20180205072258.5968-2-zyimin@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
Enhance the fault detection.
Fixup the precedence to check the destination path existance
before checking for the source accessibility.
Add the maxstbl entry to both the Query PCI Function Group
response and the PCIBusDevice structure.
Initialize the maxstbl to 128 per default until we get
the actual data from the hardware.
Signed-off-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Reviewed-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Message-Id: <1512046530-17773-5-git-send-email-pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
PCIDevice pointer has been a parameter of kvm_arch_fixup_msi_route().
So we don't need to store zpci idx in msix message data to find out the
specific zpci device. Instead, we could use pci device id to find its
corresponding zpci device.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Message-Id: <1504606380-49341-2-git-send-email-zyimin@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
If we do not provide zpci, pci reconfiguration via sclp is not available
either. I/O adapter configuration, however, should always be present.
Rename the values that refer to I/O adapter configuration (instead of only
pci) to make things clearer.
Move length checking of the sccb for I/O adapter configuration into the
common sclp code (out of the pci code). This also fixes an issue that
the pci code would refer to a field in the sccb before checking whether
it was actually long enough.
Check for the adapter type in the sccb and return unrecognized adapter
type if the guest tries to issue I/O adapter configure/deconfigure for
a type other than pci or for pci if the zpci facility is not provided.
Reviewed-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Reviewed-by: Halil Pasic <pasic@linux.vnet.ibm.com>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
The nt2 event class is pci-only - don't look for events if pci is
not in the active cpu model.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
This finishes QOM'fication of IOMMUMemoryRegion by introducing
a IOMMUMemoryRegionClass. This also provides a fastpath analog for
IOMMU_MEMORY_REGION_GET_CLASS().
This makes IOMMUMemoryRegion an abstract class.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Message-Id: <20170711035620.4232-3-aik@ozlabs.ru>
Acked-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This defines new QOM object - IOMMUMemoryRegion - with MemoryRegion
as a parent.
This moves IOMMU-related fields from MR to IOMMU MR. However to avoid
dymanic QOM casting in fast path (address_space_translate, etc),
this adds an @is_iommu boolean flag to MR and provides new helper to
do simple cast to IOMMU MR - memory_region_get_iommu. The flag
is set in the instance init callback. This defines
memory_region_is_iommu as memory_region_get_iommu()!=NULL.
This switches MemoryRegion to IOMMUMemoryRegion in most places except
the ones where MemoryRegion may be an alias.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Message-Id: <20170711035620.4232-2-aik@ozlabs.ru>
Acked-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Let's use an enum for io adapter type, and standardize its naming to
CSS_IO_ADAPTER_* by changing S390_PCIPT_ADAPTER to CSS_IO_ADAPTER_PCI.
Signed-off-by: Fei Li <sherrylf@linux.vnet.ibm.com>
Reviewed-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
S390PCIBusDevice is typedef'ed earlier in the file, before the hunks
that this patch modifies. The double typedef causes old versions of
GCC to complain.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <1485523252-88288-1-git-send-email-pbonzini@redhat.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
The PCI bus number is usually set by the host during the enumeration.
In the s390 architecture we neither get a Device Tree nor have an
enumeration understanding bridge devices.
Let's fake the enumeration on reset and set the PCI_PRIMARY_BUS,
PCI_SECONDARY_BUS and PCI_SUBORDINATE_BUS config entries for the
bridges.
Let's add the configuration of these three config entries on bridge hot
plug.
The bus number is calculated based on a new entry, bus_num of the
S390pciState device.
This commit is inspired by what spapr pci does.
Signed-off-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
After PCI multibus is supported, more than 32 PCI devices could be
plugged. The current implementation of s390_pci_find_dev_by_fh()
appears low performance if there's a huge number of PCI devices
plugged. Therefore we introduce a hashtable using idx as key to store
zpci device's pointer on account of translating fh to idx very easily.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
A function may recursively call device search functions or may call
serveral different device search function. Passing the S390pciState to
search functions as an argument instead of looking up it inside the
search functions lowers the number of calling s390_get_phb().
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
In order to support a greater number of devices we use a QTAILQ
list of devices instead of a limited array.
This leads us to change:
- every lookup function s390_pci_find_xxx() for QTAILQ
- the FH_MASK_INDEX to index up to 65536 devices
Signed-off-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
When initializing a PCI device, an address space is required during PCI
core initialization and before the call to the embedding object hotplug
callback. To provide this AS, we allocate a S390PCIIOMMU object
containing this AS. Initialization of S390PCIIOMMU object is done
before the PCI device is completely created. So that we cannot
associate the IOMMU with the device at the moment. To track the IOMMU
object, we use g_hash functions with the PCI device's bus address as a
key to provide an array of pointers indexed by the PCI device's devfn
to the allocated IOMMU.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Currently S390PCIIOMMU is a normal struct. Let's make it inherit Object
in order to take advantage of QOM. In addition, we move some stuff
related to IOMMU from S390PCIBusDevice to S390PCIIOMMU.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Acked-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
For efficiency we now assign one msix io region for each pci device
and provide it with the pointer to the zPCI device as opaque
parameter. In addition, we remove msix address space and add msix io
region as a subregion to the root memory region of pci device.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Tracked down with an ugly, brittle and probably buggy Perl script.
Also move includes converted to <...> up so they get included before
ours where that's obviously okay.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard Henderson <rth@twiddle.net>
The current implementation of hot-unplug handler is abrupt. Any pci
operation will be just rejected if pci device is unconfigured. Thus a
pci device can not be reset or destroyed in a right, smooth and safe
way.
Improve this as follows:
- Notify the guest via a HP_EVENT_DECONFIGURE_REQUEST(0x303) event in
the unplug handler, giving it a chance to deconfigure the device via
sclp and allowing us to continue hot-unplug afterwards.
- Set up a timer that will generate the HP_EVENT_CONFIGURE_TO_STBRES
(0x304) event as before if the guest did not react after an adequate
time.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Because of the refactor of s390_pci_find_dev_by_idx(), list_pci()
should be updated. We introduce a new function to get the next
available zpci device. It simplifies the code of looking up zpci
devices.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Acked-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
The uid-checking facility guarantees uniqueness of the uid within the
vm and exposes the real uid to the guest when listing pci devices.
Let's always enable it and present it to the guest in the response to
the list pci clp command.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
To support definitions of s390 pci attributes in Qemu cmdline, we have
to make current S390PCIBusDevice struct inherit DeviceState and add
three properties for it. Currently we only support definitions of uid
and fid.
'uid' is optionally defined by users, identifies a zpci device and
must be defined with a 16-bit and non-zero unique value.
'fid' ranges from 0x0 to 0xFFFFFFFF. For fid property, we introduce a
new PropertyInfo by the name of s390_pci_fid_propinfo with our special
setter and getter. As 'fid' is optional, introduce 'fid_defined' to
track whether the user specified a fid.
'target' field is to direct qemu to find the corresponding generic PCI
device. It is equal to the 'id' value of one of generic pci devices.
If the user doesn't specify 'id' parameter for a generic pci device,
its 'id' value will be generated automatically and use this value as
'target' to create an associated zpci device.
If the user did not specify 'uid' or 'fid', values are generated
automatically. 'target' is required.
In addition, if a pci device has no associated zpci device, the code
will generate a zpci device automatically for it.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Currently each zpci device holds its own DMA address space and memory
region. At the same time, all instances of zpci device are stored in
S390pciState. So duirng the initialization of S390pciState, all zpci
devices are created and then all DMA address spaces are created. Thus,
when initializing pci devices, their corresponding DMA address spaces
could be found.
But zpci qdev will be introduced later. Zpci device may be initialized
and plugged afterwards generic pci device. So we should initialize all
DMA address spaces and memory regions before initializing zpci devices.
We introduce a new struct named S390PCIIOMMU. And a new field of
S390pciState, which is an array to store all instances of S390PCIIOMMU,
is added so that qemu pci code could find the corresponding DMA
address space when initializing a generic pci device. And this should
be done before the connection of a zpci device and a generic pci
device is built.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Acked-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
To enable S390PCIBusDevice as qdev, there should be a new bus to
plug and manage all instances of S390PCIBusDevice. Due to this,
S390PCIBus is introduced.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Current code uses some fields combinatorially to indicate the state of
a s390 pci device. This patch introduces device states in order to make
the code more readable and more logical.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Present code uses some macros to structure PCI Function Handle. But
their names don't have a uniform format. Let's use FH_MASK_ as the
unified prefix.
While we're at it, differentiate the SHM bits: use different bits for
vfio and emulated devices.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
We introduce iommu_enabled field for S390PCIBusDevice struct to
track whether the iommu has been enabled for the device. This allows
us to stop temporarily changing ->configured while en/disabling the
iommu and to do conditional cleanup later.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Split s390_pcihost_iommu_configure() into separate functions for
configuring and deconfiguring in order to make the code more readable.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Split s390_sclp_configure() into separate functions for sclp
configuring and deconfiguring in order to make the code more readable.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Indicator refcounting interfaces are introduced. This patch fixes
introducing unneeded indicator mappings and failure to release
AISB mappings on deregistration.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
According to the s390 architecture, any mpcifc, pcilg, pcistg,
pcistb and rpcit instructions issued to disabled PCI functions
are rejected, and the instruction completes by setting condition
code 3. In addition, any DMA and MSIX interruption operations
are ignored.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Present code uses @size==UINT64_MAX to initialize IOMMU. It infers that it
can map any 64-bit IOVA whatsoever. But in fact, the largest DMA range for
each PCI Device on s390x is from ZPCI_SDMA_ADDR to ZPCI_EDMA_ADDR. The largest
value is returned from hardware, which is to indicate the largest range
hardware can support. But the real IOMMU size for specific PCI Device is
obtained once qemu intercepts mpcifc instruction that guest is requesting a
DMA range for that PCI Device. Therefore, before intercepting mpcifc instruction,
qemu cannot be aware of the size of IOMMU region that guest will use.
Moreover, iommu replay during device initialization for the whole region in
4k steps takes a very long time.
In conclusion, this patch intializes IOMMU region for each PCI Device when
intercept mpcifc instruction which is to register DMA range for the PCI Device.
And then, destroy IOMMU region when guest wants to deregister IOAT.
Signed-off-by: Yi Min Zhao <zyimin@linux.vnet.ibm.com>
Reviewed-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
This patch implements a pci bus for s390x together with infrastructure
to generate and handle hotplug events, to configure/unconfigure via
sclp instruction, to do iommu translations and provide s390 support for
MSI/MSI-X notification processing.
Signed-off-by: Frank Blaschka <frank.blaschka@de.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
David Hildenbrand
Yi Min Zhao
Yi Min Zhao
Pierre Morel
Cornelia Huck
Alexey Kardashevskiy
Fei Li
Paolo Bonzini
Stefan Weil
Markus Armbruster
Frank Blaschka