Commit Graph

131 Commits

Author SHA1 Message Date
Yuval Shaia abc665aae3 hw/rdma: Print backend QP number in hex format
To be consistent with other prints throughout the code fix places that
print it as decimal number.

Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Message-Id: <20180805153518.2983-12-yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
2018-08-18 18:00:48 +03:00
Yuval Shaia 292dce627b hw/rdma: Cosmetic change - move to generic function
To ease maintenance of struct comp_thread move all related code to
dedicated function.

Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Message-Id: <20180805153518.2983-11-yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
2018-08-18 18:00:39 +03:00
Yuval Shaia 67b32fe250 hw/pvrdma: Cosmetic change - indent right
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Message-Id: <20180805153518.2983-10-yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
2018-08-18 18:00:30 +03:00
Yuval Shaia 7131c4b028 hw/rdma: Reorder resource cleanup
To be consistence with allocation do the reverse order in deallocation

Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Message-Id: <20180805153518.2983-9-yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
2018-08-18 18:00:22 +03:00
Yuval Shaia 4c408c9d5b hw/rdma: Do not allocate memory for non-dma MR
There is no use in the memory allocated for non-dma MR.
Delete the code that allocates it.

Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Message-Id: <20180805153518.2983-8-yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
2018-08-18 18:00:15 +03:00
Yuval Shaia 7f99daadbd hw/rdma: Delete useless structure RdmaRmUserMR
The structure RdmaRmUserMR has no benefits, remove it an move all its
fields to struct RdmaRmMR.

Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Message-Id: <20180805153518.2983-7-yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
2018-08-18 18:00:06 +03:00
Yuval Shaia 6e7dba23af hw/pvrdma: Make default pkey 0xFFFF
0x7FFF is not the default pkey - fix it.

Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Message-Id: <20180805153518.2983-6-yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
2018-08-18 17:59:58 +03:00
Yuval Shaia eca0f2a6be hw/pvrdma: Clean CQE before use
Next CQE is fetched from CQ ring, clean it before usage as it still
carries old CQE values.

Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Message-Id: <20180805153518.2983-5-yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
2018-08-18 17:59:50 +03:00
Yuval Shaia ef846e026c hw/rdma: Modify debug macros
- Add line counter to ease navigation in log
- Print rdma instead of pvrdma

Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Message-Id: <20180805153518.2983-4-yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
2018-08-18 17:59:42 +03:00
Yuval Shaia b462a13480 hw/pvrdma: Bugfix - provide the correct attr_mask to query_qp
Calling rdma_rm_query_qp with attr_mask equals to -1 leads to error
where backend query_qp fails to retrieve the needed QP attributes.
Fix it by providing the attr_mask we got from driver.

Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Message-Id: <20180805153518.2983-3-yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
2018-08-18 17:59:33 +03:00
Yuval Shaia 751522275a hw/rdma: Make distinction between device init and start modes
There are certain operations that are well considered as part of device
configuration while others are needed only when "start" command is
triggered by the guest driver. An example of device initialization step
is msix_init and example of "device start" stage is the creation of a CQ
completion handler thread.

Driver expects such distinction - implement it.

Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Message-Id: <20180805153518.2983-2-yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
2018-08-18 17:59:23 +03:00
Philippe Mathieu-Daudé 78109066e6 hw/rdma: Use the IEC binary prefix definitions
It eases code review, unit is explicit.

Patch generated using:

  $ git grep -E '(1024|2048|4096|8192|(<<|>>).?(10|20|30))' hw/ include/hw/

and modified manually.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com>
Message-Id: <20180625124238.25339-40-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-07-02 15:41:17 +02:00
Marcel Apfelbaum d18a530d85 hw/rdma: Fix possible out of bounds access to port GID index
Make sure the backend GID index is less then port's
gid table length.

Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com>
Message-Id: <20180430200223.4119-8-marcel.apfelbaum@gmail.com>
2018-05-03 20:52:29 +03:00
Yuval Shaia 6c080b9ea6 hw/rdma: Delete duplicate definition of MAX_RM_TBL_NAME
By a mistake this constant was defined twice - remove the duplication.

Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Message-Id: <20180430200223.4119-7-marcel.apfelbaum@gmail.com>
2018-05-03 20:52:29 +03:00
Yuval Shaia 350929172b hw/rdma: Fix possible out of bounds access to regs array
Coverity (CID1390589, CID1390608).
Array size is RDMA_BAR1_REGS_SIZE, let's make sure the given address is
in range.

While there also:
1. Adjust the size of this bar to reasonable size
2. Report the size of the array with sizeof(array)

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Message-Id: <20180430200223.4119-6-marcel.apfelbaum@gmail.com>
2018-05-03 20:52:29 +03:00
Yuval Shaia c387e8a4ec hw/rdma: Fix possible out of bounds access to GID table
Array size is MAX_PORT_GIDS, let's make sure the given index is in
range.

While there limit device table size to 1.

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Message-Id: <20180430200223.4119-5-marcel.apfelbaum@gmail.com>
2018-05-03 20:52:29 +03:00
Yuval Shaia b9e34872b9 hw/rdma: Delete port's pkey table
Support for PKEY is not yet implemented. Removing the unneeded table
until a support will be added.

Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Message-Id: <20180430200223.4119-4-marcel.apfelbaum@gmail.com>
2018-05-03 20:52:29 +03:00
Marcel Apfelbaum b0197cf80a hw/rdma: Fix possible usage of a NULL pointer
Coverity CID 1390586; The cq handle is provided by the guest
and cannot be trusted to be previuosly allocated.
Fix it by exiting the completion flow.

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com>
Message-Id: <20180430200223.4119-3-marcel.apfelbaum@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-05-03 20:52:29 +03:00
Marcel Apfelbaum 1bad4957c8 hw/rdma: Fix possible munmap call on a NULL pointer
Coverity CID 1390620: we call munmap() on a NULL pointer.

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com>
Message-Id: <20180430200223.4119-2-marcel.apfelbaum@gmail.com>
2018-05-03 20:52:29 +03:00
Yuval Shaia 6f559013c8 hw/rdma: Fix 32-bit compilation
Use the correct printf formats, so that a 32-bit compile doesn't spit
out lots of warnings about %lx being incompatible with uint64_t.

Suggested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180322095220.9976-4-yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2018-03-23 18:38:55 +03:00
Yuval Shaia 94f480b8db hw/rdma: Use correct print format in CHK_ATTR macro
Macro should not cast the given variable to u64 instead it should use
the supplied format argument (fmt).

Reported-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180322095220.9976-3-yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2018-03-23 18:38:55 +03:00
Yuval Shaia 9bbb8d3577 hw/rdma: Change host_virt to void *
To avoid compilation warnings on 32-bit machines:
rdma_backend.c: In function 'rdma_backend_create_mr':
rdma_backend.c:409:37: error: cast to pointer from integer of different
size [-Werror=int-to-pointer-cast]
	mr->ibmr = ibv_reg_mr(pd->ibpd, (void *)addr, length, access);

Reported-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180322095220.9976-2-yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2018-03-23 18:38:55 +03:00
Marcel Apfelbaum 197053e212 hw/rdma: fix clang compilation errors
Fix some enum castings and extra parentheses.

Reported-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
Message-Id: <20180321140316.96045-1-marcel@redhat.com>
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com>
2018-03-23 18:38:55 +03:00
Michael S. Tsirkin 0efc9511aa rdma: fix up include directives
Our rule right now is to use <> for external headers only.
RDMA code violates that, fix it up.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2018-03-23 18:38:55 +03:00
Yuval Shaia 79cfdca7aa hw/rdma: Add support for Query QP verb to pvrdma device
This IB verb is needed by some applications - implement it.

Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2018-03-23 18:38:55 +03:00
Yuval Shaia c99f217431 hw/rdma: Add Query QP operation
This operation is needed by rdma devices - implement it.

Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2018-03-23 18:38:55 +03:00
Yuval Shaia 919ae3dd11 hw/rdma: Implementation of PVRDMA device
PVRDMA is the QEMU implementation of VMware's paravirtualized RDMA device.
It works with its Linux Kernel driver AS IS, no need for any special
guest modifications.

While it complies with the VMware device, it can also communicate with
bare metal RDMA-enabled machines and does not require an RDMA HCA in the
host, it can work with Soft-RoCE (rxe).

It does not require the whole guest RAM to be pinned allowing memory
over-commit and, even if not implemented yet, migration support will be
possible with some HW assistance.

Implementation is divided into 2 components, rdma general and pvRDMA
specific functions and structures.

The second PVRDMA sub-module - interaction with PCI layer.
- Device configuration and setup (MSIX, BARs etc).
- Setup of DSR (Device Shared Resources)
- Setup of device ring.
- Device management.

Reviewed-by: Dotan Barak <dotanb@mellanox.com>
Reviewed-by: Zhu Yanjun <yanjun.zhu@oracle.com>
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2018-02-19 13:03:24 +02:00
Yuval Shaia 98d176f8e5 hw/rdma: PVRDMA commands and data-path ops
First PVRDMA sub-module - implementation of the PVRDMA device.
- PVRDMA commands such as create CQ and create MR.
- Data path QP operations - post_send and post_recv.
- Completion handler.

Reviewed-by: Dotan Barak <dotanb@mellanox.com>
Reviewed-by: Zhu Yanjun <yanjun.zhu@oracle.com>
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2018-02-19 13:03:24 +02:00
Yuval Shaia ef6d4ccdc9 hw/rdma: Implementation of generic rdma device layers
This layer is composed of two sub-modules, backend and resource manager.
Backend sub-module is responsible for all the interaction with IB layers
such as ibverbs and umad (external libraries).
Resource manager is a collection of functions and structures to manage
RDMA resources such as QPs, CQs and MRs.

Reviewed-by: Dotan Barak <dotanb@mellanox.com>
Reviewed-by: Zhu Yanjun <yanjun.zhu@oracle.com>
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2018-02-19 13:03:24 +02:00
Yuval Shaia b3a9227769 hw/rdma: Definitions for rdma device and rdma resource manager
Definition of various structures and constants used in backend and
resource manager modules.

Reviewed-by: Dotan Barak <dotanb@mellanox.com>
Reviewed-by: Zhu Yanjun <yanjun.zhu@oracle.com>
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2018-02-19 13:03:24 +02:00
Yuval Shaia dcbf469aef hw/rdma: Add wrappers and macros
As all mapping for this device are from driver to device,
declare wrappers on top of pci_dma_*map functions.

In addition, declare macros to be used for debug messages.

Reviewed-by: Dotan Barak <dotanb@mellanox.com>
Reviewed-by: Zhu Yanjun <yanjun.zhu@oracle.com>
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2018-02-19 13:03:24 +02:00