OpenE2K/qemu-e2k
13
4
Fork 0
Code Issues 4 Pull Requests Projects 2 Releases Activity
f80196b772
qemu-e2k/docs/system/devices/nvme.rst
Niklas Cassel 146b5fa505 hw/nvme: fix example serial in documentation 
The serial prop on the controller is actually describing the nvme
subsystem serial, which has to be identical for all controllers within
the same nvme subsystem.

This is enforced since commit a859eb9f8f ("hw/nvme: enforce common
serial per subsystem").

Fix the documentation, so that people copying the qemu command line
example won't get an error on qemu start.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
2022-07-15 10:40:33 +02:00

323 lines
11 KiB
ReStructuredText
Raw Blame History

==============
NVMe Emulation
==============
QEMU provides NVMe emulation through the ``nvme``, ``nvme-ns`` and
``nvme-subsys`` devices.
See the following sections for specific information on
* `Adding NVMe Devices`_, `additional namespaces`_ and `NVM subsystems`_.
* Configuration of `Optional Features`_ such as `Controller Memory Buffer`_,
`Simple Copy`_, `Zoned Namespaces`_, `metadata`_ and `End-to-End Data
Protection`_,
Adding NVMe Devices
===================
Controller Emulation
--------------------
The QEMU emulated NVMe controller implements version 1.4 of the NVM Express
specification. All mandatory features are implement with a couple of exceptions
and limitations:
* Accounting numbers in the SMART/Health log page are reset when the device
is power cycled.
* Interrupt Coalescing is not supported and is disabled by default.
The simplest way to attach an NVMe controller on the QEMU PCI bus is to add the
following parameters:
.. code-block:: console
-drive file=nvm.img,if=none,id=nvm
-device nvme,serial=deadbeef,drive=nvm
There are a number of optional general parameters for the ``nvme`` device. Some
are mentioned here, but see ``-device nvme,help`` to list all possible
parameters.
``max_ioqpairs=UINT32`` (default: ``64``)
Set the maximum number of allowed I/O queue pairs. This replaces the
deprecated ``num_queues`` parameter.
``msix_qsize=UINT16`` (default: ``65``)
The number of MSI-X vectors that the device should support.
``mdts=UINT8`` (default: ``7``)
Set the Maximum Data Transfer Size of the device.
``use-intel-id`` (default: ``off``)
Since QEMU 5.2, the device uses a QEMU allocated "Red Hat" PCI Device and
Vendor ID. Set this to ``on`` to revert to the unallocated Intel ID
previously used.
Additional Namespaces
---------------------
In the simplest possible invocation sketched above, the device only support a
single namespace with the namespace identifier ``1``. To support multiple
namespaces and additional features, the ``nvme-ns`` device must be used.
.. code-block:: console
-device nvme,id=nvme-ctrl-0,serial=deadbeef
-drive file=nvm-1.img,if=none,id=nvm-1
-device nvme-ns,drive=nvm-1
-drive file=nvm-2.img,if=none,id=nvm-2
-device nvme-ns,drive=nvm-2
The namespaces defined by the ``nvme-ns`` device will attach to the most
recently defined ``nvme-bus`` that is created by the ``nvme`` device. Namespace
identifiers are allocated automatically, starting from ``1``.
There are a number of parameters available:
``nsid`` (default: ``0``)
Explicitly set the namespace identifier.
``uuid`` (default: *autogenerated*)
Set the UUID of the namespace. This will be reported as a "Namespace UUID"
descriptor in the Namespace Identification Descriptor List.
``eui64``
Set the EUI-64 of the namespace. This will be reported as a "IEEE Extended
Unique Identifier" descriptor in the Namespace Identification Descriptor List.
Since machine type 6.1 a non-zero default value is used if the parameter
is not provided. For earlier machine types the field defaults to 0.
``bus``
If there are more ``nvme`` devices defined, this parameter may be used to
attach the namespace to a specific ``nvme`` device (identified by an ``id``
parameter on the controller device).
NVM Subsystems
--------------
Additional features becomes available if the controller device (``nvme``) is
linked to an NVM Subsystem device (``nvme-subsys``).
The NVM Subsystem emulation allows features such as shared namespaces and
multipath I/O.
.. code-block:: console
-device nvme-subsys,id=nvme-subsys-0,nqn=subsys0
-device nvme,serial=deadbeef,subsys=nvme-subsys-0
-device nvme,serial=deadbeef,subsys=nvme-subsys-0
This will create an NVM subsystem with two controllers. Having controllers
linked to an ``nvme-subsys`` device allows additional ``nvme-ns`` parameters:
``shared`` (default: ``on`` since 6.2)
Specifies that the namespace will be attached to all controllers in the
subsystem. If set to ``off``, the namespace will remain a private namespace
and may only be attached to a single controller at a time. Shared namespaces
are always automatically attached to all controllers (also when controllers
are hotplugged).
``detached`` (default: ``off``)
If set to ``on``, the namespace will be be available in the subsystem, but
not attached to any controllers initially. A shared namespace with this set
to ``on`` will never be automatically attached to controllers.
Thus, adding
.. code-block:: console
-drive file=nvm-1.img,if=none,id=nvm-1
-device nvme-ns,drive=nvm-1,nsid=1
-drive file=nvm-2.img,if=none,id=nvm-2
-device nvme-ns,drive=nvm-2,nsid=3,shared=off,detached=on
will cause NSID 1 will be a shared namespace that is initially attached to both
controllers. NSID 3 will be a private namespace due to ``shared=off`` and only
attachable to a single controller at a time. Additionally it will not be
attached to any controller initially (due to ``detached=on``) or to hotplugged
controllers.
Optional Features
=================
Controller Memory Buffer
------------------------
``nvme`` device parameters related to the Controller Memory Buffer support:
``cmb_size_mb=UINT32`` (default: ``0``)
This adds a Controller Memory Buffer of the given size at offset zero in BAR
2.
``legacy-cmb`` (default: ``off``)
By default, the device uses the "v1.4 scheme" for the Controller Memory
Buffer support (i.e, the CMB is initially disabled and must be explicitly
enabled by the host). Set this to ``on`` to behave as a v1.3 device wrt. the
CMB.
Simple Copy
-----------
The device includes support for TP 4065 ("Simple Copy Command"). A number of
additional ``nvme-ns`` device parameters may be used to control the Copy
command limits:
``mssrl=UINT16`` (default: ``128``)
Set the Maximum Single Source Range Length (``MSSRL``). This is the maximum
number of logical blocks that may be specified in each source range.
``mcl=UINT32`` (default: ``128``)
Set the Maximum Copy Length (``MCL``). This is the maximum number of logical
blocks that may be specified in a Copy command (the total for all source
ranges).
``msrc=UINT8`` (default: ``127``)
Set the Maximum Source Range Count (``MSRC``). This is the maximum number of
source ranges that may be used in a Copy command. This is a 0's based value.
Zoned Namespaces
----------------
A namespaces may be "Zoned" as defined by TP 4053 ("Zoned Namespaces"). Set
``zoned=on`` on an ``nvme-ns`` device to configure it as a zoned namespace.
The namespace may be configured with additional parameters
``zoned.zone_size=SIZE`` (default: ``128MiB``)
Define the zone size (``ZSZE``).
``zoned.zone_capacity=SIZE`` (default: ``0``)
Define the zone capacity (``ZCAP``). If left at the default (``0``), the zone
capacity will equal the zone size.
``zoned.descr_ext_size=UINT32`` (default: ``0``)
Set the Zone Descriptor Extension Size (``ZDES``). Must be a multiple of 64
bytes.
``zoned.cross_read=BOOL`` (default: ``off``)
Set to ``on`` to allow reads to cross zone boundaries.
``zoned.max_active=UINT32`` (default: ``0``)
Set the maximum number of active resources (``MAR``). The default (``0``)
allows all zones to be active.
``zoned.max_open=UINT32`` (default: ``0``)
Set the maximum number of open resources (``MOR``). The default (``0``)
allows all zones to be open. If ``zoned.max_active`` is specified, this value
must be less than or equal to that.
``zoned.zasl=UINT8`` (default: ``0``)
Set the maximum data transfer size for the Zone Append command. Like
``mdts``, the value is specified as a power of two (2^n) and is in units of
the minimum memory page size (CAP.MPSMIN). The default value (``0``)
has this property inherit the ``mdts`` value.
Metadata
--------
The virtual namespace device supports LBA metadata in the form separate
metadata (``MPTR``-based) and extended LBAs.
``ms=UINT16`` (default: ``0``)
Defines the number of metadata bytes per LBA.
``mset=UINT8`` (default: ``0``)
Set to ``1`` to enable extended LBAs.
End-to-End Data Protection
--------------------------
The virtual namespace device supports DIF- and DIX-based protection information
(depending on ``mset``).
``pi=UINT8`` (default: ``0``)
Enable protection information of the specified type (type ``1``, ``2`` or
``3``).
``pil=UINT8`` (default: ``0``)
Controls the location of the protection information within the metadata. Set
to ``1`` to transfer protection information as the first eight bytes of
metadata. Otherwise, the protection information is transferred as the last
eight bytes.
Virtualization Enhancements and SR-IOV (Experimental Support)
-------------------------------------------------------------
The ``nvme`` device supports Single Root I/O Virtualization and Sharing
along with Virtualization Enhancements. The controller has to be linked to
an NVM Subsystem device (``nvme-subsys``) for use with SR-IOV.
A number of parameters are present (**please note, that they may be
subject to change**):
``sriov_max_vfs`` (default: ``0``)
Indicates the maximum number of PCIe virtual functions supported
by the controller. Specifying a non-zero value enables reporting of both
SR-IOV and ARI (Alternative Routing-ID Interpretation) capabilities
by the NVMe device. Virtual function controllers will not report SR-IOV.
``sriov_vq_flexible``
Indicates the total number of flexible queue resources assignable to all
the secondary controllers. Implicitly sets the number of primary
controller's private resources to ``(max_ioqpairs - sriov_vq_flexible)``.
``sriov_vi_flexible``
Indicates the total number of flexible interrupt resources assignable to
all the secondary controllers. Implicitly sets the number of primary
controller's private resources to ``(msix_qsize - sriov_vi_flexible)``.
``sriov_max_vi_per_vf`` (default: ``0``)
Indicates the maximum number of virtual interrupt resources assignable
to a secondary controller. The default ``0`` resolves to
``(sriov_vi_flexible / sriov_max_vfs)``
``sriov_max_vq_per_vf`` (default: ``0``)
Indicates the maximum number of virtual queue resources assignable to
a secondary controller. The default ``0`` resolves to
``(sriov_vq_flexible / sriov_max_vfs)``
The simplest possible invocation enables the capability to set up one VF
controller and assign an admin queue, an IO queue, and a MSI-X interrupt.
.. code-block:: console
-device nvme-subsys,id=subsys0
-device nvme,serial=deadbeef,subsys=subsys0,sriov_max_vfs=1,
sriov_vq_flexible=2,sriov_vi_flexible=1
The minimum steps required to configure a functional NVMe secondary
controller are:
* unbind flexible resources from the primary controller
.. code-block:: console
nvme virt-mgmt /dev/nvme0 -c 0 -r 1 -a 1 -n 0
nvme virt-mgmt /dev/nvme0 -c 0 -r 0 -a 1 -n 0
* perform a Function Level Reset on the primary controller to actually
release the resources
.. code-block:: console
echo 1 > /sys/bus/pci/devices/0000:01:00.0/reset
* enable VF
.. code-block:: console
echo 1 > /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs
* assign the flexible resources to the VF and set it ONLINE
.. code-block:: console
nvme virt-mgmt /dev/nvme0 -c 1 -r 1 -a 8 -n 1
nvme virt-mgmt /dev/nvme0 -c 1 -r 0 -a 8 -n 2
nvme virt-mgmt /dev/nvme0 -c 1 -r 0 -a 9 -n 0
* bind the NVMe driver to the VF
.. code-block:: console
echo 0000:01:00.1 > /sys/bus/pci/drivers/nvme/bind
Reference in New Issue View Git Blame Copy Permalink