QEMU With E2K User Support
Go to file
David Hildenbrand b2b63008b3 libvhost-user: Use most of mmap_offset as fd_offset
In the past, QEMU would create memory regions that could partially cover
hugetlb pages, making mmap() fail if we would use the mmap_offset as an
fd_offset. For that reason, we never used the mmap_offset as an offset into
the fd and instead always mapped the fd from the very start.

However, that can easily result in us mmap'ing a lot of unnecessary
parts of an fd, possibly repeatedly.

QEMU nowadays does not create memory regions that partially cover huge
pages -- it never really worked with postcopy. QEMU handles merging of
regions that partially cover huge pages (due to holes in boot memory) since
2018 in c1ece84e7c ("vhost: Huge page align and merge").

Let's be a bit careful and not unconditionally convert the
mmap_offset into an fd_offset. Instead, let's simply detect the hugetlb
size and pass as much as we can as fd_offset, making sure that we call
mmap() with a properly aligned offset.

With QEMU and a virtio-mem device that is fully plugged (50GiB using 50
memslots) the qemu-storage daemon process consumes in the VA space
1281GiB before this change and 58GiB after this change.

================ Vhost user message ================
Request: VHOST_USER_ADD_MEM_REG (37)
Flags:   0x9
Size:    40
Fds: 59
Adding region 4
    guest_phys_addr: 0x0000000200000000
    memory_size:     0x0000000040000000
    userspace_addr:  0x00007fb73bffe000
    old mmap_offset: 0x0000000080000000
    fd_offset:       0x0000000080000000
    new mmap_offset: 0x0000000000000000
    mmap_addr:       0x00007f02f1bdc000
Successfully added new region
================ Vhost user message ================
Request: VHOST_USER_ADD_MEM_REG (37)
Flags:   0x9
Size:    40
Fds: 59
Adding region 5
    guest_phys_addr: 0x0000000240000000
    memory_size:     0x0000000040000000
    userspace_addr:  0x00007fb77bffe000
    old mmap_offset: 0x00000000c0000000
    fd_offset:       0x00000000c0000000
    new mmap_offset: 0x0000000000000000
    mmap_addr:       0x00007f0284000000
Successfully added new region

Reviewed-by: Raphael Norwitz <raphael@enfabrica.net>
Acked-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20240214151701.29906-12-david@redhat.com>
Tested-by: Mario Casquero <mcasquer@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2024-03-12 17:56:55 -04:00
.github/workflows
.gitlab/issue_templates
.gitlab-ci.d gitlab-ci: add manual job to run Coverity 2024-03-08 19:11:00 +01:00
accel hw/xen: Extract 'xen_igd.h' from 'xen_pt.h' 2024-03-09 18:51:45 +01:00
audio audio: Depend on dbus_display1_dep 2024-02-16 17:27:22 +04:00
authz
backends oslib-posix: initialize backend memory objects in parallel 2024-02-06 08:15:22 +01:00
block - LUKS support for detached headers 2024-02-12 14:13:58 +00:00
bsd-user gdbstub: Call gdbserver_fork() both in parent and in child 2024-03-06 12:35:19 +00:00
chardev char: Slightly better error reporting when chardev is in use 2024-03-09 18:56:37 +03:00
common-user
configs mips: do not list individual devices from configs/ 2024-03-08 15:51:22 +01:00
contrib contrib/plugins/howvec: migrate to new per_vcpu API 2024-03-06 12:35:44 +00:00
crypto crypto: Introduce 'detached-header' field in QCryptoBlockInfoLUKS 2024-02-09 12:50:37 +00:00
disas target/riscv: honour show_opcodes when disassembling 2024-03-06 12:35:51 +00:00
docs Misc HW patch queue 2024-03-09 20:12:21 +00:00
dump dump: Fix HMP dump-guest-memory -z without -R 2024-01-30 21:20:20 +03:00
ebpf
fpu
fsdev configure, meson: rename targetos to host_os 2023-12-31 09:11:29 +01:00
gdb-xml
gdbstub gdbstub: Implement follow-fork-mode child 2024-03-06 12:35:19 +00:00
host/include
hw vdpa: fix network breakage after cancelling migration 2024-03-12 17:56:55 -04:00
include vdpa: define SVQ transitioning state for mode switching 2024-03-12 17:56:55 -04:00
io io: fsync before closing a file channel 2024-03-01 15:42:04 +08:00
libdecnumber
linux-headers linux-headers: Update to Linux v6.8-rc6 2024-03-08 20:48:03 +10:00
linux-user linux-user/riscv: Add Ztso extension to hwprobe 2024-03-08 20:48:03 +10:00
migration QAPI patches patches for 2024-03-04 2024-03-05 11:20:15 +00:00
monitor monitor/target: Include missing 'exec/memory.h' header 2024-02-13 10:59:25 +03:00
nbd
net vdpa: indicate transitional state for SVQ switching 2024-03-12 17:56:55 -04:00
pc-bios pc-bios/README: Add information about hppa-firmware 2024-03-03 06:41:19 +01:00
plugins plugins: cleanup codepath for previous inline operation 2024-03-06 12:35:50 +00:00
po
python
qapi qdev: Add a granule_mode property 2024-03-09 19:17:01 +01:00
qga qga/qapi-schema: Fix guest-set-memory-blocks documentation 2024-03-04 07:12:40 +01:00
qobject
qom hw/acpi: move object_resolve_type_unambiguous to core QOM 2024-02-27 09:36:41 +01:00
replay replay: Improve error messages about configuration conflicts 2024-03-09 18:56:36 +03:00
roms roms/hppa: Add build rules for hppa-firmware 2024-03-03 06:41:19 +01:00
scripts trivial patches for 2024-03-09 2024-03-09 20:12:05 +00:00
scsi configure, meson: rename targetos to host_os 2023-12-31 09:11:29 +01:00
semihosting {linux,bsd}-user: Introduce get_task_state() 2024-03-06 12:35:19 +00:00
stats
storage-daemon meson: remove config_targetos 2023-12-31 09:11:28 +01:00
stubs hw/xen: Remove unnecessary xen_hvm_inject_msi() stub 2024-03-09 18:51:45 +01:00
subprojects libvhost-user: Use most of mmap_offset as fd_offset 2024-03-12 17:56:55 -04:00
system Misc HW patch queue 2024-03-09 20:12:21 +00:00
target Misc HW patch queue 2024-03-09 20:12:21 +00:00
tcg tcg/optimize: fix uninitialized variable 2024-02-29 11:36:05 -10:00
tests tests/unit/test-smp-parse: Test "parameter=0" SMP configurations 2024-03-09 19:17:01 +01:00
tools
trace
ui ui/cocoa: Remove stretch_video flag 2024-03-05 12:04:41 +01:00
util oslib-posix: fix memory leak in touch_all_pages 2024-03-08 15:51:22 +01:00
.dir-locals.el
.editorconfig
.exrc
.gdbinit
.git-blame-ignore-revs
.gitattributes
.gitignore
.gitlab-ci.yml
.gitmodules
.gitpublish
.mailmap mailmap: Fix Stefan Weil email 2024-01-30 21:20:20 +03:00
.patchew.yml
.readthedocs.yml readthodocs: fully specify a build environment 2024-01-12 13:23:48 +00:00
.travis.yml
block.c block: Support detached LUKS header creation using qemu-img 2024-02-09 12:50:37 +00:00
blockdev-nbd.c
blockdev.c blockdev: Fix block_resize error reporting for op blockers 2024-03-09 18:56:37 +03:00
blockjob.c
configure configure: put all symlink creation together 2024-02-16 13:56:09 +01:00
COPYING
COPYING.LIB
cpu-common.c system/cpus: rename qemu_mutex_lock_iothread() to bql_lock() 2024-01-08 10:45:43 -05:00
cpu-target.c cpu: Remove page_size_init 2024-02-29 11:35:37 -10:00
event-loop-base.c
gitdm.config
hmp-commands-info.hx hmp: Add option to info qtree to omit details 2024-03-09 19:17:01 +01:00
hmp-commands.hx hmp: Remove deprecated 'singlestep' command 2024-01-19 11:38:32 +01:00
iothread.c iothread: Simplify expression in qemu_in_iothread() 2024-02-13 10:59:25 +03:00
job-qmp.c
job.c
Kconfig
Kconfig.host
LICENSE
MAINTAINERS hw/gpio: Implement STM32L4x5 GPIO 2024-03-07 12:19:25 +00:00
Makefile Makefile: clean qemu-iotests output 2023-12-31 09:11:28 +01:00
meson_options.txt meson: fix type of "relocatable" option 2023-12-31 09:11:27 +01:00
meson.build meson: Remove --warn-common ldflag 2024-03-08 15:51:21 +01:00
module-common.c
os-posix.c qemu_init: increase NOFILE soft limit on POSIX 2024-02-09 12:47:58 +00:00
os-win32.c
page-vary-common.c
page-vary-target.c
pythondeps.toml
qemu-bridge-helper.c
qemu-edid.c
qemu-img-cmds.hx docs/devel/docs: Document .hx file syntax 2024-01-15 17:12:22 +00:00
qemu-img.c
qemu-io-cmds.c
qemu-io.c
qemu-keymap.c
qemu-nbd.c qemu-nbd: mention --tls-hostname option in qemu-nbd --help 2024-02-13 10:59:25 +03:00
qemu-options.hx qemu-options.hx: Don't claim "-serial" has limit of 4 serial ports 2024-03-05 13:22:56 +00:00
qemu.nsi
qemu.sasl
README.rst
replication.c
trace-events
VERSION
version.rc

===========
QEMU README
===========

QEMU is a generic and open source machine & userspace emulator and
virtualizer.

QEMU is capable of emulating a complete machine in software without any
need for hardware virtualization support. By using dynamic translation,
it achieves very good performance. QEMU can also integrate with the Xen
and KVM hypervisors to provide emulated hardware while allowing the
hypervisor to manage the CPU. With hypervisor support, QEMU can achieve
near native performance for CPUs. When QEMU emulates CPUs directly it is
capable of running operating systems made for one machine (e.g. an ARMv7
board) on a different machine (e.g. an x86_64 PC board).

QEMU is also capable of providing userspace API virtualization for Linux
and BSD kernel interfaces. This allows binaries compiled against one
architecture ABI (e.g. the Linux PPC64 ABI) to be run on a host using a
different architecture ABI (e.g. the Linux x86_64 ABI). This does not
involve any hardware emulation, simply CPU and syscall emulation.

QEMU aims to fit into a variety of use cases. It can be invoked directly
by users wishing to have full control over its behaviour and settings.
It also aims to facilitate integration into higher level management
layers, by providing a stable command line interface and monitor API.
It is commonly invoked indirectly via the libvirt library when using
open source applications such as oVirt, OpenStack and virt-manager.

QEMU as a whole is released under the GNU General Public License,
version 2. For full licensing details, consult the LICENSE file.


Documentation
=============

Documentation can be found hosted online at
`<https://www.qemu.org/documentation/>`_. The documentation for the
current development version that is available at
`<https://www.qemu.org/docs/master/>`_ is generated from the ``docs/``
folder in the source tree, and is built by `Sphinx
<https://www.sphinx-doc.org/en/master/>`_.


Building
========

QEMU is multi-platform software intended to be buildable on all modern
Linux platforms, OS-X, Win32 (via the Mingw64 toolchain) and a variety
of other UNIX targets. The simple steps to build QEMU are:


.. code-block:: shell

  mkdir build
  cd build
  ../configure
  make

Additional information can also be found online via the QEMU website:

* `<https://wiki.qemu.org/Hosts/Linux>`_
* `<https://wiki.qemu.org/Hosts/Mac>`_
* `<https://wiki.qemu.org/Hosts/W32>`_


Submitting patches
==================

The QEMU source code is maintained under the GIT version control system.

.. code-block:: shell

   git clone https://gitlab.com/qemu-project/qemu.git

When submitting patches, one common approach is to use 'git
format-patch' and/or 'git send-email' to format & send the mail to the
qemu-devel@nongnu.org mailing list. All patches submitted must contain
a 'Signed-off-by' line from the author. Patches should follow the
guidelines set out in the `style section
<https://www.qemu.org/docs/master/devel/style.html>`_ of
the Developers Guide.

Additional information on submitting patches can be found online via
the QEMU website

* `<https://wiki.qemu.org/Contribute/SubmitAPatch>`_
* `<https://wiki.qemu.org/Contribute/TrivialPatches>`_

The QEMU website is also maintained under source control.

.. code-block:: shell

  git clone https://gitlab.com/qemu-project/qemu-web.git

* `<https://www.qemu.org/2017/02/04/the-new-qemu-website-is-up/>`_

A 'git-publish' utility was created to make above process less
cumbersome, and is highly recommended for making regular contributions,
or even just for sending consecutive patch series revisions. It also
requires a working 'git send-email' setup, and by default doesn't
automate everything, so you may want to go through the above steps
manually for once.

For installation instructions, please go to

*  `<https://github.com/stefanha/git-publish>`_

The workflow with 'git-publish' is:

.. code-block:: shell

  $ git checkout master -b my-feature
  $ # work on new commits, add your 'Signed-off-by' lines to each
  $ git publish

Your patch series will be sent and tagged as my-feature-v1 if you need to refer
back to it in the future.

Sending v2:

.. code-block:: shell

  $ git checkout my-feature # same topic branch
  $ # making changes to the commits (using 'git rebase', for example)
  $ git publish

Your patch series will be sent with 'v2' tag in the subject and the git tip
will be tagged as my-feature-v2.

Bug reporting
=============

The QEMU project uses GitLab issues to track bugs. Bugs
found when running code built from QEMU git or upstream released sources
should be reported via:

* `<https://gitlab.com/qemu-project/qemu/-/issues>`_

If using QEMU via an operating system vendor pre-built binary package, it
is preferable to report bugs to the vendor's own bug tracker first. If
the bug is also known to affect latest upstream code, it can also be
reported via GitLab.

For additional information on bug reporting consult:

* `<https://wiki.qemu.org/Contribute/ReportABug>`_


ChangeLog
=========

For version history and release notes, please visit
`<https://wiki.qemu.org/ChangeLog/>`_ or look at the git history for
more detailed information.


Contact
=======

The QEMU community can be contacted in a number of ways, with the two
main methods being email and IRC

* `<mailto:qemu-devel@nongnu.org>`_
* `<https://lists.nongnu.org/mailman/listinfo/qemu-devel>`_
* #qemu on irc.oftc.net

Information on additional methods of contacting the community can be
found online via the QEMU website:

* `<https://wiki.qemu.org/Contribute/StartHere>`_