QEMU With E2K User Support
Go to file
Kevin Wolf 9ec7a59b5a coroutine: Revert to constant batch size
Commit 4c41c69e changed the way the coroutine pool is sized because for
virtio-blk devices with a large queue size and heavy I/O, it was just
too small and caused coroutines to be deleted and reallocated soon
afterwards. The change made the size dynamic based on the number of
queues and the queue size of virtio-blk devices.

There are two important numbers here: Slightly simplified, when a
coroutine terminates, it is generally stored in the global release pool
up to a certain pool size, and if the pool is full, it is freed.
Conversely, when allocating a new coroutine, the coroutines in the
release pool are reused if the pool already has reached a certain
minimum size (the batch size), otherwise we allocate new coroutines.

The problem after commit 4c41c69e is that it not only increases the
maximum pool size (which is the intended effect), but also the batch
size for reusing coroutines (which is a bug). It means that in cases
with many devices and/or a large queue size (which defaults to the
number of vcpus for virtio-blk-pci), many thousand coroutines could be
sitting in the release pool without being reused.

This is not only a waste of memory and allocations, but it actually
makes the QEMU process likely to hit the vm.max_map_count limit on Linux
because each coroutine requires two mappings (its stack and the guard
page for the stack), causing it to abort() in qemu_alloc_stack() because
when the limit is hit, mprotect() starts to fail with ENOMEM.

In order to fix the problem, change the batch size back to 64 to avoid
uselessly accumulating coroutines in the release pool, but keep the
dynamic maximum pool size so that coroutines aren't freed too early
in heavy I/O scenarios.

Note that this fix doesn't strictly make it impossible to hit the limit,
but this would only happen if most of the coroutines are actually in use
at the same time, not just sitting in a pool. This is the same behaviour
as we already had before commit 4c41c69e. Fully preventing this would
require allowing qemu_coroutine_create() to return an error, but it
doesn't seem to be a scenario that people hit in practice.

Cc: qemu-stable@nongnu.org
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2079938
Fixes: 4c41c69e05
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20220510151020.105528-3-kwolf@redhat.com>
Tested-by: Hiroki Narukawa <hnarukaw@yahoo-corp.jp>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2022-05-12 12:21:30 +02:00
.github/workflows github: fix config mistake preventing repo lockdown commenting 2022-04-26 16:12:26 +01:00
.gitlab/issue_templates
.gitlab-ci.d gitlab-ci: Upgrade mingw base package. 2022-05-09 08:21:14 +02:00
accel Clean up header guards that don't match their file name 2022-05-11 16:49:06 +02:00
audio Remove qemu-common.h include from most units 2022-04-06 14:31:55 +02:00
authz
backends meson: use have_vhost_* variables to pick sources 2022-05-07 07:46:58 +02:00
block Clean up ill-advised or unusual header guards 2022-05-11 16:50:01 +02:00
bsd-user Clean up decorations and whitespace around header guards 2022-05-11 16:50:32 +02:00
capstone@f8b1b83301
chardev Clean up decorations and whitespace around header guards 2022-05-11 16:50:32 +02:00
common-user
configs hppa: use new CONFIG_HPPA_B160L option instead of CONFIG_DINO to build hppa machine 2022-05-08 18:52:36 +01:00
contrib util: rename qemu_*block() socket functions 2022-05-03 15:53:20 +04:00
crypto Clean up ill-advised or unusual header guards 2022-05-11 16:50:01 +02:00
disas disas: Remove old libopcode ppc disassembler 2022-05-09 08:21:05 +02:00
docs target-arm queue: 2022-05-09 09:33:53 -07:00
dtc@b6910bec11
dump dump/win_dump: add 32-bit guest Windows support 2022-04-22 13:41:56 +04:00
ebpf
fpu softfloat: Use FloatRelation for fracN_cmp 2022-04-26 20:01:55 -07:00
fsdev Remove qemu-common.h include from most units 2022-04-06 14:31:55 +02:00
gdb-xml
hw coroutine: Rename qemu_coroutine_inc/dec_pool_size() 2022-05-12 12:20:45 +02:00
include coroutine: Rename qemu_coroutine_inc/dec_pool_size() 2022-05-12 12:20:45 +02:00
io util: rename qemu_*block() socket functions 2022-05-03 15:53:20 +04:00
libdecnumber
linux-headers linux-headers: include missing changes from 5.17 2022-03-15 11:50:50 +01:00
linux-user Clean up decorations and whitespace around header guards 2022-05-11 16:50:32 +02:00
meson@12f9f04ba0
migration meson, configure: move RDMA options to meson 2022-04-28 08:52:20 +02:00
monitor vnc: add display-update monitor command. 2022-04-27 10:49:28 -07:00
nbd qapi: nbd-export: allow select bitmaps by node/name pair 2022-04-26 13:15:19 -05:00
net meson: use have_vhost_* variables to pick sources 2022-05-07 07:46:58 +02:00
pc-bios ppc/pnv: Update skiboot to v7.0 2022-04-20 17:58:35 -03:00
plugins Clean up header guards that don't match their file name 2022-05-11 16:49:06 +02:00
po
python python/qmp: remove pylint workaround from legacy.py 2022-04-21 11:01:00 -04:00
qapi Pull request 2022-05-09 11:07:04 -07:00
qga qga-vss: always build qga-vss.tlb when qga-vss.dll is built 2022-05-07 07:46:58 +02:00
qobject include/qapi: add g_autoptr support for qobject types 2022-04-06 10:50:38 +02:00
qom include: add qemu/keyval.h 2022-04-21 17:03:51 +04:00
replay Use g_new() & friends where that makes obvious sense 2022-03-21 15:44:44 +01:00
roms ppc/pnv: Update skiboot to v7.0 2022-04-20 17:58:35 -03:00
scripts doc: remove hxtool-conv.pl 2022-05-09 08:21:14 +02:00
scsi util: replace qemu_get_local_state_pathname() 2022-04-21 17:09:09 +04:00
semihosting semihosting: clean up handling of expanded argv 2022-03-23 10:38:09 +00:00
slirp@a88d9ace23
softmmu Warn user if the vga flag is passed but no vga device is created 2022-05-09 08:21:14 +02:00
storage-daemon include: rename qemu-common.h qemu/help-texts.h 2022-04-21 16:58:24 +04:00
stubs Move error_printf_unless_qmp() with monitor unit 2022-04-21 17:09:09 +04:00
subprojects/libvhost-user vhost-user: Don't pass file descriptor for VHOST_USER_REM_MEM_REG 2022-05-04 15:55:23 +02:00
target Clean up decorations and whitespace around header guards 2022-05-11 16:50:32 +02:00
tcg Normalize header guard symbol definition 2022-05-11 16:50:26 +02:00
tests Clean up ill-advised or unusual header guards 2022-05-11 16:50:01 +02:00
tools Clean up header guards that don't match their file name 2022-05-11 16:49:06 +02:00
trace error: use GLib to remember the program name 2022-03-22 14:46:18 +04:00
ui Clean up ill-advised or unusual header guards 2022-05-11 16:50:01 +02:00
util coroutine: Revert to constant batch size 2022-05-12 12:21:30 +02:00
.cirrus.yml cirrus/win32: upgrade mingw base packages 2022-05-09 08:21:14 +02:00
.dir-locals.el
.editorconfig
.exrc
.gdbinit
.gitattributes gitattributes: Cover Objective-C source files 2022-03-29 00:15:14 +02:00
.gitignore
.gitlab-ci.yml
.gitmodules
.gitpublish
.mailmap MAINTAINERS/.mailmap: update email for Leif Lindholm 2022-05-09 11:47:53 +01:00
.patchew.yml
.readthedocs.yml
.travis.yml gitlab: disable accelerated zlib for s390x 2022-03-22 17:07:30 +00:00
block.c block: Classify bdrv_get_flags() as I/O function 2022-05-04 15:55:23 +02:00
blockdev-nbd.c qapi: nbd-export: allow select bitmaps by node/name pair 2022-04-26 13:15:19 -05:00
blockdev.c Replace qemu_gettimeofday() with g_get_real_time() 2022-04-06 10:50:37 +02:00
blockjob.c
configure configure, meson: move vhost options to Meson 2022-05-07 07:46:59 +02:00
COPYING
COPYING.LIB
cpu.c util/log: Remove qemu_log_close 2022-04-20 10:51:11 -07:00
cpus-common.c Use g_new() & friends where that makes obvious sense 2022-03-21 15:44:44 +01:00
disas.c disas: Remove old libopcode ppc disassembler 2022-05-09 08:21:05 +02:00
event-loop-base.c util/event-loop-base: Introduce options to set the thread pool size 2022-05-09 10:43:23 +01:00
gdbstub.c whpx: Added support for breakpoints and stepping 2022-04-06 14:31:55 +02:00
gitdm.config
hmp-commands-info.hx mos6522: add "info via" HMP command for debugging 2022-03-09 09:28:28 +00:00
hmp-commands.hx vnc: add display-update monitor command. 2022-04-27 10:49:28 -07:00
iothread.c util/event-loop-base: Introduce options to set the thread pool size 2022-05-09 10:43:23 +01:00
job-qmp.c
job.c job.h: assertions in the callers of JobDriver function pointers 2022-03-04 18:18:26 +01:00
Kconfig
Kconfig.host meson: use have_vhost_* variables to pick sources 2022-05-07 07:46:58 +02:00
LICENSE
MAINTAINERS target-arm queue: 2022-05-09 09:33:53 -07:00
Makefile
memory_ldst.c.inc
meson_options.txt configure, meson: move vhost options to Meson 2022-05-07 07:46:59 +02:00
meson.build util/main-loop: Introduce the main loop into QOM 2022-05-09 10:43:23 +01:00
module-common.c
os-posix.c os-posix: replace pipe()+cloexec with g_unix_open_pipe(CLOEXEC) 2022-05-03 15:46:17 +04:00
os-win32.c Remove qemu-common.h include from most units 2022-04-06 14:31:55 +02:00
page-vary-common.c Remove qemu-common.h include from most units 2022-04-06 14:31:55 +02:00
page-vary.c include: move target page bits declaration to page-vary.h 2022-04-06 14:31:43 +02:00
qemu-bridge-helper.c
qemu-edid.c
qemu-img-cmds.hx
qemu-img.c qapi: rename BlockDirtyBitmapMergeSource to BlockDirtyBitmapOrStr 2022-04-26 13:13:50 -05:00
qemu-io-cmds.c nbd patches for 2022-03-07 2022-03-09 11:38:29 +00:00
qemu-io.c include: rename qemu-common.h qemu/help-texts.h 2022-04-21 16:58:24 +04:00
qemu-keymap.c
qemu-nbd.c Replace qemu_pipe() with g_unix_open_pipe() 2022-05-03 15:17:56 +04:00
qemu-options.hx qemu-options: Limit the -xen options to x86 and arm 2022-05-09 08:21:14 +02:00
qemu.nsi nsis installer: Fix mouse-over descriptions for emulators 2022-03-18 10:55:15 +00:00
qemu.sasl
README.rst
replication.c
trace-events
VERSION Open 7.1 development tree 2022-04-19 18:21:23 -07:00
version.rc

===========
QEMU README
===========

QEMU is a generic and open source machine & userspace emulator and
virtualizer.

QEMU is capable of emulating a complete machine in software without any
need for hardware virtualization support. By using dynamic translation,
it achieves very good performance. QEMU can also integrate with the Xen
and KVM hypervisors to provide emulated hardware while allowing the
hypervisor to manage the CPU. With hypervisor support, QEMU can achieve
near native performance for CPUs. When QEMU emulates CPUs directly it is
capable of running operating systems made for one machine (e.g. an ARMv7
board) on a different machine (e.g. an x86_64 PC board).

QEMU is also capable of providing userspace API virtualization for Linux
and BSD kernel interfaces. This allows binaries compiled against one
architecture ABI (e.g. the Linux PPC64 ABI) to be run on a host using a
different architecture ABI (e.g. the Linux x86_64 ABI). This does not
involve any hardware emulation, simply CPU and syscall emulation.

QEMU aims to fit into a variety of use cases. It can be invoked directly
by users wishing to have full control over its behaviour and settings.
It also aims to facilitate integration into higher level management
layers, by providing a stable command line interface and monitor API.
It is commonly invoked indirectly via the libvirt library when using
open source applications such as oVirt, OpenStack and virt-manager.

QEMU as a whole is released under the GNU General Public License,
version 2. For full licensing details, consult the LICENSE file.


Documentation
=============

Documentation can be found hosted online at
`<https://www.qemu.org/documentation/>`_. The documentation for the
current development version that is available at
`<https://www.qemu.org/docs/master/>`_ is generated from the ``docs/``
folder in the source tree, and is built by `Sphinx
<https://www.sphinx-doc.org/en/master/>_`.


Building
========

QEMU is multi-platform software intended to be buildable on all modern
Linux platforms, OS-X, Win32 (via the Mingw64 toolchain) and a variety
of other UNIX targets. The simple steps to build QEMU are:


.. code-block:: shell

  mkdir build
  cd build
  ../configure
  make

Additional information can also be found online via the QEMU website:

* `<https://wiki.qemu.org/Hosts/Linux>`_
* `<https://wiki.qemu.org/Hosts/Mac>`_
* `<https://wiki.qemu.org/Hosts/W32>`_


Submitting patches
==================

The QEMU source code is maintained under the GIT version control system.

.. code-block:: shell

   git clone https://gitlab.com/qemu-project/qemu.git

When submitting patches, one common approach is to use 'git
format-patch' and/or 'git send-email' to format & send the mail to the
qemu-devel@nongnu.org mailing list. All patches submitted must contain
a 'Signed-off-by' line from the author. Patches should follow the
guidelines set out in the `style section
<https://www.qemu.org/docs/master/devel/style.html>` of
the Developers Guide.

Additional information on submitting patches can be found online via
the QEMU website

* `<https://wiki.qemu.org/Contribute/SubmitAPatch>`_
* `<https://wiki.qemu.org/Contribute/TrivialPatches>`_

The QEMU website is also maintained under source control.

.. code-block:: shell

  git clone https://gitlab.com/qemu-project/qemu-web.git

* `<https://www.qemu.org/2017/02/04/the-new-qemu-website-is-up/>`_

A 'git-publish' utility was created to make above process less
cumbersome, and is highly recommended for making regular contributions,
or even just for sending consecutive patch series revisions. It also
requires a working 'git send-email' setup, and by default doesn't
automate everything, so you may want to go through the above steps
manually for once.

For installation instructions, please go to

*  `<https://github.com/stefanha/git-publish>`_

The workflow with 'git-publish' is:

.. code-block:: shell

  $ git checkout master -b my-feature
  $ # work on new commits, add your 'Signed-off-by' lines to each
  $ git publish

Your patch series will be sent and tagged as my-feature-v1 if you need to refer
back to it in the future.

Sending v2:

.. code-block:: shell

  $ git checkout my-feature # same topic branch
  $ # making changes to the commits (using 'git rebase', for example)
  $ git publish

Your patch series will be sent with 'v2' tag in the subject and the git tip
will be tagged as my-feature-v2.

Bug reporting
=============

The QEMU project uses GitLab issues to track bugs. Bugs
found when running code built from QEMU git or upstream released sources
should be reported via:

* `<https://gitlab.com/qemu-project/qemu/-/issues>`_

If using QEMU via an operating system vendor pre-built binary package, it
is preferable to report bugs to the vendor's own bug tracker first. If
the bug is also known to affect latest upstream code, it can also be
reported via GitLab.

For additional information on bug reporting consult:

* `<https://wiki.qemu.org/Contribute/ReportABug>`_


ChangeLog
=========

For version history and release notes, please visit
`<https://wiki.qemu.org/ChangeLog/>`_ or look at the git history for
more detailed information.


Contact
=======

The QEMU community can be contacted in a number of ways, with the two
main methods being email and IRC

* `<mailto:qemu-devel@nongnu.org>`_
* `<https://lists.nongnu.org/mailman/listinfo/qemu-devel>`_
* #qemu on irc.oftc.net

Information on additional methods of contacting the community can be
found online via the QEMU website:

* `<https://wiki.qemu.org/Contribute/StartHere>`_