Commit Graph

1119 Commits

Author SHA1 Message Date
Vladimir Sementsov-Ogievskiy 0c88f1970c hbitmap: drop meta bitmaps as they are unused
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20200205112041.6003-5-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>
2020-03-18 14:03:46 -04:00
Vladimir Sementsov-Ogievskiy 30b8346cc3 hbitmap: unpublish hbitmap_iter_skip_words
Function is internal and even commented as internal. Drop its
definition from .h file.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20200205112041.6003-4-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>
2020-03-18 14:03:46 -04:00
Vladimir Sementsov-Ogievskiy be24c7140c hbitmap: move hbitmap_iter_next_word to hbitmap.c
The function is definitely internal (it's not used by third party and
it has complicated interface). Move it to .c file.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20200205112041.6003-3-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>
2020-03-18 14:03:46 -04:00
Stefan Hajnoczi ac90871cf8 lockable: add QemuRecMutex support
The polymorphic locking macros don't support QemuRecMutex yet.  Add it
so that lock guards can be used with QemuRecMutex.

Convert TCG plugins functions that benefit from these macros.  Manual
qemu_rec_mutex_lock/unlock() callers are left unmodified in cases where
clarity would not improve by switching to the macros.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-03-17 15:18:45 +01:00
Stefan Hajnoczi 3284c3ddc4 lockable: add lock guards
This patch introduces two lock guard macros that automatically unlock a
lock object (QemuMutex and others):

  void f(void) {
      QEMU_LOCK_GUARD(&mutex);
      if (!may_fail()) {
          return; /* automatically unlocks mutex */
      }
      ...
  }

and:

  WITH_QEMU_LOCK_GUARD(&mutex) {
      if (!may_fail()) {
          return; /* automatically unlocks mutex */
      }
  }
  /* automatically unlocks mutex here */
  ...

Convert qemu-timer.c functions that benefit from these macros as an
example.  Manual qemu_mutex_lock/unlock() callers are left unmodified in
cases where clarity would not improve by switching to the macros.

Many other QemuMutex users remain in the codebase that might benefit
from lock guards.  Over time they can be converted, if that is
desirable.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
[Use QEMU_MAKE_LOCKABLE_NONNULL. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-03-17 15:18:45 +01:00
Paolo Bonzini 8834dcf47e lockable: add QEMU_MAKE_LOCKABLE_NONNULL
This will be needed for lock guards, because if the lock is NULL the
dummy for loop of the lock guard never runs.  This can cause confusion
and dummy warnings in the compiler, but even if it did not, aborting
with a NULL pointer dereference is a less surprising behavior.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-03-17 15:18:34 +01:00
Robert Hoo 27f08ea1c7 util: add util function buffer_zero_avx512()
And intialize buffer_is_zero() with it, when Intel AVX512F is
available on host.

This function utilizes Intel AVX512 fundamental instructions which
is faster than its implementation with AVX2 (in my unit test, with
4K buffer, on CascadeLake SP, ~36% faster, buffer_zero_avx512() V.S.
buffer_zero_avx2()).

Signed-off-by: Robert Hoo <robert.hu@linux.intel.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-03-16 23:02:21 +01:00
Peter Maydell 5931ed5641 Block patches for the 5.0 softfreeze:
- qemu-img measure for LUKS
 - Improve block-copy's performance by reducing inter-request
   dependencies
 - Make curl's detection of accept-ranges more robust
 - Memleak fixes
 - iotest fix
 -----BEGIN PGP SIGNATURE-----
 
 iQFGBAABCAAwFiEEkb62CjDbPohX0Rgp9AfbAGHVz0AFAl5o5UUSHG1yZWl0ekBy
 ZWRoYXQuY29tAAoJEPQH2wBh1c9AnpMIALI1JGJ+kywe+BNTphOKtFGuIuHku/K4
 2GujC9WjX4to1xyozockXcP/AlFCk1yicbxW8hxPMxgtNmLh6E6Y2vRNT4/Fnpjc
 KeyFw062rtD/O6H8jchfX1wIaMBDBASM2GGUtonbfs9mdYmFnEPg/9jo0eX/b9Qp
 kEDFyVonJsAGFT7gWYB7p2RxToqC/JQBdenacKuYeKkLQOMvUHChERktnRhSfRt6
 bJIBkQ1e8xFuuFfnEUQR2GkvzzVw8k7kkY5gSN5m1kBCi3D1i/Pee5CFkYnQFiAD
 66GGKH5OIFt83t7msqu1zhb7kobZ1OH671nMSTdqTRncRnQfLWloCkk=
 =D7LF
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-03-11' into staging

Block patches for the 5.0 softfreeze:
- qemu-img measure for LUKS
- Improve block-copy's performance by reducing inter-request
  dependencies
- Make curl's detection of accept-ranges more robust
- Memleak fixes
- iotest fix

# gpg: Signature made Wed 11 Mar 2020 13:19:01 GMT
# gpg:                using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg:                issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-03-11:
  block/block-copy: hide structure definitions
  block/block-copy: reduce intersecting request lock
  block/block-copy: rename start to offset in interfaces
  block/block-copy: refactor interfaces to use bytes instead of end
  block/block-copy: factor out find_conflicting_inflight_req
  block/block-copy: use block_status
  block/block-copy: specialcase first copy_range request
  block/block-copy: fix progress calculation
  job: refactor progress to separate object
  block/qcow2-threads: fix qcow2_decompress
  qemu-img: free memory before re-assign
  block/qcow2: do free crypto_opts in qcow2_close()
  iotests: Fix nonportable use of od --endian
  block/curl: HTTP header field names are case insensitive
  block/curl: HTTP header fields allow whitespace around values
  iotests: add 288 luks qemu-img measure test
  qemu-img: allow qemu-img measure --object without a filename
  luks: implement .bdrv_measure()
  luks: extract qcrypto_block_calculate_payload_offset()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-03-11 17:06:40 +00:00
Vladimir Sementsov-Ogievskiy 01fe1ca945 job: refactor progress to separate object
We need it in separate to pass to the block-copy object in the next
commit.

Cc: qemu-stable@nongnu.org
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200311103004.7649-2-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-03-11 12:42:30 +01:00
Stefan Hajnoczi a31ca6801c qemu/queue.h: clear linked list pointers on remove
Do not leave stale linked list pointers around after removal.  It's
safer to set them to NULL so that use-after-removal results in an
immediate segfault.

The RCU queue removal macros are unchanged since nodes may still be
traversed after removal.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Link: https://lore.kernel.org/r/20200224103406.1894923-2-stefanha@redhat.com
Message-Id: <20200224103406.1894923-2-stefanha@redhat.com>
2020-03-09 16:39:20 +00:00
Zenghui Yu b59ea3640c compiler.h: Don't use compile-time assert when __NO_INLINE__ is defined
Our robot reported the following compile-time warning while compiling
Qemu with -fno-inline cflags:

In function 'load_memop',
    inlined from 'load_helper' at /qemu/accel/tcg/cputlb.c:1578:20,
    inlined from 'full_ldub_mmu' at /qemu/accel/tcg/cputlb.c:1624:12:
/qemu/accel/tcg/cputlb.c:1502:9: error: call to 'qemu_build_not_reached' declared with attribute error: code path is reachable
         qemu_build_not_reached();
         ^~~~~~~~~~~~~~~~~~~~~~~~
    [...]

It looks like a false-positive because only (MO_UB ^ MO_BSWAP) will
hit the default case in load_memop() while need_swap (size > 1) has
already ensured that MO_UB is not involved.

So the thing is that compilers get confused by the -fno-inline and
just can't accurately evaluate memop_size(op) at compile time, and
then the qemu_build_not_reached() is wrongly triggered by (MO_UB ^
MO_BSWAP).  Let's carefully don't use the compile-time assert when
no functions will be inlined into their callers.

Reported-by: Euler Robot <euler.robot@huawei.com>
Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Zenghui Yu <yuzenghui@huawei.com>
Message-Id: <20200205141545.180-1-yuzenghui@huawei.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2020-02-28 10:58:41 -08:00
Juan Quintela a2d07731e7 migration: Add support for modules
So we don't have to compile everything in, or have ifdefs

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-02-28 09:24:43 +01:00
Yoshinori Sato ed04c8b14c qemu/bitops.h: Add extract8 and extract16
Signed-off-by: Yoshinori Sato <ysato@users.sourceforge.jp>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200212130311.127515-3-ysato@users.sourceforge.jp>
Message-Id: <20200225124710.14152-14-alex.bennee@linaro.org>
2020-02-25 20:20:23 +00:00
Alexander Bulekov e785e50a5e fuzz: add FUZZ_TARGET module type
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-id: 20200220041118.23264-5-alxndr@bu.edu
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22 08:26:47 +00:00
Stefan Hajnoczi 4749079ce0 aio-posix: make AioHandler deletion O(1)
It is not necessary to scan all AioHandlers for deletion.  Keep a list
of deleted handlers instead of scanning the full list of all handlers.

The AioHandler->deleted field can be dropped.  Let's check if the
handler has been inserted into the deleted list instead.  Add a new
QLIST_IS_INSERTED() API for this check.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Sergio Lopez <slp@redhat.com>
Message-id: 20200214171712.541358-5-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22 08:26:47 +00:00
Stefan Hajnoczi 195ed8cb36 qemu/queue.h: add QLIST_SAFE_REMOVE()
QLIST_REMOVE() assumes the element is in a list.  It also leaves the
element's linked list pointers dangling.

Introduce a safe version of QLIST_REMOVE() and convert open-coded
instances of this pattern.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Sergio Lopez <slp@redhat.com>
Message-id: 20200214171712.541358-4-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22 08:26:47 +00:00
Paolo Bonzini 8c3570e339 rcu_queue: add QSLIST functions
QSLIST is the only family of lists for which we do not have RCU-friendly accessors,
add them.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200220103828.24525-1-pbonzini@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22 08:26:47 +00:00
Peter Maydell a8c6af67e1 ppc patch queue 2020-02-21
Here's the next patch of ppc target patches.  Highlights are:
   * Some fixes for CAS / unplug interactions
   * Remove some leaks of device trees
   * Some fixes for the PHB3 and PHB4 devices
   * Support for NVDIMMs on the pseries machine type
   * Assorted other fixes and cleanups
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEdfRlhq5hpmzETofcbDjKyiDZs5IFAl5PUAwACgkQbDjKyiDZ
 s5KBiBAAzUs/K1ygQuvcNTChOeu+3rBMe5XWRSRPIhqIZWnebssptwiMfqjVQXHB
 1unLmmkUU8ov9cXWiRiAF/mrmBX6Kw2UbnNzjatvB7L1BX8b3Z848vYKb0mTZG6g
 Q7E8nyqQqmWUtzeF6B0sNf597oPalHO5j6r5e/Pl5hjZGkjDXihB5tKePBRucnOO
 wCsWSsoafaBJwq2w8KBahfgLYdQdCW2G2GHcfGouobzravEuOaxpknoLJiUw65Wa
 Gf6PFrJE5XxDbn+zxkvs6RvPnNMkfFxn0fYbmPghjAM58Unz2LGyf9RRd5b6beuu
 fYsoZPxX+gOFVjBBwjTmj+VxSL1Fw+b2xOrd1uf7zVNyJNO8AsxRWQMD7bLFTGSZ
 95KkP5wSjcw2sm8eTtKkUAOP9YcTBJP1mjfjX/9rSGASCoOv6DJ71YOtyI4pzh5S
 DYLYm2EL3A0BxqxLodSx4xcTrh3IHAUXcteYr+ndGOy7MHzZGR88gtUmuAoFcgKc
 N0xV8dRoC5gNgLLxQ7zMh9q6qQHr0n13rRFKQ075piU4ce7JUzSR+EXCfQh2HLv+
 GYM+FpZo2zFsWZxXDQE1fsLaCfz533HsplevAM2J/bGWAokYMEZKV6jV7DKXgxW0
 6SrrSdxJrEThb198j1jSKCs0vwLDPcvV1GV6/sWStHBH1totRhY=
 =hrAW
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-5.0-20200221' into staging

ppc patch queue 2020-02-21

Here's the next patch of ppc target patches.  Highlights are:
  * Some fixes for CAS / unplug interactions
  * Remove some leaks of device trees
  * Some fixes for the PHB3 and PHB4 devices
  * Support for NVDIMMs on the pseries machine type
  * Assorted other fixes and cleanups

# gpg: Signature made Fri 21 Feb 2020 03:35:40 GMT
# gpg:                using RSA key 75F46586AE61A66CC44E87DC6C38CACA20D9B392
# gpg: Good signature from "David Gibson <david@gibson.dropbear.id.au>" [full]
# gpg:                 aka "David Gibson (Red Hat) <dgibson@redhat.com>" [full]
# gpg:                 aka "David Gibson (ozlabs.org) <dgibson@ozlabs.org>" [full]
# gpg:                 aka "David Gibson (kernel.org) <dwg@kernel.org>" [unknown]
# Primary key fingerprint: 75F4 6586 AE61 A66C C44E  87DC 6C38 CACA 20D9 B392

* remotes/dgibson/tags/ppc-for-5.0-20200221:
  hw/ppc/virtex_ml507:fix leak of fdevice tree blob
  spapr: Fix handling of unplugged devices during CAS and migration
  spapr: Don't use spapr_drc_needed() in CAS code
  ppc: free 'fdt' after reset the machine
  target/ppc/cpu.h: Clean up comments in the struct CPUPPCState definition
  target/ppc/cpu.h: Move fpu related members closer in cpu env
  target/ppc: Fix typo in comments
  spapr: Allow changing offset for -kernel image
  pnv/phb3: Add missing break statement
  pnv/phb4: Fix error path in pnv_pec_realize()
  pnv/phb3: Convert 1u to 1ull
  target/ppc/cpu.h: Remove duplicate includes
  spapr: Add Hcalls to support PAPR NVDIMM device
  spapr: Add NVDIMM device support
  nvdimm: add uuid property to nvdimm
  mem: move nvdimm_device_list to utilities
  ppc: function to setup latest class options
  ppc/pnv: Fix PCI_EXPRESS dependency
  qtest: Fix rtas dependencies
  spapr/rtas: Print message from "ibm,os-term"

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-21 14:20:42 +00:00
Shivaprasad G Bhat 3f350f6bb3 mem: move nvdimm_device_list to utilities
nvdimm_device_list is required for parsing the list for devices
in subsequent patches. Move it to common utility area.

Signed-off-by: Shivaprasad G Bhat <sbhat@linux.ibm.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Message-Id: <158131055857.2897.15658377276504711773.stgit@lep8c.aus.stglabs.ibm.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2020-02-21 09:15:03 +11:00
Josh Kunz 4b25a50674 linux-user: Use `qemu_log' for strace
This change switches linux-user strace logging to use the newer `qemu_log`
logging subsystem rather than the older `gemu_log` (notice the "g")
logger. `qemu_log` has several advantages, namely that it allows logging
to a file, and provides a more unified interface for configuration
of logging (via the QEMU_LOG environment variable or options).

This change introduces a new log mask: `LOG_STRACE` which is used for
logging of user-mode strace messages.

Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Josh Kunz <jkz@google.com>
Message-Id: <20200204025416.111409-3-jkz@google.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2020-02-19 11:17:40 +01:00
Peter Maydell a284f798f3 Remove support for CLOCK_MONOTONIC not being defined
Some older parts of QEMU's codebase assume that CLOCK_MONOTONIC
might not be defined by the host OS, and have workarounds to
deal with this. However, more recently (notably in commit
50290c002c for qemu-img in mid-2019, but also much
earlier in 2011 in commit 22795174a3 for ui/spice-display.c)
we've written code that assumes CLOCK_MONOTONIC is always
defined. The only host OS anybody's ever noticed this on
is OSX 10.11 and earlier, which we don't support.

So we can assume that all our host OSes have the #define,
and we can remove some now-unnecessary ifdefs.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200201172252.6605-1-peter.maydell@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-12 16:23:01 +01:00
Luc Michel e261b36810 seqlock: fix seqlock_write_unlock_impl function
The seqlock write unlock function was incorrectly calling
seqlock_write_begin() instead of seqlock_write_end(), and was releasing
the lock before incrementing the sequence. This could lead to a race
condition and a corrupted sequence number becoming odd even though the
lock is not held.

Signed-off-by: Luc Michel <luc.michel@greensocs.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200129144948.2161551-1-luc.michel@greensocs.com>
Fixes: 988fcafc73 ("seqlock: add QemuLockable support", 2018-08-23)
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-12 16:23:01 +01:00
Aravinda Prasad 19e067e032 Wrapper function to wait on condition for the main loop mutex
Introduce a wrapper function to wait on condition for
the main loop mutex. This function atomically releases
the main loop mutex and causes the calling thread to
block on the condition. This wrapper is required because
qemu_global_mutex is a static variable.

Signed-off-by: Aravinda Prasad <arawinda.p@gmail.com>
Signed-off-by: Ganesh Goudar <ganeshgr@linux.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Greg Kurz <groug@kaod.org>
Message-Id: <20200130184423.20519-2-ganeshgr@linux.ibm.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2020-02-03 11:33:10 +11:00
Eric Auger a085664f21 migration: Simplify get_qlist
Instead of inserting read elements at the head and
then reversing the list, it is simpler to add
each element after the previous one. Introduce
QLIST_RAW_INSERT_AFTER helper and use it in
get_qlist().

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Suggested-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2020-01-29 11:28:59 +01:00
Eric Auger 4746dbf8a9 migration: Support QLIST migration
Support QLIST migration using the same principle as QTAILQ:
94869d5c52 ("migration: migrate QTAILQ").

The VMSTATE_QLIST_V macro has the same proto as VMSTATE_QTAILQ_V.
The change mainly resides in QLIST RAW macros: QLIST_RAW_INSERT_HEAD
and QLIST_RAW_REVERSE.

Tests also are provided.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2020-01-20 09:10:23 +01:00
Marc-André Lureau 1e419ee68f chardev: generate an internal id when none given
Internally, qemu may create chardev without ID. Those will not be
looked up with qemu_chr_find(), which prevents using qdev_prop_set_chr().

Use id_generate(), to generate an internal name (prefixed with #), so
no conflict exist with user-named chardev.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: xiaoqiang zhao <zxq_yx_007@163.com>
2020-01-07 16:50:09 +04:00
Peter Maydell c4d1069c25 Add dbus-vmstate
Hi,
 
 With external processes or helpers participating to the VM support, it
 becomes necessary to handle their migration. Various options exist to
 transfer their state:
 1) as the VM memory, RAM or devices (we could say that's how
    vhost-user devices can be handled today, they are expected to
    restore from ring state)
 2) other "vmstate" (as with TPM emulator state blobs)
 3) left to be handled by management layer
 
 1) is not practical, since an external processes may legitimatelly
 need arbitrary state date to back a device or a service, or may not
 even have an associated device.
 
 2) needs ad-hoc code for each helper, but is simple and working
 
 3) is complicated for management layer, QEMU has the migration timing
 
 The proposed "dbus-vmstate" object will connect to a given D-Bus
 address, and save/load from org.qemu.VMState1 owners on migration.
 
 Thus helpers can easily have their state migrated with QEMU, without
 implementing ad-hoc support (such as done for TPM emulation)
 
 D-Bus is ubiquitous on Linux (it is systemd IPC), and can be made to
 work on various other OSes. There are several implementations and good
 bindings for various languages.  (the tests/dbus-vmstate-test.c is a
 good example of how simple the implementation of services can be, even
 in C)
 
 dbus-vmstate is put into use by the libvirt series "[PATCH 00/23] Use
 a slirp helper process".
 
 v2:
  - fix build with broken mingw-glib
 -----BEGIN PGP SIGNATURE-----
 
 iQJQBAABCAA6FiEEh6m9kz+HxgbSdvYt2ujhCXWWnOUFAl4TR5ccHG1hcmNhbmRy
 ZS5sdXJlYXVAcmVkaGF0LmNvbQAKCRDa6OEJdZac5R6EEACFTd4hDG8i/GnxCFut
 MGcTusJr+2IklIT/K0qpLf0axNUoIqycwv8m0T9QhoG8h+9lMykOd1YJpNetT5qK
 gifOF2gcPK/9WIdFbX7dLSUAWpzO6fG/RzKK65Nc1uJSnXlb8JV0BU/6FrfCE+3U
 Bg5PvVtxxtwejQfQPOI7bPxOqxr/SmjUGcbFgacMAMG0Lm/VG/92kdoC6Z4Xf/bd
 FcAeiO2CiPoGXG5zD4WF1emwxnSu65PgcFpSpqvvFlmDbYlTwoMt4VWxTfkAzbAM
 IES7j2IbhUEe3p0hvMTqmmsmds1QNCBgnQI/LtQiXPTnbfpBcZ0wT6QsSZXWvHz8
 ClA9OAimxyELblTGjD9vsi3G5m2DQS+NdfPOX7hfHouVQzDJJaS8jxDItpPgXwSO
 fZ9mUO8ps3N2YTakuKNBP/IzDOuyExrBg80GF+HbEc59Uhj8Yq/awyz1XsqjQzVP
 54+TUjwC8HZxVWgMeqiJ1njPTfRJo6uAnguLbfAXj8P9vaXLtsy/3JGsmKiziXXW
 XzvQDzhfOMjm7Uo7vN7Hp3X/UYJxnaQ3dViqZnv/gqG6yv+igVlqyrTx2IBhN2NW
 DZt3c7VqVUBYFShLgfy0zDjzM/s7mFkQKCFHUsBqIwODugYEc3TTdAa60QYjX5i9
 negngax45KM6nF3tq74fJpwWVw==
 =M4kD
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/elmarco/tags/dbus-vmstate7-pull-request' into staging

Add dbus-vmstate

Hi,

With external processes or helpers participating to the VM support, it
becomes necessary to handle their migration. Various options exist to
transfer their state:
1) as the VM memory, RAM or devices (we could say that's how
   vhost-user devices can be handled today, they are expected to
   restore from ring state)
2) other "vmstate" (as with TPM emulator state blobs)
3) left to be handled by management layer

1) is not practical, since an external processes may legitimatelly
need arbitrary state date to back a device or a service, or may not
even have an associated device.

2) needs ad-hoc code for each helper, but is simple and working

3) is complicated for management layer, QEMU has the migration timing

The proposed "dbus-vmstate" object will connect to a given D-Bus
address, and save/load from org.qemu.VMState1 owners on migration.

Thus helpers can easily have their state migrated with QEMU, without
implementing ad-hoc support (such as done for TPM emulation)

D-Bus is ubiquitous on Linux (it is systemd IPC), and can be made to
work on various other OSes. There are several implementations and good
bindings for various languages.  (the tests/dbus-vmstate-test.c is a
good example of how simple the implementation of services can be, even
in C)

dbus-vmstate is put into use by the libvirt series "[PATCH 00/23] Use
a slirp helper process".

v2:
 - fix build with broken mingw-glib

# gpg: Signature made Mon 06 Jan 2020 14:43:35 GMT
# gpg:                using RSA key 87A9BD933F87C606D276F62DDAE8E10975969CE5
# gpg:                issuer "marcandre.lureau@redhat.com"
# gpg: Good signature from "Marc-André Lureau <marcandre.lureau@redhat.com>" [full]
# gpg:                 aka "Marc-André Lureau <marcandre.lureau@gmail.com>" [full]
# Primary key fingerprint: 87A9 BD93 3F87 C606 D276  F62D DAE8 E109 7596 9CE5

* remotes/elmarco/tags/dbus-vmstate7-pull-request:
  tests: add dbus-vmstate-test
  tests: add migration-helpers unit
  dockerfiles: add dbus-daemon to some of latest distributions
  configure: add GDBUS_CODEGEN
  Add dbus-vmstate object
  util: add dbus helper unit
  docs: start a document to describe D-Bus usage
  vmstate: replace DeviceState with VMStateIf
  vmstate: add qom interface to get id

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-01-06 18:22:42 +00:00
Marc-André Lureau a5021d6991 util: add dbus helper unit
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-06 18:41:32 +04:00
Marc-André Lureau ee13240e60 osdep: add qemu_unlink()
Add a helper function to match qemu_open() which may return files
under the /dev/fdset prefix. Those shouldn't be removed, since it's
only a qemu namespace.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-02 16:29:32 +04:00
Peter Maydell f59b31e6d0 Various testing and logging updates
- test tci with Travis
   - enable multiarch testing in Travis
   - default to out-of-tree builds
   - make changing logfile safe via RCU
   - remove redundant tests
   - remove gtester test from docker
   - convert DEBUG_MMAP to tracepoints
   - remove hand rolled glob function
   - trigger tcg re-configure when needed
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAl37M6gACgkQ+9DbCVqe
 KkQ0Rwf/d0lGDPQN1Uf0zqZRQQmDCqWVuqHhZJ5xWDjbyVT2eUwR07TvNZeUKEWX
 iO+u6S7Tv91oLjZN5WjhaiuSjtJaEzCdcpkIJAWXLP/lzse37HEwvLBsdg71y+46
 LNvBrJRPpQotdb7fjr8RgCwc1qg2Bz15ekSn7XIA175zTMmUsshLJBVhLbGNqrVm
 F2UmjB9oFJ0+nzrcEnpFmWw7xvVrX1dImZXv5C2pvuHF7efSjGwiFviTRZgDjOGs
 V7HiWRV1QcgTigncncxTMbhMTKTVKK+e7O+y0DZWt/NSrT/yLDy5rcwySpmvu6C+
 cRmh/0tMo1KAhiz8Xy8LookhVj6hdA==
 =OAkV
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-tesing-and-misc-191219-1' into staging

Various testing and logging updates

  - test tci with Travis
  - enable multiarch testing in Travis
  - default to out-of-tree builds
  - make changing logfile safe via RCU
  - remove redundant tests
  - remove gtester test from docker
  - convert DEBUG_MMAP to tracepoints
  - remove hand rolled glob function
  - trigger tcg re-configure when needed

# gpg: Signature made Thu 19 Dec 2019 08:24:08 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-tesing-and-misc-191219-1: (25 commits)
  tests/tcg: ensure we re-configure if configure.sh is updated
  trace: replace hand-crafted pattern_glob with g_pattern_match_simple
  linux-user: convert target_munmap debug to a tracepoint
  linux-user: log page table changes under -d page
  linux-user: add target_mmap_complete tracepoint
  linux-user: convert target_mmap debug to tracepoint
  linux-user: convert target_mprotect debug to tracepoint
  travis.yml: Remove the redundant clang-with-MAIN_SOFTMMU_TARGETS entry
  docker: gtester is no longer used
  Added tests for close and change of logfile.
  Add use of RCU for qemu_logfile.
  qemu_log_lock/unlock now preserves the qemu_logfile handle.
  Add a mutex to guarantee single writer to qemu_logfile handle.
  Cleaned up flow of code in qemu_set_log(), to simplify and clarify.
  Fix double free issue in qemu_set_log_filename().
  ci: build out-of-tree
  travis.yml: Enable builds on arm64, ppc64le and s390x
  tests/test-util-filemonitor: Skip test on non-x86 Travis containers
  tests/hd-geo-test: Skip test when images can not be created
  iotests: Skip test 079 if it is not possible to create large files
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-12-20 14:00:49 +00:00
Peter Maydell 4800819827 * More uses of RCU_READ_LOCK_GUARD (Dave, myself)
* QOM doc improvments (Greg)
 * Cleanups from the Meson conversion (Marc-André)
 * Support for multiple -accel options (myself)
 * Many x86 machine cleanup (Philippe, myself)
 * tests/migration-test cleanup (Juan)
 * PC machine removal and next round of deprecation (Thomas)
 * kernel-doc integration (Peter, myself)
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQEcBAABAgAGBQJd+YJGAAoJEL/70l94x66D0YYIAIZpS6i6NYJC8KHCl49fjI7U
 qHDN7MiKYTU+l3i0+iGmQL6XN5ClAY0pXkY5LBFIDpsohHR5f4jdrIKjyvcHzuIM
 gx/NLsiA45/niHYrn/hEo0P7CwGTrrdWL+SVmScnKcwYiBzMO/uYblxlbUBKLPNn
 eGaKQmEkvlUBR9GS6S1+jYg8234ZRZ4+12t5dqqADBQ7Kc0wn6KC5yebIoQxCgVc
 9F5Ezdkl7befrTI7El3EC6aT18bKhIBZIs1PT/hzqzlGFhBuKM7uKDb43Yx8c7XQ
 bk5vzHmblPAgQyK4OETQ+DM745AOk6vBiJZbR9nrDUXWvUkrEXTQZMJKU0FXdlE=
 =hyYX
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* More uses of RCU_READ_LOCK_GUARD (Dave, myself)
* QOM doc improvments (Greg)
* Cleanups from the Meson conversion (Marc-André)
* Support for multiple -accel options (myself)
* Many x86 machine cleanup (Philippe, myself)
* tests/migration-test cleanup (Juan)
* PC machine removal and next round of deprecation (Thomas)
* kernel-doc integration (Peter, myself)

# gpg: Signature made Wed 18 Dec 2019 01:35:02 GMT
# gpg:                using RSA key BFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream: (87 commits)
  vga: cleanup mapping of VRAM for non-PCI VGA
  hw/display: Remove "rombar" hack from vga-pci and vmware_vga
  hw/pci: Remove the "command_serr_enable" property
  hw/audio: Remove the "use_broken_id" hack from the AC97 device
  hw/i386: Remove the deprecated machines 0.12 up to 0.15
  hw/pci-host: Add Kconfig entry to select the IGD Passthrough Host Bridge
  hw/pci-host/i440fx: Extract the IGD passthrough host bridge device
  hw/pci-host/i440fx: Use definitions instead of magic values
  hw/pci-host/i440fx: Use size_t to iterate over ARRAY_SIZE()
  hw/pci-host/i440fx: Extract PCII440FXState to "hw/pci-host/i440fx.h"
  hw/pci-host/i440fx: Correct the header description
  Fix some comment spelling errors.
  target/i386: remove unused pci-assign codes
  WHPX: refactor load library
  migration: check length directly to make sure the range is aligned
  memory: include MemoryListener documentation and some missing function parameters
  docs: add memory API reference
  memory.h: Silence kernel-doc complaints
  docs: Create bitops.rst as example of kernel-docs
  bitops.h: Silence kernel-doc complaints
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-12-20 11:20:25 +00:00
Robert Foley 7606488c0e Add use of RCU for qemu_logfile.
This now allows changing the logfile while logging is active,
and also solves the issue of a seg fault while changing the logfile.

Any read access to the qemu_logfile handle will use
the rcu_read_lock()/unlock() around the use of the handle.
To fetch the handle we will use atomic_rcu_read().
We also in many cases do a check for validity of the
logfile handle before using it to deal with the case where the
file is closed and set to NULL.

The cases where we write to the qemu_logfile will use atomic_rcu_set().
Writers will also use call_rcu() with a newly added qemu_logfile_free
function for freeing/closing when readers have finished.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20191118211528.3221-6-robert.foley@linaro.org>
2019-12-18 20:18:02 +00:00
Robert Foley fc59d2d870 qemu_log_lock/unlock now preserves the qemu_logfile handle.
qemu_log_lock() now returns a handle and qemu_log_unlock() receives a
handle to unlock.  This allows for changing the handle during logging
and ensures the lock() and unlock() are for the same file.

Also in target/tilegx/translate.c removed the qemu_log_lock()/unlock()
calls (and the log("\n")), since the translator can longjmp out of the
loop if it attempts to translate an instruction in an inaccessible page.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20191118211528.3221-5-robert.foley@linaro.org>
2019-12-18 20:18:02 +00:00
Markus Armbruster deda497b84 error: Fix -msg timestamp default
-msg parameter "timestamp" defaults to "off" if you don't specify msg,
and to "on" if you do.  Messed up right in commit 5e2ac51917 "add
timestamp to error_report()".  Mostly harmless, because "timestamp" is
the only parameter, so "if you do" is "-msg ''", which nobody does.

Change the default to "off" no matter what.

While there, rename enable_timestamp_msg to error_with_timestamp, and
polish documentation.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191010081508.8978-1-armbru@redhat.com>
2019-12-18 08:36:15 +01:00
Peter Maydell 7d41d76471 bitops.h: Silence kernel-doc complaints
Fix the problems with kernel-doc/sphinx syntax in the
doc comments for the shuffle and unshuffle functions:
 * mismatch between comment and prototype for argument name
 * the inline bit patterns need to be marked up so they
   are processed properly and rendered as monospace

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20190521122519.12573-6-peter.maydell@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-12-17 19:36:57 +01:00
Beata Michalska 61c490e25e Memory: Enable writeback for given memory region
Add an option to trigger memory writeback to sync given memory region
with the corresponding backing store, case one is available.
This extends the support for persistent memory, allowing syncing on-demand.

Signed-off-by: Beata Michalska <beata.michalska@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20191121000843.24844-3-beata.michalska@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-12-16 10:46:35 +00:00
Alex Bennée 3fb356cc86 tcg plugins: expose an API version concept
This is a very simple versioning API which allows the plugin
infrastructure to check the API a plugin was built against. We also
expose a min/cur API version to the plugin via the info block in case
it wants to avoid using old deprecated APIs in the future.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Robert Foley <robert.foley@linaro.org>
2019-11-12 14:32:55 +00:00
Andrew Jones 0df9142d27 target/arm/cpu64: max cpu: Introduce sve<N> properties
Introduce cpu properties to give fine control over SVE vector lengths.
We introduce a property for each valid length up to the current
maximum supported, which is 2048-bits. The properties are named, e.g.
sve128, sve256, sve384, sve512, ..., where the number is the number of
bits. See the updates to docs/arm-cpu-features.rst for a description
of the semantics and for example uses.

Note, as sve-max-vq is still present and we'd like to be able to
support qmp_query_cpu_model_expansion with guests launched with e.g.
-cpu max,sve-max-vq=8 on their command lines, then we do allow
sve-max-vq and sve<N> properties to be provided at the same time, but
this is not recommended, and is why sve-max-vq is not mentioned in the
document.  If sve-max-vq is provided then it enables all lengths smaller
than and including the max and disables all lengths larger. It also has
the side-effect that no larger lengths may be enabled and that the max
itself cannot be disabled. Smaller non-power-of-two lengths may,
however, be disabled, e.g. -cpu max,sve-max-vq=4,sve384=off provides a
guest the vector lengths 128, 256, and 512 bits.

This patch has been co-authored with Richard Henderson, who reworked
the target/arm/cpu64.c changes in order to push all the validation and
auto-enabling/disabling steps into the finalizer, resulting in a nice
LOC reduction.

Signed-off-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Tested-by: Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
Reviewed-by: Beata Michalska <beata.michalska@linaro.org>
Message-id: 20191031142734.8590-5-drjones@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-11-01 20:40:59 +00:00
Peter Maydell 68d8ef4ec5 TCG Plugins initial implementation
- use --enable-plugins @ configure
   - low impact introspection (-plugin empty.so to measure overhead)
   - plugins cannot alter guest state
   - example plugins included in source tree (tests/plugins)
   - -d plugin to enable plugin output in logs
   - check-tcg runs extra tests when plugins enabled
   - documentation in docs/devel/plugins.rst
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAl23BZMACgkQ+9DbCVqe
 KkRPegf/QHygZ4ER2jOaWEookxiOEcik+dzQKVGNqLNXeMLvo5fGjGVpFoFxSgfv
 ZvCAL4xbW44zsYlVfh59tfn4Tu9qK7s7/qM3WXpHsmuvEuhoWef0Lt2jSe+D46Rs
 KeG/aX+rHLUR8rr9eCgE+1/MQmxPUj3VUonkUpNkk2ebBbSNoLSOudB4DD9Vcyl7
 Pya1kPvA6W9bwI20ZSWihE7flg13o62Pp+LgAFLrsfxXOxOMkPrU8Pp+B0Dvr+hL
 5Oh0clZLhiRi75x+KVGZ90TVsoftdjYoOWGMOudS/+NNmqKT1NTLm0K1WJYyRMQ1
 V0ne4/OcGNq7x8gcOx/xs09ADu5/VA==
 =UXR/
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-tcg-plugins-281019-4' into staging

TCG Plugins initial implementation

  - use --enable-plugins @ configure
  - low impact introspection (-plugin empty.so to measure overhead)
  - plugins cannot alter guest state
  - example plugins included in source tree (tests/plugins)
  - -d plugin to enable plugin output in logs
  - check-tcg runs extra tests when plugins enabled
  - documentation in docs/devel/plugins.rst

# gpg: Signature made Mon 28 Oct 2019 15:13:23 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-tcg-plugins-281019-4: (57 commits)
  travis.yml: enable linux-gcc-debug-tcg cache
  MAINTAINERS: add me for the TCG plugins code
  scripts/checkpatch.pl: don't complain about (foo, /* empty */)
  .travis.yml: add --enable-plugins tests
  include/exec: wrap cpu_ldst.h in CONFIG_TCG
  accel/stubs: reduce headers from tcg-stub
  tests/plugin: add hotpages to analyse memory access patterns
  tests/plugin: add instruction execution breakdown
  tests/plugin: add a hotblocks plugin
  tests/tcg: enable plugin testing
  tests/tcg: drop test-i386-fprem from TESTS when not SLOW
  tests/tcg: move "virtual" tests to EXTRA_TESTS
  tests/tcg: set QEMU_OPTS for all cris runs
  tests/tcg/Makefile.target: fix path to config-host.mak
  tests/plugin: add sample plugins
  linux-user: support -plugin option
  vl: support -plugin option
  plugin: add qemu_plugin_outs helper
  plugin: add qemu_plugin_insn_disas helper
  plugin: expand the plugin_init function to include an info block
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-10-30 14:10:32 +00:00
Alex Bennée ca76a66975 plugin: add qemu_plugin_outs helper
Having the plugins grab stdout and spew stuff there is a bit ugly and
certainly makes the tests look ugly. Provide a hook back into QEMU
which can be redirected as needed.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Aaron Lindsay <aaron@os.amperecomputing.com>
2019-10-28 15:12:38 +00:00
Alex Bennée cbafa2362a plugin: add qemu_plugin_insn_disas helper
Give the plugins access to the QEMU dissasembler so they don't have to
re-invent the wheel. We generate a warning when there are spare bytes
in the decode buffer. This is usually due to the front end loading in
more bytes than decoded.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-10-28 15:12:38 +00:00
Alex Bennée 5901b2e15b plugin: expand the plugin_init function to include an info block
This provides a limited amount of info to plugins about the guest
system that will allow them to make some additional decisions on
setup.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-10-28 15:12:38 +00:00
Alex Bennée 235537fa73 plugins: implement helpers for resolving hwaddr
We need to keep a local per-cpu copy of the data as other threads may
be running. Currently we can provide insight as to if the access was
IO or not and give the offset into a given device (usually the main
RAMBlock). We store enough information to get details such as the
MemoryRegion which might be useful in later expansions to the API.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-10-28 15:12:38 +00:00
Emilio G. Cota 050ec8cc18 queue: add QTAILQ_REMOVE_SEVERAL
This is faster than removing elements one by one.

Will gain a user soon.

Signed-off-by: Emilio G. Cota <cota@braap.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-10-28 15:12:38 +00:00
Alex Bennée 5c5d69b0d5 plugin: add implementation of the api
[AJB: split from the core code commit]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-10-28 15:12:38 +00:00
Emilio G. Cota 54cb65d858 plugin: add core code
Signed-off-by: Emilio G. Cota <cota@braap.org>
[AJB: moved directory and merged various fixes]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-10-28 15:12:38 +00:00
Emilio G. Cota 975c455346 plugin: add user-facing API
Add the API first to ease review.

Signed-off-by: Emilio G. Cota <cota@braap.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-10-28 15:12:38 +00:00
Peter Maydell aaffb85335 Block patches for softfreeze:
- iotest patches
 - Improve performance of the mirror block job in write-blocking mode
 - Limit memory usage for the backup block job
 - Add discard and write-zeroes support to the NVMe host block driver
 - Fix a bug in the mirror job
 - Prevent the qcow2 driver from creating technically non-compliant qcow2
   v3 images (where there is not enough extra data for snapshot table
   entries)
 - Allow callers of bdrv_truncate() (etc.) to determine whether the file
   must be resized to the exact given size or whether it is OK for block
   devices not to shrink
 -----BEGIN PGP SIGNATURE-----
 
 iQFGBAABCAAwFiEEkb62CjDbPohX0Rgp9AfbAGHVz0AFAl2224ESHG1yZWl0ekBy
 ZWRoYXQuY29tAAoJEPQH2wBh1c9AeXMH/RXKEX4BZYMRKCe41P18tJC9Bl2x0T20
 YeOsZVvpARlr7o/36BF2kGFF4MnL0OQ+9ELuyROX865rk/VL2rWqnHDE5oQM889a
 dFwMs+0zvNbig3iLNcw0H5OkE2mrdM+a1EUdn/lBe/39Z8dPqPxRGqIYHq38Ugdu
 emwSy1nWen7o0f71HRJfyVtI3KcrzXx71FrA/FY2yL/eHz+zRYGZj2SpAdFPkXP/
 lgaz+m0tWhnSW1QzEOXB0Gh69ULt/DczCinYmv5qUY1noW5TPPtiDNCQTts5O4ba
 oJsR3AJv5/l9m65JTmiyQSqnQfPcstrQ5FqOcSnP637cfqUFyWsvdks=
 =L7v1
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-10-28' into staging

Block patches for softfreeze:
- iotest patches
- Improve performance of the mirror block job in write-blocking mode
- Limit memory usage for the backup block job
- Add discard and write-zeroes support to the NVMe host block driver
- Fix a bug in the mirror job
- Prevent the qcow2 driver from creating technically non-compliant qcow2
  v3 images (where there is not enough extra data for snapshot table
  entries)
- Allow callers of bdrv_truncate() (etc.) to determine whether the file
  must be resized to the exact given size or whether it is OK for block
  devices not to shrink

# gpg: Signature made Mon 28 Oct 2019 12:13:53 GMT
# gpg:                using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg:                issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2019-10-28: (69 commits)
  qemu-iotests: restrict 264 to qcow2 only
  Revert "qemu-img: Check post-truncation size"
  block: Pass truncate exact=true where reasonable
  block: Let format drivers pass @exact
  block: Evaluate @exact in protocol drivers
  block: Add @exact parameter to bdrv_co_truncate()
  block: Do not truncate file node when formatting
  block/cor: Drop cor_co_truncate()
  block: Handle filter truncation like native impl.
  iotests: Test qcow2's snapshot table handling
  iotests: Add peek_file* functions
  qcow2: Fix v3 snapshot table entry compliancy
  qcow2: Repair snapshot table with too many entries
  qcow2: Fix overly long snapshot tables
  qcow2: Keep track of the snapshot table length
  qcow2: Fix broken snapshot table entries
  qcow2: Add qcow2_check_fix_snapshot_table()
  qcow2: Separate qcow2_check_read_snapshot_table()
  qcow2: Write v3-compliant snapshot list on upgrade
  qcow2: Put qcow2_upgrade() into its own function
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-10-28 14:40:01 +00:00
Max Reitz 5d5b33c080 include: Move endof() up from hw/virtio/virtio.h
endof() is a useful macro, we can make use of it outside of virtio.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-2-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-28 11:51:07 +01:00
Vladimir Sementsov-Ogievskiy f16ba00de9 util: introduce SharedResource
Introduce an API for some shared splittable resource, like memory.
It's going to be used by backup. Backup uses both read/write io and
copy_range. copy_range may consume memory implictly, so the new API is
abstract: it doesn't allocate any real memory but only hands out
tickets.

The idea is that we have some total amount of something and callers
should wait in coroutine queue if there is not enough of the resource
at the moment.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191022111805.3432-5-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-28 11:22:31 +01:00
Wei Yang 038adc2f58 core: replace getpagesize() with qemu_real_host_page_size
There are three page size in qemu:

  real host page size
  host page size
  target page size

All of them have dedicate variable to represent. For the last two, we
use the same form in the whole qemu project, while for the first one we
use two forms: qemu_real_host_page_size and getpagesize().

qemu_real_host_page_size is defined to be a replacement of
getpagesize(), so let it serve the role.

[Note] Not fully tested for some arch or device.

Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Message-Id: <20191013021145.16011-3-richardw.yang@linux.intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-10-26 15:38:06 +02:00
Kevin Wolf 944f3d5dd2 coroutine: Add qemu_co_mutex_assert_locked()
Some functions require that the caller holds a certain CoMutex for them
to operate correctly. Add a function so that they can assert the lock is
really held.

Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Tested-by: Michael Weiser <michael.weiser@gmx.de>
Reviewed-by: Michael Weiser <michael.weiser@gmx.de>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2019-10-25 15:18:38 +02:00
Vladimir Sementsov-Ogievskiy 3d692649d1 qemu-coroutine-sleep: introduce qemu_co_sleep_wake
Introduce a function to gracefully wake a coroutine sleeping in
qemu_co_sleep_ns().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191009084158.15614-2-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2019-10-22 09:22:07 -05:00
Vladimir Sementsov-Ogievskiy 48557b1383 util/hbitmap: strict hbitmap_reset
hbitmap_reset has an unobvious property: it rounds requested region up.
It may provoke bugs, like in recently fixed write-blocking mode of
mirror: user calls reset on unaligned region, not keeping in mind that
there are possible unrelated dirty bytes, covered by rounded-up region
and information of this unrelated "dirtiness" will be lost.

Make hbitmap_reset strict: assert that arguments are aligned, allowing
only one exception when @start + @count == hb->orig_size. It's needed
to comfort users of hbitmap_next_dirty_area, which cares about
hb->orig_size.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20190806152611.280389-1-vsementsov@virtuozzo.com>
[Maintainer edit: Max's suggestions from on-list. --js]
[Maintainer edit: Eric's suggestion for aligned macro. --js]
Signed-off-by: John Snow <jsnow@redhat.com>
2019-10-17 17:02:32 -04:00
Dr. David Alan Gilbert 5626f8c6d4 rcu: Add automatically released rcu_read_lock variants
RCU_READ_LOCK_GUARD() takes the rcu_read_lock and then uses glib's
g_auto infrastructure (and thus whatever the compiler's hooks are) to
release it on all exits of the block.

WITH_RCU_READ_LOCK_GUARD() is similar but is used as a wrapper for the
lock, i.e.:

   WITH_RCU_READ_LOCK_GUARD() {
       stuff under lock
   }

Note the 'unused' attribute is needed to work around clang bug:
  https://bugs.llvm.org/show_bug.cgi?id=43482

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20191007143642.301445-2-dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2019-10-11 14:19:49 +01:00
Richard Henderson 560e36d5a4 qemu/compiler.h: Add qemu_build_not_reached
Use this as a compile-time assert that a particular
code path is not reachable.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-09-25 10:19:57 -07:00
Richard Henderson c6b716cdc0 cputlb: Disable __always_inline__ without optimization
This forced inlining can result in missing symbols,
which makes a debugging build harder to follow.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-09-25 10:12:33 -07:00
Paul Durrant 374752a26b xen / notify: introduce a new XenWatchList abstraction
Xenstore watch call-backs are already abstracted away from XenBus using
the XenWatch data structure but the associated NotifierList manipulation
and file handle registration is still open coded in various xen_bus_...()
functions.
This patch creates a new XenWatchList data structure to allow these
interactions to be abstracted away from XenBus as well. This is in
preparation for a subsequent patch which will introduce separate watch lists
for XenBus and XenDevice objects.

NOTE: This patch also introduces a new notifier_list_empty() helper function
      for the purposes of adding an assertion that a XenWatchList is not
      freed whilst its associated NotifierList is still occupied.

Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Reviewed-by: Anthony Perard <anthony.perard@citrix.com>
Message-Id: <20190913082159.31338-2-paul.durrant@citrix.com>
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
2019-09-24 12:18:47 +01:00
Philippe Mathieu-Daudé 31e404151b cutils: Move size_to_str() from "qemu-common.h" to "qemu/cutils.h"
"qemu/cutils.h" contains various qemu_strtosz_*() functions
useful to convert strings to size. It seems natural to have
the opposite usage (from size to string) there too.

The function definition is already in util/cutils.c.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20190903120555.7551-1-philmd@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-09-19 11:57:34 +02:00
Yury Kotov 3dcc9c6ec4 qemu-thread: Add qemu_cond_timedwait
The new function is needed to implement conditional sleep for CPU
throttling. It's possible to reuse qemu_sem_timedwait, but it's more
difficult than just add qemu_cond_timedwait.

Also moved compute_abs_deadline function up the code to reuse it in
qemu_cond_timedwait_impl win32.

Signed-off-by: Yury Kotov <yury-kotov@yandex-team.ru>
Message-Id: <20190909131335.16848-2-yury-kotov@yandex-team.ru>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-09-16 17:13:06 +02:00
Stefan Hajnoczi 72d41eb4b8 memory: fetch pmem size in get_file_size()
Neither stat(2) nor lseek(2) report the size of Linux devdax pmem
character device nodes.  Commit 314aec4a6e
("hostmem-file: reject invalid pmem file sizes") added code to
hostmem-file.c to fetch the size from sysfs and compare against the
user-provided size=NUM parameter:

  if (backend->size > size) {
      error_setg(errp, "size property %" PRIu64 " is larger than "
                 "pmem file \"%s\" size %" PRIu64, backend->size,
                 fb->mem_path, size);
      return;
  }

It turns out that exec.c:qemu_ram_alloc_from_fd() already has an
equivalent size check but it skips devdax pmem character devices because
lseek(2) returns 0:

  if (file_size > 0 && file_size < size) {
      error_setg(errp, "backing store %s size 0x%" PRIx64
                 " does not match 'size' option 0x" RAM_ADDR_FMT,
                 mem_path, file_size, size);
      return NULL;
  }

This patch moves the devdax pmem file size code into get_file_size() so
that we check the memory size in a single place:
qemu_ram_alloc_from_fd().  This simplifies the code and makes it more
general.

This also fixes the problem that hostmem-file only checks the devdax
pmem file size when the pmem=on parameter is given.  An unchecked
size=NUM parameter can lead to SIGBUS in QEMU so we must always fetch
the file size for Linux devdax pmem character device nodes.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20190830093056.12572-1-stefanha@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-09-16 12:32:21 +02:00
Peter Maydell 3d9442ee1d Block layer patches:
- qcow2: Allow overwriting multiple compressed clusters at once for
   better performance
 - nfs: add support for nfs_umount
 - file-posix: write_zeroes fixes
 - qemu-io, blockdev-create, pr-manager: Fix crashes and memory leaks
 - qcow2: Fix the calculation of the maximum L2 cache size
 - vpc: Fix return code for vpc_co_create()
 - blockjob: Code cleanup
 - iotests improvements (e.g. for use with valgrind)
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJde20nAAoJEH8JsnLIjy/WilcP/RGqhScaROcvr584XRJ+/r0p
 Wx/r1dfAL7uWaCrUt0Z7BtoAQb0vTJLAmShRvyfyDqSurPoTCGunQhXqHC/eX3oK
 fo6iB/fjXKCEsEjKcgamxv6it9rz3wjqeQLLakHZW4Z62yXAfyFZE/vXYdx6IS5B
 UlVI2gOjz3lfGWZCBd1rOQNnOOTtSeBkTMPndrpri5m5gvnZFBaCcAldUtrQwvau
 YQnzPhOsDlz00gR9NqDehQZcBCoNZnhhOMdcGWrvcMbabJ/yY/iqgCVxYq7m2Sol
 olcMw/Mg8Z/Mp2qr4hOUmxoHSEX9y7mZ7hCPRlw/3NdiEQpquIfpILa8CvmO8jqJ
 ZHnLWILf2ZCmU75vTM2ilhvgErtEmYbE39jzRYF/dNfBTwfx9Yp2eFbG57OWqNyJ
 sEJB3JY+RuPyqP/QX7jAelnL2jtSmyhmV1dBYIpD0cib2+VTCgdgWMoMNlXRXG+8
 9XoRVfBNHvybR7o7xw7xUTmGdkkAZDbNlzGH6ekIn3HCiDqdpL7SUkV7kBUCV9ie
 YKcZMRrwOiYr8agi4SlctyUPY/h1nFJFPKSBdPcKSf353sjZgpzYeubXUJd+kTBX
 orkkJVq9+WcqEKScvqzTraf4CN5FHP7J1rvQdxbP6RWgkY64j4gkMScGv7qu29K2
 WNQrD2Ij+KKC0zprkCx0
 =yYOk
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- qcow2: Allow overwriting multiple compressed clusters at once for
  better performance
- nfs: add support for nfs_umount
- file-posix: write_zeroes fixes
- qemu-io, blockdev-create, pr-manager: Fix crashes and memory leaks
- qcow2: Fix the calculation of the maximum L2 cache size
- vpc: Fix return code for vpc_co_create()
- blockjob: Code cleanup
- iotests improvements (e.g. for use with valgrind)

# gpg: Signature made Fri 13 Sep 2019 11:19:19 BST
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (23 commits)
  qcow2: Stop overwriting compressed clusters one by one
  block/create: Do not abort if a block driver is not available
  qemu-io: Don't leak pattern file in error path
  iotests: extend sleeping time under Valgrind
  iotests: extended timeout under Valgrind
  iotests: Valgrind fails with nonexistent directory
  iotests: Add casenotrun report to bash tests
  iotests: exclude killed processes from running under Valgrind
  iotests: allow Valgrind checking all QEMU processes
  block/nfs: add support for nfs_umount
  block/nfs: tear down aio before nfs_close
  iotests: skip 232 when run tests as root
  iotests: Test blockdev-create for vpc
  iotests: Restrict nbd Python tests to nbd
  iotests: Restrict file Python tests to file
  iotests: Add supported protocols to execute_test()
  vpc: Return 0 from vpc_co_create() on success
  file-posix: Fix has_write_zeroes after NO_FALLBACK
  pr-manager: Fix invalid g_free() crash bug
  iotests: Test reverse sub-cluster qcow2 writes
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-09-13 13:43:42 +01:00
Shu-Chun Weng 9bdfa4d23f linux-user: add memfd_create
Add support for the memfd_create syscall. If the host does not have the
libc wrapper, translate to a direct syscall with NC-macro.

Buglink: https://bugs.launchpad.net/qemu/+bug/1734792
Signed-off-by: Shu-Chun Weng <scw@google.com>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20190819180947.180725-1-scw@google.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-09-11 08:46:17 +02:00
Vladimir Sementsov-Ogievskiy bb0c940993 job: drop job_drain
In job_finish_sync job_enter should be enough for a job to make some
progress and draining is a wrong tool for it. So use job_enter directly
here and drop job_drain with all related staff not used more.

Suggested-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Tested-by: John Snow <jsnow@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-09-10 08:58:43 +02:00
Juan Quintela e5b6353cf2 socket: Add backlog parameter to socket_listen
Current parameter was always one.  We continue with that value for now
in all callers.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
---
Moved trace to socket_listen
2019-09-03 23:24:42 +02:00
Vladimir Sementsov-Ogievskiy 5396234b96 block/qcow2: implement .bdrv_co_pwritev(_compressed)_part
Implement and use new interface to get rid of hd_qiov.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20190604161514.262241-13-vsementsov@virtuozzo.com
Message-Id: <20190604161514.262241-13-vsementsov@virtuozzo.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2019-08-27 14:58:42 +01:00
Vladimir Sementsov-Ogievskiy f76889e7b9 util/iov: improve qemu_iovec_is_zero
We'll need to check a part of qiov soon, so implement it now.

Optimization with align down to 4 * sizeof(long) is dropped due to:
1. It is strange: it aligns length of the buffer, but where is a
   guarantee that buffer pointer is aligned itself?
2. buffer_is_zero() is a better place for optimizations and it has
   them.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20190604161514.262241-3-vsementsov@virtuozzo.com
Message-Id: <20190604161514.262241-3-vsementsov@virtuozzo.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2019-08-27 14:52:45 +01:00
Vladimir Sementsov-Ogievskiy d953169d48 util/iov: introduce qemu_iovec_init_extended
Introduce new initialization API, to create requests with padding. Will
be used in the following patch. New API uses qemu_iovec_init_buf if
resulting io vector has only one element, to avoid extra allocations.
So, we need to update qemu_iovec_destroy to support destroying such
QIOVs.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20190604161514.262241-2-vsementsov@virtuozzo.com
Message-Id: <20190604161514.262241-2-vsementsov@virtuozzo.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2019-08-27 14:52:45 +01:00
Marc-André Lureau 81d8ccb1be module: return success on module load
Let the caller know of load success.

Note that this also changes slightly the behaviour of the function to
try loading on subsequent calls if the previous ones failed.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-08-21 16:29:57 +02:00
Pavel Dovgalyuk dcb1578069 util/qemu-timer: refactor deadline calculation for external timers
icount-based record/replay uses qemu_clock_deadline_ns_all to measure
the period until vCPU may be interrupted.
This function takes in account the virtual timers, because they belong
to the virtual devices that may generate interrupt request or affect
the virtual machine state.
However, there are a subset of virtual timers, that are marked with
'external' flag. These do not change the virtual machine state and
only based on virtual clock. Calculating the deadling using the external
timers breaks the determinism, because they do not belong to the replayed
part of the virtual machine.
This patch fixes the deadline calculation for this case by adding
new parameter for skipping the external timers when it is needed.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>

--

v2 changes:
 - added new parameter for timer attribute mask
Message-Id: <156404426682.18669.17014100602930969222.stgit@pasha-Precision-3630-Tower>

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-08-20 17:26:21 +02:00
Dr. David Alan Gilbert 3c2d4c8aa6 timer: last, remove last bits of last
The reset notifiers kept a 'last' counter to notice jumps;
now that we've remove the notifier we don't need to keep 'last'.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190724115823.4199-5-dgilbert@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-08-20 17:26:21 +02:00
Dr. David Alan Gilbert 4ea9a0e3db timer: Remove reset notifiers
Remove the reset notifer from the core qemu-timer code.
The only user was mc146818 and we've just remove it's use.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190724115823.4199-3-dgilbert@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-08-20 17:26:20 +02:00
Markus Armbruster 2f780b6a91 sysemu: Move the VMChangeStateEntry typedef to qemu/typedefs.h
In my "build everything" tree, changing sysemu/sysemu.h triggers a
recompile of some 1800 out of 6600 objects (not counting tests and
objects that don't depend on qemu/osdep.h, down from 5400 due to the
previous commit).

Several headers include sysemu/sysemu.h just to get typedef
VMChangeStateEntry.  Move it from sysemu/sysemu.h to qemu/typedefs.h.
Spell its structure tag the same while there.  Drop the now
superfluous includes of sysemu/sysemu.h from headers.

Touching sysemu/sysemu.h now recompiles some 1100 objects.
qemu/uuid.h also drops from 1800 to 1100, and
qapi/qapi-types-run-state.h from 5000 to 4400.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190812052359.30071-29-armbru@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-08-16 13:31:53 +02:00
Markus Armbruster 7969dd9169 Include sysemu/hostmem.h less
Move the HostMemoryBackend typedef from sysemu/hostmem.h to
qemu/typedefs.h.  This renders a few inclusions of sysemu/hostmem.h
superfluous; drop them.

Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190812052359.30071-25-armbru@redhat.com>
2019-08-16 13:31:53 +02:00
Markus Armbruster dc5e9ac716 Include qemu/queue.h slightly less
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190812052359.30071-20-armbru@redhat.com>
2019-08-16 13:31:52 +02:00
Markus Armbruster d645427057 Include migration/vmstate.h less
In my "build everything" tree, changing migration/vmstate.h triggers a
recompile of some 2700 out of 6600 objects (not counting tests and
objects that don't depend on qemu/osdep.h).

hw/hw.h supposedly includes it for convenience.  Several other headers
include it just to get VMStateDescription.  The previous commit made
that unnecessary.

Include migration/vmstate.h only where it's still needed.  Touching it
now recompiles only some 1600 objects.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20190812052359.30071-16-armbru@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-08-16 13:31:52 +02:00
Markus Armbruster 8a9358cc6e migration: Move the VMStateDescription typedef to typedefs.h
We declare incomplete struct VMStateDescription in a couple of places
so we don't have to include migration/vmstate.h for the typedef.
That's fine with me.  However, the next commit will drop
migration/vmstate.h from a massive number of compiles.  Move the
typedef to qemu/typedefs.h now, so I don't have to insert struct in
front of VMStateDescription all over the place then.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190812052359.30071-15-armbru@redhat.com>
2019-08-16 13:31:52 +02:00
Markus Armbruster 64552b6be4 Include hw/irq.h a lot less
In my "build everything" tree, changing hw/irq.h triggers a recompile
of some 5400 out of 6600 objects (not counting tests and objects that
don't depend on qemu/osdep.h).

hw/hw.h supposedly includes it for convenience.  Several other headers
include it just to get qemu_irq and.or qemu_irq_handler.

Move the qemu_irq and qemu_irq_handler typedefs from hw/irq.h to
qemu/typedefs.h, and then include hw/irq.h only where it's still
needed.  Touching it now recompiles only some 500 objects.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190812052359.30071-13-armbru@redhat.com>
2019-08-16 13:31:52 +02:00
Markus Armbruster 2a28720d77 typedefs: Separate incomplete types and function types
While there, rewrite the obsolete file comment.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190812052359.30071-12-armbru@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
[File comment rewritten, commit message adjusted]
2019-08-16 13:31:52 +02:00
Markus Armbruster cbcbba9805 queue: Drop superfluous #include qemu/atomic.h
When commit 5f7d05ecfd added QLIST_INSERT_HEAD_RCU() to qemu/queue.h,
it had to include qemu/atomic.h.  Commit 341774fe6c removed
QLIST_INSERT_HEAD_RCU() again, but neglected to remove the #include.
Do that now.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20190812052359.30071-6-armbru@redhat.com>
2019-08-16 13:31:51 +02:00
Markus Armbruster 2ae16a6aa4 Include generated QAPI headers less
Some of the generated qapi-types-MODULE.h are included all over the
place.  Changing a QAPI type can trigger massive recompiling.  Top
scorers recompile more than 1000 out of some 6600 objects (not
counting tests and objects that don't depend on qemu/osdep.h):

    6300 qapi/qapi-builtin-types.h
    5700 qapi/qapi-types-run-state.h
    3900 qapi/qapi-types-common.h
    3300 qapi/qapi-types-sockets.h
    3000 qapi/qapi-types-misc.h
    3000 qapi/qapi-types-crypto.h
    3000 qapi/qapi-types-job.h
    3000 qapi/qapi-types-block-core.h
    2800 qapi/qapi-types-block.h
    1300 qapi/qapi-types-net.h

Clean up headers to include generated QAPI headers only where needed.
Impact is negligible except for hw/qdev-properties.h.

This header includes qapi/qapi-types-block.h and
qapi/qapi-types-misc.h.  They are used only in expansions of property
definition macros such as DEFINE_PROP_BLOCKDEV_ON_ERROR() and
DEFINE_PROP_OFF_AUTO().  Moving their inclusion from
hw/qdev-properties.h to the users of these macros avoids pointless
recompiles.  This is how other property definition macros, such as
DEFINE_PROP_NETDEV(), already work.

Improves things for some of the top scorers:

    3600 qapi/qapi-types-common.h
    2800 qapi/qapi-types-sockets.h
     900 qapi/qapi-types-misc.h
    2200 qapi/qapi-types-crypto.h
    2100 qapi/qapi-types-job.h
    2100 qapi/qapi-types-block-core.h
     270 qapi/qapi-types-block.h

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190812052359.30071-3-armbru@redhat.com>
2019-08-16 13:31:51 +02:00
Markus Armbruster ec150c7e09 include: Make headers more self-contained
Back in 2016, we discussed[1] rules for headers, and these were
generally liked:

1. Have a carefully curated header that's included everywhere first.  We
   got that already thanks to Peter: osdep.h.

2. Headers should normally include everything they need beyond osdep.h.
   If exceptions are needed for some reason, they must be documented in
   the header.  If all that's needed from a header is typedefs, put
   those into qemu/typedefs.h instead of including the header.

3. Cyclic inclusion is forbidden.

This patch gets include/ closer to obeying 2.

It's actually extracted from my "[RFC] Baby steps towards saner
headers" series[2], which demonstrates a possible path towards
checking 2 automatically.  It passes the RFC test there.

[1] Message-ID: <87h9g8j57d.fsf@blackfin.pond.sub.org>
    https://lists.nongnu.org/archive/html/qemu-devel/2016-03/msg03345.html
[2] Message-Id: <20190711122827.18970-1-armbru@redhat.com>
    https://lists.nongnu.org/archive/html/qemu-devel/2019-07/msg02715.html

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20190812052359.30071-2-armbru@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-08-16 13:31:51 +02:00
Peter Xu ad37f24d57 bitmap: Add bitmap_copy_with_{src|dst}_offset()
These helpers copy the source bitmap to destination bitmap with a
shift either on the src or dst bitmap.

Meanwhile, we never have bitmap tests but we should.

This patch also introduces the initial test cases for utils/bitmap.c
but it only tests the newly introduced functions.

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20190603065056.25211-5-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

---

Bitmap test used sizeof(unsigned long) instead of BITS_PER_LONG.
2019-07-15 15:39:02 +02:00
Richard Henderson 359896dfa4 include/qemu/atomic.h: Add signal_barrier
We have some potential race conditions vs our user-exec signal
handler that will be solved with this barrier.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-07-14 12:19:00 +02:00
Markus Armbruster a8d2532645 Include qemu-common.h exactly where needed
No header includes qemu-common.h after this commit, as prescribed by
qemu-common.h's file comment.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190523143508.25387-5-armbru@redhat.com>
[Rebased with conflicts resolved automatically, except for
include/hw/arm/xlnx-zynqmp.h hw/arm/nrf51_soc.c hw/arm/msf2-soc.c
block/qcow2-refcount.c block/qcow2-cluster.c block/qcow2-cache.c
target/arm/cpu.h target/lm32/cpu.h target/m68k/cpu.h target/mips/cpu.h
target/moxie/cpu.h target/nios2/cpu.h target/openrisc/cpu.h
target/riscv/cpu.h target/tilegx/cpu.h target/tricore/cpu.h
target/unicore32/cpu.h target/xtensa/cpu.h; bsd-user/main.c and
net/tap-bsd.c fixed up]
2019-06-12 13:20:20 +02:00
Markus Armbruster 856dfd8a03 qemu-common: Move qemu_isalnum() etc. to qemu/ctype.h
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190523143508.25387-3-armbru@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-06-11 20:22:09 +02:00
Richard Henderson 8d8404f156 util: Add qemu_guest_getrandom and associated routines
This routine is intended to produce high-quality random numbers to the
guest.  Normally, such numbers are crypto quality from the host, but a
command-line option can force the use of a fully deterministic sequence
for use while debugging.

Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-22 12:38:54 -04:00
Markus Armbruster 6834c3f410 Clean up decorations and whitespace around header guards
Cleaned up with scripts/clean-header-guards.pl.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190315145123.28030-9-armbru@redhat.com>
2019-05-13 08:58:55 +02:00
Markus Armbruster 177d9e0da0 Normalize header guard symbol definition.
We commonly define the header guard symbol without an explicit value.
Normalize the exceptions.

Done with scripts/clean-header-guards.pl.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190315145123.28030-8-armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-05-13 08:58:55 +02:00
Markus Armbruster a8b991b52d Clean up ill-advised or unusual header guards
Leading underscores are ill-advised because such identifiers are
reserved.  Trailing underscores are merely ugly.  Strip both.

Our header guards commonly end in _H.  Normalize the exceptions.

Done with scripts/clean-header-guards.pl.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190315145123.28030-7-armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[Changes to slirp/ dropped, as we're about to spin it off]
2019-05-13 08:58:55 +02:00
Markus Armbruster 58ea30f514 Clean up header guards that don't match their file name
Header guard symbols should match their file name to make guard
collisions less likely.

Cleaned up with scripts/clean-header-guards.pl, followed by some
renaming of new guard symbols picked by the script to better ones.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190315145123.28030-6-armbru@redhat.com>
[Rebase to master: update include/hw/net/ne2000-isa.h]
2019-05-13 08:58:55 +02:00
Cao Jiaxi 946376c21b osdep: Fix mingw compilation regarding stdio formats
I encountered the following compilation error on mingw:

/mnt/d/qemu/include/qemu/osdep.h:97:9: error: '__USE_MINGW_ANSI_STDIO' macro redefined [-Werror,-Wmacro-redefined]
 #define __USE_MINGW_ANSI_STDIO 1
        ^
/mnt/d/llvm-mingw/aarch64-w64-mingw32/include/_mingw.h:433:9: note: previous definition is here
 #define __USE_MINGW_ANSI_STDIO 0      /* was not defined so it should be 0 */

It turns out that __USE_MINGW_ANSI_STDIO must be set before any
system headers are included, not just before stdio.h.

Signed-off-by: Cao Jiaxi <driver1998@foxmail.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Message-id: 20190503003719.10233-1-driver1998@foxmail.com
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-05-07 12:55:03 +01:00
Cao Jiaxi 48bb55bfaa QEMU_PACKED: Remove gcc_struct attribute in Windows non x86 targets
gcc_struct is for x86 only, and it generates an warning on ARM64 Clang/MinGW targets.

Signed-off-by: Cao Jiaxi <driver1998@foxmail.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20190503003618.10089-1-driver1998@foxmail.com
[PMM: dropped the slirp change as slirp is now a submodule]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-05-07 12:55:02 +01:00
Philippe Mathieu-Daudé a331dd0299 hw/devices: Move TI touchscreen declarations into a new header
Since uWireSlave is only used in this new header, there is no
need to expose it via "qemu/typedefs.h".

Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20190412165416.7977-9-philmd@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-04-29 17:57:21 +01:00
Zhang Yi 2ac0f1621c util/mmap-alloc: Add a 'is_pmem' parameter to qemu_ram_mmap
besides the existing 'shared' flags, we are going to add
'is_pmem' to qemu_ram_mmap(), which indicated the memory backend
file is a persist memory.

Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
Signed-off-by: Zhang Yi <yi.z.zhang@linux.intel.com>
Reviewed-by: Pankaj Gupta <pagupta@redhat.com>
Message-Id: <786c46862cfeb253ee0ea2f44d62ffe76edb7fa4.1549555521.git.yi.z.zhang@linux.intel.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Pankaj Gupta <pagupta@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-04-25 14:17:36 -03:00
Markus Armbruster ede9a8a656 include: Move fprintf_function to disas/
The previous commits have eliminated fprintf_function outside
disassemblers, simplifying code and cleaning up the ugly type-punning
fprintf_function seems to attract.  Move fprintf_function to
include/disas/dis-asm.h to reduce the temptation to abuse it.

I considered renaming it to fprintf_ftype (reverting that part of
commit 6e2d864edf, v0.14.0) to get us closer to binutils, but I
figure the fork is too distant to make this worthwhile.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190417191805.28198-18-armbru@redhat.com>
2019-04-18 22:18:59 +02:00
Markus Armbruster 19aaa4c3fd qemu-print: New qemu_fprintf(), qemu_vfprintf()
Code that doesn't want to know about current monitor vs. stdout
vs. stderr takes an fprintf_function callback and a FILE * argument to
pass to it.  Actual arguments are either fprintf() and stdout or
stderr, or monitor_fprintf() and the current monitor cast to FILE *.
monitor_fprintf() casts it right back, and is otherwise identical to
monitor_printf().  The type-punning is ugly.

New qemu_fprintf() and qemu_vprintf() address this need without type
punning: they are like fprintf() and vfprintf(), except they print to
the current monitor when passed a null FILE *.  The next commits will
put them to use.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190417191805.28198-14-armbru@redhat.com>
2019-04-18 22:18:59 +02:00
Markus Armbruster ac7ff4cf5f qsp: Simplify how qsp_report() prints
qsp_report() takes an fprintf()-like callback and a FILE * to pass to
it.

Its only caller hmp_sync_profile() passes monitor_fprintf() and the
current monitor cast to FILE *.  monitor_fprintf() casts it right
back, and is otherwise identical to monitor_printf().  The
type-punning is ugly.

Drop the callback, and call qemu_printf() instead.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190417191805.28198-7-armbru@redhat.com>
2019-04-18 22:18:59 +02:00
Markus Armbruster 15ce35fcf1 include: Include fprintf-fn.h only where needed
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190417191805.28198-2-armbru@redhat.com>
2019-04-18 22:18:59 +02:00
Markus Armbruster 637de4dba2 qemu-print: New qemu_printf(), qemu_vprintf() etc.
We commonly want to print to the current monitor if we have one, else
to stdout/stderr.  For stderr, have error_printf().  For stdout, all
we have is monitor_vfprintf(), which is rather unwieldy.  We often
print to stderr just because error_printf() is easier.

New qemu_printf() and qemu_vprintf() do exactly what's needed.  The
next commits will put them to use.

Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190417190641.26814-12-armbru@redhat.com>
2019-04-18 22:18:59 +02:00