Commit Graph

66145 Commits

Author SHA1 Message Date
Cleber Rosa 755ee70ff7 configure: keep track of Python version
Some functionality is dependent on the Python version
detected/configured on configure.  While it's possible to run the
Python version later and check for the version, doing it once is
preferable.  Also, it's a relevant information to keep in build logs,
as the overall behavior of the build can be affected by it.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20181109150710.31085-2-crosa@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-01-17 17:52:40 -02:00
Philippe Mathieu-Daudé 9ab5a24b52 scripts: Remove unused python imports
Reported-by: LGTM code review
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20181108143422.15955-1-philmd@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-01-17 17:52:40 -02:00
Thomas Huth 285cecd655 scripts/device-crash-test: Remove known crashes
Looks like we've fixed them all already in the past months, e.g. with:

 f7d6bfcdc0
 spapr_pci: fail gracefully with non-pseries machine types

 2363d5ee23
 hw/ppc/spapr_cpu_core: Add a proper check for spapr machine

 ef0e8fc768
 iommu: Don't crash if machine is not PC_MACHINE

 8929fc3a55
 hw/block/pflash_cfi*.c: fix confusing assert fail message

... so we can remove these entries from the ERROR_WHITELIST now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <1541510826-21031-1-git-send-email-thuth@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-01-17 17:52:39 -02:00
Peter Maydell 681d61362d Pull request
-----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJcPoIpAAoJEH3vgQaq/DkONy8P/28kkLogU55oNitKnt8oFxAE
 sfDBnEk+3rJV6L+fPN0TlqDnKDV0iEylpUTsCTEdL4OAtBaR1v0lhQzhGQIE0bHy
 6fk6nEiCgUVhDoQSGGFXqBq/Z6eYu6nsOfPF+SoS1Auv1ISB7KZdHR6K0dRcOUKk
 L3uIGDmop5+c1j5deG+90MYj38wvu7NwD2zt4kiMdYNUQHT4XtE591yvIUVMBo+s
 gE3vunsyu8++iS8NvxM7yJzsZStRoCRvks6/aBuGISfCWuJUpycdI6QpK+phitDs
 SdjN8jlAcY947o3xKwaKI8Brit+xoVDCw/0RaTwh14s4gBj7lI3B+XmlRGHiz/mw
 Ry8bqrzs/wo8Thr263rVr1gBnQQKcb4mIPEa7pqELvdnANdI3kOs78vwBw/wW2/+
 CTxAcrgyHItod5zKyxqiMDcoirtvHF55TIEpVuGhTfixOVQGNazBsP3IqfCCh03C
 Bx8SZEVP9PEV+LB/Y3v4n+xuJaCBrbAA0KQ5XMSuhE/nLatela9gcuwe9QNFnM4p
 G0nN4hQ/mksja1oj5VfRTmx4YrY67PAK+D+TBjUvKu4luUW+2cOTkqSDbRXwhplp
 kFF52alkv6m0SLWHUwOMXsgZBnNl7goIT4so4RfR2X3pH3/yj24ce8df7oKRGXMN
 ASBY5X3kqalOzglPNTp1
 =4Y4H
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into staging

Pull request

# gpg: Signature made Wed 16 Jan 2019 01:00:25 GMT
# gpg:                using RSA key 7DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>"
# Primary key fingerprint: FAEB 9711 A12C F475 812F  18F2 88A9 064D 1835 61EB
#      Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76  CBD0 7DEF 8106 AAFC 390E

* remotes/jnsnow/tags/bitmaps-pull-request:
  Revert "hbitmap: Add @advance param to hbitmap_iter_next()"
  Revert "test-hbitmap: Add non-advancing iter_next tests"
  Revert "block/dirty-bitmap: Add bdrv_dirty_iter_next_area"
  block/mirror: fix and improve do_sync_target_write
  tests: add tests for hbitmap_next_dirty_area
  dirty-bitmap: add bdrv_dirty_bitmap_next_dirty_area
  tests: add tests for hbitmap_next_zero with specified end parameter
  dirty-bitmap: improve bdrv_dirty_bitmap_next_zero

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-01-17 12:48:42 +00:00
Vladimir Sementsov-Ogievskiy 19c021e194 Revert "hbitmap: Add @advance param to hbitmap_iter_next()"
This reverts commit a33fbb4f8b.

The functionality is unused.

Note: in addition to automatic revert, drop second parameter in
hbitmap_iter_next() call from hbitmap_next_dirty_area() too.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2019-01-15 18:26:50 -05:00
Vladimir Sementsov-Ogievskiy 4294c4ab48 Revert "test-hbitmap: Add non-advancing iter_next tests"
This reverts commit 269576848e.

The functionality is unused. Drop tests.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2019-01-15 18:26:50 -05:00
Vladimir Sementsov-Ogievskiy 166cd55125 Revert "block/dirty-bitmap: Add bdrv_dirty_iter_next_area"
This reverts commit 72d10a9421.

The function is unused now.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2019-01-15 18:26:50 -05:00
Vladimir Sementsov-Ogievskiy 1eaf1b0fdf block/mirror: fix and improve do_sync_target_write
Use bdrv_dirty_bitmap_next_dirty_area() instead of
bdrv_dirty_iter_next_area(), because of the following problems of
bdrv_dirty_iter_next_area():

1. Using HBitmap iterators we should carefully handle unaligned offset,
as first call to hbitmap_iter_next() may return a value less than
original offset (actually, it will be original offset rounded down to
bitmap granularity). This handling is not done in
do_sync_target_write().

2. bdrv_dirty_iter_next_area() handles unaligned max_offset
incorrectly:

look at the code:
    if (max_offset == iter->bitmap->size) {
        /* If max_offset points to the image end, round it up by the
         * bitmap granularity */
        gran_max_offset = ROUND_UP(max_offset, granularity);
    } else {
        gran_max_offset = max_offset;
    }

    ret = hbitmap_iter_next(&iter->hbi, false);
    if (ret < 0 || ret + granularity > gran_max_offset) {
        return false;
    }

and assume that max_offset != iter->bitmap->size but still unaligned.
if 0 < ret < max_offset we found dirty area, but the function can
return false in this case (if ret + granularity > max_offset).

3. bdrv_dirty_iter_next_area() uses inefficient loop to find the end of
the dirty area. Let's use more efficient hbitmap_next_zero instead
(bdrv_dirty_bitmap_next_dirty_area() do so)

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-15 18:26:50 -05:00
Vladimir Sementsov-Ogievskiy bb6a0ec10e tests: add tests for hbitmap_next_dirty_area
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-15 18:26:50 -05:00
Vladimir Sementsov-Ogievskiy a78a1a48cd dirty-bitmap: add bdrv_dirty_bitmap_next_dirty_area
The function alters bdrv_dirty_iter_next_area(), which is wrong and
less efficient (see further commit
"block/mirror: fix and improve do_sync_target_write" for description).

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-15 18:26:50 -05:00
Vladimir Sementsov-Ogievskiy fa9c2da294 tests: add tests for hbitmap_next_zero with specified end parameter
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-15 18:26:49 -05:00
Vladimir Sementsov-Ogievskiy 76d570dc49 dirty-bitmap: improve bdrv_dirty_bitmap_next_zero
Add bytes parameter to the function, to limit searched range.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-15 18:26:49 -05:00
Peter Maydell 6f2f34177a slirp updates
Gerd Hoffmann (1):
   slirp: add tftp tracing
 
 Marc-André Lureau (61):
   slirp: associate slirp_output callback with the Slirp context
   slirp: remove do_pty from fork_exec()
   slirp: replace ex_pty with ex_chardev
   slirp: use a dedicated field for chardev pointer
   slirp: remove unused EMU_RSH
   slirp: rename /extra/chardev
   slirp: move internal function declarations
   slirp: remove Monitor dependency, return a string for info
   slirp: fix slirp_add_exec() leaks
   slirp: replace the poor-man string split with g_strsplit()
   slirp: remove dead declarations
   slirp: move socket pair creation in helper function
   slirp: remove unused M_TRAILINGSPACE
   slirp: use a callback structure to interface with qemu
   slirp: remove PROBE_CONN dead-code
   slirp: remove FULL_BOLT
   slirp: remove the disabled readv()/writev() code path
   slirp: remove HAVE_SYS_SIGNAL_H
   slirp: remove unused HAVE_SYS_BITYPES_H
   slirp: remove NO_UNIX_SOCKETS
   slirp: remove unused HAVE_SYS_STROPTS_H
   slirp: remove unused HAVE_ARPA_INET_H
   slirp: remove unused HAVE_SYS_WAIT_H
   slirp: remove unused HAVE_SYS_SELECT_H
   slirp: remove HAVE_SYS_IOCTL_H
   slirp: remove HAVE_SYS_FILIO_H
   slirp: remove unused DECLARE_IOVEC
   slirp: remove unused HAVE_INET_ATON
   slirp: replace HOST_WORDS_BIGENDIAN with glib equivalent
   slirp: replace SIZEOF_CHAR_P with glib equivalent
   slirp: replace compile time DO_KEEPALIVE
   slirp: remove unused global slirp_instance
   slirp: replace error_report() with g_critical()
   slirp: improve a bit the debug macros
   slirp: add a callback to log guest errors
   slirp: remove #if notdef dead code
   slirp: remove unused sbflush()
   slirp: NULL is defined by stddef.h
   slirp: remove dead TCP_ACK_HACK code
   slirp: replace ARRAY_SIZE with G_N_ELEMENTS
   net: do not depend on slirp internals
   glib-compat: add g_spawn_async_with_fds() fallback
   slirp: simplify fork_exec()
   slirp: replace error_report() with g_critical()
   slirp: drop <Vista compatibility
   slirp: rename exec_list
   slirp: use virtual time for packet expiration
   slirp: replace a fprintf with g_critical()
   slirp: replace some fprintf() with DEBUG_MISC
   slirp: replace a DEBUG block with WITH_ICMP_ERROR_MSG
   slirp: no need to make DPRINTF conditional on DEBUG
   slirp: always build with debug statements
   slirp: introduce SLIRP_DEBUG environment variable
   slirp: use %p for pointers format
   slirp: remove remaining DEBUG blocks
   slirp: replace DEBUG_ARGS with DEBUG_ARG
   slirp: factor out guestfwd addition checks
   slirp: add clock_get_ns() callback
   build-sys: use a separate slirp-obj-y && slirp.mo
   slirp: set G_LOG_DOMAIN
   slirp: call into g_debug() for DEBUG macros
 
 Prasad J Pandit (1):
   slirp: check data length while emulating ident function
 
 Samuel Thibault (2):
   slirp: Enable fork_exec support on Windows
   slirp: Mark debugging calls as unlikely
 
  Makefile              |   5 +-
  Makefile.objs         |   4 +-
  Makefile.target       |   5 +-
  include/glib-compat.h |  56 +++++++++
  net/colo-compare.c    |  11 +-
  net/colo.c            |   1 +
  net/colo.h            |   7 +-
  net/filter-rewriter.c |   9 +-
  net/slirp.c           |  61 +++++----
  net/util.h            |  55 ++++++++
  slirp/Makefile.objs   |  37 +++++-
  slirp/arp_table.c     |  12 +-
  slirp/bootp.c         |  10 +-
  slirp/cksum.c         |   8 +-
  slirp/debug.h         |  47 ++++---
  slirp/dhcpv6.c        |  17 ++-
  slirp/if.c            |   4 +-
  slirp/ip.h            |  10 +-
  slirp/ip6.h           |   3 +-
  slirp/ip6_icmp.c      |  27 ++--
  slirp/ip6_icmp.h      |   6 +-
  slirp/ip6_input.c     |   2 +-
  slirp/ip6_output.c    |   4 +-
  slirp/ip_icmp.c       |  31 ++---
  slirp/ip_input.c      | 200 -----------------------------
  slirp/libslirp.h      |  27 ++--
  slirp/main.h          |  33 -----
  slirp/mbuf.c          |   2 +-
  slirp/mbuf.h          |   1 -
  slirp/misc.c          | 286 +++++++++++++++++-------------------------
  slirp/misc.h          |  13 +-
  slirp/ncsi.c          |   4 +-
  slirp/ndp_table.c     |  32 +++--
  slirp/sbuf.h          |   1 -
  slirp/slirp.c         | 177 +++++++++++++-------------
  slirp/slirp.h         |  45 ++-----
  slirp/slirp_config.h  |  86 -------------
  slirp/socket.c        |  53 +++-----
  slirp/socket.h        |   2 +-
  slirp/tcp.h           |   4 +-
  slirp/tcp_input.c     |  84 ++-----------
  slirp/tcp_output.c    |   2 +-
  slirp/tcp_subr.c      |  22 ++--
  slirp/tcp_timer.c     |   2 +-
  slirp/tftp.c          |   7 +-
  slirp/trace-events    |   5 +
  slirp/udp.c           |   5 +-
  slirp/udp6.c          |  11 +-
  stubs/slirp.c         |   2 +-
  49 files changed, 603 insertions(+), 935 deletions(-)
  delete mode 100644 slirp/slirp_config.h
  create mode 100644 slirp/trace-events
 
 --
 2.20.1
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEE5h27FdQXK97JfpLZ21UOifD6VPMFAlw9ErAACgkQ21UOifD6
 VPN9Yg//aAff/bSkpBXYZrtFj+EtNxFyHubloruEfsalra5FmLoiPpsCIeF0Ta1O
 MkglD3RCi1/wt91O+/ccerNUL4ZD9F7lLorg7Zyor2F/l+hfrY8hJwalP47Owa88
 DUNrwYvVyKFU6k0EW3zwzYHYpnfjLNBhdgGGqwrZJ8kDOLNNsIGKvpge4BqfhmZU
 K0vfjTuAyJ4rHJYFn9T8m0LSdM4wnzPDtkPy1KL4UF2rxTM8EXKht1NtiT7QEp8U
 klQ8Or60PzXzJY3QU8W+ameM3YUP3YOWCLlRpt9+K65/ZY5OJI0pWNFHgqGX5pDZ
 NtL5XeUJqmgHza2TtSIeG1VEf/z9OKnQNFWdAv/DSEGxxrnnQF8YLsj3IUAIAP7V
 5yZ8X7MtfMpmsKA9Z7JBg0T5s/sVGo68yWxNwdUTjiWD2GWwLYxWeq5VPHYc5EjC
 0Sl8pPkXXBxAitZvsRghR+/Nduhyc2aZ06Oe/nhOydovijsq7eSsom2rV125UcXM
 DQulu+Zaqhkys5oXmrUXMmLYzz0ES1YKFRMPBjspbz6FtDTpRpkhEmvOozEiptaY
 2+kztjbLgnwN/3oeag+fL4dyD2Px537wmIzZJue+e2L4+YpSOdpbKCnr54vwZ88I
 ubcm8WPsE4FZiNm8PxuOXjRq+T/rvgucfcmNu61+X2fJU+1MpNo=
 =54xe
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/thibault/tags/samuel-thibault' into staging

slirp updates

Gerd Hoffmann (1):
  slirp: add tftp tracing

Marc-André Lureau (61):
  slirp: associate slirp_output callback with the Slirp context
  slirp: remove do_pty from fork_exec()
  slirp: replace ex_pty with ex_chardev
  slirp: use a dedicated field for chardev pointer
  slirp: remove unused EMU_RSH
  slirp: rename /extra/chardev
  slirp: move internal function declarations
  slirp: remove Monitor dependency, return a string for info
  slirp: fix slirp_add_exec() leaks
  slirp: replace the poor-man string split with g_strsplit()
  slirp: remove dead declarations
  slirp: move socket pair creation in helper function
  slirp: remove unused M_TRAILINGSPACE
  slirp: use a callback structure to interface with qemu
  slirp: remove PROBE_CONN dead-code
  slirp: remove FULL_BOLT
  slirp: remove the disabled readv()/writev() code path
  slirp: remove HAVE_SYS_SIGNAL_H
  slirp: remove unused HAVE_SYS_BITYPES_H
  slirp: remove NO_UNIX_SOCKETS
  slirp: remove unused HAVE_SYS_STROPTS_H
  slirp: remove unused HAVE_ARPA_INET_H
  slirp: remove unused HAVE_SYS_WAIT_H
  slirp: remove unused HAVE_SYS_SELECT_H
  slirp: remove HAVE_SYS_IOCTL_H
  slirp: remove HAVE_SYS_FILIO_H
  slirp: remove unused DECLARE_IOVEC
  slirp: remove unused HAVE_INET_ATON
  slirp: replace HOST_WORDS_BIGENDIAN with glib equivalent
  slirp: replace SIZEOF_CHAR_P with glib equivalent
  slirp: replace compile time DO_KEEPALIVE
  slirp: remove unused global slirp_instance
  slirp: replace error_report() with g_critical()
  slirp: improve a bit the debug macros
  slirp: add a callback to log guest errors
  slirp: remove #if notdef dead code
  slirp: remove unused sbflush()
  slirp: NULL is defined by stddef.h
  slirp: remove dead TCP_ACK_HACK code
  slirp: replace ARRAY_SIZE with G_N_ELEMENTS
  net: do not depend on slirp internals
  glib-compat: add g_spawn_async_with_fds() fallback
  slirp: simplify fork_exec()
  slirp: replace error_report() with g_critical()
  slirp: drop <Vista compatibility
  slirp: rename exec_list
  slirp: use virtual time for packet expiration
  slirp: replace a fprintf with g_critical()
  slirp: replace some fprintf() with DEBUG_MISC
  slirp: replace a DEBUG block with WITH_ICMP_ERROR_MSG
  slirp: no need to make DPRINTF conditional on DEBUG
  slirp: always build with debug statements
  slirp: introduce SLIRP_DEBUG environment variable
  slirp: use %p for pointers format
  slirp: remove remaining DEBUG blocks
  slirp: replace DEBUG_ARGS with DEBUG_ARG
  slirp: factor out guestfwd addition checks
  slirp: add clock_get_ns() callback
  build-sys: use a separate slirp-obj-y && slirp.mo
  slirp: set G_LOG_DOMAIN
  slirp: call into g_debug() for DEBUG macros

Prasad J Pandit (1):
  slirp: check data length while emulating ident function

Samuel Thibault (2):
  slirp: Enable fork_exec support on Windows
  slirp: Mark debugging calls as unlikely

 Makefile              |   5 +-
 Makefile.objs         |   4 +-
 Makefile.target       |   5 +-
 include/glib-compat.h |  56 +++++++++
 net/colo-compare.c    |  11 +-
 net/colo.c            |   1 +
 net/colo.h            |   7 +-
 net/filter-rewriter.c |   9 +-
 net/slirp.c           |  61 +++++----
 net/util.h            |  55 ++++++++
 slirp/Makefile.objs   |  37 +++++-
 slirp/arp_table.c     |  12 +-
 slirp/bootp.c         |  10 +-
 slirp/cksum.c         |   8 +-
 slirp/debug.h         |  47 ++++---
 slirp/dhcpv6.c        |  17 ++-
 slirp/if.c            |   4 +-
 slirp/ip.h            |  10 +-
 slirp/ip6.h           |   3 +-
 slirp/ip6_icmp.c      |  27 ++--
 slirp/ip6_icmp.h      |   6 +-
 slirp/ip6_input.c     |   2 +-
 slirp/ip6_output.c    |   4 +-
 slirp/ip_icmp.c       |  31 ++---
 slirp/ip_input.c      | 200 -----------------------------
 slirp/libslirp.h      |  27 ++--
 slirp/main.h          |  33 -----
 slirp/mbuf.c          |   2 +-
 slirp/mbuf.h          |   1 -
 slirp/misc.c          | 286 +++++++++++++++++-------------------------
 slirp/misc.h          |  13 +-
 slirp/ncsi.c          |   4 +-
 slirp/ndp_table.c     |  32 +++--
 slirp/sbuf.h          |   1 -
 slirp/slirp.c         | 177 +++++++++++++-------------
 slirp/slirp.h         |  45 ++-----
 slirp/slirp_config.h  |  86 -------------
 slirp/socket.c        |  53 +++-----
 slirp/socket.h        |   2 +-
 slirp/tcp.h           |   4 +-
 slirp/tcp_input.c     |  84 ++-----------
 slirp/tcp_output.c    |   2 +-
 slirp/tcp_subr.c      |  22 ++--
 slirp/tcp_timer.c     |   2 +-
 slirp/tftp.c          |   7 +-
 slirp/trace-events    |   5 +
 slirp/udp.c           |   5 +-
 slirp/udp6.c          |  11 +-
 stubs/slirp.c         |   2 +-
 49 files changed, 603 insertions(+), 935 deletions(-)
 delete mode 100644 slirp/slirp_config.h
 create mode 100644 slirp/trace-events

--
2.20.1

# gpg: Signature made Mon 14 Jan 2019 22:52:32 GMT
# gpg:                using RSA key DB550E89F0FA54F3
# gpg: Good signature from "Samuel Thibault <samuel.thibault@aquilenet.fr>"
# gpg:                 aka "Samuel Thibault <sthibault@debian.org>"
# gpg:                 aka "Samuel Thibault <samuel.thibault@gnu.org>"
# gpg:                 aka "Samuel Thibault <samuel.thibault@inria.fr>"
# gpg:                 aka "Samuel Thibault <samuel.thibault@labri.fr>"
# gpg:                 aka "Samuel Thibault <samuel.thibault@ens-lyon.org>"
# gpg:                 aka "Samuel Thibault <samuel.thibault@u-bordeaux.fr>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 900C B024 B679 31D4 0F82  304B D017 8C76 7D06 9EE6
#      Subkey fingerprint: E61D BB15 D417 2BDE C97E  92D9 DB55 0E89 F0FA 54F3

* remotes/thibault/tags/samuel-thibault: (65 commits)
  slirp: check data length while emulating ident function
  slirp: Mark debugging calls as unlikely
  slirp: call into g_debug() for DEBUG macros
  slirp: set G_LOG_DOMAIN
  build-sys: use a separate slirp-obj-y && slirp.mo
  slirp: add clock_get_ns() callback
  slirp: factor out guestfwd addition checks
  slirp: replace DEBUG_ARGS with DEBUG_ARG
  slirp: remove remaining DEBUG blocks
  slirp: use %p for pointers format
  slirp: introduce SLIRP_DEBUG environment variable
  slirp: always build with debug statements
  slirp: no need to make DPRINTF conditional on DEBUG
  slirp: replace a DEBUG block with WITH_ICMP_ERROR_MSG
  slirp: replace some fprintf() with DEBUG_MISC
  slirp: replace a fprintf with g_critical()
  slirp: use virtual time for packet expiration
  slirp: rename exec_list
  slirp: drop <Vista compatibility
  slirp: Enable fork_exec support on Windows
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-01-15 18:32:57 +00:00
Peter Maydell 4b9f0b0f7c Pull request
No user-visible changes.
 -----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJcPLmTAAoJEJykq7OBq3PIxOEH/jQItQRDlB2PCSyojNkvMpXJ
 LVnSc4a5edUUt9e+YvI3+DYX7WJ43VtxfurgOSwS2gFds92ckD9+U5CrFbxsLCoL
 7no+cC8fgNSqp+lmIX0c7nexgTLRU99Qmov3ixTKVzE6WrBjHN5fHRZlMIY8wXlE
 NCXnAzM8uoxDXRyK/a2dQ7ckGaYtXn9O8x7N+B+HyVI+0yyGArjZTZUcC2ok+Smf
 wzGfxl/iQDxLWw0kfl6adhTC2riVFUzsSshqKnDEqgpBerwT6WSiWvmxLhpetf5E
 2C4wEmGDgdt5kLC+OU3XxDfLaNAN/W0kpcszkZODstkcSrQQ2B0tcbJigzruua4=
 =rosm
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

No user-visible changes.

# gpg: Signature made Mon 14 Jan 2019 16:32:19 GMT
# gpg:                using RSA key 9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
  aio-posix: Fix concurrent aio_poll/set_fd_handler.
  aio-posix: Unregister fd from ctx epoll when removing fd_handler.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-01-15 17:24:00 +00:00
Peter Maydell 44ba601063 nbd patches for 2019-01-14
Promote bitmap/NBD interfaces to stable for use in incremental
 backups. Add 'qemu-nbd --bitmap'.
 
 - John Snow: 0/11 bitmaps: remove x- prefix from QMP api
 - Philippe Mathieu-Daudé: qemu-nbd: Rename 'exp' variable clashing with math::exp() symbol
 - Eric Blake: 0/8 Promote x-nbd-server-add-bitmap to stable
 -----BEGIN PGP SIGNATURE-----
 
 iQEcBAABCAAGBQJcPLU5AAoJEKeha0olJ0Nqt4AH/2JXm2Ky9lth8QdVzPodh0cl
 xeBteQ51LVjneAlQuT5u5LR6+0ujljVCDUPaUfAcxhWJagofzPVIdSe+kOnQEtgN
 e7zf1pFxvLkITTl2iu0/HFxBz1pJb+ZayZqZ5zOHfj7LVSafwNC7MbW/t5yueiXl
 fPbxZqF4+C7lN9woS4p6zgB5berQuWHdHxsLLbkxFigr9JRiH3t8qm1tsa0nd/AZ
 uCyVAzqxFBkchh86uZ8TisUU7XqR+ab99/feOEkMWj2ctra7HfkJeCkPOZialzOS
 5A4UBZSDzCwr10NtHBMQhvSmvBFghm4L3u9faejg81TTTpbIQtcUoXu+ClGIHVM=
 =cYBw
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-01-14' into staging

nbd patches for 2019-01-14

Promote bitmap/NBD interfaces to stable for use in incremental
backups. Add 'qemu-nbd --bitmap'.

- John Snow: 0/11 bitmaps: remove x- prefix from QMP api
- Philippe Mathieu-Daudé: qemu-nbd: Rename 'exp' variable clashing with math::exp() symbol
- Eric Blake: 0/8 Promote x-nbd-server-add-bitmap to stable

# gpg: Signature made Mon 14 Jan 2019 16:13:45 GMT
# gpg:                using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>"
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>"
# gpg:                 aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-01-14:
  qemu-nbd: Add --bitmap=NAME option
  nbd: Merge nbd_export_bitmap into nbd_export_new
  nbd: Remove x-nbd-server-add-bitmap
  nbd: Allow bitmap export during QMP nbd-server-add
  nbd: Merge nbd_export_set_name into nbd_export_new
  nbd: Only require disabled bitmap for read-only exports
  nbd: Forbid nbd-server-stop when server is not running
  nbd: Add some error case testing to iotests 223
  qemu-nbd: Rename 'exp' variable clashing with math::exp() symbol
  iotests: add iotest 236 for testing bitmap merge
  iotests: implement pretty-print for log and qmp_log
  iotests: change qmp_log filters to expect QMP objects only
  iotests: remove default filters from qmp_log
  iotests: add qmp recursive sorting function
  iotests: add filter_generated_node_ids
  iotests.py: don't abort if IMGKEYSECRET is undefined
  block: remove 'x' prefix from experimental bitmap APIs
  blockdev: n-ary bitmap merge
  block/dirty-bitmap: remove assertion from restore
  blockdev: abort transactions in reverse order

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-01-15 14:19:18 +00:00
Peter Maydell b2f7c27f56 gitdm updates with 2018 year end stats:
git log --numstat --after="1/1/2018 00:00" --before="31/12/2018 23:59" | ~/src/gitdm.git/gitdm -n -l 10
 
   Top changeset contributors by employer
   Red Hat                   3091 (43.3%)
   Linaro                    1201 (16.8%)
   (None)                     484 (6.8%)
   IBM                        426 (6.0%)
   Academics (various)        186 (2.6%)
   Virtuozzo                  172 (2.4%)
   Wave Computing             118 (1.7%)
   Igalia                     109 (1.5%)
   Xilinx                     102 (1.4%)
   Cadence Design Systems      80 (1.1%)
 
   Top lines changed by employer
   Red Hat                   140523 (30.3%)
   Cadence Design Systems    81010 (17.5%)
   Linaro                    78098 (16.8%)
   Wave Computing            33134 (7.1%)
   IBM                       18918 (4.1%)
   SiFive                    14436 (3.1%)
   Academics (various)       11995 (2.6%)
   (None)                    11458 (2.5%)
   Virtuozzo                 10770 (2.3%)
   Oracle                    6698 (1.4%)
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAlw8tBQACgkQ+9DbCVqe
 KkTqwgf/bRYA/G4IxQdId2rshuvXbNOtRfl5hSX1jBH6DpiPMQTWQRszjrEUYrAH
 yRleRTgC/wpQZKF3IW6EdbIf5IWM08bTJ4nx9VaTD8c6RJDt+S6dmFGSSZFH2YQa
 Gngt06K43SBfFBUMUBk/TJmThlhqv72UBBAEnCN6xax+8281KaESdgAmGCZTL0nn
 EuGZYscQXB1JrUxIWI/gNFss+hcpym+zK10GbmnEojjLONLmOk2mlDwfluFWBiiS
 x0awcJLl2eAc8M6dv7o+pQ5h81whndbWt/Oj5qTVh+rqTHzTWrEMPvf6CAsD2HWK
 QrWjxM5BzEhTuv905HOBuWyWHdUd9A==
 =YG3/
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-misc-gitdm-next-140119-1' into staging

gitdm updates with 2018 year end stats:

  git log --numstat --after="1/1/2018 00:00" --before="31/12/2018 23:59" | ~/src/gitdm.git/gitdm -n -l 10

  Top changeset contributors by employer
  Red Hat                   3091 (43.3%)
  Linaro                    1201 (16.8%)
  (None)                     484 (6.8%)
  IBM                        426 (6.0%)
  Academics (various)        186 (2.6%)
  Virtuozzo                  172 (2.4%)
  Wave Computing             118 (1.7%)
  Igalia                     109 (1.5%)
  Xilinx                     102 (1.4%)
  Cadence Design Systems      80 (1.1%)

  Top lines changed by employer
  Red Hat                   140523 (30.3%)
  Cadence Design Systems    81010 (17.5%)
  Linaro                    78098 (16.8%)
  Wave Computing            33134 (7.1%)
  IBM                       18918 (4.1%)
  SiFive                    14436 (3.1%)
  Academics (various)       11995 (2.6%)
  (None)                    11458 (2.5%)
  Virtuozzo                 10770 (2.3%)
  Oracle                    6698 (1.4%)

# gpg: Signature made Mon 14 Jan 2019 16:08:52 GMT
# gpg:                using RSA key FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>"
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-misc-gitdm-next-140119-1:
  MAINTAINERS: add myself as a route for gitdm updates
  contrib/gitdm: add another name to WaveComp map
  contrib/gitdm: add two more IBM'ers to group-map-ibm
  contrib/gitdm: Add other IBMers
  contrib/gitdm: add Nokia and Proxmox to the domain-map

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-01-15 13:16:58 +00:00
Prasad J Pandit a7104eda7d slirp: check data length while emulating ident function
While emulating identification protocol, tcp_emu() does not check
available space in the 'sc_rcv->sb_data' buffer. It could lead to
heap buffer overflow issue. Add check to avoid it.

Reported-by: Kira <864786842@qq.com>
Signed-off-by: Prasad J Pandit <pjp@fedoraproject.org>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
2019-01-14 23:36:38 +01:00
Samuel Thibault 55ef9c617b slirp: Mark debugging calls as unlikely
to get them out of the hot path.

Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2019-01-14 23:36:38 +01:00
Marc-André Lureau 226ea7a964 slirp: call into g_debug() for DEBUG macros
Make slirp use GLib logging, instead of fprintf(), so that
applications can filter log, process it etc.

With recent versions of glib, G_MESSAGES_DEBUG must be set to "all" or
"Slirp" to see slirp debug messages.

Reformat DEBUG_MISC & DEBUG_ERROR calls to not need \n ending.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
2019-01-14 23:36:38 +01:00
Marc-André Lureau 2a2d3e4a25 slirp: set G_LOG_DOMAIN
We are moving to g_log() facilities to log errors and probably debug
messages too. Let's have the "Slirp" prefix on messages slirp produces.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
2019-01-14 23:36:38 +01:00
Marc-André Lureau 1fdf39927f build-sys: use a separate slirp-obj-y && slirp.mo
This will allow to have cflags for the whole slirp.mo -objs.
It makes it possible to build tests that links only with
slirp-obj-y (and not the whole common-obj).

It is also a step towards building slirp as a shared library, although
this requires a bit more thoughts to build with
net/slirp.o (CONFIG_SLIRP would need to be 'm') and other build issues.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
2019-01-14 23:36:32 +01:00
Peter Maydell eb0667fe7f A bunch of fixes for testing:
- Various Travis updates
   - "stable" SID snapshot for docker
   - avoid :latest docker tags
   - g_usleep fix for some tests
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAlw8o9cACgkQ+9DbCVqe
 KkSmMQf/TRnnbaf/SIrYpAA6yGcAJtEnAU8f/CH3idAkEPv3+UtTkdSHor1u3A2G
 BJaplFhYTOL/BwjUl1sctGYa8jVR6e1SS2rLRtIG7d+DyvmpFJNx9gKEdSkAuwbk
 BVNElqFfKCezQeM9IrjhzXd7InoW7PNQMrKRrMvuECddLc0Ds9CTvOSY1WUpOlDh
 rTEjvVhPu8LdZkhie/0D+zAzv8++iUDdImd1w7m3ilVPtyMflhdtaQ1Qpv6M/ldW
 cmbIIiGq2xsC126Piyq2gguCNV8nt/a8vGavkCCIXNZ07eWVRxoMuGzlvVBhX9qV
 oll92Fs9R8ZBROz5qpt/Uqt1+rBjNg==
 =XPd+
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-next-140119-1' into staging

A bunch of fixes for testing:

  - Various Travis updates
  - "stable" SID snapshot for docker
  - avoid :latest docker tags
  - g_usleep fix for some tests

# gpg: Signature made Mon 14 Jan 2019 14:59:35 GMT
# gpg:                using RSA key FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>"
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-next-140119-1: (21 commits)
  Revert "tests: Disable qht-bench parallel test when using gprof"
  tests: use g_usleep instead of rem = sleep(time)
  tests/docker: remove SID_AGE test hack
  tests/docker: update our Travis image
  travis: bump to Xenial baseline
  docker: Use a stable snapshot for Debian Sid
  travis: remove matrix settings that duplicate global settings
  travis: run tests in verbose mode
  travis: stop using container based envs
  travis: stop redefining the script commands
  travis: use homebrew addon for MacOSX
  travis: don't clone git submodules upfront
  travis: standardize the syntax used for env variables
  travis: define all the build matrix entries in one place
  travis: add whitespace between each major section & matrix entry
  tests: use in-place sed magic for enabling deb-src in travis image
  tests: update Fedora i386 cross image to Fedora 29
  tests: update Fedora dockerfile to use Fedora 29
  tests: remove obsolete 'debian' dockerfile
  tests: run ldconfig after installing extra software
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-01-14 19:28:12 +00:00
Peter Maydell 89bd861c2b x86 queue, 2019-01-14
* Reenable RDTSCP support on Opteron_G[345] CPU models CPU models
   (Borislav Petkov)
 * host-phys-bits-limit option for better control of 5-level EPT
   (Eduardo Habkost)
 * Disable MPX support on named CPU models (Paolo Bonzini)
 * expose HV_CPUID_ENLIGHTMENT_INFO.EAX and HV_CPUID_NESTED_FEATURES.EAX
   as feature words (Vitaly Kuznetsov)
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABCAAGBQJcPJ3TAAoJECgHk2+YTcWmNlkP/1zlFjCo1em9RLGxeZrkzFna
 ugH/tZxqWQb3MfzNSsj51DGy2NB2WT2BaCtJKWjhzdDTLqveFaFCUKfhGyYETIHU
 OgNbUCjxBkv3vvWJ8c8w7pcc/atPZL7OCqIEbgJ/SwcSl0hF3bJrRJYYuA5dYeWx
 KewMRinFy3VEa+qlTGSAvpyp5C6XxIOPI6COQvu/CVlLmdlj+gURbaTtWHpZ3USU
 tTxSJLjbAPUlULAk57Wh2r5LpEgGzuBts9VosiQWzREz03rqYTk6JPjyxQKEasaz
 2lhjQWriRwcPNpgSSRfdnwMxlKisgLwWf/A9StoNjHWlzcTn6HR7HFP5O6YgKcEV
 EkAhu9UGeltqOHGJPReO6y5J8VHHj7FJZE5PfDE3mYKgmEsn+1O7gOr9amy5PCT9
 a7zozGHWOqJMb05XbB4CxENmgoNRKE9rMhwe5Jjc9dxrHOtgcXoUO9YNYvPE6ZyC
 6GxKdn0rAzZldwuFj8orZZeYLUlfoV0frNWfNKi8FdBad7z5HalVtB7jjaWRw6eI
 mOabTZPV6aaiyHgiR/YGBd3h9kWex9zJWxVDZMoRRFBNFSs8HFKzbVrwv/g6e83B
 2ZUrQ4MLI5cgU8zZ2XdTPx7SVV+Nxr6wWXtx55Sn9LB8IGMdEqZ06W+iZfZgF2PV
 miQR5IsbOapmIBssV1Mo
 =S08V
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ehabkost/tags/x86-next-pull-request' into staging

x86 queue, 2019-01-14

* Reenable RDTSCP support on Opteron_G[345] CPU models CPU models
  (Borislav Petkov)
* host-phys-bits-limit option for better control of 5-level EPT
  (Eduardo Habkost)
* Disable MPX support on named CPU models (Paolo Bonzini)
* expose HV_CPUID_ENLIGHTMENT_INFO.EAX and HV_CPUID_NESTED_FEATURES.EAX
  as feature words (Vitaly Kuznetsov)

# gpg: Signature made Mon 14 Jan 2019 14:33:55 GMT
# gpg:                using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>"
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/x86-next-pull-request:
  i386/kvm: add a comment explaining why .feat_names are commented out for Hyper-V feature bits
  x86: host-phys-bits-limit option
  target/i386: Disable MPX support on named CPU models
  target-i386: Reenable RDTSCP support on Opteron_G[345] CPU models CPU models
  i386/kvm: expose HV_CPUID_ENLIGHTMENT_INFO.EAX and HV_CPUID_NESTED_FEATURES.EAX as feature words

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-01-14 17:35:00 +00:00
Eric Blake 636192c4b6 qemu-nbd: Add --bitmap=NAME option
Having to fire up qemu, then use QMP commands for nbd-server-start
and nbd-server-add, just to expose a persistent dirty bitmap, is
rather tedious.  Make it possible to expose a dirty bitmap using
just qemu-nbd (of course, for now this only works when qemu-nbd is
visiting a BDS formatted as qcow2).

Of course, any good feature also needs unit testing, so expand
iotest 223 to cover it.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190111194720.15671-9-eblake@redhat.com>
2019-01-14 10:09:46 -06:00
Eric Blake 678ba275c7 nbd: Merge nbd_export_bitmap into nbd_export_new
We only have one caller that wants to export a bitmap name,
which it does right after creation of the export. But there is
still a brief window of time where an NBD client could see the
export but not the dirty bitmap, which a robust client would
have to interpret as meaning the entire image should be treated
as dirty.  Better is to eliminate the window entirely, by
inlining nbd_export_bitmap() into nbd_export_new(), and refusing
to create the bitmap in the first place if the requested bitmap
can't be located.

We also no longer need logic for setting a different bitmap
name compared to the bitmap being exported.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20190111194720.15671-8-eblake@redhat.com>
2019-01-14 10:09:46 -06:00
Eric Blake 7dc570b380 nbd: Remove x-nbd-server-add-bitmap
Now that nbd-server-add can do the same functionality (well, other
than making the exported bitmap name different than the underlying
bitamp - but we argued that was not essential, since it is just as
easy to create a new non-persistent bitmap with the desired name),
we no longer need the experimental separate command.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20190111194720.15671-7-eblake@redhat.com>
2019-01-14 10:09:46 -06:00
Eric Blake 5fcbeb0681 nbd: Allow bitmap export during QMP nbd-server-add
With the experimental x-nbd-server-add-bitmap command, there was
a window of time where an NBD client could see the export but not
the associated dirty bitmap, which can cause a client that planned
on using the dirty bitmap to be forced to treat the entire image
as dirty as a safety fallback.  Furthermore, if the QMP client
successfully exports a disk but then fails to add the bitmap, it
has to take on the burden of removing the export.  Since we don't
allow changing the exposed dirty bitmap (whether to a different
bitmap, or removing advertisement of the bitmap), it is nicer to
make the bitmap tied to the export at the time the export is
created, with automatic failure to export if the bitmap is not
available.

The experimental command included an optional 'bitmap-export-name'
field for remapping the name exposed over NBD to be different from
the bitmap name stored on disk.  However, my libvirt demo code
for implementing differential backups on top of persistent bitmaps
did not need to take advantage of that feature (it is instead
possible to create a new temporary bitmap with the desired name,
use block-dirty-bitmap-merge to merge one or more persistent
bitmaps into the temporary, then associate the temporary with the
NBD export, if control is needed over the exported bitmap name).
Hence, I'm not copying that part of the experiment over to the
stable addition. For more details on the libvirt demo, see
https://www.redhat.com/archives/libvir-list/2018-October/msg01254.html,
https://kvmforum2018.sched.com/event/FzuB/facilitating-incremental-backup-eric-blake-red-hat

This patch focuses on the user interface, and reduces (but does
not completely eliminate) the window where an NBD client can see
the export but not the dirty bitmap, with less work to clean up
after errors.  Later patches will add further cleanups now that
this interface is declared stable via a single QMP command,
including removing the race window.

Update test 223 to use the new interface.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20190111194720.15671-6-eblake@redhat.com>
2019-01-14 10:09:46 -06:00
Eric Blake 3fa4c76590 nbd: Merge nbd_export_set_name into nbd_export_new
The existing NBD code had a weird split where nbd_export_new()
created an export but did not add it to the list of exported
names until a later nbd_export_set_name() came along and grabbed
a second reference on the object; later, the first call to
nbd_export_close() drops the second reference while removing
the export from the list.  This is in part because the QAPI
NbdServerRemoveNode enum documents the possibility of adding a
mode where we could do a soft disconnect: preventing new clients,
but waiting for existing clients to gracefully quit, based on
the mode used when calling nbd_export_close().

But in spite of all that, note that we never change the name of
an NBD export while it is exposed, which means it is easier to
just inline the process of setting the name as part of creating
the export.

Inline the contents of nbd_export_set_name() and
nbd_export_set_description() into the two points in an export
lifecycle where they matter, then adjust both callers to pass
the name up front.  Note that for creation, all callers pass a
non-NULL name, (passing NULL at creation was for old style
servers, but we removed support for that in commit 7f7dfe2a),
so we can add an assert and do things unconditionally; but for
cleanup, because of the dual nature of nbd_export_close(), we
still have to be careful to avoid use-after-free.  Along the
way, add a comment reminding ourselves of the potential of
adding a middle mode disconnect.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20190111194720.15671-5-eblake@redhat.com>
2019-01-14 10:09:46 -06:00
Eric Blake 702aa50d61 nbd: Only require disabled bitmap for read-only exports
Our initial implementation of x-nbd-server-add-bitmap put
in a restriction because of incremental backups: in that
usage, we are exporting one qcow2 file (the temporary overlay
target of a blockdev-backup sync:none job) and a dirty bitmap
owned by a second qcow2 file (the source of the
blockdev-backup, which is the backing file of the temporary).
While both qcow2 files are still writable (the target in
order to capture copy-on-write of old contents, and the
source in order to track live guest writes in the meantime),
the NBD client expects to see constant data, including the
dirty bitmap.  An enabled bitmap in the source would be
modified by guest writes, which is at odds with the NBD
export being a read-only constant view, hence the initial
code choice of enforcing a disabled bitmap (the intent is
that the exposed bitmap was disabled in the same transaction
that started the blockdev-backup job, although we don't want
to track enough state to actually enforce that).

However, consider the case of a bitmap contained in a read-only
node (including when the bitmap is found in a backing layer of
the active image).  Because the node can't be modified, the
bitmap won't change due to writes, regardless of whether it is
still enabled.  Forbidding the export unless the bitmap is
disabled is awkward, paritcularly since we can't change the
bitmap to be disabled (because the node is read-only).

Alternatively, consider the case of live storage migration,
where management directs the destination to create a writable
NBD server, then performs a drive-mirror from the source to
the target, prior to doing the rest of the live migration.
Since storage migration can be time-consuming, it may be wise
to let the destination include a dirty bitmap to track which
portions it has already received, where even if the migration
is interrupted and restarted, the source can query the
destination block status in order to potentially minimize
re-sending data that has not changed in the meantime on a
second attempt. Such code has not been written, and might not
be trivial (after all, a cluster being marked dirty in the
bitmap does not necessarily guarantee it has the desired
contents), but it makes sense that letting an active dirty
bitmap be exposed and changing alongside writes may prove
useful in the future.

Solve both issues by gating the restriction against a
disabled bitmap to only happen when the caller has requested
a read-only export, and where the BDS that owns the bitmap
(whether or not it is the BDS handed to nbd_export_new() or
from its backing chain) is still writable.  We could drop
the check altogether (if management apps are prepared to
deal with a changing bitmap even on a read-only image), but
for now keeping a check for the read-only case still stands
a chance of preventing management errors.

Update iotest 223 to show the looser behavior by leaving
a bitmap enabled the whole run; note that we have to tear
down and re-export a node when handling an error.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190111194720.15671-4-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-14 10:09:46 -06:00
Eric Blake 7801c3a7fd nbd: Forbid nbd-server-stop when server is not running
Since we already forbid other nbd-server commands when not
in the right state, it is unlikely that any caller was relying
on a second stop to behave as a silent no-op.  Update iotest
223 to show the improved behavior.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190111194720.15671-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-14 10:09:46 -06:00
Eric Blake 2d2fd67428 nbd: Add some error case testing to iotests 223
Testing success paths is important, but it's also nice to highlight
expected failure handling, to show that we don't crash, and so that
upcoming tests that change behavior can demonstrate the resulting
effects on error paths.

Add the following errors:
Attempting to export without a running server
Attempting to start a second server
Attempting to export a bad node name
Attempting to export a name that is already exported
Attempting to export an enabled bitmap
Attempting to remove an already removed export
Attempting to quit server a second time

All of these properly complain except for a second server-stop,
which will be fixed next.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190111194720.15671-2-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-01-14 10:09:46 -06:00
Philippe Mathieu-Daudé 9d97658020 qemu-nbd: Rename 'exp' variable clashing with math::exp() symbol
The use of a variable named 'exp' prevents includes to import <math.h>.

Rename it to avoid:

  qemu-nbd.c:64:19: error: ‘exp’ redeclared as different kind of symbol
   static NBDExport *exp;
                     ^~~
  In file included from /usr/include/features.h:428,
                   from /usr/include/bits/libc-header-start.h:33,
                   from /usr/include/stdint.h:26,
                   from /usr/lib/gcc/x86_64-redhat-linux/8/include/stdint.h:9,
                   from /source/qemu/include/qemu/osdep.h:80,
                   from /source/qemu/qemu-nbd.c:19:
  /usr/include/bits/mathcalls.h:95:1: note: previous declaration of ‘exp’ was here
    __MATHCALL_VEC (exp,, (_Mdouble_ __x));
    ^~~~~~~~~~~~~~

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190111163519.11457-1-philmd@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2019-01-14 10:09:46 -06:00
John Snow 14da540f2a iotests: add iotest 236 for testing bitmap merge
New interface, new smoke test.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20181221093529.23855-12-jsnow@redhat.com>
[eblake: fix last-minute change to echo text]
Signed-off-by: Eric Blake <eblake@redhat.com>
2019-01-14 10:09:46 -06:00
John Snow 55cd64eab5 iotests: implement pretty-print for log and qmp_log
If iotests have lines exceeding >998 characters long, git doesn't
want to send it plaintext to the list. We can solve this by allowing
the iotests to use pretty printed QMP output that we can match against
instead.

As a bonus, it's much nicer for human eyes too.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181221093529.23855-11-jsnow@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2019-01-14 10:09:46 -06:00
John Snow 08fcd6111e iotests: change qmp_log filters to expect QMP objects only
As laid out in the previous commit's message:

```
Several places in iotests deal with serializing objects into JSON
strings, but to add pretty-printing it seems desirable to localize
all of those cases.

log() seems like a good candidate for that centralized behavior.
log() can already serialize json objects, but when it does so,
it assumes filters=[] operates on QMP objects, not strings.

qmp_log currently operates by dumping outgoing and incoming QMP
objects into strings and filtering them assuming that filters=[]
are string filters.
```

Therefore:

Change qmp_log to treat filters as if they're always qmp object filters,
then change the logging call to rely on log()'s ability to serialize QMP
objects, so we're not duplicating that effort.

Add a qmp version of filter_testfiles and adjust the only caller using
it for qmp_log to use the qmp version.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20181221093529.23855-10-jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2019-01-14 10:09:46 -06:00
John Snow f8ca8609d8 iotests: remove default filters from qmp_log
Several places in iotests deal with serializing objects into JSON
strings, but to add pretty-printing it seems desirable to localize
all of those cases.

log() seems like a good candidate for that centralized behavior.
log() can already serialize json objects, but when it does so,
it assumes filters=[] operates on QMP objects, not strings.

qmp_log currently operates by dumping outgoing and incoming QMP
objects into strings and filtering them assuming that filters=[]
are string filters.

To have qmp_log use log's serialization, qmp_log will need to
accept only qmp filters, not text filters.

However, only a single caller of qmp_log actually requires any
filters at all. I remove the default filter and add it explicitly
to the caller in preparation for refactoring qmp_log to use rich
filters instead.

test 206 is amended to name the filter explicitly and the default
is removed.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181221093529.23855-9-jsnow@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2019-01-14 10:09:46 -06:00
John Snow 0706e87d72 iotests: add qmp recursive sorting function
Python before 3.6 does not sort dictionaries (including kwargs).
Therefore, printing QMP objects involves sorting the keys to have
a predictable ordering in the iotests output. This means that
iotests output will sometimes show arguments in an order not
specified by the test author.

Presently, we accomplish this by using json.dumps' sort_keys argument,
where we only serialize the arguments dictionary, but not the command.

However, if we want to pretty-print QMP objects being sent to the
QEMU process, we need to build the entire command before logging it.
Ordinarily, this would then involve "arguments" being sorted above
"execute", which would necessitate a rather ugly and harder-to-read
change to many iotests outputs.

To facilitate pretty-printing AND maintaining predictable output AND
having "arguments" sort after "execute", add a custom sort function
that takes a dictionary and recursively builds an OrderedDict that
maintains the specific key order we wish to see in iotests output.

The qmp_log function uses this to build a QMP object that keeps
"execute" above "arguments", but sorts all keys and keys in any
subdicts in "arguments" lexicographically to maintain consistent
iotests output, with no incompatible changes to any current test.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181221093529.23855-8-jsnow@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2019-01-14 10:09:46 -06:00
John Snow fa1151f811 iotests: add filter_generated_node_ids
To mimic the common filter of the same name, but for the python tests.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181221093529.23855-7-jsnow@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2019-01-14 10:09:46 -06:00
John Snow 58ebcb65d8 iotests.py: don't abort if IMGKEYSECRET is undefined
Instead of using os.environ[], use .get with a default of empty string
to match the setup in check to allow us to import the iotests module
(for debugging, say) without needing a crafted environment just to
import the module.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181221093529.23855-6-jsnow@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2019-01-14 10:09:46 -06:00
John Snow 0e2b7f0983 block: remove 'x' prefix from experimental bitmap APIs
The 'x' prefix was added because I was uncertain of the direction we'd
take for the libvirt API. With the general approach solidified, I feel
comfortable committing to this API for 4.0.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181221093529.23855-5-jsnow@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2019-01-14 10:09:46 -06:00
John Snow 360d4e4e9a blockdev: n-ary bitmap merge
Especially outside of transactions, it is helpful to provide
all-or-nothing semantics for bitmap merges. This facilitates
the coalescing of multiple bitmaps into a single target for
the "checkpoint" interpretation when assembling bitmaps that
represent arbitrary points in time from component bitmaps.

This is an incompatible change from the preliminary version
of the API.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181221093529.23855-4-jsnow@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2019-01-14 10:09:46 -06:00
John Snow 07d5a8df6a block/dirty-bitmap: remove assertion from restore
When making a backup of a dirty bitmap (for transactions), we want to
restore that backup whether or not the bitmap is enabled.

It is perfectly valid to write into bitmaps that are disabled. It is
only illegitimate for the guest to have done so.

Remove this assertion.

Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20181221093529.23855-3-jsnow@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2019-01-14 10:09:46 -06:00
John Snow f4de0f8c40 blockdev: abort transactions in reverse order
Presently, we abort transactions in the same order they were processed in.
Bitmap commands, though, attempt to restore backup data structures on abort.

That's not valid, they need to be aborted in reverse chronological order.

Replace the QSIMPLEQ data structure with a QTAILQ one, so we can iterate
in reverse for the abort phase of the transaction.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181221093529.23855-2-jsnow@redhat.com>
[eblake: rebase]
Signed-off-by: Eric Blake <eblake@redhat.com>
2019-01-14 10:08:47 -06:00
Alex Bennée 92329a7e70 MAINTAINERS: add myself as a route for gitdm updates
Seeing as I'll get pegged by get_maintainers.pl anyway I might as well
make the support status of the data mining config official.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-01-14 16:06:57 +00:00
Alex Bennée cb8d955df7 contrib/gitdm: add another name to WaveComp map
Requested by Aleksandar.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-01-14 16:06:56 +00:00
Alex Bennée f7329232b7 contrib/gitdm: add two more IBM'ers to group-map-ibm
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
2019-01-14 16:06:56 +00:00
Joel Stanley 843c14831f contrib/gitdm: Add other IBMers
Here are some IBMers who use their personal addresses when submitting
patches.

Signed-off-by: Joel Stanley <joel@jms.id.au>
Acked-by: Andrew Jeffery <andrew@aj.id.au>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
2019-01-14 16:06:56 +00:00
Alex Bennée 3f209fc7c4 contrib/gitdm: add Nokia and Proxmox to the domain-map
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
2019-01-14 16:06:56 +00:00
Alex Bennée a36270a4d1 Revert "tests: Disable qht-bench parallel test when using gprof"
This reverts commit ce2eefd7c2. The
underlying cause was fixed with eb4f8e100f.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-01-14 14:55:32 +00:00
Alex Bennée eb4f8e100f tests: use g_usleep instead of rem = sleep(time)
Relying on sleep to always return having slept isn't safe as a signal
may have occurred. If signals are constantly incoming the program will
never reach its termination condition. This is believed to be the
mechanism causing time outs for qht-test in Travis.

The glib g_usleep() deals with all of this for us so lets use it instead.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
2019-01-14 14:52:30 +00:00