Commit Graph

11224 Commits

Author SHA1 Message Date
Peter Maydell 1ed9228f63 nbd patches for 2021-02-02
- more cleanup from iotest python conversion
 - progress towards consistent use of signed 64-bit types through block layer
 - fix some crashes related to NBD reconnect
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAmAasREACgkQp6FrSiUn
 Q2q0bAgAqkkrQtMhTcQ6cZenLhBc7qIojOybnOnO8hXb7SpndHjE9f9OF0f0+OOm
 9vk24KyohvCFc7IwMHA/lccvHYH4Ftb8VPgYEs+U+iCxzdyM8M1goy8tbOp82Fgg
 8ahxRbBP4VTkBUiXNcCxO2KbBQL2m+YkLKvfbw+XN0KbxER6hoikfNsSty97/3su
 ZKtfEqBPH7Tl7kJz0JBfGTpCSPa6cXFzIwxK+m8pq7IKeNrICg1Uu0VR49QfTWlh
 2ptIxvKx+QkxLV+QoW7m/fqXFaIFye0tQAwTr/5b3/jBtAkgg8Yznn0TMgiqwXMr
 SuxbUpjNWqBtjjhwavSje020VhrD2w==
 =7eUe
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2021-02-02-v2' into staging

nbd patches for 2021-02-02

- more cleanup from iotest python conversion
- progress towards consistent use of signed 64-bit types through block layer
- fix some crashes related to NBD reconnect

# gpg: Signature made Wed 03 Feb 2021 14:20:01 GMT
# gpg:                using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg:                 aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2021-02-02-v2:
  nbd: make nbd_read* return -EIO on error
  block/nbd: only enter connection coroutine if it's present
  block/nbd: only detach existing iochannel from aio_context
  block/io: use int64_t bytes in copy_range
  block/io: support int64_t bytes in read/write wrappers
  block/io: support int64_t bytes in bdrv_co_p{read,write}v_part()
  block/io: support int64_t bytes in bdrv_aligned_preadv()
  block/io: support int64_t bytes in bdrv_co_do_copy_on_readv()
  block/io: support int64_t bytes in bdrv_aligned_pwritev()
  block/io: support int64_t bytes in bdrv_co_do_pwrite_zeroes()
  block/io: use int64_t bytes in driver wrappers
  block: use int64_t as bytes type in tracked requests
  block/io: improve bdrv_check_request: check qiov too
  block/throttle-groups: throttle_group_co_io_limits_intercept(): 64bit bytes
  block/io: bdrv_pad_request(): support qemu_iovec_init_extended failure
  block/io: refactor bdrv_pad_request(): move bdrv_pad_request() up
  block: fix theoretical overflow in bdrv_init_padding()
  util/iov: make qemu_iovec_init_extended() honest
  block: refactor bdrv_check_request: add errp
  iotests: Fix expected whitespace for 185

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-03 14:52:12 +00:00
Roman Kagan 5082fc82a6 nbd: make nbd_read* return -EIO on error
NBD reconnect logic considers the error code from the functions that
read NBD messages to tell if reconnect should be attempted or not: it is
attempted on -EIO, otherwise the client transitions to NBD_CLIENT_QUIT
state (see nbd_channel_error).  This error code is propagated from the
primitives like nbd_read.

The problem, however, is that nbd_read itself turns every error into -1
rather than -EIO.  As a result, if the NBD server happens to die while
sending the message, the client in QEMU receives less data than it
expects, considers it as a fatal error, and wouldn't attempt
reestablishing the connection.

Fix it by turning every negative return from qio_channel_read_all into
-EIO returned from nbd_read.  Apparently that was the original behavior,
but got broken later.  Also adjust nbd_readXX to follow.

Fixes: e6798f06a6 ("nbd: generalize usage of nbd_read")
Signed-off-by: Roman Kagan <rvkagan@yandex-team.ru>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210129073859.683063-4-rvkagan@yandex-team.ru>
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-02-03 08:17:12 -06:00
Vladimir Sementsov-Ogievskiy a5215b8fdf block/io: use int64_t bytes in copy_range
We are generally moving to int64_t for both offset and bytes parameters
on all io paths.

Main motivation is realization of 64-bit write_zeroes operation for
fast zeroing large disk chunks, up to the whole disk.

We chose signed type, to be consistent with off_t (which is signed) and
with possibility for signed return type (where negative value means
error).

So, convert now copy_range parameters which are already 64bit to signed
type.

It's safe as we don't work with requests overflowing BDRV_MAX_LENGTH
(which is less than INT64_MAX), and do check the requests in
bdrv_co_copy_range_internal() (by bdrv_check_request32(), which calls
bdrv_check_request()).

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201211183934.169161-17-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-02-03 08:17:12 -06:00
Vladimir Sementsov-Ogievskiy e9e52efdc5 block/io: support int64_t bytes in read/write wrappers
We are generally moving to int64_t for both offset and bytes parameters
on all io paths.

Main motivation is realization of 64-bit write_zeroes operation for
fast zeroing large disk chunks, up to the whole disk.

We chose signed type, to be consistent with off_t (which is signed) and
with possibility for signed return type (where negative value means
error).

Now, since bdrv_co_preadv_part() and bdrv_co_pwritev_part() have been
updated, update all their wrappers.

For all of them type of 'bytes' is widening, so callers are safe. We
have update request_fn in blkverify.c simultaneously. Still it's just a
pointer to one of bdrv_co_pwritev() or bdrv_co_preadv(), and type is
widening for callers of the request_fn anyway.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201211183934.169161-16-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: grammar tweak]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-02-03 08:17:12 -06:00
Vladimir Sementsov-Ogievskiy 37e9403ea8 block/io: support int64_t bytes in bdrv_co_p{read,write}v_part()
We are generally moving to int64_t for both offset and bytes parameters
on all io paths.

Main motivation is realization of 64-bit write_zeroes operation for
fast zeroing large disk chunks, up to the whole disk.

We chose signed type, to be consistent with off_t (which is signed) and
with possibility for signed return type (where negative value means
error).

So, prepare bdrv_co_preadv_part() and bdrv_co_pwritev_part() and their
remaining dependencies now.

bdrv_pad_request() is updated simultaneously, as pointer to bytes passed
to it both from bdrv_co_pwritev_part() and bdrv_co_preadv_part().

So, all callers of bdrv_pad_request() are updated to pass 64bit bytes.
bdrv_pad_request() is already good for 64bit requests, add
corresponding assertion.

Look at bdrv_co_preadv_part() and bdrv_co_pwritev_part().
Type is widening, so callers are safe. Let's look inside the functions.

In bdrv_co_preadv_part() and bdrv_aligned_pwritev() we only pass bytes
to other already int64_t interfaces (and some obviously safe
calculations), it's OK.

In bdrv_co_do_zero_pwritev() aligned_bytes may become large now, still
it's passed to bdrv_aligned_pwritev which supports int64_t bytes.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201211183934.169161-15-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-02-03 08:17:11 -06:00
Eric Blake 8024726459 block: use int64_t as bytes type in tracked requests
We are generally moving to int64_t for both offset and bytes parameters
on all io paths.

Main motivation is realization of 64-bit write_zeroes operation for
fast zeroing large disk chunks, up to the whole disk.

We chose signed type, to be consistent with off_t (which is signed) and
with possibility for signed return type (where negative value means
error).

All requests in block/io must not overflow BDRV_MAX_LENGTH, all
external users of BdrvTrackedRequest already have corresponding
assertions, so we are safe. Add some assertions still.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201211183934.169161-9-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-02-03 08:14:15 -06:00
Vladimir Sementsov-Ogievskiy 801625e69d block/throttle-groups: throttle_group_co_io_limits_intercept(): 64bit bytes
The function is called from 64bit io handlers, and bytes is just passed
to throttle_account() which is 64bit too (unsigned though). So, let's
convert intermediate argument to 64bit too.

This patch is a first in the 64-bit-blocklayer series, so we are
generally moving to int64_t for both offset and bytes parameters on all
io paths. Main motivation is realization of 64-bit write_zeroes
operation for fast zeroing large disk chunks, up to the whole disk.

We chose signed type, to be consistent with off_t (which is signed) and
with possibility for signed return type (where negative value means
error).

Patch-correctness audit by Eric Blake:

  Caller has 32-bit, this patch now causes widening which is safe:
  block/block-backend.c: blk_do_preadv() passes 'unsigned int'
  block/block-backend.c: blk_do_pwritev_part() passes 'unsigned int'
  block/throttle.c: throttle_co_pwrite_zeroes() passes 'int'
  block/throttle.c: throttle_co_pdiscard() passes 'int'

  Caller has 64-bit, this patch fixes potential bug where pre-patch
  could narrow, except it's easy enough to trace that callers are still
  capped at 2G actions:
  block/throttle.c: throttle_co_preadv() passes 'uint64_t'
  block/throttle.c: throttle_co_pwritev() passes 'uint64_t'

  Implementation in question: block/throttle-groups.c
  throttle_group_co_io_limits_intercept() takes 'unsigned int bytes'
  and uses it: argument to util/throttle.c throttle_account(uint64_t)

  All safe: it patches a latent bug, and does not introduce any 64-bit
  gotchas once throttle_co_p{read,write}v are relaxed, and assuming
  throttle_account() is not buggy.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-Id: <20201211183934.169161-7-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-02-03 08:14:00 -06:00
Vladimir Sementsov-Ogievskiy 4c002cef0e util/iov: make qemu_iovec_init_extended() honest
Actually, we can't extend the io vector in all cases. Handle possible
MAX_IOV and size_t overflows.

For now add assertion to callers (actually they rely on success anyway)
and fix them in the following patch.

Add also some additional good assertions to qemu_iovec_init_slice()
while being here.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201211183934.169161-3-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-02-03 08:00:33 -06:00
Vladimir Sementsov-Ogievskiy 69b55e03f7 block: refactor bdrv_check_request: add errp
It's better to pass &error_abort than just assert that result is 0: on
crash, we'll immediately see the reason in the backtrace.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201211183934.169161-2-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: fix iotest 206 fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-02-03 08:00:33 -06:00
Peter Maydell 99ae0cd90d target-arm queue:
* hw/intc/arm_gic: Allow to use QTest without crashing
  * hw/char/exynos4210_uart: Fix buffer size reporting with FIFO disabled
  * hw/char/exynos4210_uart: Fix missing call to report ready for input
  * hw/arm/smmuv3: Fix addr_mask for range-based invalidation
  * hw/ssi/imx_spi: Fix various minor bugs
  * hw/intc/arm_gic: Fix interrupt ID in GICD_SGIR register
  * hw/arm: Add missing Kconfig dependencies
  * hw/arm: Display CPU type in machine description
 -----BEGIN PGP SIGNATURE-----
 
 iQJNBAABCAA3FiEE4aXFk81BneKOgxXPPCUl7RQ2DN4FAmAaeAQZHHBldGVyLm1h
 eWRlbGxAbGluYXJvLm9yZwAKCRA8JSXtFDYM3tWFD/9VTSSTmGMtSLGpuVt6t07x
 zqaFvRe+xUjrunwt25yx9tu9o4txXTk6mekgAz51QSeijESVIQQUKArZjbLWxRjl
 EXZAedOCF+f+lpzdQCO/GZtsHOfcWa158qm51NlEIM3cn1NiSASs0ky3r52MjjAR
 g0NFYTiZNplq8ah/0RljhHMhnAHzUbp/IErIxknWOVKvaH45+eji7mKxUk2vaCXB
 L5HCEzGCbPCqMMi3DFcwm9nNIYRu7X0hs9nR0AXTvdXbCoDqSyD4dEpjEcK/2IFM
 4zbS4NFRD7ndjD0C502+EUFav3tfd5/UfIjNg3oquMTDQrMCvh5Y1i3II0lVVWe5
 eSfbqV2eBzJHBanf7P64fRpk9mBNduJ8BZrozICvkCJxj5y3nTofKA9hXeGaDAdy
 7sA7Uzkpb1vnnMFzYk/0t2D6BSSFiknYuDHnfMY0nRoHsuDvY5yHw9Tt181D+qST
 UyLcmS8BB227WGgQPKSsFiUu7U423BIoiD5pp8cRyAg+FojqH6BZrcVluUWng6Ru
 ZVG0349lTmZ4mgv2hZI1qqIh80o40TI6K7OerQuMkZlq+4xseZvyb6+SQHaQ/j3I
 CKBiN15y/UzE5eivG6Siq3MR2myCBCSevTxDw4s+zR9FhoRQqFKoDhPGXTR9btWo
 1mgDNUQfyWtCcdoJKb7jEg==
 =kWoi
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20210203' into staging

target-arm queue:
 * hw/intc/arm_gic: Allow to use QTest without crashing
 * hw/char/exynos4210_uart: Fix buffer size reporting with FIFO disabled
 * hw/char/exynos4210_uart: Fix missing call to report ready for input
 * hw/arm/smmuv3: Fix addr_mask for range-based invalidation
 * hw/ssi/imx_spi: Fix various minor bugs
 * hw/intc/arm_gic: Fix interrupt ID in GICD_SGIR register
 * hw/arm: Add missing Kconfig dependencies
 * hw/arm: Display CPU type in machine description

# gpg: Signature made Wed 03 Feb 2021 10:16:36 GMT
# gpg:                using RSA key E1A5C593CD419DE28E8315CF3C2525ED14360CDE
# gpg:                issuer "peter.maydell@linaro.org"
# gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>" [ultimate]
# gpg:                 aka "Peter Maydell <pmaydell@gmail.com>" [ultimate]
# gpg:                 aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>" [ultimate]
# Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83  15CF 3C25 25ED 1436 0CDE

* remotes/pmaydell/tags/pull-target-arm-20210203: (21 commits)
  hw/arm: Display CPU type in machine description
  hw/net/can: ZynqMP CAN device requires PTIMER
  hw/arm/xlnx-versal: Versal SoC requires ZynqMP peripherals
  hw/arm/xlnx-versal: Versal SoC requires ZDMA
  hw/arm/exynos4210: Add missing dependency on OR_IRQ
  hw/arm/stm32f405_soc: Add missing dependency on OR_IRQ
  hw/intc/arm_gic: Fix interrupt ID in GICD_SGIR register
  hw/ssi: imx_spi: Correct tx and rx fifo endianness
  hw/ssi: imx_spi: Correct the burst length > 32 bit transfer logic
  hw/ssi: imx_spi: Round up the burst length to be multiple of 8
  hw/ssi: imx_spi: Disable chip selects when controller is disabled
  hw/ssi: imx_spi: Rework imx_spi_write() to handle block disabled
  hw/ssi: imx_spi: Rework imx_spi_read() to handle block disabled
  hw/ssi: imx_spi: Rework imx_spi_reset() to keep CONREG register value
  hw/ssi: imx_spi: Remove pointless variable initialization
  hw/ssi: imx_spi: Remove imx_spi_update_irq() in imx_spi_reset()
  hw/ssi: imx_spi: Use a macro for number of chip selects supported
  hw/arm/smmuv3: Fix addr_mask for range-based invalidation
  hw/char/exynos4210_uart: Fix missing call to report ready for input
  hw/char/exynos4210_uart: Fix buffer size reporting with FIFO disabled
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-03 12:55:44 +00:00
Peter Maydell 8360ebeb4f Machine queue, 2021-02-02
Feature:
 * nvdimm: read-only file support (Stefan Hajnoczi)
 -----BEGIN PGP SIGNATURE-----
 
 iQJIBAABCAAyFiEEWjIv1avE09usz9GqKAeTb5hNxaYFAmAZp5kUHGVoYWJrb3N0
 QHJlZGhhdC5jb20ACgkQKAeTb5hNxaYSeBAAp2lr8eKwsEju6Qpmo+OGo6rG/ORM
 KWRtwwLhHfjj2iFipURAkoepOqmtD/xd7fwItdTwi6zZxzr2eNwKqdlc9T93onzX
 P1yXCNHrz0NmmDB8LbC9Bw2NwubghAYgkuWo+fWMxNU92r+ObnvEc3Otx7P7sGvb
 Lr0yFRANgbJBLw1JTB9kMcWVDhiJ8tMPO9drhUWiMjl4kOaLNyrjUpafNzaleIaO
 GRtIXWts2Waq8wIz3XgSqWDytr075v2vWfvbR4JeAQxFArFbplR3BUuDa8G4nf6O
 nB+DERSMw+fOB2f6ZSYaAqSNwCgIrtwjhkZID0EXzNr03GPuonRnKLRZXYHvlEPT
 0qLaQeeLX+RXr6vGIjPJceQ3PyPlgDMVgVAKPS6Fvx4a3vYzAKfIj6E85QDNhuUJ
 FD37bS3iHpGxMAiuw1Ju1xJdkdzpX11h3nruo8K3sFciMK0CjkfHQKwGcw4XtKE1
 U56PEp+hKmoPe3z9CT1QmlJXlAJ+3NCAAXp6yt5yFDMqU4qxD2ns9+ts/S75o3bo
 P9pTBiaSAlUyOF1a0+0tOMny/7yz+Eb+Dw4CszZ6cCErVC/wNuJXYsF3BCT2roPN
 0d5CKwA717Jh7JmxhSlGXAW4d5mSMn8RHbFnHd5ZKFiIAh4+wnKfXrD9vB+Vq4fr
 Nja+LAk7QoURKjc=
 =TJ6/
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ehabkost-gl/tags/machine-next-pull-request' into staging

Machine queue, 2021-02-02

Feature:
* nvdimm: read-only file support (Stefan Hajnoczi)

# gpg: Signature made Tue 02 Feb 2021 19:27:21 GMT
# gpg:                using RSA key 5A322FD5ABC4D3DBACCFD1AA2807936F984DC5A6
# gpg:                issuer "ehabkost@redhat.com"
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" [full]
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost-gl/tags/machine-next-pull-request:
  nvdimm: check -object memory-backend-file, readonly=on option
  hostmem-file: add readonly=on|off option
  memory: add readonly support to memory_region_init_ram_from_file()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-03 09:54:21 +00:00
Bin Meng 1da79ecc7a hw/ssi: imx_spi: Use a macro for number of chip selects supported
Avoid using a magic number (4) everywhere for the number of chip
selects supported.

Signed-off-by: Bin Meng <bin.meng@windriver.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-id: 20210129132323.30946-2-bmeng.cn@gmail.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-02 17:00:54 +00:00
Stefan Hajnoczi 369d6dc4de memory: add readonly support to memory_region_init_ram_from_file()
There is currently no way to open(O_RDONLY) and mmap(PROT_READ) when
creating a memory region from a file. This functionality is needed since
the underlying host file may not allow writing.

Add a bool readonly argument to memory_region_init_ram_from_file() and
the APIs it calls.

Extend memory_region_init_ram_from_file() rather than introducing a
memory_region_init_rom_from_file() API so that callers can easily make a
choice between read/write and read-only at runtime without calling
different APIs.

No new RAMBlock flag is introduced for read-only because it's unclear
whether RAMBlocks need to know that they are read-only. Pass a bool
readonly argument instead.

Both of these design decisions can be changed in the future. It just
seemed like the simplest approach to me.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Liam Merwick <liam.merwick@oracle.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Message-Id: <20210104171320.575838-2-stefanha@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2021-02-01 17:07:34 -05:00
Stefan Hajnoczi 651d588f03 error: rename error_with_timestamp to message_with_timestamp
The -msg timestamp=on|off option controls whether a timestamp is printed
with error_report() messages. The "-msg" name suggests that this option
has a wider effect than just error_report(). The next patch extends it
to the 'log' trace backend, so rename the variable from
error_with_timestamp to message_with_timestamp.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210125113507.224287-2-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-02-01 10:50:55 +00:00
Peter Maydell 74208cd252 * Replace --enable/disable-git-update with --with-git-submodules
to allow improved control over use of git submodules
 * Deprecate the -enable-fips option
 * Ensure docs use prefer format for bool options
 * Clarify platform support rules
 * Misc fixes to keymap conversions
 * Fix misc problems on macOS
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE2vOm/bJrYpEtDo4/vobrtBUQT98FAmAUQXUACgkQvobrtBUQ
 T9/9Zw//VV3z7MI70HjWk17TUmkKLGFbmrc3xvtS6EMZbhfXpsR8qMfSjoRxP4Gn
 CgeftOa/lK5XjL7FcEUmcZeO8jZNE/+S5wgP2upP5Ae1zuDeyaSPTGR6UYs7WqSa
 P1kQsot2sAsZCP7Lko0jSv7rEDInK1bcdWr0a/xR5M2TvKiXoEdbElIZdwe3yLbC
 qWPWiv2pp6z2eGtJK/9VBxXjP0tlkNAR+jz8p/8RSZEEKf3aSq6HvVae+bHU86nX
 9t4baZk3ZuuwkmHJrHJP/72BjjitqEOiydtSsOCVYmY2GOclYeTnezG/vm75cGIK
 ej6hwFGElAEQk+KU9g7HOXi1eqXncSBjP0xWqc8PaZLW7wW/Uv8UV94fk+SCkIQ0
 GVBBLP8PzySL+w33NN7sv14cWeBpsjy9arMr4okZ80klFlS2hx/FqnUmhFhoN8ub
 ptIPUcSJoL+6pjxatV2yJ6bHkVga9PiwtLC8/rqETYPJz0b756xlPaEGd+0B1P0V
 ZMraE7lCgS0kJM6w0xF3R7JlGVtKouj4ZePCK5MGjpByBAK/vn/MbhbEmcyh+uJD
 r6sakKwqLjDLKdmQDd+biq3q6mvKE8ZKaUj8dXYdvxXSxxqXZ8DnrNpj/UkDTi3S
 chlTT1qUYrp2MVYqGmFjZ0XcEdEUR1I39rUs+wRxE3RgtE56M1A=
 =g6bC
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/berrange-gitlab/tags/misc-fixes-pull-request' into staging

* Replace --enable/disable-git-update with --with-git-submodules
  to allow improved control over use of git submodules
* Deprecate the -enable-fips option
* Ensure docs use prefer format for bool options
* Clarify platform support rules
* Misc fixes to keymap conversions
* Fix misc problems on macOS

# gpg: Signature made Fri 29 Jan 2021 17:10:13 GMT
# gpg:                using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" [full]
# gpg:                 aka "Daniel P. Berrange <berrange@redhat.com>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E  8E3F BE86 EBB4 1510 4FDF

* remotes/berrange-gitlab/tags/misc-fixes-pull-request:
  tests: Replace deprecated ASN1 code
  tests: Fix runtime error in test-authz-pam
  ui: update keycodemapdb submodule commit
  crypto: Add spaces around operator
  configure: replace --enable/disable-git-update with --with-git-submodules
  docs: fix missing backslash in certtool shell example
  docs: simplify and clarify the platform support rules
  Prefer 'on' | 'off' over 'yes' | 'no' for bool options
  os: deprecate the -enable-fips option and QEMU's FIPS enforcement
  crypto: Fix memory leaks in set_loaded for tls-*
  crypto: Forbid broken unloading of secrets
  crypto: Move USER_CREATABLE to secret_common base class
  crypto: Fix some code style problems, add spaces around operator

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-29 19:51:25 +00:00
Daniel P. Berrangé 4d7beeab38 Prefer 'on' | 'off' over 'yes' | 'no' for bool options
Update some docs and test cases to use 'on' | 'off' as the preferred
value for bool options.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2021-01-29 17:07:53 +00:00
Peter Maydell 38867d0b7e arm: Remove frq properties on CMSDK timer, dualtimer, watchdog, ARMSSE
Now no users are setting the frq properties on the CMSDK timer,
dualtimer, watchdog or ARMSSE SoC devices, we can remove the
properties and the struct fields that back them.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Luc Michel <luc@lmichel.fr>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20210128114145.20536-25-peter.maydell@linaro.org
Message-id: 20210121190622.22000-25-peter.maydell@linaro.org
2021-01-29 15:54:44 +00:00
Peter Maydell efc34aaa82 hw/arm/mps2: Inline CMSDK_APB_TIMER creation
The old-style convenience function cmsdk_apb_timer_create() for
creating CMSDK_APB_TIMER objects is used in only two places in
mps2.c.  Most of the rest of the code in that file uses the new
"initialize in place" coding style.

We want to connect up a Clock object which should be done between the
object creation and realization; rather than adding a Clock* argument
to the convenience function, convert the timer creation code in
mps2.c to the same style as is used already for the watchdog,
dualtimer and other devices, and delete the now-unused convenience
function.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Luc Michel <luc@lmichel.fr>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20210128114145.20536-13-peter.maydell@linaro.org
Message-id: 20210121190622.22000-13-peter.maydell@linaro.org
2021-01-29 15:54:43 +00:00
Peter Maydell 8fd34dc0c4 hw/arm/armsse: Wire up clocks
Create two input clocks on the ARMSSE devices, one for the normal
MAINCLK, and one for the 32KHz S32KCLK, and wire these up to the
appropriate devices.  The old property-based clock frequency setting
will remain in place until conversion is complete.

This is a migration compatibility break for machines mps2-an505,
mps2-an521, musca-a, musca-b1.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Luc Michel <luc@lmichel.fr>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20210128114145.20536-12-peter.maydell@linaro.org
Message-id: 20210121190622.22000-12-peter.maydell@linaro.org
2021-01-29 15:54:42 +00:00
Peter Maydell 13059a3a10 hw/arm/armsse: Rename "MAINCLK" property to "MAINCLK_FRQ"
While we transition the ARMSSE code from integer properties
specifying clock frequencies to Clock objects, we want to have the
device provide both at once.  We want the final name of the main
input Clock to be "MAINCLK", following the hardware name.
Unfortunately creating an input Clock with a name X creates an
under-the-hood QOM property X; for "MAINCLK" this clashes with the
existing UINT32 property of that name.

Rename the UINT32 property to MAINCLK_FRQ so it can coexist with the
MAINCLK Clock; once the transition is complete MAINCLK_FRQ will be
deleted.

Commit created with:
 perl -p -i -e 's/MAINCLK/MAINCLK_FRQ/g' hw/arm/{armsse,mps2-tz,musca}.c include/hw/arm/armsse.h

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Luc Michel <luc@lmichel.fr>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20210128114145.20536-11-peter.maydell@linaro.org
Message-id: 20210121190622.22000-11-peter.maydell@linaro.org
2021-01-29 15:54:42 +00:00
Peter Maydell eeae0b2bf4 hw/watchdog/cmsdk-apb-watchdog: Add Clock input
As the first step in converting the CMSDK_APB_TIMER device to the
Clock framework, add a Clock input.  For the moment we do nothing
with this clock; we will change the behaviour from using the
wdogclk-frq property to using the Clock once all the users of this
device have been converted to wire up the Clock.

This is a migration compatibility break for machines mps2-an385,
mps2-an386, mps2-an500, mps2-an511, mps2-an505, mps2-an521, musca-a,
musca-b1, lm3s811evb, lm3s6965evb.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Luc Michel <luc@lmichel.fr>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20210128114145.20536-10-peter.maydell@linaro.org
Message-id: 20210121190622.22000-10-peter.maydell@linaro.org
2021-01-29 15:54:42 +00:00
Peter Maydell 55fd0f84a2 hw/timer/cmsdk-apb-dualtimer: Add Clock input
As the first step in converting the CMSDK_APB_DUALTIMER device to the
Clock framework, add a Clock input.  For the moment we do nothing
with this clock; we will change the behaviour from using the pclk-frq
property to using the Clock once all the users of this device have
been converted to wire up the Clock.

We take the opportunity to correct the name of the clock input to
match the hardware -- the dual timer names the clock which drives the
timers TIMCLK.  (It does also have a 'pclk' input, which is used only
for the register and APB bus logic; on the SSE-200 these clocks are
both connected together.)

This is a migration compatibility break for machines mps2-an385,
mps2-an386, mps2-an500, mps2-an511, mps2-an505, mps2-an521, musca-a,
musca-b1.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Luc Michel <luc@lmichel.fr>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20210128114145.20536-9-peter.maydell@linaro.org
Message-id: 20210121190622.22000-9-peter.maydell@linaro.org
2021-01-29 15:54:42 +00:00
Peter Maydell 7cc378edee hw/timer/cmsdk-apb-timer: Add Clock input
As the first step in converting the CMSDK_APB_TIMER device to the
Clock framework, add a Clock input.  For the moment we do nothing
with this clock; we will change the behaviour from using the pclk-frq
property to using the Clock once all the users of this device have
been converted to wire up the Clock.

Since the device doesn't already have a doc comment for its "QEMU
interface", we add one including the new Clock.

This is a migration compatibility break for machines mps2-an505,
mps2-an521, musca-a, musca-b1.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Luc Michel <luc@lmichel.fr>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20210128114145.20536-8-peter.maydell@linaro.org
Message-id: 20210121190622.22000-8-peter.maydell@linaro.org
2021-01-29 15:54:42 +00:00
Peter Maydell b56d351e25 hw/timer/cmsdk-apb-timer: Rename CMSDKAPBTIMER struct to CMSDKAPBTimer
The state struct for the CMSDK APB timer device doesn't follow our
usual naming convention of camelcase -- "CMSDK" and "APB" are both
acronyms, but "TIMER" is not so should not be all-uppercase.
Globally rename the struct to "CMSDKAPBTimer" (bringing it into line
with CMSDKAPBWatchdog and CMSDKAPBDualTimer; CMSDKAPBUART remains
as-is because "UART" is an acronym).

Commit created with:
 perl -p -i -e 's/CMSDKAPBTIMER/CMSDKAPBTimer/g' hw/timer/cmsdk-apb-timer.c include/hw/arm/armsse.h include/hw/timer/cmsdk-apb-timer.h

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Luc Michel <luc@lmichel.fr>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20210128114145.20536-7-peter.maydell@linaro.org
Message-id: 20210121190622.22000-7-peter.maydell@linaro.org
2021-01-29 15:54:42 +00:00
Peter Maydell 132b10251c clock: Add new clock_has_source() function
Add a function for checking whether a clock has a source.  This is
useful for devices which have input clocks that must be wired up by
the board as it allows them to fail in realize rather than ploughing
on with a zero-period clock.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Luc Michel <luc@lmichel.fr>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20210128114145.20536-3-peter.maydell@linaro.org
Message-id: 20210121190622.22000-3-peter.maydell@linaro.org
2021-01-29 15:54:42 +00:00
Peter Maydell ad140dadd5 ptimer: Add new ptimer_set_period_from_clock() function
The ptimer API currently provides two methods for setting the period:
ptimer_set_period(), which takes a period in nanoseconds, and
ptimer_set_freq(), which takes a frequency in Hz.  Neither of these
lines up nicely with the Clock API, because although both the Clock
and the ptimer track the frequency using a representation of whole
and fractional nanoseconds, conversion via either period-in-ns or
frequency-in-Hz will introduce a rounding error.

Add a new function ptimer_set_period_from_clock() which takes the
Clock object directly to avoid the rounding issues.  This includes a
facility for the user to specify that there is a frequency divider
between the Clock proper and the timer, as some timer devices like
the CMSDK APB dualtimer need this.

To avoid having to drag in clock.h from ptimer.h we add the Clock
type to typedefs.h.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Luc Michel <luc@lmichel.fr>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20210128114145.20536-2-peter.maydell@linaro.org
Message-id: 20210121190622.22000-2-peter.maydell@linaro.org
2021-01-29 15:54:42 +00:00
Mihai Carabas d097b3dcb6 hw/misc/pvpanic: add PCI interface support
Add PCI interface support for PVPANIC device. Create a new file pvpanic-pci.c
where the PCI specific routines reside and update the build system with the new
files and config structure.

Signed-off-by: Mihai Carabas <mihai.carabas@oracle.com>
Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Mihai Carabas <mihai.carabas@oracle.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-29 10:47:28 +00:00
Mihai Carabas 677726ef1e hw/misc/pvpanic: split-out generic and bus dependent code
To ease the PCI device addition in next patches, split the code as follows:
- generic code (read/write/setup) is being kept in pvpanic.c
- ISA dependent code moved to pvpanic-isa.c

Also, rename:
- ISA_PVPANIC_DEVICE -> PVPANIC_ISA_DEVICE.
- TYPE_PVPANIC -> TYPE_PVPANIC_ISA.
- MemoryRegion io -> mr.
- pvpanic_ioport_* in pvpanic_*.

Update the build system with the new files and config structure.

Signed-off-by: Mihai Carabas <mihai.carabas@oracle.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-29 10:47:28 +00:00
Joelle van Dyne 1ad27f7d93 osdep: build with non-working system() function
Build without error on hosts without a working system(). If system()
is called, return -1 with ENOSYS.

Signed-off-by: Joelle van Dyne <j@getutm.app>
Message-id: 20210126012457.39046-6-j@getutm.app
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-29 10:47:28 +00:00
Maxim Uvarov daa726d926 arm-virt: add secure pl061 for reset/power down
Add secure pl061 for reset/power down machine from
the secure world (Arm Trusted Firmware). Connect it
with gpio-pwr driver.

Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
Reviewed-by: Andrew Jones <drjones@redhat.com>
[PMM: Added mention of the new device to the documentation]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-29 10:47:28 +00:00
Peter Maydell 7e7eb9f852 QAPI patches patches for 2021-01-28
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmASY10SHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZT4M4P+gKN64+WaErotLHHsqtiA0aoTwbTFXin
 OEyR5du0+PjX96qYHHV+ZDn5uxxKI57/SRNooPndjU63sYgAbNApfsu+wUDZC844
 qMSlrmyw2+Lw1EIykoLXK49+pEDU0XpVIciL5+zEdtCgjiJRjrOOJ/JRBcKoQNHn
 UArGNQ8y0D+0i8uXyJjyvQeHdz6KUr9sX1vqwRGMt9axEMDJks0+Si4Zg3z2wlWJ
 Sc3WsXEhikxK1qkF2/6VsopOgNGB0UUvV6q1GO6ngdqag1Hb6mACzSv9mtIShGjh
 a2MISBhxF8h4wfO8U5TiS9vBgYR3elA3kRGsn4FOfD3sSilt/SWLPHWXdlO1aL2E
 TollRPtYBqn2YIYQP1SEp7NIqaWC/QaGkP/mH8Jvv0YlL64RK879lv6KiHKzfvI7
 HBD7WGZBwMQqPczuw308tqDTQPKUsPDYoEJAFRywkLry86wL8DBOlkQ0lWUjF06s
 UQk/i09nhrcNLo0GbmgAOHUVj4m03zLyMW/fYmsQ8xe9/b6GBwJvtm2v5wKwg0HE
 ixxj4oBIk5YV5Xwt7DKLkT0voPAAgNK13a6ywzbyfsigwaJaO9tLtZ0PMuaT9kgs
 b/OBdeeIYpFdIT/DlcMWpIFi53VYe0McX8MmprHcMZb1133wk5Z5gk+FAWLMifrw
 2ltmoUPoB1dC
 =djiE
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2021-01-28' into staging

QAPI patches patches for 2021-01-28

# gpg: Signature made Thu 28 Jan 2021 07:10:21 GMT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2021-01-28:
  qapi: More complex uses of QAPI_LIST_APPEND
  qapi: Use QAPI_LIST_APPEND in trivial cases
  qapi: Introduce QAPI_LIST_APPEND
  qapi: A couple more QAPI_LIST_PREPEND() stragglers
  net: Clarify early exit condition

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-28 22:43:18 +00:00
Eric Blake dc13f40c6b qapi: Introduce QAPI_LIST_APPEND
Similar to the existing QAPI_LIST_PREPEND, but designed for use where
we want to preserve insertion order.  Callers will be added in
upcoming patches.  Note the difference in signature: PREPEND takes
List*, APPEND takes List**.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210113221013.390592-4-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-01-28 08:08:45 +01:00
Kevin Wolf 86b1cf3227 block: Separate blk_is_writable() and blk_supports_write_perm()
Currently, blk_is_read_only() tells whether a given BlockBackend can
only be used in read-only mode because its root node is read-only. Some
callers actually try to answer a slightly different question: Is the
BlockBackend configured to be writable, by taking write permissions on
the root node?

This can differ, for example, for CD-ROM devices which don't take write
permissions, but may be backed by a writable image file. scsi-cd allows
write requests to the drive if blk_is_read_only() returns false.
However, the write request will immediately run into an assertion
failure because the write permission is missing.

This patch introduces separate functions for both questions.
blk_supports_write_perm() answers the question whether the block
node/image file can support writable devices, whereas blk_is_writable()
tells whether the BlockBackend is currently configured to be writable.

All calls of blk_is_read_only() are converted to one of the two new
functions.

Fixes: https://bugs.launchpad.net/bugs/1906693
Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210118123448.307825-2-kwolf@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-01-27 20:45:20 +01:00
Vladimir Sementsov-Ogievskiy 143a6384f5 block/block-copy: drop unused argument of block_copy()
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-21-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy 5b49c2bdc1 block/block-copy: drop unused block_copy_set_progress_callback()
Drop unused code.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-20-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy e0323a045f blockjob: add set_speed to BlockJobDriver
We are going to use async block-copy call in backup, so we'll need to
passthrough setting backup speed to block-copy call.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-9-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy a6d23d56df block/block-copy: add block_copy_cancel
Add function to cancel running async block-copy call. It will be used
in backup.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-8-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy 7e032df0ea block/block-copy: add ratelimit to block-copy
We are going to directly use one async block-copy operation for backup
job, so we need rate limiter.

We want to maintain current backup behavior: only background copying is
limited and copy-before-write operations only participate in limit
calculation. Therefore we need one rate limiter for block-copy state
and boolean flag for block-copy call state for actual limitation.

Note, that we can't just calculate each chunk in limiter after
successful copying: it will not save us from starting a lot of async
sub-requests which will exceed limit too much. Instead let's use the
following scheme on sub-request creation:
1. If at the moment limit is not exceeded, create the request and
account it immediately.
2. If at the moment limit is already exceeded, drop create sub-request
and handle limit instead (by sleep).
With this approach we'll never exceed the limit more than by one
sub-request (which pretty much matches current backup behavior).

Note also, that if there is in-flight block-copy async call,
block_copy_kick() should be used after set-speed to apply new setup
faster. For that block_copy_kick() published in this patch.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-7-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy 26be9d62dd block/block-copy: add max_chunk and max_workers parameters
They will be used for backup.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-5-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy de4641b46b block/block-copy: implement block_copy_async
We'll need async block-copy invocation to use in backup directly.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-4-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy 86c6a3b690 qapi: backup: add perf.use-copy-range parameter
Experiments show, that copy_range is not always making things faster.
So, to make experimentation simpler, let's add a parameter. Some more
perf parameters will be added soon, so here is a new struct.

For now, add new backup qmp parameter with x- prefix for the following
reasons:

 - We are going to add more performance parameters, some will be
   related to the whole block-copy process, some only to background
   copying in backup (ignored for copy-before-write operations).
 - On the other hand, we are going to use block-copy interface in other
   block jobs, which will need performance options as well.. And it
   should be the same structure or at least somehow related.

So, there are too much unclean things about how the interface and now
we need the new options mostly for testing. Let's keep them
experimental for a while.

In do_backup_common() new x-perf parameter handled in a way to
make further options addition simpler.

We add use-copy-range with default=true, and we'll change the default
in further patch, after moving backup to use block-copy.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-2-vsementsov@virtuozzo.com>
[mreitz: s/5\.2/6.0/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy 7f4a396d76 qapi: block-stream: add "bottom" argument
The code already don't freeze base node and we try to make it prepared
for the situation when base node is changed during the operation. In
other words, block-stream doesn't own base node.

Let's introduce a new interface which should replace the current one,
which will in better relations with the code. Specifying bottom node
instead of base, and requiring it to be non-filter gives us the
following benefits:

 - drop difference between above_base and base_overlay, which will be
   renamed to just bottom, when old interface dropped

 - clean way to work with parallel streams/commits on the same backing
   chain, which otherwise become a problem when we introduce a filter
   for stream job

 - cleaner interface. Nobody will surprised the fact that base node may
   disappear during block-stream, when there is no word about "base" in
   the interface.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201216061703.70908-11-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Andrey Shinkevich e275458b29 copy-on-read: skip non-guest reads if no copy needed
If the flag BDRV_REQ_PREFETCH was set, skip idling read/write
operations in COR-driver. It can be taken into account for the
COR-algorithms optimization. That check is being made during the
block stream job by the moment.

Add the BDRV_REQ_PREFETCH flag to the supported_read_flags of the
COR-filter.

block: Modify the comment for the flag BDRV_REQ_PREFETCH as we are
going to use it alone and pass it to the COR-filter driver for further
processing.

Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201216061703.70908-9-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Andrey Shinkevich 897dd0ec4f block: include supported_read_flags into BDS structure
Add the new member supported_read_flags to the BlockDriverState
structure. It will control the flags set for copy-on-read operations.
Make the block generic layer evaluate supported read flags before they
go to a block driver.

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 [vsementsov: use assert instead of abort]
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201216061703.70908-8-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Andrey Shinkevich 880747a887 qapi: add filter-node-name to block-stream
Provide the possibility to pass the 'filter-node-name' parameter to the
block-stream job as it is done for the commit block job.

Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  [vsementsov: comment indentation, s/Since: 5.2/Since: 6.0/]
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201216061703.70908-5-vsementsov@virtuozzo.com>
[mreitz: s/commit/stream/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 11:26:54 +01:00
Andrey Shinkevich 8872ef78ab block: add API function to insert a node
Provide API for insertion a node to backing chain.

Suggested-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201216061703.70908-3-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 11:26:54 +01:00
Peter Maydell 31ee895047 -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
 
 iQEcBAABAgAGBQJgDonvAAoJEO8Ells5jWIR++8IAJBTpKDSwdwG1ySn5m6tHZHM
 GdFD2Ao8R8VsonJzw9i/1Gfn9T0icndRU6VC+nJEdvSLXUDoAQdmMmLCk4oml41g
 gzPzsc6bFKwKB4DG3RyixbEoUsMl7jQRoVYOgWy1Ezx2daAGG9PjuDMdBU4tNCFv
 IxkVpOz3RClhzaMz3O0OkLKljooRcvhePB+8FuZCOnBzxDJ9RU1KXDpv5kYMswFL
 9f+D2UK474IcaAQCzKXuHrs/k3utMEY32udjqQdgb9y/8jZYJuEZSB7pyXozuSAO
 QSl62FxbAwfx85hVbMqnIa2Yd629SiVVHsotavAZz60fcBaQgrNbBTWDNlCrWJo=
 =XN54
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging

# gpg: Signature made Mon 25 Jan 2021 09:05:51 GMT
# gpg:                using RSA key EF04965B398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <jasowang@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 215D 46F4 8246 689E C77F  3562 EF04 965B 398D 6211

* remotes/jasowang/tags/net-pull-request:
  net: checksum: Introduce fine control over checksum type
  net: checksum: Add IP header checksum calculation
  net: checksum: Skip fragmented IP packets
  net: Fix handling of id in netdev_add and netdev_del

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-25 15:56:13 +00:00
Peter Maydell 55d98950a6 SD/MMC patches
- Various improvements for SD cards in SPI mode (Bin Meng)
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAmANx6cACgkQ4+MsLN6t
 wN5OTRAAvBZrBn3YktziZZds4HpKpBdEC/lAmlYNBCl6cn6gpAfrYz1WjpKm+DrA
 0tfDeanoqUnWNReYwFRyQHzpWtIjIGo1K5tLbBVGE3qL1DtoZliDMA94RAGZu9UW
 vrdWFxFtRFJ6Yqs0JjIhY2c+K9y7UcYRqATihbl/TpQNLSlVKblKnP1GPKZWqpRx
 RL+sdAzwXhtXLzaJ/Jnk4XDTibNsLsRMWsa0rKM2o6181NqumYDj6gWOFfZWADji
 lScwZzU0gWxYEarruUWaMMDxxB/1OXGH5Rd+bpDTrqVJV9qgsEEVj1VrJVfCPQFk
 nInd0X4cAp+Mq4x901eovWcF+nT/zNWS/vJ0JiJKlxciz3Oev0kJLPJ/3YssLK3k
 LYrhb20Py5ug41XYnpOKLcXR8CBKyqRlmwp8U330lCooLDxhy2hXaU41B0Dte3M3
 CgngnOKmr2xizdWKy8L9GFvcQIPv1w9tRIOm/Z3CaU4JNaDSZo8vSUMFAtzsiW7B
 dB6TOXcYxQZEPt1u6dO5KUDetd7m2pRMQ+or5lZa3d5w57kpAzuLRyiyXWv1npQc
 4nVf6fS/tqmkqOjZkyj3lliAKdDkmEfWoiSRvUjHeddetGFd5VJ8IjPdf6pDGj1G
 H1ix3N1JIrGpBmKwVrjTbxeGBGuD4vhetQeMQ498exmzaiYEmgk=
 =B3sY
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/sdmmc-20210124' into staging

SD/MMC patches

- Various improvements for SD cards in SPI mode (Bin Meng)

# gpg: Signature made Sun 24 Jan 2021 19:16:55 GMT
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/sdmmc-20210124:
  hw/sd: sd.h: Cosmetic change of using spaces
  hw/sd: ssi-sd: Use macros for the dummy value and tokens in the transfer
  hw/sd: ssi-sd: Fix the wrong command index for STOP_TRANSMISSION
  hw/sd: ssi-sd: Add a state representing Nac
  hw/sd: ssi-sd: Suffix a data block with CRC16
  util: Add CRC16 (CCITT) calculation routines
  hw/sd: sd: Drop sd_crc16()
  hw/sd: sd: Support CMD59 for SPI mode
  hw/sd: ssi-sd: Fix incorrect card response sequence

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-25 11:52:00 +00:00
Bin Meng f574633529 net: checksum: Introduce fine control over checksum type
At present net_checksum_calculate() blindly calculates all types of
checksums (IP, TCP, UDP). Some NICs may have a per type setting in
their BDs to control what checksum should be offloaded. To support
such hardware behavior, introduce a 'csum_flag' parameter to the
net_checksum_calculate() API to allow fine control over what type
checksum is calculated.

Existing users of this API are updated accordingly.

Signed-off-by: Bin Meng <bin.meng@windriver.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
2021-01-25 17:04:56 +08:00
Peter Maydell e672f1d397 Fix tcg constant temp overflow.
Fix running during atomic single-step.
 Partial support for apple silicon.
 Cleanups for accel/tcg.
 -----BEGIN PGP SIGNATURE-----
 
 iQFRBAABCgA7FiEEekgeeIaLTbaoWgXAZN846K9+IV8FAmANt7kdHHJpY2hhcmQu
 aGVuZGVyc29uQGxpbmFyby5vcmcACgkQZN846K9+IV/qwQgApfJEYmEURz2ETsBv
 z1D/lVM6sjVky7Rk0GiCePf9GZhpXuLK0tEDxWPhHp8d466KfvU3iiXLv0xGbFV5
 MBYXPw2uJWLfYNqfi3YGJYSGAdESKJSx27hMMb7RkElbbKqmFGVyPTkZG0VDoiCZ
 hEUMGFXa1e0WjTQchm7AUw8CtATHUTmC2wkVqA+Z+BNqoU2WKbvy6f5QgM2CdWEX
 bFoC/K0o0vIr8OrwS76ZyVDwbGG7s8r9gA2B9a2Nw73/n4q6DdZPwcUIR23blHa/
 A9NmdU4HRNt07nfJN9DQ2qe9utNkoaFj7SpRW2+RJCVkTz5hqbiZTO1c3ZJa0MTi
 QbyMNQ==
 =KGcn
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/rth-gitlab/tags/pull-tcg-20210124' into staging

Fix tcg constant temp overflow.
Fix running during atomic single-step.
Partial support for apple silicon.
Cleanups for accel/tcg.

# gpg: Signature made Sun 24 Jan 2021 18:08:57 GMT
# gpg:                using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg:                issuer "richard.henderson@linaro.org"
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [full]
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A  05C0 64DF 38E8 AF7E 215F

* remotes/rth-gitlab/tags/pull-tcg-20210124:
  tcg: Restart code generation when we run out of temps
  tcg: Toggle page execution for Apple Silicon
  accel/tcg: Restrict cpu_io_recompile() from other accelerators
  accel/tcg: Declare missing cpu_loop_exit*() stubs
  accel/tcg: Restrict tb_gen_code() from other accelerators
  accel/tcg: Move tb_flush_jmp_cache() to cputlb.c
  accel/tcg: Make cpu_gen_init() static
  tcg: Optimize inline dup_const for MO_64
  qemu/compiler: Split out qemu_build_not_reached_always
  tcg: update the cpu running flag in cpu_exec_step_atomic

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-24 19:36:45 +00:00