* Documentation for the decorators in the "acceptance" tests
* One small rework of a libqtest function
-----BEGIN PGP SIGNATURE-----
iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmARBpMRHHRodXRoQHJl
ZGhhdC5jb20ACgkQLtnXdP5wLbXrmg//ap1z2taLpgUdMVxImdMieqsfzSxMUt75
vkmQvk+rquK8VqwVKGaMsps7aANXDFD2etjAx0zp/9uS6OApNRegRFNb5eOr2oUR
RBhnNYkabmAvabn7lFmb2FZ84cCA3c3rY0JsZ44eYcdOOQVLuKMH6//m7gPztmkA
BNYh8riAmOTVjL81cJVmA/hKUO7P1eR2+604gRr4t7mP9/I72QG3f12o062c+dWA
xXNK8gtqSjgfofZeqU8HYHkJoyi/3N6aGHE4XmojPbE0fzj1gjxlHm0LpIhrpwbI
hxkj3fwVh2e89dhMXCYhFKrYjUEQuJPUdLmMRl2KcD9AmNFtM3S+WSi6SX8m9CXR
FBYpz5zOk7HHDZKJZ17U0xr5NZ8E0iULc0uLAVm5BqkmiT3DqjaWb0JYe/4rUT7Y
XxOX7MHA5AByRfv6IBfKCmRPUXM16W4tvWYHsCzU7ViV7eOxNLl4yY+3YvyfigHf
5MENOAkr3kdL54sDOsARCLs8D6zyXyKyspt13SAoRg7cGxS3WpSKYVKxYeX+VXbs
S/mUoKFT1IE8PkiDnQzw5PypPxgc3700AlrIAk9ZQiWol5tfq8QKBf4a9LSpV5Uu
02OlI3ufkdZwALgw9ZJNsSR0hX/rcALwOxlQ4kBcWsT45kxwch0337H+U6bUVnca
AJpJfDzlXI0=
=CNCA
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2021-01-27' into staging
* Patches to speed up and improve the gitlab-CI
* Documentation for the decorators in the "acceptance" tests
* One small rework of a libqtest function
# gpg: Signature made Wed 27 Jan 2021 06:22:11 GMT
# gpg: using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg: issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg: aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg: aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg: aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3 EAB9 2ED9 D774 FE70 2DB5
* remotes/huth-gitlab/tags/pull-request-2021-01-27:
libqtest: Rework qtest_rsp()
docs/devel: Explain how acceptance tests can be skipped
gitlab-ci.yml: Avoid recompiling the sources in the test jobs
gitlab-ci.yml: Exclude some redundant targets in build-without-default-features
meson: Do not build optional libraries by default
configure: Only check for audio drivers if system-mode is selected
gitlab-ci.yml: Avoid some submodules to speed up the CI a little bit
gitlab-ci: Test building linux-user targets on CentOS 7
tests/docker: Install static libc package in CentOS 7
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Restrict tci_write_reg16() to 64-bit hosts to fix on 32-bit ones:
[520/1115] Compiling C object libqemu-arm-linux-user.fa.p/tcg_tci.c.o
FAILED: libqemu-arm-linux-user.fa.p/tcg_tci.c.o
tcg/tci.c:132:1: error: 'tci_write_reg16' defined but not used [-Werror=unused-function]
tci_write_reg16(tcg_target_ulong *regs, TCGReg index, uint16_t value)
^~~~~~~~~~~~~~~
Fixes: 2f160e0f97 ("tci: Add implementation for INDEX_op_ld16u_i64")
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Message-Id: <20210123094107.2340222-1-f4bug@amsat.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
I've already moved my repositories to gitlab for extra CI coverage,
and I won't use the ones at github anymore.
Signed-off-by: Greg Kurz <groug@kaod.org>
Reviewed-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Message-Id: <161071814430.152031.14540382419012818908.stgit@bahia.lan>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
In commit 2f487a3d40 we fixed a problem observed with using the
vmware-vga device and the VNC UI frontend in a belt-and-braces
manner:
* we made the VNC frontend handle non-multiple-of-16 surface widths
* we rounded up the vmware-vga display width to a multiple of 16
However this introduced a spurious dependency of a device model on a
UI frontend header. vmware-vga isn't special and should not care
about what UI frontend it is using, and the VNC frontend needs to
handle arbitrary surface widths because other display device models
could use them. Moreover, even if the maximum width in vmware-vga is
made a multiple of 16, the guest itself can always program a
different width.
Remove the dependency on the VNC header. Since we have been using
the rounded-up width value since 2014, stick with it rather than
introducing a behaviour change, but don't calculate it by rounding up
to VNC_DIRTY_BITS_PER_PIXEL any more.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20210112161608.16055-1-peter.maydell@linaro.org>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Unlike other pseudo-encodings these don't break gtk-vnc
because older versions don't suport the extended desktop
resize extension in the first place.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210125104041.495274-3-kraxel@redhat.com>
This reverts commit 9e1632ad07.
Older gtk-vnc versions can't deal with non-incremental update
requests sending pseudo-encodings, so trying to send full server
state (including desktop size, cursor etc. which is done using
pseudo-encodings) doesn't fly. Return to old behavior to send
those only for new connects and when changes happen.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210125104041.495274-2-kraxel@redhat.com>
Using the cfg.use_non_secure bitfield and the MMU access type, we can determine
if the access should be secure or not.
Signed-off-by: Joe Komlodi <komlodi@xilinx.com>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Tested-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Message-Id: <1611274735-303873-4-git-send-email-komlodi@xilinx.com>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Using MMUAccessType makes it more clear what the variable's use is.
No functional change.
Signed-off-by: Joe Komlodi <komlodi@xilinx.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Tested-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Message-Id: <1611274735-303873-3-git-send-email-komlodi@xilinx.com>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
This property is used to control the security of the following interfaces
on MicroBlaze:
M_AXI_DP - data interface
M_AXI_IP - instruction interface
M_AXI_DC - dcache interface
M_AXI_IC - icache interface
It works by enabling or disabling the use of the non_secure[3:0] signals.
Interfaces and their corresponding values are taken from:
https://www.xilinx.com/support/documentation/sw_manuals/xilinx2020_2/ug984-vivado-microblaze-ref.pdf
page 153.
Signed-off-by: Joe Komlodi <komlodi@xilinx.com>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Tested-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Message-Id: <1611274735-303873-2-git-send-email-komlodi@xilinx.com>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
qtest_rsp() is used in two different ways: (1) return some arguments
to caller, which the caller must free, and (2) return no arguments to
caller. Passing non-zero @expected_args gets you (1), and passing
zero gets you (2).
Having "the return value must be freed" depend on an argument this way
is less than ideal.
Provide separate functions for the two ways: (1) qtest_rsp_args()
takes @expected_args (possibly zero), and returns that number of
arguments. Caller must free the return value always. (2) qtest_rsp()
assumes zero, and returns nothing.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210126151649.2220902-1-armbru@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Documented under the "Acceptance tests using the Avocado Framework"
section in testing.rst how environment variables are used to skip tests.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210115210022.417996-1-wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Currently, our check-system-* jobs are recompiling the whole sources
again. This happens due to the fact that the jobs are checking out
the whole source tree and required submodules again, and only try
to use the "build" directory with the binaries and object files as an
artifact from the previous stage - which simply does not work right
anymore (with the current version of meson). Due to some changed
time stamps, meson/ninja are always trying to rebuild the whole tree.
In the long run, we could likely use "meson test --no-rebuild", but
there is still some work going on in that area to improve the user
experience. So until this has been done, simply avoid recompiling the
sources with a trick: pass NINJA=":" to the make process in the test
jobs. Also check out the submodules manually before updating the
timestamps in the build folder, so that the binaries are definitely
newer that all the source files.
This saves ca. 10 - 15 minutes of precious CI cycles in each run.
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20210126065757.403853-1-thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The build-without-default-features job is running quite long and sometimes
already hits the 1h time limit. Exclude some targets which do not provide
additional test coverage here (since we e.g. also already test other targets
of the same type, just with different endianess, or a 64-bit superset) to
avoid that we hit the timeout here so easily.
Message-Id: <20210126172345.15947-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
- Make backup block jobs use asynchronous requests with the block-copy
module
- Use COR filter node for stream block jobs
- Make coroutine-sigaltstack’s qemu_coroutine_new() function thread-safe
- Report error string when file locking fails with an unexpected errno
- iotest fixes, additions, and some refactoring
-----BEGIN PGP SIGNATURE-----
iQFGBAABCAAwFiEEkb62CjDbPohX0Rgp9AfbAGHVz0AFAmAQJFsSHG1yZWl0ekBy
ZWRoYXQuY29tAAoJEPQH2wBh1c9AWZEIALwZwQU82RE+gezeDK7XPqBYuGMOVzbJ
wdvQLnAx+vEo2idLlWm4RE7GhibhTB5iRaiIGmpZhvja/+2BrZHve4GHD5+GBolF
QvLoIdfkPqhORhuAMc1+8rkP4yqv1bLBuOB6TLh15Bw6B6Q7clQbDGqubK2HIwgb
h/RcmMj/93ZN27WM589Zdk5i/1kE11sOyCKNP+UQ3hnKz7ecqtY7pHYvFS3Q9e6l
xpc/QyFhTw/OHfaUViL6W0MWhdqhf/mBlXqzvFoqVWEfdJs+56E1BeUsHRUSxTjs
2v6y7AC3s9gZC/9Icpui3mXvxAQQPTsaF0UfjEPiW+q+NiC07FtdqHo=
=S1Dc
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2021-01-26' into staging
Block patches:
- Make backup block jobs use asynchronous requests with the block-copy
module
- Use COR filter node for stream block jobs
- Make coroutine-sigaltstack’s qemu_coroutine_new() function thread-safe
- Report error string when file locking fails with an unexpected errno
- iotest fixes, additions, and some refactoring
# gpg: Signature made Tue 26 Jan 2021 14:16:59 GMT
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40
* remotes/maxreitz/tags/pull-block-2021-01-26: (53 commits)
iotests/178: Pass value to invalid option
iotests/118: Drop 'change' test
iotests: Add test for the regression fixed in c8bf9a9169
block: report errno when flock fcntl fails
simplebench: add bench-backup.py
simplebench: bench_block_job: add cmd_options argument
simplebench/bench_block_job: use correct shebang line with python3
block/block-copy: drop unused argument of block_copy()
block/block-copy: drop unused block_copy_set_progress_callback()
qapi: backup: disable copy_range by default
backup: move to block-copy
block/backup: drop extra gotos from backup_run()
block/block-copy: make progress_bytes_callback optional
iotests: 257: prepare for backup over block-copy
iotests: 219: prepare for backup over block-copy
iotests: 185: prepare for backup over block-copy
iotests/129: Limit backup's max-chunk/max-workers
iotests: 56: prepare for backup over block-copy
qapi: backup: add max-chunk and max-workers to x-perf struct
job: call job_enter from job_pause
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
The following libraries will be selected if a feature requires it:
- capstone
- fdt
- SLiRP
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210122204441.2145197-5-philmd@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Since the meson build system rework, the configure script prefers the
git submodules over the system libraries. So we are testing compilation
with capstone, fdt and libslirp as a submodule all over the place,
burning CPU cycles by recompiling these third party modules and wasting
some network bandwidth in the CI by cloning the submodules each time.
Let's stop doing that in at least a couple of jobs and use the system
libraries instead.
While we're at it, also install meson in the Fedora container, since
it is new enough already, so we do not need to check out the meson
submodule here.
Message-Id: <20210121174451.658924-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add a configuration tested by Peter Maydell (see [1] and [2])
but not covered in our CI [3]:
[705/2910] Compiling C object libqemu-arm-linux-user.fa.p/linux-user_strace.c.o
FAILED: libqemu-arm-linux-user.fa.p/linux-user_strace.c.o
../linux-user/strace.c: In function 'do_print_sockopt':
../linux-user/strace.c:2831:14: error: 'IPV6_ADDR_PREFERENCES' undeclared (first use in this function)
case IPV6_ADDR_PREFERENCES:
^
This job currently takes 31 minutes 32 seconds ([4]).
[1] https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg05086.html
[2] https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg05379.html
[3] https://gitlab.com/philmd/qemu/-/jobs/977408284
[4] https://gitlab.com/philmd/qemu/-/jobs/978223286
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210121172829.1643620-3-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
We need to install the static libc package to be able to run
the TCG tests:
$ make check-tcg
...
BUILD TCG tests for x86_64-softmmu
BUILD x86_64-softmmu guest-tests with cc
/usr/bin/ld: hello: warning: allocated section `.notes' not in segment
/usr/bin/ld: memory: warning: allocated section `.notes' not in segment
BUILD TCG tests for x86_64-linux-user
BUILD x86_64-linux-user guest-tests with cc
/usr/bin/ld: cannot find -lpthread
/usr/bin/ld: cannot find -lc
collect2: error: ld returned 1 exit status
make[2]: *** [threadcount] Error 1
make[1]: *** [cross-build-guest-tests] Error 2
make: *** [build-tcg-tests-x86_64-linux-user] Error 2
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210121172829.1643620-2-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
ccd3b3b811 has deprecated short-hand boolean options (i.e., options
with values). All options without values are interpreted as boolean
options, so this includes the invalid option "snapshot.foo" used in
iotest 178.
So after ccd3b3b811, 178 fails with:
+qemu-img: warning: short-form boolean option 'snapshot.foo' deprecated
+Please use snapshot.foo=on instead
Suppress that deprecation warning by passing some value to it (it does
not matter which, because the option is invalid anyway).
Fixes: ccd3b3b811
("qemu-option: warn for short-form boolean options")
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210126123834.115915-1-mreitz@redhat.com>
Commit 0afec75734 removed the 'change' QMP command, so we can no
longer test it in 118.
Fixes: 0afec75734
('qmp: remove deprecated "change" command')
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210126104833.57026-1-mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
When a call to fcntl(2) for the purpose of adding file locks fails
with an error other than EAGAIN or EACCES, report the error returned
by fcntl.
EAGAIN or EACCES are elided as they are considered to be common
failures, indicating that a conflicting lock is held by another
process.
No errors are elided when removing file locks.
Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Message-Id: <20210113164447.2545785-1-david.edmondson@oracle.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-22-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-21-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Drop unused code.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-20-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Further commit will add a benchmark
(scripts/simplebench/bench-backup.py), which will show that backup
works better with async parallel requests (previous commit) and
disabled copy_range. So, let's disable copy_range by default.
Note: the option was added several commits ago with default to true,
to follow old behavior (the feature was enabled unconditionally), and
only now we are going to change the default behavior.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-19-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
This brings async request handling and block-status driven chunk sizes
to backup out of the box, which improves backup performance.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-18-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-17-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
We are going to stop use of this callback in the following commit.
Still the callback handling code will be dropped in a separate commit.
So, for now let's make it optional.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-16-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Iotest 257 dumps a lot of in-progress information of backup job, such
as offset and bitmap dirtiness. Further commit will move backup to be
one block-copy call, which will introduce async parallel requests
instead of plain cluster-by-cluster copying. To keep things
deterministic, allow only one worker (only one copy request at a time)
for this test.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-15-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
The further change of moving backup to be a one block-copy call will
make copying chunk-size and cluster-size two separate things. So, even
with 64k cluster sized qcow2 image, default chunk would be 1M.
Test 219 depends on specified chunk-size. Update it for explicit
chunk-size for backup as for mirror.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-14-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
The further change of moving backup to be a one block-copy call will
make copying chunk-size and cluster-size two separate things. So, even
with 64k cluster sized qcow2 image, default chunk would be 1M.
185 test however assumes, that with speed limited to 64K, one iteration
would result in offset=64K. It will change, as first iteration would
result in offset=1M independently of speed.
So, let's explicitly specify, what test wants: set max-chunk to 64K, so
that one iteration is 64K. Note, that we don't need to limit
max-workers, as block-copy rate limiter will handle the situation and
wouldn't start new workers when speed limit is obviously reached.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-13-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Right now, this does not change anything, because backup ignores
max-chunk and max-workers. However, as soon as backup is switched over
to block-copy for the background copying process, we will need it to
keep 129 passing.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210120102043.28346-1-mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
After introducing parallel async copy requests instead of plain
cluster-by-cluster copying loop, we'll have to wait for paused status,
as we need to wait for several parallel request. So, let's gently wait
instead of just asserting that job already paused.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-12-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Add new parameters to configure future backup features. The patch
doesn't introduce aio backup requests (so we actually have only one
worker) neither requests larger than one cluster. Still, formally we
satisfy these maximums anyway, so add the parameters now, to facilitate
further patch which will really change backup job behavior.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-11-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
If main job coroutine called job_yield (while some background process
is in progress), we should give it a chance to call job_pause_point().
It will be used in backup, when moved on async block-copy.
Note, that job_user_pause is not enough: we want to handle
child_job_drained_begin() as well, which call job_pause().
Still, if job is already in job_do_yield() in job_pause_point() we
should not enter it.
iotest 109 output is modified: on stop we do bdrv_drain_all() which now
triggers job pause immediately (and pause after ready is standby).
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210116214705.822267-10-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
We are going to use async block-copy call in backup, so we'll need to
passthrough setting backup speed to block-copy call.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-9-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Add function to cancel running async block-copy call. It will be used
in backup.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-8-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
We are going to directly use one async block-copy operation for backup
job, so we need rate limiter.
We want to maintain current backup behavior: only background copying is
limited and copy-before-write operations only participate in limit
calculation. Therefore we need one rate limiter for block-copy state
and boolean flag for block-copy call state for actual limitation.
Note, that we can't just calculate each chunk in limiter after
successful copying: it will not save us from starting a lot of async
sub-requests which will exceed limit too much. Instead let's use the
following scheme on sub-request creation:
1. If at the moment limit is not exceeded, create the request and
account it immediately.
2. If at the moment limit is already exceeded, drop create sub-request
and handle limit instead (by sleep).
With this approach we'll never exceed the limit more than by one
sub-request (which pretty much matches current backup behavior).
Note also, that if there is in-flight block-copy async call,
block_copy_kick() should be used after set-speed to apply new setup
faster. For that block_copy_kick() published in this patch.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-7-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
It simplifies debugging.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-6-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
They will be used for backup.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-5-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
We'll need async block-copy invocation to use in backup directly.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-4-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Refactor common path to use BlockCopyCallState pointer as parameter, to
prepare it for use in asynchronous block-copy (at least, we'll need to
run block-copy in a coroutine, passing the whole parameters as one
pointer).
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-3-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Experiments show, that copy_range is not always making things faster.
So, to make experimentation simpler, let's add a parameter. Some more
perf parameters will be added soon, so here is a new struct.
For now, add new backup qmp parameter with x- prefix for the following
reasons:
- We are going to add more performance parameters, some will be
related to the whole block-copy process, some only to background
copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other
block jobs, which will need performance options as well.. And it
should be the same structure or at least somehow related.
So, there are too much unclean things about how the interface and now
we need the new options mostly for testing. Let's keep them
experimental for a while.
In do_backup_common() new x-perf parameter handled in a way to
make further options addition simpler.
We add use-copy-range with default=true, and we'll change the default
in further patch, after moving backup to use block-copy.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-2-vsementsov@virtuozzo.com>
[mreitz: s/5\.2/6.0/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Disposition (action) for any given signal is global for the process.
When two threads run coroutine-sigaltstack's qemu_coroutine_new()
concurrently, they may interfere with each other: One of them may revert
the SIGUSR2 handler to SIG_DFL, between the other thread (a) setting up
coroutine_trampoline() as the handler and (b) raising SIGUSR2. That
SIGUSR2 will then terminate the QEMU process abnormally.
We have to ensure that only one thread at a time can modify the
process-global SIGUSR2 handler. To do so, wrap the whole section where
that is done in a mutex.
Alternatively, we could for example have the SIGUSR2 handler always be
coroutine_trampoline(), so there would be no need to invoke sigaction()
in qemu_coroutine_new(). Laszlo has posted a patch to do so here:
https://lists.nongnu.org/archive/html/qemu-devel/2021-01/msg05962.html
However, given that coroutine-sigaltstack is more of a fallback
implementation for platforms that do not support ucontext, that change
may be a bit too invasive to be comfortable with it. The mutex proposed
here may negatively impact performance, but the change is much simpler.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210125120305.19520-1-mreitz@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
And consequentially drop it from 297's skip list.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210118105720.14824-11-mreitz@redhat.com>
Peter Maydell
Philippe Mathieu-Daudé
Greg Kurz
Gerd Hoffmann
Joe Komlodi
Markus Armbruster
Wainer dos Santos Moschetta
Thomas Huth
Philippe Mathieu-Daudé
Max Reitz
Alberto Garcia
David Edmondson
Vladimir Sementsov-Ogievskiy