Commit Graph

77275 Commits

Author SHA1 Message Date
Lukas Straub
92c932de6c migration/colo.c: Relaunch failover even if there was an error
If vmstate_loading is true, secondary_vm_do_failover will set failover
status to FAILOVER_STATUS_RELAUNCH and return success without initiating
failover. However, if there is an error during the vmstate_loading
section, failover isn't relaunched. Instead we then wait for
failover on colo_incoming_sem.

Fix this by relaunching failover even if there was an error. Also,
to make this work properly, set vmstate_loading to false when
returning during the vmstate_loading section.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Message-Id: <f60b0a8e2fadaaec792e04819dfc46951842d6ba.1589193382.git.lukasstraub2@web.de>
Reviewed-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-06-01 18:44:27 +01:00
Lukas Straub
24fa16f8cc migration/colo.c: Flush ram cache only after receiving device state
If we suceed in receiving ram state, but fail receiving the device
state, there will be a mismatch between the two.

Fix this by flushing the ram cache only after the vmstate has been
received.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Message-Id: <3289d007d494cb0e2f05b1cf4ae6a78d300fede3.1589193382.git.lukasstraub2@web.de>
Reviewed-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-06-01 18:44:27 +01:00
Lukas Straub
786d8b8e38 migration/colo.c: Use cpu_synchronize_all_states()
cpu_synchronize_all_pre_loadvm() marks all vcpus as dirty, so the
registers are loaded from CPUState before we continue running
the vm. However if we failover during checkpoint, CPUState is not
initialized and the registers are loaded with garbage. This causes
guest hangs and crashes.

Fix this by using cpu_synchronize_all_states(), which initializes
CPUState from the current cpu registers additionally to marking
the vcpus as dirty.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Message-Id: <9675031ce557b73ebd10e7bd20ebbf57f30b177c.1589193382.git.lukasstraub2@web.de>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-06-01 18:44:27 +01:00
Lukas Straub
bb70b66ed7 migration/colo.c: Use event instead of semaphore
If multiple packets miscompare in a short timeframe, the semaphore
value will be increased multiple times. This causes multiple
checkpoints even if one would be sufficient.

Fix this by using a event instead of a semaphore for triggering
checkpoints. Now, checkpoint requests will be ignored until the
checkpoint event is sent to colo-compare (which releases the
miscompared packets).

Benchmark results (iperf3):
Client-to-server tcp:
without patch: ~66 Mbit/s
with patch: ~61 Mbit/s
Server-to-client tcp:
without patch: ~702 Kbit/s
with patch: ~16 Mbit/s

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Message-Id: <fd601ba1beb524aada54ba66e87ebfc12cf4574b.1589193382.git.lukasstraub2@web.de>
Reviewed-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-06-01 18:44:27 +01:00
Philippe Mathieu-Daudé
e0d138aa9b migration/vmstate: Remove unnecessary MemoryRegion forward declaration
"migration/vmstate.h" only uses pointer to MemoryRegion, which
is already forward declared in "qemu/typedefs.h".

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200530165512.15225-1-f4bug@amsat.org>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-06-01 18:44:27 +01:00
Miklos Szeredi
93bb3d8d4c virtiofsd: remove symlink fallbacks
Path lookup in the kernel has special rules for looking up magic symlinks
under /proc.  If a filesystem operation is instructed to follow symlinks
(e.g. via AT_SYMLINK_FOLLOW or lack of AT_SYMLINK_NOFOLLOW), and the final
component is such a proc symlink, then the target of the magic symlink is
used for the operation, even if the target itself is a symlink.  I.e. path
lookup is always terminated after following a final magic symlink.

I was erronously assuming that in the above case the target symlink would
also be followed, and so workarounds were added for a couple of operations
to handle the symlink case.  Since the symlink can be handled simply by
following the proc symlink, these workardouds are not needed.

Also remove the "norace" option, which disabled the workarounds.

Commit bdfd667883 ("virtiofsd: Fix xattr operations") already dealt with
the same issue for xattr operations.

Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
Message-Id: <20200514140736.20561-1-mszeredi@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-06-01 18:44:27 +01:00
Dr. David Alan Gilbert
7d2ef6dcc1 hmp: Simplify qom-set
Simplify qom_set by making it use qmp_qom_set and the JSON parser.

(qemu) qom-get /machine smm
"auto"
(qemu) qom-set /machine smm "auto"

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20200520151108.160598-3-dgilbert@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  With 's'->'S' type change suggested by Paolo and Markus
2020-06-01 18:44:27 +01:00
Dr. David Alan Gilbert
89cf4fe34f hmp: Implement qom-get HMP command
This started off as Andreas Färber's implementation from
March 2015, but after feedback from Paolo and Markus it morphed into
using the json output which handles structs reasonably.

Use with qom-list to find the members of an object.

(qemu) qom-get /backend/console[0]/device/vga.rom[0] size
65536
(qemu) qom-get /machine smm
"auto"
(qemu) qom-get /machine rtc-time
{
    "tm_year": 120,
    "tm_sec": 51,
    "tm_hour": 9,
    "tm_min": 50,
    "tm_mon": 4,
    "tm_mday": 20
}
(qemu) qom-get /machine frob
Error: Property '.frob' not found

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20200520151108.160598-2-dgilbert@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-06-01 18:44:27 +01:00
Pan Nengyuan
2f0c285aaa migration/rdma: cleanup rdma context before g_free to avoid memleaks
When error happen in initializing 'rdma_return_path', we should cleanup rdma context
before g_free(rdma) to avoid some memleaks. This patch fix that.

Reported-by: Euler Robot <euler.robot@huawei.com>
Signed-off-by: Pan Nengyuan <pannengyuan@huawei.com>
Message-Id: <20200508100755.7875-3-pannengyuan@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-06-01 18:44:27 +01:00
Pan Nengyuan
3b59ee7227 migration/rdma: fix potential nullptr access in rdma_start_incoming_migration
'rdma' is NULL when taking the first error branch in rdma_start_incoming_migration.
And it will cause a null pointer access in label 'err'. Fix that.

Fixes: 59c59c67ee
Signed-off-by: Pan Nengyuan <pannengyuan@huawei.com>
Message-Id: <20200508100755.7875-2-pannengyuan@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  Note this is CID 1428762
2020-06-01 18:44:27 +01:00
Peter Maydell
7ea32024c6 MIPS queue for June 1st, 2020
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQEcBAABAgAGBQJe1OaVAAoJENSXKoln91pltZMH/1n6ahhwKSi4KsSUqjatn/GQ
 Co5l2TmwEFRPnAnfurl0B5RnEkFCtjVh2ePGKy1TW2xydTnLEpDnVm+R9C4MHurJ
 H/tjdb275p7ZcJpK9A/cV9UjH0MKJs3mRGQHj6hNP7QU6wtV43N/jzrZTdyPE24k
 ikd9K0KzK0NGBOfVBz6v0dMYemS28VOGREgjAAOL/Tvxb6GYMzB/MR075J9C9Nrm
 HlD59Dob7hvpCc62IuR/Ac6c1DrxEeur4rD/Uc43jcK9h5t/mh+H/Yk4O0vX0WRk
 WRpVcb8FIz/osBc37nRRtjoEqw4THcoExq3W46iMxy/Xgniks4bAuYFi/+/BcUs=
 =lKcr
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-june-01-2020' into staging

MIPS queue for June 1st, 2020

# gpg: Signature made Mon 01 Jun 2020 12:29:25 BST
# gpg:                using RSA key D4972A8967F75A65
# gpg: Good signature from "Aleksandar Markovic <amarkovic@wavecomp.com>" [full]
# Primary key fingerprint: 8526 FBF1 5DA3 811F 4A01  DD75 D497 2A89 67F7 5A65

* remotes/amarkovic/tags/mips-queue-june-01-2020:
  hw/mips: fuloong2e: Set preferred page size to 16KB
  target/mips: Support variable page size
  target/mips: Add more CP0 register for save/restore
  hw/mips: Add CPU IRQ3 delivery for KVM
  configure: Add KVM target support for MIPS64
  tests/Makefile: Fix description of "make check"

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-06-01 13:43:59 +01:00
Huacai Chen
a08d60bc6c hw/mips: fuloong2e: Set preferred page size to 16KB
Loongson processor prefers 16KB page size in system emulator, so let's
define mc->minimum_page_bits to 14.

Cc: Jiaxun Yang <jiaxun.yang@flygoat.com>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <1586337380-25217-2-git-send-email-chenhc@lemote.com>
2020-06-01 13:28:21 +02:00
Huacai Chen
ee3863b9d4 target/mips: Support variable page size
Traditionally, MIPS use 4KB page size, but Loongson prefer 16KB page
size in system emulator. So, let's define TARGET_PAGE_BITS_VARY and
TARGET_PAGE_BITS_MIN to support variable page size.

Cc: Jiaxun Yang <jiaxun.yang@flygoat.com>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <1586337380-25217-1-git-send-email-chenhc@lemote.com>
2020-06-01 13:28:21 +02:00
Huacai Chen
7e0896b0e1 target/mips: Add more CP0 register for save/restore
Add more CP0 register for save/restore, including: EBase, XContext,
PageGrain, PWBase, PWSize, PWField, PWCtl, Config*, KScratch1~KScratch6.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <1588501221-1205-6-git-send-email-chenhc@lemote.com>
2020-06-01 13:28:21 +02:00
Huacai Chen
c3173a35bc hw/mips: Add CPU IRQ3 delivery for KVM
Currently, KVM/MIPS only deliver I/O interrupt via IP2, this patch add
IP3 delivery as well, because Loongson-3 based machine use both IRQ2
(CPU's IP2) and IRQ3 (CPU's IP3).

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <1588501221-1205-4-git-send-email-chenhc@lemote.com>
2020-06-01 13:28:21 +02:00
Huacai Chen
aa2953fd16 configure: Add KVM target support for MIPS64
Preparing for Loongson-3 virtualization, add KVM target support for
MIPS64 in configure script.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <1588501221-1205-2-git-send-email-chenhc@lemote.com>
2020-06-01 13:28:21 +02:00
Huacai Chen
9865f39db0 tests/Makefile: Fix description of "make check"
The description of "make check" is out-of-date, so fix it by adding
block and softfloat.

Reviewed-by: Claudio Fontana <cfontana@suse.de>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <1588674291-6486-1-git-send-email-chenhc@lemote.com>
2020-06-01 13:28:21 +02:00
Peter Maydell
b73f417aae Python queue:
* migration acceptance test fix
 * introduce pylintrc & flake8 config
 * various cleanups (Python3, style)
 * vm-test can set QEMU_LOCAL=1 to use locally built binaries
 * refactored BootLinuxBase & LinuxKernelTest acceptance classes
 
 https://gitlab.com/philmd/qemu/pipelines/151323210
 https://travis-ci.org/github/philmd/qemu/builds/693157969
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAl7T3U8ACgkQ4+MsLN6t
 wN4JnhAAif0Lw06lEl+ZksD0d4YLyBp079BNJEUWvflsivHY9RIn4e+eXdL3Q8m9
 roBzlsV71C7Ufbp26LFthcnvLaq0JH7RhBUVUoQAI7XZ46HAr3KRyOBJQP7LCr5N
 4Z97y8hqfSdchwpYbxkEbPy58caCRIneqIvg0sp8XuyXpDpVDqP11rXTg4fgqi7i
 1+D1yjr+wgaa7Vvf4sYzOw4D5zD2Mh+zMyDFI9d7yajs/4RH9k+iZteV7baLRQ5Q
 xkC0yqHDGp+uzEF4mk+5VUiZDvDUUxnkuFYKc6mFcahKzhrxLpEsvhnPFZ+vr4ib
 1DDmSr6ihf37wBzowHgAkmTwAiGmVEobu/2h93JXJesWw0TKRT74w1ftZKEIY1v4
 1Hka38gV0LULOAOjiy+aKNJqpJ/eipds94MvllRLHCgbB4H9VKBd4ts6linn+xsM
 CUebvUOgiVzH+hYbLJ1EBLFhbsmQ+yvopbQtLIlyFpKTFhdE1dA3vfb9NV0iqfOL
 fxaP/WaibKEFF5H40H7Ro+H7cT2+hF8MyByBT6q/UzoDURkZxeswqd2ww2VcUw2M
 X6h3/Hzek8PtZ+md3G6Hb1mJccfBHElrSgXAjrZ0WLOy4ZV7Y+/QrE8ooJwIKGKZ
 NinXrUocDl8xfRNWjynImzqma5TdaLW5tOmx6yTSK1R3lQh2z7A=
 =7xHS
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/python-next-20200531' into staging

Python queue:

* migration acceptance test fix
* introduce pylintrc & flake8 config
* various cleanups (Python3, style)
* vm-test can set QEMU_LOCAL=1 to use locally built binaries
* refactored BootLinuxBase & LinuxKernelTest acceptance classes

https://gitlab.com/philmd/qemu/pipelines/151323210
https://travis-ci.org/github/philmd/qemu/builds/693157969

# gpg: Signature made Sun 31 May 2020 17:37:35 BST
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/python-next-20200531: (25 commits)
  tests/acceptance: refactor boot_linux to allow code reuse
  tests/acceptance: refactor boot_linux_console test to allow code reuse
  tests/acceptance: allow console interaction with specific VMs
  tests/acceptance/migration.py: Wait for both sides
  tests/migration/guestperf: Use Python 3 interpreter
  tests/vm: allow wait_ssh() to specify command
  tests/vm: Add ability to select QEMU from current build
  tests/vm: Pass --debug through for vm-boot-ssh
  python/qemu/qtest: Check before accessing _qtest
  python/qemu/qmp: assert sockfile is not None
  python/qemu/qmp: use True/False for non/blocking modes
  python/qemu: Adjust traceback typing
  python/qemu: fix socket.makefile() typing
  python/qemu: remove Python2 style super() calls
  python/qemu: delint; add flake8 config
  python/qemu: delint and add pylintrc
  python/qemu/machine: remove logging configuration
  python/qemu/machine: add kill() method
  python: remove more instances of sys.version_info
  scripts/qmp: Fix shebang and imports
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-05-31 21:49:07 +01:00
Peter Maydell
4ec2a1f53e - Replace hw_error() with qemu_log_mask() in the m68k coldfire machine code
-----BEGIN PGP SIGNATURE-----
 
 iQJHBAABCAAxFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAl7SDukTHGh1dGhAdHV4
 ZmFtaWx5Lm9yZwAKCRAu2dd0/nAttW0KEACfK6zFpX6KA0BhdCaa0H12VO7q/U/c
 I8k5sTxGIN76fWBVL2d3PVd3v5RJkzfG3xYv/oeG0ZVvUYLw9um72fFrGrY+gzNp
 mpRle4WphP/ndt2yXf1/WNOfbng1H0cQgdxng0ET9VlWZLh2TP4Snyi5vWal6jK1
 uMjct28iLYW/8Ob+VTzRSPh8DIWhJiU0344EqrJFzlM/biyLb+yDStAfHKo/kBdV
 w1HVBwRr0IIPrwPpZFFS5QgKz1vebVcGJkKPAoRGZgre0tipJ5UXEXZ2Y1NRof7W
 dWF8VKA6NYKW/9NeFl1sD0xG02OUyD9XSjKpQUt8SqXITQW/V2x4ZD451j48DCUk
 Me+Usef7qsLU14pT20Dp8bzvNUt4fnlUo3DL4TUlGA3Tve8vIHgvqbV7sQjvOhI5
 imXARkB09bCAvhFWBRlQBDYFPlxbfpDE+xRSrwvjkZCWjIQgOSesL82vNDWhIAZC
 W5Ud6MT0JX/SzyvWt3kdq10LpxRsSFKFDUlKH+70zNlnc5UMjl9FQgQCJEbvawsh
 xPfxT2xzEGqpfrjjTlcB3098+zCd1btXkXok8757uhU+fx+c50swhWlqsELKICLB
 6VcCBOymUrpzpXXtlPx8GtDTIoCIJDiLXWVR7xmOHLkX0MfZbn1bZ6yBo6BM9PZZ
 M2u9Ad190FYr7Q==
 =hsg4
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2020-05-30' into staging

- Replace hw_error() with qemu_log_mask() in the m68k coldfire machine code

# gpg: Signature made Sat 30 May 2020 08:44:41 BST
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "huth@tuxfamily.org"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2020-05-30:
  hw/m68k/mcf52xx: Replace hw_error() by qemu_log_mask()
  hw/m68k/mcf5206: Reduce m5206_mbar_read/write() offset arg to 16-bit

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-05-31 20:43:45 +01:00
Pavel Dovgalyuk
1c80c87c8c tests/acceptance: refactor boot_linux to allow code reuse
This patch moves image downloading functions to the separate class to allow
reusing them from record/replay tests.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <159073593167.20809.17582679291556188984.stgit@pasha-ThinkPad-X280>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
Pavel Dovgalyuk
12121c496f tests/acceptance: refactor boot_linux_console test to allow code reuse
This patch splits code in BootLinuxConsole class into two different
classes to allow reusing it by record/replay tests.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <159073588490.20809.13942096070255577558.stgit@pasha-ThinkPad-X280>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
Pavel Dovgalyuk
a5ba86d423 tests/acceptance: allow console interaction with specific VMs
Console interaction in avocado scripts was possible only with single
default VM.
This patch modifies the function parameters to allow passing a specific
VM as a parameter to interact with it.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <159073587933.20809.5122618715976660635.stgit@pasha-ThinkPad-X280>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
Dr. David Alan Gilbert
2c9120a223 tests/acceptance/migration.py: Wait for both sides
When the source finishes migration the destination will still be
receiving the data sent by the source, so it might not have quite
finished yet, so won't quite have reached 'completed'.
This lead to occasional asserts in the next few checks.

After the source has finished, check the destination as well.
(We can't just switch to checking the destination, because it doesn't
give a status until it has started receiving the migration).

Reported-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200528112404.121972-1-dgilbert@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
Philippe Mathieu-Daudé
83389e22c5 tests/migration/guestperf: Use Python 3 interpreter
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-7-philmd@redhat.com>
2020-05-31 18:25:31 +02:00
Robert Foley
6ee982c9ab tests/vm: allow wait_ssh() to specify command
This allows for waiting for completion of arbitrary commands.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200529203458.1038-7-robert.foley@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
Robert Foley
e56c45047b tests/vm: Add ability to select QEMU from current build
Added a new special variable QEMU_LOCAL=1, which
will indicate to take the QEMU binary from the current
build.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200529203458.1038-6-robert.foley@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
Robert Foley
d5326a2437 tests/vm: Pass --debug through for vm-boot-ssh
This helps debug issues that occur during the boot sequence.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200529203458.1038-5-robert.foley@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
John Snow
c95dddce49 python/qemu/qtest: Check before accessing _qtest
It can be None; so add assertions or exceptions where appropriate to
guard the access accordingly.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-30-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
John Snow
de210ec53c python/qemu/qmp: assert sockfile is not None
In truth, if you don't do this, you'll just get a TypeError
exception. Now, you'll get an AssertionError.

Is this tangibly better? No.
Does mypy complain less? Yes.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-21-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
John Snow
7af67d694e python/qemu/qmp: use True/False for non/blocking modes
The type system doesn't want integers.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-15-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
John Snow
1dda0404d8 python/qemu: Adjust traceback typing
mypy considers it incorrect to use `bool` to statically return false,
because it will assume that it could conceivably return True, and gives
different analysis in that case. Use a None return to achieve the same
effect, but make mypy happy.

Note: Pylint considers function signatures as code that might trip the
duplicate-code checker. I'd rather not disable this as it does not
trigger often in practice, so I'm disabling it as a one-off and filed a
change request; see https://github.com/PyCQA/pylint/issues/3619

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-14-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
John Snow
0add048fbd python/qemu: fix socket.makefile() typing
Note:

A bug in typeshed (https://github.com/python/typeshed/issues/3977)
misinterprets the type of makefile(). Work around this by explicitly
stating that we are opening a text-mode file.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-13-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
John Snow
3797dbcbb7 python/qemu: remove Python2 style super() calls
Use the Python3 style instead.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-12-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
John Snow
8dfac2edb2 python/qemu: delint; add flake8 config
Mostly, ignore the "no bare except" rule, because flake8 is not
contextual and cannot determine if we re-raise. Pylint can, though, so
always prefer pylint for that.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200528222129.23826-5-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:30 +02:00
John Snow
9b8ccd6d5b python/qemu: delint and add pylintrc
Bring our these files up to speed with pylint 2.5.0.
Add a pylintrc file to formalize which pylint subset
we are targeting.

The similarity ignore is there to suppress similarity
reports across imports, which for typing constants,
are going to trigger this report erroneously.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200528222129.23826-4-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:30 +02:00
John Snow
053774bdec python/qemu/machine: remove logging configuration
Python 3.5 and above do not print a warning when logging is not
configured. As a library, it's best practice to leave logging
configuration to the client executable.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-22-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:30 +02:00
Vladimir Sementsov-Ogievskiy
e0e925a611 python/qemu/machine: add kill() method
Add method to hard-kill vm, without any quit commands.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Message-Id: <20200217150246.29180-19-vsementsov@virtuozzo.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:07 +02:00
John Snow
2d110c1149 python: remove more instances of sys.version_info
We guarantee 3.5+ everywhere; remove more dead checks. In general, try
to avoid using version checks and instead prefer to attempt behavior
when possible.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514035230.25756-1-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:07 +02:00
John Snow
c7b942d7f8 scripts/qmp: Fix shebang and imports
There's more wrong with these scripts; They are in various stages of
disrepair. That's beyond the scope of this current patchset.

This just mechanically corrects the imports and the shebangs, as part of
ensuring that the python/qemu/lib refactoring didn't break anything
needlessly.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20200528222129.23826-2-jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 13:56:46 +02:00
Philippe Mathieu-Daudé
5aa628045d scripts/modules/module_block: Use Python 3 interpreter & add pseudo-main
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-6-philmd@redhat.com>
2020-05-31 13:56:46 +02:00
Philippe Mathieu-Daudé
e57a707a82 scripts/kvm/vmxcap: Use Python 3 interpreter and add pseudo-main()
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-5-philmd@redhat.com>
2020-05-31 13:56:46 +02:00
Philippe Mathieu-Daudé
06d4c71f48 scripts/qmp: Use Python 3 interpreter
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-4-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 13:56:46 +02:00
Philippe Mathieu-Daudé
806c200ef4 scripts/qemu-gdb: Use Python 3 interpreter
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-3-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 13:56:46 +02:00
Philippe Mathieu-Daudé
6acc5c0469 scripts/qemugdb: Remove shebang header
These scripts are loaded as plugin by GDB (and they don't
have any __main__ entry point). Remove the shebang header.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-2-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 13:56:46 +02:00
Philippe Mathieu-Daudé
b809667808 hw/m68k/mcf52xx: Replace hw_error() by qemu_log_mask()
hw_error() calls exit(). This a bit overkill when we can log
the accesses as unimplemented or guest error.

When fuzzing the devices, we don't want the whole process to
exit. Replace some hw_error() calls by qemu_log_mask().

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200526094052.1723-3-f4bug@amsat.org>
Reviewed-by: Thomas Huth <huth@tuxfamily.org>
Signed-off-by: Thomas Huth <huth@tuxfamily.org>
2020-05-30 09:17:46 +02:00
Philippe Mathieu-Daudé
ccff1ae4df hw/m68k/mcf5206: Reduce m5206_mbar_read/write() offset arg to 16-bit
All calls to m5206_mbar_read/m5206_mbar_write are used with
'offset = hwaddr & 0x3ff', so we are sure the offset fits
in 16-bit.

Suggested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200526094052.1723-2-f4bug@amsat.org>
Signed-off-by: Thomas Huth <huth@tuxfamily.org>
2020-05-30 09:16:28 +02:00
Peter Maydell
ce20db593f bitmaps patches for 2020-05-26
- fix non-blockdev migration of bitmaps when mirror job is in use
 - add bitmap sizing to 'qemu-img measure'
 - add 'qemu-img convert --bitmaps'
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAl7QAA8ACgkQp6FrSiUn
 Q2pRxgf+N+X929X+o6kELEFBXMckOn8LC94A/6RjZKYywY3nV1ynESYflwznbYqx
 xQzXA2fTHK0O9n9av9pSMqs612HvxQyCJ51btit9QyOKW79//5QiNBkmVzXRwaN7
 hNnq5eHgDtH7O0jJPbJ1kxY6oufaudl/npEbBINlXknkIyvtChRfMhkAEOcV0N/y
 OxnmFm02r3TENoDmLdCmmi6iSDBvUl2th4p5ICZ0D9qUOnL83NNnMoYNF1goH25Z
 AlJWLe577mV5rw3w+HpoPutJZffPPiveCgW04WaEOhF4b0RzJFYazw9DM7PQiC+T
 hDzL3m15ZmgocgDLbbXEJiCVsDUwTA==
 =83Zm
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-05-26-v3' into staging

bitmaps patches for 2020-05-26

- fix non-blockdev migration of bitmaps when mirror job is in use
- add bitmap sizing to 'qemu-img measure'
- add 'qemu-img convert --bitmaps'

# gpg: Signature made Thu 28 May 2020 19:16:47 BST
# gpg:                using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg:                 aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-bitmaps-2020-05-26-v3:
  iotests: Add test 291 to for qemu-img bitmap coverage
  qemu-img: Add convert --bitmaps option
  qemu-img: Factor out code for merging bitmaps
  qcow2: Expose bitmaps' size during measure
  iotests: Fix test 178
  migration: forbid bitmap migration by generated node-name
  migration: add_bitmaps_to_list: check disk name once
  iotests: 194: test also migration of dirty bitmap
  migration: fix bitmaps pre-blockdev migration with mirror job
  block/dirty-bitmap: add bdrv_has_named_bitmaps helper
  migration: refactor init_dirty_bitmap_migration

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-05-29 19:25:54 +01:00
Peter Maydell
c86274bc2e Testing and one plugin fix:
- support alternates for genisoimage to test/vm
   - add clang++ to clang tests
   - fix record/replay smoke test
   - enable more softfloat tests
   - better detection of hung gdb
   - upgrade aarch64 tcg test x-compile to gcc-10
   - fix plugin cpu_index clash vs threads
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAl7OazAACgkQ+9DbCVqe
 KkRFQwf/VPIy5u556hyXsXLW4cZeJG8zx4fOUhkLTecDL1ch6A+9io1EIyWCmkyU
 GmyuR7Lg5J3f5gsOHydNTiOKu3/VKYkjFAGhGGjz2jDHFz7tA3UWgMO58Nce0g5R
 IxNGD0IrJwyM0m1H5tB8NEkd7AAOnpGslr0YoE5eX1MHpeXSiDmXxIvYUA/7nktY
 AFld8wNh9IG1Gg8Ww5Iyxn4fuxggEtzFFvvjrZVZq5rKdDgv+9D3M2cIjNvfmSGV
 zIBjZBOFwopGk9cNMTk91oo4mEqH3BBrCHsxAmOnyf4WzH4us8TdRCr9dHKzxiFt
 YN/3iOP2IUZqstGGV/PY50WI0RQEoQ==
 =2Krq
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-tcg-plugins-270520-1' into staging

Testing and one plugin fix:

  - support alternates for genisoimage to test/vm
  - add clang++ to clang tests
  - fix record/replay smoke test
  - enable more softfloat tests
  - better detection of hung gdb
  - upgrade aarch64 tcg test x-compile to gcc-10
  - fix plugin cpu_index clash vs threads

# gpg: Signature made Wed 27 May 2020 14:29:20 BST
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-tcg-plugins-270520-1:
  tests/tcg: add new threadcount test
  linux-user: properly "unrealize" vCPU object
  cpus-common: ensure auto-assigned cpu_indexes don't clash
  tests/docker: use a gcc-10 based image for arm64 tests
  tests/docker: add debian11 base image
  tests/tcg: better detect confused gdb which can't connect
  tests/fp: split and audit the conversion tests
  tests/fp: enable extf80_le_quite tests
  tests/tcg: fix invocation of the memory record/replay tests
  travis.yml: Use clang++ in the Clang tests
  tests/vm: pass --genisoimage to basevm script
  configure: add alternate binary for genisoimage

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-05-29 17:41:45 +01:00
Eric Blake
cf2d1203dc iotests: Add test 291 to for qemu-img bitmap coverage
Add a new test covering the 'qemu-img bitmap' subcommand, as well as
'qemu-img convert --bitmaps', both added in recent patches.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200521192137.1120211-6-eblake@redhat.com>
2020-05-28 13:16:30 -05:00
Eric Blake
15e39ad950 qemu-img: Add convert --bitmaps option
Make it easier to copy all the persistent bitmaps of (the top layer
of) a source image along with its guest-visible contents, by adding a
boolean flag for use with qemu-img convert.  This is basically
shorthand, as the same effect could be accomplished with a series of
'qemu-img bitmap --add' and 'qemu-img bitmap --merge -b source'
commands, or by their corresponding QMP commands.

Note that this command will fail in the same scenarios where 'qemu-img
measure' omits a 'bitmaps size:' line, namely, when either the source
or the destination lacks persistent bitmap support altogether.

See also https://bugzilla.redhat.com/show_bug.cgi?id=1779893

While touching this, clean up a couple coding issues spotted in the
same function: an extra blank line, and merging back-to-back 'if
(!skip_create)' blocks.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521192137.1120211-5-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2020-05-28 13:16:30 -05:00