Commit Graph

90420 Commits

Author SHA1 Message Date
Peter Maydell 6b54a31bf7 Python Pull request
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+ber27ys35W+dsvQfe+BBqr8OQ4FAmFSGncACgkQfe+BBqr8
 OQ6AaxAAmR8SaIGtOy/OdbtIFGbnT2m4h+NThvHQL+gjfoGovPd6BiK7p0ZfjOlK
 mq/NlJ3vKkhdqmM2OvLbK9bbrUkS/fB8kd7mxJusEb1FnvuPWPAA++p+IijvUKxO
 igGxdL+nVYXIdBTUd/WawUXWlOUnjS93e7odKnYmGKbLPVF3t5N107zs1BDiTUCw
 /l1lAJEd7VqAyg6MJ59IreN04YG3uUHXKEfqxKUiYBuuxdl9so6cnZ8uLRpY8BFd
 2JImO08X/7GZxANsdBQtTRJTicY/NZ0H8Vv70bspQJq/cCTKpiQsB2HCvBtA3KFE
 n6S/CXasQphbG/4EcZ/BBqQ8w/v8pBHJJHIRFEbjCKBBl7hrthQoZrkkajDRxHxD
 lvMI9jl6iOXJLPfx4nhIjqecl2W+hUTt8J9S75y+tvSypCDtykbF4CuIvMWwbC1d
 A8Q4X8KB4vG0OWMrnfQVCHM7zxcbHeWoD3dhcZEQZcfDYlBIzVr1s2wipRQP05D/
 OYjw21/QLkbY6ZAwS8RKApbnQtQS8GbV+hrflK9ZUmTuhUWPTd1qcUv1tuSDgMTl
 2pyathUX9Qz90glXb2zPtmX+vLlaf2Gaf/fi/OSKLV+h3pPojUQ+QsCOedsI2MOX
 BxvM0Q8gm3L7K/94tArovuScbiERwmg0kQqdyyZEun7BbkipZH4=
 =OlV7
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/jsnow-gitlab/tags/python-pull-request' into staging

Python Pull request

# gpg: Signature made Mon 27 Sep 2021 20:24:39 BST
# gpg:                using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F  18F2 88A9 064D 1835 61EB
#      Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76  CBD0 7DEF 8106 AAFC 390E

* remotes/jsnow-gitlab/tags/python-pull-request: (32 commits)
  python/aqmp-tui: Add syntax highlighting
  python: add optional pygments dependency
  python: Add entry point for aqmp-tui
  python/aqmp-tui: Add AQMP TUI
  python: Add dependencies for AQMP TUI
  python/aqmp: Add Coverage.py support
  python/aqmp: add LineProtocol tests
  python/aqmp: add AsyncProtocol unit tests
  python: bump avocado to v90.0
  python/aqmp: add scary message
  python/aqmp: add asyncio_run compatibility wrapper
  python/aqmp: add _raw() execution interface
  python/aqmp: add execute() interfaces
  python/aqmp: Add message routing to QMP protocol
  python/pylint: disable no-member check
  python/aqmp: add QMP protocol support
  python/pylint: disable too-many-function-args
  python/aqmp: add QMP event support
  python/aqmp: add well-known QMP object models
  python/aqmp: add QMP Message format
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-09-28 13:07:32 +01:00
Peter Maydell 14f02d8a9e Integration testing patches
- More Linux kernel record/replay tests (Pavel Dovgalyuk)
 - Various fixes (Willian Rampazzo, Cleber Rosa)
 - Split machine_ppc.py per machine (David Gibson)
 - Add AVOCADO_TESTS command line environment variable (Willian Rampazzo)
 - Test PowerPC PowerNV 8/9 machines (Cédric Le Goater)
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAmFR/jMACgkQ4+MsLN6t
 wN7iPw//QqstsMTQX7PvjbFxi2VkQtTCTMlgOkqdovs4w5ThHxS9lSvEZeJ0bsMM
 1Uutufg7CzVoZuB1aKewiJ5qZ0y3QrNPhdE2hcxFLCZogCmN6Iuyr92URfhP0TIe
 eMLqiHtPuIsq1zApTxvdn5DMBZnonMFE3rX3wWaK6wHPELNai+28hJr+M3pe98Kt
 8xhIKkQ0Cz3QI41wkDzbpwyvYmDuLbSzsCUFp+8codxFHKer873cKGs80CSTNscI
 s+ZDVHc/ZfX0JXtn3XTnb65A4wuSz8r7PPT5vZ12AbgG6UEzaSOqyx+NcomDV9+S
 8jJMI5fWeyrk1S4xSs96ALuYYP50gi8SXZ59WXpduQvMji4pB5EQ3EWtcackZe8p
 JAPYFEISJa/sLG//6oHlRcYXTHti/aAw4L6a6sLzocYd9qEMsGKr5AlsYkm+TYFF
 uO4YjcGYJazvJg1n86YgEIzfxcJt7haltUhNcmYltXOtHEJFd7+VRzZosy8v/zQ5
 NuPEt5jfsZdRyrnEg7FowI2Y6N9LJqPz9PBVgyIuRKPD86Irw/yDrl92ECnp8f8j
 VEpKgXrp2stffBhedD4pUvG+4OfeNndvo5ZnV9mSp6LKGa+4hUHMtMpnASpw8gfT
 Cul7cgtRfexvXKmXvYKCjrwwJ0VO0cQVRd2wzwBlM+5txycvbdg=
 =1mf1
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd/tags/integration-testing-20210927' into staging

Integration testing patches

- More Linux kernel record/replay tests (Pavel Dovgalyuk)
- Various fixes (Willian Rampazzo, Cleber Rosa)
- Split machine_ppc.py per machine (David Gibson)
- Add AVOCADO_TESTS command line environment variable (Willian Rampazzo)
- Test PowerPC PowerNV 8/9 machines (Cédric Le Goater)

# gpg: Signature made Mon 27 Sep 2021 18:24:03 BST
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd/tags/integration-testing-20210927: (22 commits)
  tests/acceptance: Test powernv machines
  tests/acceptance/ppc_prep_40p.py: clean up unused import
  tests/acceptance/ppc_prep_40p.py: NetBSD 7.1.2 location update
  tests/acceptance/boot_xen.py: removed unused import
  acceptance/tests/vnc.py: use explicit syntax for enabling passwords
  Acceptance Tests: improve check-acceptance description
  qemu: Split machine_ppc.py acceptance tests
  tests/Makefile: add AVOCADO_TESTS option to make check-acceptance
  docs/devel/testing: add instruction to run a single acceptance test
  tests/Makefile: allow control over tags during check-acceptance
  avocado_qemu: fix inheritance order on LinuxTest class
  avocado_qemu: explicitly return None to avoid R1710
  avocado_qemu: tweak ssh connect method
  avocado_qemu: fix import module based on isort
  avocado_qemu: standardize super() call following PEP3135
  Acceptance Tests: add standard clean up at test tearDown()
  Acceptance tests: add myself as a reviewer for the acceptance tests
  tests/acceptance: Linux boot test for record/replay
  tests/acceptance: add replay kernel test for alpha
  tests/acceptance: add replay kernel test for nios2
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-09-27 19:52:43 +01:00
Cédric Le Goater 4c5fc0c5fc tests/acceptance: Test powernv machines
Fetch the OpenPOWER images to boot the powernv8 and powernv9 machines
with a simple PCI layout.

Cc: Cleber Rosa <crosa@redhat.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210817093036.1288791-1-clg@kaod.org>
2021-09-27 19:21:37 +02:00
Cleber Rosa 3d2ec56550 tests/acceptance/ppc_prep_40p.py: clean up unused import
Just a removal of an unused imported symbol.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210924185506.2542588-16-crosa@redhat.com>
2021-09-27 19:06:47 +02:00
Cleber Rosa 089f25877f tests/acceptance/ppc_prep_40p.py: NetBSD 7.1.2 location update
The NetBSD-7.1.2-prep.iso is no longer available on the CDN, but it's
still available in the archive.

Let's update its location so that users without the file on cache can
still fetch it and run the test.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210924185506.2542588-15-crosa@redhat.com>
2021-09-27 19:06:47 +02:00
Cleber Rosa 6f1f86cfa7 tests/acceptance/boot_xen.py: removed unused import
Just a clean up for an unused import.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210924185506.2542588-13-crosa@redhat.com>
2021-09-27 19:06:47 +02:00
Cleber Rosa 0f981d8792 acceptance/tests/vnc.py: use explicit syntax for enabling passwords
This matches the command line on 82a17d1d67, where the "on" or "off"
should be explicitly given.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210924185506.2542588-9-crosa@redhat.com>
2021-09-27 19:06:47 +02:00
Cleber Rosa 181e1ab2ad Acceptance Tests: improve check-acceptance description
The "check-acceptance" make rule won't necessarily run *all* available
tests, because it employs a filter based on the currently configured
targets.  This change in the description of the rule makes that
behavior extra clear.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210924185506.2542588-3-crosa@redhat.com>
2021-09-27 19:06:47 +02:00
David Gibson c5f0a81650 qemu: Split machine_ppc.py acceptance tests
machine_ppc.py contains tests for 3 different ppc based machine types.  It
is listed in MAINTAINERS along with the PPC TCG cpu code.  That's not
really accurate though, since it's really more about testing those machines
than the CPUs.

Therefore, split it up into separate files for the separate machine types,
and list those along with their machine types in MAINTAINERS.

Suggested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210927044808.73391-2-david@gibson.dropbear.id.au>
2021-09-27 19:06:47 +02:00
Willian Rampazzo 94c714620b tests/Makefile: add AVOCADO_TESTS option to make check-acceptance
Add the possibility of running all the tests from a single file, or
multiple files, running a single test within a file or multiple tests
within multiple files using `make check-acceptance` and the
AVOCADO_TESTS environment variable.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210923161141.232208-4-willianr@redhat.com>
2021-09-27 19:06:47 +02:00
Willian Rampazzo 6676f18fa5 docs/devel/testing: add instruction to run a single acceptance test
Add instructions to the Acceptance tests section about running a
single test file or a test within the test file.

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210923161141.232208-3-willianr@redhat.com>
2021-09-27 19:06:47 +02:00
Willian Rampazzo 23022794de tests/Makefile: allow control over tags during check-acceptance
Although it is possible to run a specific test using the avocado
command-line, a user may want to use a specific tag while running the
``make check-acceptance`` during the development or debugging.

This allows using the AVOCADO_TAGS environment variable where the user
takes total control of which tests should run based on the tags defined.

This also makes the check-acceptance command flexible to restrict tests
based on tags while running on CI.

e.g.:

AVOCADO_TAGS="foo bar baz" make check-acceptance

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210923161141.232208-2-willianr@redhat.com>
2021-09-27 19:06:47 +02:00
Willian Rampazzo e519df437a avocado_qemu: fix inheritance order on LinuxTest class
Class hierarchy on Python is defined from right to left. Although the
current code is not harmful, let's fix it to avoid problems in the future.

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-7-willianr@redhat.com>
2021-09-27 19:06:42 +02:00
Willian Rampazzo 6c58af2e17 avocado_qemu: explicitly return None to avoid R1710
The linter is complaining the `pick_default_qemu_bin` is not explicitly
returning None. Fix it to explicitly return None and avoid R1710
inconsistent-return-statements.

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-6-willianr@redhat.com>
2021-09-27 19:06:36 +02:00
Willian Rampazzo 22e82e0982 avocado_qemu: tweak ssh connect method
The current implementation will crash if the connection fails as the
`time` module is not imported. Fix the import problem. While here,
tweaks the connection to wait progressively when the connection fails.

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[PMD: Reworded description]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-5-willianr@redhat.com>
2021-09-27 19:06:30 +02:00
Willian Rampazzo e410bac070 avocado_qemu: fix import module based on isort
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-4-willianr@redhat.com>
2021-09-27 19:06:25 +02:00
Willian Rampazzo 20bf915418 avocado_qemu: standardize super() call following PEP3135
PEP3135 states when calling super(), there is no need to use arguments.
This changes the calls on avocado_qemu to standardize according to
PEP3135 and avoid warnings from linters.

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-3-willianr@redhat.com>
2021-09-27 19:06:15 +02:00
Cleber Rosa 1f1fcf0c57 Acceptance Tests: add standard clean up at test tearDown()
The avocado.Test class, used as the basis of the avocado_qemu.Test
class, performs a clean of temporary directories up as part of its own
tearDown() implementation.

But the avocado_qemu.Test class is currently missing the same clean
up, as it implemented its own tearDown() method without resorting to
the upper class behavior.

This brings avocado_qemu.Test behavior in sync with the standard
avocado.Test behavior and prevents temporary directories from
cluttering the test results directory (unless instructed to do so with
Avocado's "--keep-tmp" option).

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
[willianr: respin to new Python super format]
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-2-willianr@redhat.com>
2021-09-27 19:06:15 +02:00
Willian Rampazzo 8adacf7fa9 Acceptance tests: add myself as a reviewer for the acceptance tests
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210809192907.42138-1-willianr@redhat.com>
2021-09-27 19:06:06 +02:00
Pavel Dovgalyuk 82184f4018 tests/acceptance: Linux boot test for record/replay
This patch adds a test for record/replay, which boots Linux
image from the disk and interacts with the network.
The idea and code of this test is borrowed from boot_linux.py
This test includes only x86_64 platform. Other platforms and
machines will be added later after testing and improving
record/replay to completely support them.

Each test consists of the following phases:
 - downloading the disk image
 - recording the execution
 - replaying the execution

Replay does not validates the output, but waits until QEMU
finishes the execution. This is reasonable, because
QEMU usually hangs when replay goes wrong.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <162737554047.1735673.13133593401566029378.stgit@pasha-ThinkPad-X280>
2021-09-27 18:55:41 +02:00
Pavel Dovgalyuk 7f7c382a4d tests/acceptance: add replay kernel test for alpha
This patch adds record/replay test which boots Linux
kernel on alpha platform. The test uses kernel binaries
taken from boot_linux_console test.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <162737553482.1735673.10021851966976933952.stgit@pasha-ThinkPad-X280>
2021-09-27 18:51:40 +02:00
Pavel Dovgalyuk ce9771f5a4 tests/acceptance: add replay kernel test for nios2
This patch adds record/replay test which boots Linux
kernel on nios2 platform. The test uses kernel binaries
taken from boot_linux_console test.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <162737552919.1735673.12493523185952280539.stgit@pasha-ThinkPad-X280>
2021-09-27 18:51:40 +02:00
Pavel Dovgalyuk 27f551135e tests/acceptance: add replay kernel test for openrisc
This patch adds record/replay test which boots Linux
kernel on openrisc platform. The test uses kernel binaries
taken from boot_linux_console test.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <162737552350.1735673.14603125561530143423.stgit@pasha-ThinkPad-X280>
2021-09-27 18:51:40 +02:00
Pavel Dovgalyuk f7ade77931 tests/acceptance: add replay kernel test for s390x
This patch adds record/replay test which boots Linux
kernel on s390x platform. The test uses kernel binaries
taken from boot_linux_console test.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
[PMD: Drop default '-smp 1' as suggested by Thomas]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <162737551785.1735673.6775108576116333386.stgit@pasha-ThinkPad-X280>
2021-09-27 18:51:36 +02:00
G S Niteesh Babu 99e45a6131 python/aqmp-tui: Add syntax highlighting
Add syntax highlighting for the incoming and outgoing QMP messages.
This is achieved using the pygments module which was added in a
previous commit.

The current implementation is a really simple one which doesn't
allow for any configuration. In future this has to be improved
to allow for easier theme config using an external config of
some sort.

Signed-off-by: G S Niteesh Babu <niteesh.gs@gmail.com>
Message-Id: <20210823220746.28295-6-niteesh.gs@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
G S Niteesh Babu f37c34d601 python: add optional pygments dependency
Added pygments as optional dependency for AQMP TUI.
This is required for the upcoming syntax highlighting feature
in AQMP TUI.
The dependency has also been added in the devel optional group.

Added mypy 'ignore_missing_imports' for pygments since it does
not have any type stubs.

Signed-off-by: G S Niteesh Babu <niteesh.gs@gmail.com>
Message-Id: <20210823220746.28295-5-niteesh.gs@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
G S Niteesh Babu 35755f7d4f python: Add entry point for aqmp-tui
Add an entry point for aqmp-tui. This will allow it to be run from
the command line using "aqmp-tui localhost:1234"
More options available in the TUI can be found using "aqmp-tui -h"

Signed-off-by: G S Niteesh Babu <niteesh.gs@gmail.com>
Message-Id: <20210823220746.28295-4-niteesh.gs@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
G S Niteesh Babu aeb6b48a47 python/aqmp-tui: Add AQMP TUI
Added AQMP TUI.

Implements the follwing basic features:
1) Command transmission/reception.
2) Shows events asynchronously.
3) Shows server status in the bottom status bar.
4) Automatic retries on disconnects and error conditions.

Also added type annotations and necessary pylint/mypy configurations.

Signed-off-by: G S Niteesh Babu <niteesh.gs@gmail.com>
Message-Id: <20210823220746.28295-3-niteesh.gs@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
G S Niteesh Babu 974e2f4722 python: Add dependencies for AQMP TUI
Added dependencies for the upcoming AQMP TUI under the optional
'tui' group.

The same dependencies have also been added under the devel group
since no work around has been found for optional groups to imply
other optional groups.

Signed-off-by: G S Niteesh Babu <niteesh.gs@gmail.com>
Message-Id: <20210823220746.28295-2-niteesh.gs@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow a4ffaecd57 python/aqmp: Add Coverage.py support
I'm not exposing this via the Makefile help, it's not likely to be
useful to passersby. Switch the avocado runner to the 'legacy' runner
for now, as the new runner seems to obscure coverage reports, again.

Usage is to enter your venv of choice and then:
`make check-coverage && xdg-open htmlcov/index.html`.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210915162955.333025-28-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow 8193b9d148 python/aqmp: add LineProtocol tests
Tests a real connect, a real accept, and really sending and receiving a
message over a UNIX socket.

Brings coverage of protocol.py up to ~93%.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210915162955.333025-27-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow a1f71b61ea python/aqmp: add AsyncProtocol unit tests
This tests most of protocol.py -- From a hacked up Coverage.py run, it's
at about 86%. There's a few error cases that aren't very well tested
yet, they're hard to induce artificially so far. I'm working on it.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210915162955.333025-26-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow 4320f7172f python: bump avocado to v90.0
Avocado v90 includes improved support for running async unit tests. The
workaround that existed prior to v90 causes the unit tests to fail
afterwards, however, so upgrade our minimum version pin to the very
latest and greatest.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210915162955.333025-25-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow ed6d4d7a95 python/aqmp: add scary message
Add a warning whenever AQMP is used to steer people gently away from
using it for the time-being.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210915162955.333025-24-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow debbabd77f python/aqmp: add asyncio_run compatibility wrapper
As a convenience. It isn't used by the library itself, but it is used by
the test suite. It will also come in handy for users of the library
still on Python 3.6.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210915162955.333025-23-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow 41f4f92260 python/aqmp: add _raw() execution interface
This is added in anticipation of wanting it for a synchronous wrapper
for the iotest interface. Normally, execute() and execute_msg() both
raise QMP errors in the form of Python exceptions.

Many iotests expect the entire reply as-is. To reduce churn there, add a
private execution interface that will ease transition churn. However, I
do not wish to encourage its use, so it will remain a private interface.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210915162955.333025-22-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow e0fea0b3ac python/aqmp: add execute() interfaces
Add execute() and execute_msg().

_execute() is split into _issue() and _reply() halves so that
hypothetical subclasses of QMP that want to support different execution
paradigms can do so.

I anticipate a synchronous interface may have need of separating the
send/reply phases. However, I do not wish to expose that interface here
and want to actively discourage it, so they remain private interfaces.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210915162955.333025-21-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow 577737be55 python/aqmp: Add message routing to QMP protocol
Add the ability to handle and route messages in qmp_protocol.py. The
interface for actually sending anything still isn't added until next
commit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210915162955.333025-20-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow 4cd17f375d python/pylint: disable no-member check
mypy handles this better -- but we only need the workaround because
pylint under Python 3.6 does not understand that a MutableMapping really
does have a .get() method attached.

We could remove this again once 3.7 is our minimum.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210915162955.333025-19-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow c67d696b85 python/aqmp: add QMP protocol support
The star of our show!

Add most of the QMP protocol, sans support for actually executing
commands. No problem, that happens in the next several commits.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210915162955.333025-18-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow 29a8ea9ba2 python/pylint: disable too-many-function-args
too-many-function-args seems prone to failure when considering
things like Method Resolution Order, which mypy gets correct. When
dealing with multiple inheritance, pylint doesn't seem to understand
which method will actually get called, while mypy does.

Remove the less powerful, redundant check.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210915162955.333025-17-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow b3cda213a7 python/aqmp: add QMP event support
This class was designed as a "mix-in" primarily so that the feature
could be given its own treatment in its own python module.

It gets quite a bit too long otherwise.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210915162955.333025-16-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow ad07299941 python/aqmp: add well-known QMP object models
The QMP spec doesn't define very many objects that are iron-clad in
their format, but there are a few. This module makes it trivial to
validate them without relying on an external third-party library.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210915162955.333025-15-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow 08f98a2231 python/aqmp: add QMP Message format
The Message class is here primarily to serve as a solid type to use for
mypy static typing for unambiguous annotation and documentation.

We can also stuff JSON serialization and deserialization into this class
itself so it can be re-used even outside this infrastructure.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20210915162955.333025-14-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow 762bd4d7a7 python/aqmp: add AsyncProtocol._readline() method
This is added as a courtesy: many protocols are line-based, including
QMP. Putting it in AsyncProtocol lets us keep the QMP class
implementation just a pinch more abstract.

(And, if we decide to add a QTEST implementation later, it will need
this, too. (Yes, I have a QTEST implementation.))

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20210915162955.333025-13-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow 12c7a57f5b python/aqmp: add _cb_inbound and _cb_outbound logging hooks
Add hooks designed to log/filter incoming/outgoing messages. The primary
intent for these is to be able to support iotests which may want to log
messages with specific filters for reproducible output.

Another use is for plugging into Urwid frameworks; all messages in/out
can be automatically added to a rendering list for the purposes of a
qmp-shell like tool.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20210915162955.333025-12-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow 2686ac1316 python/aqmp: add configurable read buffer limit
QMP can transmit some pretty big messages, and the default limit of 64KB
isn't sufficient. Make sure that we can configure it.

Reported-by: G S Niteesh Babu <niteesh.gs@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20210915162955.333025-11-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow 774c64a58d python/aqmp: add AsyncProtocol.accept() method
It's a little messier than connect, because it wasn't designed to accept
*precisely one* connection. Such is life.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20210915162955.333025-10-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow 50e533061f python/aqmp: add logging to AsyncProtocol
Give the connection and the reader/writer tasks nicknames, and add
logging statements throughout.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20210915162955.333025-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00
John Snow c1408345af python/aqmp: Add logging utility helpers
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20210915162955.333025-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-09-27 12:10:29 -04:00