Commit Graph

211 Commits

Author SHA1 Message Date
Daniel P. Berrangé
7a21bee2aa misc: fix commonly doubled up words
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220707163720.1421716-5-berrange@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-08-01 11:58:02 +02:00
Thomas Huth
9b0ecfaba5 python/qemu/qmp/legacy: Replace 'returns-whitelist' with the correct type
'returns-whitelist' has been renamed to 'command-returns-exceptions' in
commit b86df37478 ("qapi: Rename pragma *-whitelist to *-exceptions").

Message-Id: <20220711095721.61280-1-thuth@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-07-18 20:28:06 +02:00
Peter Maydell
9323e79f10 Fix 'writeable' typos
We have about 30 instances of the typo/variant spelling 'writeable',
and over 500 of the more common 'writable'.  Standardize on the
latter.

Change produced with:

  sed -i -e 's/\([Ww][Rr][Ii][Tt]\)[Ee]\([Aa][Bb][Ll][Ee]\)/\1\2/g' $(git grep -il writeable)

and then hand-undoing the instance in linux-headers/linux/kvm.h.

Most of these changes are in comments or documentation; the
exceptions are:
 * a local variable in accel/hvf/hvf-accel-ops.c
 * a local variable in accel/kvm/kvm-all.c
 * the PMCR_WRITABLE_MASK macro in target/arm/internals.h
 * the EPT_VIOLATION_GPA_WRITABLE macro in target/i386/hvf/vmcs.h
   (which is never used anywhere)
 * the AR_TYPE_WRITABLE_MASK macro in target/i386/hvf/vmx.h
   (which is never used anywhere)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Message-id: 20220505095015.2714666-1-peter.maydell@linaro.org
2022-06-08 19:38:47 +01:00
John Snow
e7874a50ff python: update for mypy 0.950
typeshed (included in mypy) recently updated to improve the typing for
WriteTransport objects. I was working around this, but now there's a
version where I shouldn't work around it.

Unfortunately this creates some minor ugliness if I want to support both
pre- and post-0.950 versions. For now, for my sanity, just disable the
unused-ignores warning.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220526000921.1581503-2-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-06 09:26:54 +02:00
John Snow
47430775ed python/qmp: remove pylint workaround from legacy.py
Pylint upgraded recently (2.13.z) and having a pylint: disable comment
in the middle of an argument field causes it some grief (It appears to
stop parsing when it encounters it, causing some syntax problems). Since
the duplicate line threshold was bumped up in 22305c2a08, we don't
need this workaround anymore. Drop it.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@openvz.org>
Message-id: 20220330172812.3427355-10-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-04-21 11:01:00 -04:00
John Snow
b1a9b1f7a6 python: rename 'aqmp-tui' to 'qmp-tui'
This is the last vestige of the "aqmp" moniker surviving in the tree; remove it.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@openvz.org>
Message-id: 20220330172812.3427355-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-04-21 11:01:00 -04:00
John Snow
37094b6dd5 python: rename qemu.aqmp to qemu.qmp
Now that we are fully switched over to the new QMP library, move it back
over the old namespace. This is being done primarily so that we may
upload this package simply as "qemu.qmp" without introducing confusion
over whether or not "aqmp" is a new protocol or not.

The trade-off is increased confusion inside the QEMU developer
tree. Sorry!

Note: the 'private' member "_aqmp" in legacy.py also changes to "_qmp";
not out of necessity, but just to remove any traces of the "aqmp"
name.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Acked-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@openvz.org>
Message-id: 20220330172812.3427355-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-04-21 11:01:00 -04:00
John Snow
adaca6e085 python: remove the old QMP package
Thank you for your service!

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Message-id: 20220330172812.3427355-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-04-21 11:01:00 -04:00
John Snow
b0654f4f98 python/aqmp: copy qmp docstrings to qemu.aqmp.legacy
Copy the docstrings out of qemu.qmp, adjusting them as necessary to
more accurately reflect the current state of this class.

(Licensing: This is copying and modifying GPLv2-only licensed docstrings
into a GPLv2-only file.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Message-id: 20220330172812.3427355-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-04-21 11:01:00 -04:00
John Snow
0c78ebf722 python/aqmp: fully separate from qmp.QEMUMonitorProtocol
After this patch, qemu.aqmp.legacy.QEMUMonitorProtocol no longer
inherits from qemu.qmp.QEMUMonitorProtocol. To do this, several
inherited methods need to be explicitly re-defined.

(Licensing: This is copying and modifying GPLv2-only code into a
GPLv2-only file.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Message-id: 20220330172812.3427355-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-04-21 11:01:00 -04:00
John Snow
9fcd3930e0 python/aqmp: take QMPBadPortError and parse_address from qemu.qmp
Shift these definitions over from the qmp package to the async qmp
package.

(Licensing: this is a lateral move, from GPLv2 (only) to GPLv2 (only))

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Message-id: 20220330172812.3427355-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-04-21 11:01:00 -04:00
John Snow
445c9d4e3d python/aqmp-tui: relicense as LGPLv2+
aqmp-tui, the async QMP text user interface tool, is presently licensed
as GPLv2+. I intend to include this tool as an add-on to an LGPLv2+
library package hosted on PyPI.org. I've selected LGPLv2+ to maximize
compatibility with other licenses while retaining a copyleft license.

To keep licensing matters simple, I'd like to relicense this tool as
LGPLv2+ as well in order to keep the resultant license of the hosted
release files simple -- even if library users won't "link against" this
command line tool.

Therefore, I am asking permission to loosen the license.

Niteesh is effectively the sole author of this code, with scattered
lines from myself.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: G S Niteesh Babu <niteesh.gs@gmail.com>
Message-id: 20220325200438.2556381-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-04-21 11:01:00 -04:00
John Snow
0e08b94700 python/qmp-shell: relicense as LGPLv2+
qmp-shell is presently licensed as GPLv2 (only). I intend to include
this tool as an add-on to an LGPLv2+ library package hosted on
PyPI.org. I've selected LGPLv2+ to maximize compatibility with other
licenses while retaining a copyleft license.

To keep licensing matters simple, I'd like to relicense this tool as
LGPLv2+ as well in order to keep the resultant license of the hosted
release files simple -- even if library users won't "link against" this
command line tool.

Therefore, I am asking permission from the current authors of this
tool to loosen the license. At present, those people are:

- John Snow (me!), 411/609
- Luiz Capitulino, Author, 97/609
- Daniel Berrangé, 81/609
- Eduardo Habkost, 10/609
- Marc-André Lureau, 6/609
- Fam Zheng, 3/609
- Cleber Rosa, 1/609

(All of which appear to have been written under redhat.com addresses.)

Eduardo's fixes are largely automated from 2to3 conversion tools and may
not necessarily constitute authorship, but his signature would put to
rest any questions.

Cleber's changes concern a single import statement change. Also won't
hurt to ask.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Acked-by: Fam Zheng <fam@euphon.net>
Acked-by: Luiz Capitulino <lcapitulino@redhat.com>
Acked-by: Eduardo Habkost <eduardo@habkost.net>
Acked-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20220325200438.2556381-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-04-21 11:01:00 -04:00
John Snow
9dcea96d08 python/aqmp: relicense as LGPLv2+
I am the sole author of all of the async QMP code (python/qemu/aqmp)
with the following exceptions:

python/qemu/aqmp/qmp_shell.py and python/qemu/aqmp/legacy.py were
written by Luiz Capitulino (et al) and are already licensed separately
as GPLv2 (only).

aqmp_tui.py was written by Niteesh Babu G S and is licensed as GPLv2+.

I wish to relicense as LGPLv2+ in order to provide as much flexibility
as I reasonably can, while retaining a copyleft license. It is my belief
that LGPLv2+ is a suitable license for the Python ecosystem that aligns
with the goals and philosophy of the QEMU project.

The intent is to eventually drop legacy.py, leaving only library code
that is LGPLv2+.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20220325200438.2556381-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-04-21 11:01:00 -04:00
John Snow
380fc8f32e python/aqmp: add explicit GPLv2 license to legacy.py
The legacy.py module is heavily based on the QMP module by Luiz
Capitulino (et al) which is licensed as explicit GPLv2-only. The async
QMP package is currently licensed similarly, but I intend to relicense
the async package to the more flexible LGPLv2+.

In preparation for that change, make the license on legacy.py explicit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20220325200438.2556381-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-04-21 11:01:00 -04:00
John Snow
a4225303a1 python/machine: permanently switch to AQMP
Remove the QEMU_PYTHON_LEGACY_QMP environment variable, making the
switch from sync qmp to async qmp permanent. Update exceptions and
import paths as necessary.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Acked-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20220321203315.909411-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-04-21 11:01:00 -04:00
John Snow
062fd1dad2 python/utils: add VerboseProcessError
This adds an Exception that extends the Python stdlib
subprocess.CalledProcessError.

The difference is that the str() method of this exception also adds the
stdout/stderr logs. In effect, if this exception goes unhandled, Python
will print the output in a visually distinct wrapper to the terminal so
that it's easy to spot in a sea of traceback information.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220321201618.903471-3-jsnow@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-03-22 10:14:23 +01:00
John Snow
be73231ba8 python/utils: add add_visual_margin() text decoration utility
>>> print(add_visual_margin(msg, width=72, name="Commit Message"))
┏━ Commit Message ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
┃ add_visual_margin() takes a chunk of text and wraps it in a visual
┃ container that force-wraps to a specified width. An optional title
┃ label may be given, and any of the individual glyphs used to draw the
┃ box may be replaced or specified as well.
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Acked-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220321201618.903471-2-jsnow@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-03-22 10:14:22 +01:00
John Snow
4c1fe7003c python/aqmp: drop _bind_hack()
_bind_hack() was a quick fix to allow async QMP to call bind(2) prior to
calling listen(2) and accept(2). This wasn't sufficient to fully address
the race condition present in synchronous clients.

With the race condition in legacy.py fixed (see the previous commit),
there are no longer any users of _bind_hack(). Drop it.

Fixes: b0b662bb2b
Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-11-jsnow@redhat.com
[Expanded commit message. --js]
Signed-off-by: John Snow <jsnow@redhat.com>
2022-03-07 14:36:41 -05:00
John Snow
673856f9d8 python/aqmp: fix race condition in legacy.py
legacy.py provides a synchronous model. iotests frequently uses this
paradigm:

 - create QMP client object
 - start QEMU process
 - await connection from QEMU process

In the switch from sync to async QMP, the QMP client object stopped
calling bind() and listen() during the QMP object creation step, which
creates a race condition if the QEMU process dials in too quickly.

With refactoring out of the way, restore the former behavior of calling
bind() and listen() during __init__() to fix this race condition.

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-10-jsnow@redhat.com
[Expanded commit message. --js]
Signed-off-by: John Snow <jsnow@redhat.com>
2022-03-07 14:36:41 -05:00
John Snow
481607c7d3 python/aqmp: add start_server() and accept() methods
Add start_server() and accept() methods that can be used instead of
start_server_and_accept() to allow more fine-grained control over the
incoming connection process.

(Eagle-eyed reviewers will surely notice that it's a bit weird that
"CONNECTING" is a state that's shared between both the start_server()
and connect() states. That's absolutely true, and it's very true that
checking on the presence of _accepted as an indicator of state is a
hack. That's also very certainly true. But ... this keeps client code an
awful lot simpler, as it doesn't have to care exactly *how* the
connection is being made, just that it *is*. Is it worth disrupting that
simplicity in order to provide a better state guard on `accept()`? Hm.)

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-03-07 14:36:41 -05:00
John Snow
32c5abf051 python/aqmp: stop the server during disconnect()
Before we allow the full separation of starting the server and accepting
new connections, make sure that the disconnect cleans up the server and
its new state, too.

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-03-07 14:36:41 -05:00
John Snow
5e9902a030 python/aqmp: refactor _do_accept() into two distinct steps
Refactor _do_accept() into _do_start_server() and _do_accept(). As of
this commit, the former calls the latter, but in subsequent commits
they'll be split apart.

(So please forgive the misnomer for _do_start_server(); it will live up
to its name shortly, and the docstring will be updated then too. I'm
just cutting down on some churn.)

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-03-07 14:36:41 -05:00
John Snow
1b9c8cb6ce python/aqmp: squelch pylint warning for too many lines
I would really like to keep this under 1000 lines, I promise. Doesn't
look like it's gonna happen.

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-03-07 14:36:41 -05:00
John Snow
830e6fd36e python/aqmp: split _client_connected_cb() out as _incoming()
As part of disentangling the monolithic nature of _do_accept(), split
out the incoming callback to prepare for factoring out the "wait for a
peer" step. Namely, this means using an event signal we can wait on from
outside of this method.

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-03-07 14:36:41 -05:00
John Snow
68a6cf3ffe python/aqmp: remove _new_session and _establish_connection
These two methods attempted to entirely envelop the logic of
establishing a connection to a peer start to finish. However, we need to
break apart the incoming connection step into more granular steps. We
will no longer be able to reasonably constrain the logic inside of these
helper functions.

So, remove them - with _session_guard(), they no longer serve a real
purpose.

Although the public API doesn't change, the internal API does. Now that
there are no intermediary methods between e.g. connect() and
_do_connect(), there's no hook where the runstate is set. As a result,
the test suite changes a little to cope with the new semantics of
_do_accept() and _do_connect().

Lastly, take some pieces of the now-deleted docstrings and move
them up to the public interface level. They were a little more detailed,
and it won't hurt to keep them.

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-03-07 14:36:41 -05:00
John Snow
0ba4e76b23 python/aqmp: rename 'accept()' to 'start_server_and_accept()'
Previously, I had a method named "accept()" that under-the-hood calls
bind(2), listen(2) *and* accept(2). I meant this as a simplification and
counterpart to the one-shot "connect()" method.

This is confusing to readers who expect accept() to mean *just*
accept(2). Since I need to split apart the "accept()" method into
multiple methods anyway (one of which strongly resembling accept(2)), it
feels pertinent to rename this method *now*.

Rename this all-in-one method "start_server_and_accept()" instead.

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-03-07 14:36:41 -05:00
John Snow
40196c2393 python/aqmp: add _session_guard()
In _new_session, there's a fairly complex except clause that's used to
give semantic errors to callers of accept() and connect(). We need to
create a new two-step replacement for accept(), so factoring out this
piece of logic will be useful.

Bolster the comments and docstring here to try and demystify what's
going on in this fairly delicate piece of Python magic.

(If we were using Python 3.7+, this would be an @asynccontextmanager. We
don't have that very nice piece of magic, however, so this must take an
Awaitable to manage the Exception contexts properly. We pay the price
for platform compatibility.)

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220225205948.3693480-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-03-07 14:36:41 -05:00
Daniel P. Berrangé
5c66d7d8de python: support recording QMP session to a file
When running QMP commands with very large response payloads, it is often
not easy to spot the info you want. If we can save the response to a
file then tools like 'grep' or 'jq' can be used to extract information.

For convenience of processing, we merge the QMP command and response
dictionaries together:

  {
      "arguments": {},
      "execute": "query-kvm",
      "return": {
          "enabled": false,
          "present": true
      }
  }

Example usage

  $ ./scripts/qmp/qmp-shell-wrap -l q.log -p -- ./build/qemu-system-x86_64 -display none
  Welcome to the QMP low-level shell!
  Connected
  (QEMU) query-kvm
  {
      "return": {
          "enabled": false,
          "present": true
      }
  }
  (QEMU) query-mice
  {
      "return": [
          {
              "absolute": false,
              "current": true,
              "index": 2,
              "name": "QEMU PS/2 Mouse"
          }
      ]
  }

 $ jq --slurp '. | to_entries[] | select(.value.execute == "query-kvm") |
               .value.return.enabled' < q.log
   false

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220128161157.36261-3-berrange@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-02-23 17:07:26 -05:00
Daniel P. Berrangé
439125293c python: introduce qmp-shell-wrap convenience tool
With the current 'qmp-shell' tool developers must first spawn QEMU with
a suitable -qmp arg and then spawn qmp-shell in a separate terminal
pointing to the right socket.

With 'qmp-shell-wrap' developers can ignore QMP sockets entirely and
just pass the QEMU command and arguments they want. The program will
listen on a UNIX socket and tell QEMU to connect QMP to that.

For example, this:

 # qmp-shell-wrap -- qemu-system-x86_64 -display none

Is roughly equivalent of running:

 # qemu-system-x86_64 -display none -qmp qmp-shell-1234 &
 # qmp-shell qmp-shell-1234

Except that 'qmp-shell-wrap' switches the socket peers around so that
it is the UNIX socket server and QEMU is the socket client. This makes
QEMU reliably go away when qmp-shell-wrap exits, closing the server
socket.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20220128161157.36261-2-berrange@redhat.com
[Edited for rebase. --js]
Signed-off-by: John Snow <jsnow@redhat.com>
2022-02-23 17:07:26 -05:00
John Snow
b0b662bb2b python/aqmp: add socket bind step to legacy.py
The synchronous QMP library would bind to the server address during
__init__(). The new library delays this to the accept() call, because
binding occurs inside of the call to start_[unix_]server(), which is an
async method -- so it cannot happen during __init__ anymore.

Python 3.7+ adds the ability to create the server (and thus the bind()
call) and begin the active listening in separate steps, but we don't
have that functionality in 3.6, our current minimum.

Therefore ... Add a temporary workaround that allows the synchronous
version of the client to bind the socket in advance, guaranteeing that
there will be a UNIX socket in the filesystem ready for the QEMU client
to connect to without a race condition.

(Yes, it's a bit ugly. Fixing it more nicely will have to wait until our
minimum Python version is 3.7+.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20220201041134.1237016-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-02-02 14:12:22 -05:00
John Snow
50465f94d2 python/machine: raise VMLaunchFailure exception from launch()
This allows us to pack in some extra information about the failure,
which guarantees that if the caller did not *intentionally* cause a
failure (by capturing this Exception), some pretty good clues will be
printed at the bottom of the traceback information.

This will help make failures in the event of a non-negative return code
more obvious when they go unhandled; the current behavior in
_post_shutdown() is to print a warning message only in the event of
signal-based terminations (for negative return codes).

(Note: In Python, catching BaseException instead of Exception catches a
broader array of Exception events, including SystemExit and
KeyboardInterrupt. We do not want to "wrap" such exceptions as a
VMLaunchFailure, because that will 'downgrade' the exception from a
BaseException to a regular Exception. We do, however, want to perform
cleanup in either case, so catch on the broadest scope and
wrap-and-re-raise only in the more targeted scope.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20220201041134.1237016-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-02-02 14:12:22 -05:00
John Snow
fa73e6e4ca python/aqmp: Fix negotiation with pre-"oob" QEMU
QEMU versions prior to the "oob" capability *also* can't accept the
"enable" keyword argument at all. Fix the handshake process with older
QEMU versions.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20220201041134.1237016-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-02-02 14:12:22 -05:00
John Snow
fd9c3a6219 python: move qmp-shell under the AQMP package
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
0347c4c4cf python: move qmp utilities to python/qemu/utils
In order to upload a QMP package to PyPI, I want to remove any scripts
that I am not 100% confident I want to support upstream, beyond our
castle walls.

Move most of our QMP utilities into the utils package so we can split
them out from the PyPI upload.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
f3efd12930 python/qmp: switch qmp-shell to AQMP
We have a replacement for async QMP, but it doesn't have feature parity
yet. For now, then, port the old tool onto the new backend.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2022-01-21 16:01:31 -05:00
John Snow
8d6cdc5118 python/qmp: switch qom tools to AQMP
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
26db07516f python/qmp: switch qemu-ga-client to AQMP
Async QMP always raises a "ConnectError" on any connection error which
houses the cause in a second exception. We can check if this root cause
was python's ConnectionError to determine a fairly similar condition to
the original error check here.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
7017f3853a python/qemu-ga-client: don't use deprecated CLI syntax in usage comment
Cleanup related to commit ccd3b3b811, "qemu-option: warn for
short-form boolean options".

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
6e7751dc38 python/aqmp: rename AQMPError to QMPError
This is in preparation for renaming qemu.aqmp to qemu.qmp. I should have
done this from this from the very beginning, but it's a convenient time
to make sure this churn is taken care of.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2022-01-21 16:01:31 -05:00
John Snow
728dcac5e3 python/aqmp: add SocketAddrT to package root
It's a commonly needed definition, it can be re-exported by the root.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
0e6bfd8b96 python/aqmp: copy type definitions from qmp
Copy the remaining type definitions from QMP into the qemu.aqmp.legacy
module. Now, users that require the legacy interface don't need to
import anything else but qemu.aqmp.legacy wrapper.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
3b5bf136f5 python/aqmp: handle asyncio.TimeoutError on execute()
This exception can be injected into any await statement. If we are
canceled via timeout, we want to clear the pending execution record on
our way out.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2022-01-21 16:01:31 -05:00
John Snow
3bc72e3aed python/aqmp: add __del__ method to legacy interface
asyncio can complain *very* loudly if you forget to back out of things
gracefully before the garbage collector starts destroying objects that
contain live references to asyncio Tasks.

The usual fix is just to remember to call aqmp.disconnect(), but for the
sake of the legacy wrapper and quick, one-off scripts where a graceful
shutdown is not necessarily of paramount imporance, add a courtesy
cleanup that will trigger prior to seeing screenfuls of confusing
asyncio tracebacks.

Note that we can't *always* save you from yourself; depending on when
the GC runs, you might just seriously be out of luck. The best we can do
in this case is to gently remind you to clean up after yourself.

(Still much better than multiple pages of incomprehensible python
warnings for the crime of forgetting to put your toys away.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
dc6877bd2e python/aqmp: fix docstring typo
Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
2022-01-21 16:01:31 -05:00
John Snow
366d33158c python: update type hints for mypy 0.930
Mypy 0.930, released Dec 22, changes the way argparse objects are
considered. Crafting a definition that works under Python 3.6 and an
older mypy alongside newer versions simultaneously is ... difficult,
so... eh. Stub it out with an 'Any' definition to get the CI moving
again.

Oh well.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Message-id: 20220110191349.1841027-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-01-10 18:22:59 -05:00
John Snow
42d73f2894 Python/aqmp: fix type definitions for mypy 0.920
0.920 (Released 2021-12-15) is not entirely happy with the
way that I was defining _FutureT:

qemu/aqmp/protocol.py:601: error: Item "object" of the upper bound
"Optional[Future[Any]]" of type variable "_FutureT" has no attribute
"done"

Update it with something a little mechanically simpler that works better
across a wider array of mypy versions.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20220110191349.1841027-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-01-10 18:22:44 -05:00
John Snow
f75b20e4f1 python/aqmp: use absolute import statement
pylint's dependency astroid appears to have bugs in 2.9.1 and 2.9.2 (Dec
31 and Jan 3) that appear to erroneously expect the qemu namespace to
have an __init__.py file. astroid 2.9.3 (Jan 9) avoids that problem, but
appears to not understand a relative import within a namespace package.

Update the relative import - it was worth changing anyway, because these
packages will eventually be packaged and distributed separately.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Message-id: 20220110191349.1841027-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2022-01-10 18:22:33 -05:00
John Snow
a57cb3e23d python/aqmp: fix send_fd_scm for python 3.6.x
3.6 doesn't play keepaway with the socket object, so we don't need to go
fishing for it on this version. In fact, so long as 'sendmsg' is still
available, it's probably preferable to just use that method and only go
fishing for forbidden details when we absolutely have to.

Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-id: 20211118204620.1897674-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-22 18:41:21 -05:00
John Snow
1611e6cf4e python/machine: handle "fast" QEMU terminations
In the case that the QEMU process actually launches -- but then dies so
quickly that we can't establish a QMP connection to it -- QEMUMachine
currently calls _post_shutdown() assuming that it never launched the VM
process.

This isn't true, though: it "merely" may have failed to establish a QMP
connection and the process is in the middle of its own exit path.

If we don't wait for the subprocess, the caller may get a bogus `None`
return for .exitcode(). This behavior was observed from
device-crash-test; after the switch to Async QMP, the timings were
changed such that it was now seemingly possible to witness the failure
of "vm.launch()" *prior* to the exitcode becoming available.

The semantic of the `_launched` property is changed in this
patch. Instead of representing the condition "launch() executed
successfully", it will now represent "has forked a child process
successfully". This way, wait() when called in the exit path won't
become a no-op.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-id: 20211118204620.1897674-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-22 18:41:17 -05:00