Commit Graph

191 Commits

Author SHA1 Message Date
John Snow
7137a96099 qapi: Prefer explicit relative imports
All of the QAPI include statements are changed to be package-aware, as
explicit relative imports.

A quirk of Python packages is that the name of the package exists only
*outside* of the package. This means that to a module inside of the qapi
folder, there is inherently no such thing as the "qapi" package. The
reason these imports work is because the "qapi" package exists in the
context of the caller -- the execution shim, where sys.path includes a
directory that has a 'qapi' folder in it.

When we write "from qapi import sibling", we are NOT referencing the folder
'qapi', but rather "any package named qapi in sys.path". If you should
so happen to have a 'qapi' package in your path, it will use *that*
package.

When we write "from .sibling import foo", we always reference explicitly
our sibling module; guaranteeing consistency in *where* we are importing
these modules from.

This can be useful when working with virtual environments and packages
in development mode. In development mode, a package is installed as a
series of symlinks that forwards to your same source files. The problem
arises because code quality checkers will follow "import qapi.x" to the
"installed" version instead of the sibling file and -- even though they
are the same file -- they have different module paths, and this causes
cyclic import problems, false positive type mismatch errors, and more.

It can also be useful when dealing with hierarchical packages, e.g. if
we allow qemu.core.qmp, qemu.qapi.parser, etc.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201009161558.107041-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-10 11:37:47 +02:00
John Snow
a76ab215ec qapi: move generator entrypoint into package
As part of delinting and adding type hints to the QAPI generator, it's
helpful for the entrypoint to be part of the package, only leaving a
very tiny entrypoint shim outside of the package.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201009161558.107041-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[invalid_char() renamed to invalid_prefix_char()]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-10 11:37:47 +02:00
John Snow
adcb9b36c9 qapi: modify docstrings to be sphinx-compatible
A precise style guide and a package-wide overhaul is forthcoming pending
further discussion and consensus. For now, merely avoid obvious errors
that cause Sphinx documentation build problems, using a style loosely
based on PEP 257 and Sphinx Autodoc. It is chosen for interoperability
with our existing Sphinx framework, and because it has loose recognition
in the Pycharm IDE.

See also:

https://www.python.org/dev/peps/pep-0257/
https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#info-field-lists
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20201009161558.107041-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-10 11:37:46 +02:00
Kevin Wolf
04f22362f1 qapi: Add a 'coroutine' flag for commands
This patch adds a new 'coroutine' flag to QMP command definitions that
tells the QMP dispatcher that the command handler is safe to be run in a
coroutine.

The documentation of the new flag pretends that this flag is already
used as intended, which it isn't yet after this patch. We'll implement
this in another patch in this series.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201005155855.256490-9-kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-09 07:08:19 +02:00
Peter Maydell
a27ff0a249 scripts/qapi: Remove texinfo generation support
We no longer use the generated texinfo format documentation,
so delete the code that generates it, and the test case for
the generation.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200925162316.21205-17-peter.maydell@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-09-29 17:55:39 +02:00
Peter Maydell
a69a6d4b4d scripts/qapi/parser.py: improve doc comment indent handling
Make the handling of indentation in doc comments more sophisticated,
so that when we see a section like:

Notes: some text
       some more text
          indented line 3

we save it for the doc-comment processing code as:

some text
some more text
   indented line 3

and when we see a section with the heading on its own line:

Notes:

some text
some more text
   indented text

we also accept that and save it in the same form.

If we detect that the comment document text is not indented as much
as we expect it to be, we throw a parse error.  (We don't complain
about over-indented sections, because for rST this can be legitimate
markup.)

The golden reference for the doc comment text is updated to remove
the two 'wrong' indents; these now form a test case that we correctly
stripped leading whitespace from an indented multi-line argument
definition.

We update the documentation in docs/devel/qapi-code-gen.txt to
describe the new indentation rules.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200925162316.21205-6-peter.maydell@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Whitespace between sentences tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-09-29 17:55:39 +02:00
Peter Maydell
99dff36d3a scripts/qapi: Move doc-comment whitespace stripping to doc.py
As we accumulate lines from doc comments when parsing the JSON, the
QAPIDoc class generally strips leading and trailing whitespace using
line.strip() when it calls _append_freeform().  This is fine for
Texinfo, but for rST leading whitespace is significant.  We'd like to
move to having the text in doc comments be rST format rather than a
custom syntax, so move the removal of leading whitespace from the
QAPIDoc class to the texinfo-specific processing code in
texi_format() in qapi/doc.py.

(Trailing whitespace will always be stripped by the rstrip() in
Section::append regardless.)

In a followup commit we will make the whitespace in the lines of doc
comment sections more consistently follow the input source.

There is no change to the generated .texi files before and after this
commit.

Because the qapi-schema test checks the exact values of the
documentation comments against a reference, we need to update that
reference to match the new whitespace.  In the first four places this
is now correctly checking that we did put in the amount of whitespace
to pass a rST-formatted list to the backend; in the last two places
the extra whitespace is 'wrong' and will go away again in the
following commit.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200925162316.21205-5-peter.maydell@linaro.org>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-09-29 17:55:39 +02:00
Markus Armbruster
dcdc07a97c qapi: Make section headings start a new doc comment block
Our current QAPI doc-comment markup allows section headers (introduced
with a leading '=' or '==') anywhere in a free-form documentation
comment.  This works for Texinfo because the generator simply prints a
Texinfo section command at that point in the output stream.  For rST
generation, since we're assembling a tree of docutils nodes, this is
awkward because a new section implies starting a new section node at
the top level of the tree and generating text into there.

Make section headers start a new free-form documentation block, so the
future rST document generator doesn't have to look at every line in
free-form blocks and handle headings in odd places.

This change makes no difference to the generated Texinfo.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200320091805.5585-3-armbru@redhat.com>
2020-09-07 16:35:16 +02:00
Markus Armbruster
d98884b75d qapi: Reject section markup in definition documentation
Section markup in definition documentation makes no sense and can
produce invalid Texinfo.  Reject.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200320091805.5585-2-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-09-07 16:35:16 +02:00
Daniel P. Berrangé
221db5daf6 qapi: enable use of g_autoptr with QAPI types
Currently QAPI generates a type and function for free'ing it:

  typedef struct QCryptoBlockCreateOptions QCryptoBlockCreateOptions;
  void qapi_free_QCryptoBlockCreateOptions(QCryptoBlockCreateOptions *obj);

This is used in the traditional manner:

  QCryptoBlockCreateOptions *opts = NULL;

  opts = g_new0(QCryptoBlockCreateOptions, 1);

  ....do stuff with opts...

  qapi_free_QCryptoBlockCreateOptions(opts);

Since bumping the min glib to 2.48, QEMU has incrementally adopted the
use of g_auto/g_autoptr. This allows the compiler to run a function to
free a variable when it goes out of scope, the benefit being the
compiler can guarantee it is freed in all possible code ptahs.

This benefit is applicable to QAPI types too, and given the seriously
long method names for some qapi_free_XXXX() functions, is much less
typing. This change thus makes the code generator emit:

 G_DEFINE_AUTOPTR_CLEANUP_FUNC(QCryptoBlockCreateOptions,
                              qapi_free_QCryptoBlockCreateOptions)

The above code example now becomes

  g_autoptr(QCryptoBlockCreateOptions) opts = NULL;

  opts = g_new0(QCryptoBlockCreateOptions, 1);

  ....do stuff with opts...

Note, if the local pointer needs to live beyond the scope holding the
variable, then g_steal_pointer can be used. This is useful to return the
pointer to the caller in the success codepath, while letting it be freed
in all error codepaths.

  return g_steal_pointer(&opts);

The crypto/block.h header needs updating to avoid symbol clash now that
the g_autoptr support is a standard QAPI feature.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200723153845.2934357-1-berrange@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-09-03 09:38:36 +02:00
Markus Armbruster
cbf97d5b79 qapi: Fix visit_type_STRUCT() not to fail for null object
To make deallocating partially constructed objects work, the
visit_type_STRUCT() need to succeed without doing anything when passed
a null object.

Commit cdd2b228b9 "qapi: Smooth visitor error checking in generated
code" broke that.  To reproduce, run tests/test-qobject-input-visitor
with AddressSanitizer:

    ==4353==ERROR: LeakSanitizer: detected memory leaks

    Direct leak of 16 byte(s) in 1 object(s) allocated from:
	#0 0x7f192d0c5d28 in __interceptor_calloc (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xded28)
	#1 0x7f192cd21b10 in g_malloc0 (/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x51b10)
	#2 0x556725f6bbee in visit_next_list qapi/qapi-visit-core.c:86
	#3 0x556725f49e15 in visit_type_UserDefOneList tests/test-qapi-visit.c:474
	#4 0x556725f4489b in test_visitor_in_fail_struct_in_list tests/test-qobject-input-visitor.c:1086
	#5 0x7f192cd42f29  (/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x72f29)

    SUMMARY: AddressSanitizer: 16 byte(s) leaked in 1 allocation(s).

Test case /visitor/input/fail/struct-in-list feeds a list with a bad
element to the QObject input visitor.  Visiting that element duly
fails, and aborts the visit with the list only partially constructed:
the faulty object is null.  Cleaning up the partially constructed list
visits that null object, fails, and aborts the visit before the list
node gets freed.

Fix the the generated visit_type_STRUCT() to succeed for null objects.

Fixes: cdd2b228b9
Reported-by: Li Qiang <liq3ea@163.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200716150617.4027356-1-armbru@redhat.com>
Tested-by: Li Qiang <liq3ea@gmail.com>
Reviewed-by: Li Qiang <liq3ea@gmail.com>
2020-07-21 14:38:23 +02:00
Markus Armbruster
cdd2b228b9 qapi: Smooth visitor error checking in generated code
Use visitor functions' return values to check for failure.  Eliminate
error_propagate() that are now unnecessary.  Delete @err that are now
unused.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200707160613.848843-41-armbru@redhat.com>
2020-07-10 15:18:08 +02:00
Markus Armbruster
012d4c96e2 qapi: Make visitor functions taking Error ** return bool, not void
See recent commit "error: Document Error API usage rules" for
rationale.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200707160613.848843-18-armbru@redhat.com>
2020-07-10 15:18:08 +02:00
Markus Armbruster
89bf68f933 qapi: Generate simpler marshalling code when no arguments
When command FOO has no arguments, its generated qmp_marshal_FOO() is
a bit confusing.  Make it simpler:

     visit_start_struct(v, NULL, NULL, 0, &err);
     if (err) {
         goto out;
     }
-
-    if (!err) {
-        visit_check_struct(v, &err);
-    }
+    visit_check_struct(v, &err);
     visit_end_struct(v, NULL);
     if (err) {
         goto out;
     }

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200424084338.26803-16-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-04-30 07:26:41 +02:00
Markus Armbruster
2061487bdb qapi: Disallow qmp_marshal_FOO(NULL, ...)
For QMP commands without arguments, gen_marshal() laboriously
generates a qmp_marshal_FOO() that copes with null @args.  Turns
there's just one caller that passes null instead of an empty QDict.
Adjust that caller, and simplify gen_marshal().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200424084338.26803-15-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-04-30 07:26:41 +02:00
Markus Armbruster
7111a86e1b qapi: Assert non-input visitors see only valid alternate tags
An alternate type's visit_type_FOO() fails when it runs into an
invalid ->type.

This is appropriate with an input visitor: visit_start_alternate()
sets ->type according to the input, and bad input can lead to bad
->type.

It should never happen with an output, clone or dealloc visitor: if it
did, the alternate being output, cloned or deallocated would be messed
up beyond repair.  Assert that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200424084338.26803-12-armbru@redhat.com>
2020-04-30 07:26:40 +02:00
Markus Armbruster
c978bd5226 qapi: Clean up visitor's recovery from input with invalid type
An alternate type's visit_type_FOO() fails when it runs into an
invalid ->type.  If it's an input visit, we then need to free the the
object we got from visit_start_alternate().  We do that with
qapi_free_FOO(), which uses the dealloc visitor.

Trouble is that object is in a bad state: its ->type is invalid.  So
the dealloc visitor will run into the same error again, and the error
recovery skips deallocating the alternate's (invalid) alternative.
Works, because qapi_free_FOO() ignores the error.

Avoid it instead: free the messed up object with by g_free().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200424084338.26803-11-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-04-30 07:26:40 +02:00
Markus Armbruster
8e08bf4ea2 qapi: Assert incomplete object occurs only in dealloc visitor
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200424084338.26803-7-armbru@redhat.com>
2020-04-30 06:51:15 +02:00
Markus Armbruster
f965e8fea6 qapi: New special feature flag "deprecated"
Unlike regular feature flags, the new special feature flag
"deprecated" is recognized by the QAPI generator.  For now, it's only
permitted with commands, events, and struct members.  It will be put
to use shortly.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200317115459.31821-26-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Doc typo fixed]
2020-03-17 21:42:47 +01:00
Markus Armbruster
84ab008687 qapi: Add feature flags to struct members
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20200317115459.31821-21-armbru@redhat.com>
2020-03-17 21:25:47 +01:00
Markus Armbruster
645178c069 qapi/schema: Call QAPIDoc.connect_member() in just one place
The .connect_doc() of classes that have QAPISchemaMember connect them
to their documentation.  Change them to delegate the actual work to
new QAPISchemaMember.connect_doc().  Matches the .connect_doc() that
already exist.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20200317115459.31821-20-armbru@redhat.com>
2020-03-17 21:25:47 +01:00
Markus Armbruster
5858fd1a02 qapi/schema: Rename QAPISchemaObjectType{Variant,Variants}
QAPISchemaObjectTypeVariants represents both object type and alternate
type variants.  Rename to QAPISchemaVariants.

Rename QAPISchemaObjectTypeVariant the same way.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20200317115459.31821-19-armbru@redhat.com>
2020-03-17 21:25:47 +01:00
Markus Armbruster
226b5be6d4 qapi/schema: Reorder classes so related ones are together
Move QAPISchemaAlternateType up some, so that all QAPISchemaFOOType
are together.  Move QAPISchemaObjectTypeVariants right behind its
users.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20200317115459.31821-18-armbru@redhat.com>
2020-03-17 21:25:47 +01:00
Markus Armbruster
ed30f58dde qapi/schema: Change _make_features() to a take feature list
QAPISchema._make_features() takes a definition expression, and
extracts its 'features' member.  The other ._make_FOO() leave
destructuring expressions to their callers.  Change ._make_features()
to match them.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200317115459.31821-17-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-17 21:25:41 +01:00
Markus Armbruster
24cfd6addd qapi/introspect: Factor out _make_tree()
The value of @qmp_schema_qlit is generated from an expression tree.
Tree nodes are created in several places.  Factor out the common code
into _make_tree().  This isn't much of a win now.  It will pay off
when we add feature flags in the next few commits.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20200317115459.31821-16-armbru@redhat.com>
2020-03-17 19:58:34 +01:00
Markus Armbruster
2e8a843d19 qapi/introspect: Rename *qlit* to reduce confusion
We generate the value of qmp_schema_qlit from an expression tree.  The
function doing that is named to_qlit(), and its inputs are accumulated
in QAPISchemaGenIntrospectVisitor._qlits.  We call both its input and
its output "qlit".  This is confusing.

Use "tree" for input, and "qlit" only for output: rename to_qlit() to
_tree_to_qlit(), ._qlits to ._trees, ._gen_qlit() to ._gen_tree().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20200317115459.31821-15-armbru@redhat.com>
2020-03-17 19:58:34 +01:00
Markus Armbruster
7b3bc9e28f qapi: Consistently put @features parameter right after @ifcond
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200317115459.31821-14-armbru@redhat.com>
2020-03-17 19:58:34 +01:00
Markus Armbruster
013b4efc9b qapi: Add feature flags to remaining definitions
In v4.1.0, we added feature flags just to struct types (commit
6a8c0b5102^..f3ed93d545), to satisfy an immediate need (commit
c9d4070991 "file-posix: Add dynamic-auto-read-only QAPI feature").  In
v4.2.0, we added them to commands (commit 23394b4c39 "qapi: Add
feature flags to commands") to satisfy another immediate need (commit
d76744e65e "qapi: Allow introspecting fix for savevm's cooperation
with blockdev").

Add them to the remaining definitions: enumeration types, union types,
alternate types, and events.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200317115459.31821-13-armbru@redhat.com>
2020-03-17 19:58:34 +01:00
Markus Armbruster
e4405b3069 qapi/schema: Clean up around QAPISchemaEntity.connect_doc()
QAPISchemaEntity calls doc.connect_feature() in .check().  Improper
since commit ee1e6a1f6c split .connect_doc() off .check().  Move the
call.  Requires making the children call super().connect_doc() as they
should.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200317115459.31821-12-armbru@redhat.com>
2020-03-17 19:58:34 +01:00
Kevin Wolf
2af282ec51 qemu-storage-daemon: Add --monitor option
This adds and parses the --monitor option, so that a QMP monitor can be
used in the storage daemon. The monitor offers commands defined in the
QAPI schema at storage-daemon/qapi/qapi-schema.json.

The --monitor options currently allows to create multiple monitors with
the same ID. This part of the interface is considered unstable. We will
reject such configurations as soon as we have a design for the monitor
subsystem to perform these checks. (In the system emulator, we depend on
QemuOpts rejecting duplicate IDs.)

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200224143008.13362-21-kwolf@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-03-06 17:21:28 +01:00
Markus Armbruster
8ec0e1a4e6 qapi: Brush off some (py)lint
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200304155932.20452-5-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2020-03-05 09:24:11 +01:00
Markus Armbruster
2cae67bcb5 qapi: Use super() now we have Python 3
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200304155932.20452-4-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2020-03-05 09:24:11 +01:00
Markus Armbruster
ed39c03e2f qapi: Drop conditionals for Python 2
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200304155932.20452-3-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2020-03-05 09:24:11 +01:00
Markus Armbruster
baa310f1bb qapi: Inheriting from object is pointless with Python 3, drop
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200304155932.20452-2-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2020-03-05 09:24:11 +01:00
Paolo Bonzini
423edd9a31 drop "from __future__ import print_function"
This is only needed for Python 2, which we do not support anymore.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200204160604.19883-1-pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:15:16 +01:00
Markus Armbruster
43d1455cf8 qapi: Fix code generation with Python 3.5
Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
modules" switched QAPISchema.visit() from

    for entity in self._entity_list:

effectively to

    for mod in self._module_dict.values():
        for entity in mod._entity_list:

Visits in the same order as long as .values() is in insertion order.
That's the case only for Python 3.6 and later.  Before, it's in some
arbitrary order, which results in broken generated code.

Fix by making self._module_dict an OrderedDict rather than a dict.

Fixes: 3e7fb5811b
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Thomas Huth <thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: BALATON Zoltan <balaton@eik.bme.hu>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20200116202558.31473-1-armbru@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-01-20 12:17:38 +00:00
Markus Armbruster
3bef3aaec9 qapi: Simplify QAPISchemaModularCVisitor
Since the previous commit, QAPISchemaVisitor.visit_module() is called
just once.  Simplify QAPISchemaModularCVisitor accordingly.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-7-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-01-14 11:01:58 +01:00
Markus Armbruster
3e7fb5811b qapi: Fix code generation for empty modules
When a sub-module doesn't contain any definitions, we don't generate
code for it, but we do generate the #include.

We generate code only for modules that get visited.
QAPISchema.visit() visits only modules that have definitions.  It can
visit modules multiple times.

Clean this up as follows.  Collect entities in their QAPISchemaModule.
Have QAPISchema.visit() call QAPISchemaModule.visit() for each module.
Have QAPISchemaModule.visit() call .visit_module() for itself, and
QAPISchemaEntity.visit() for each of its entities.  This way, we visit
each module exactly once.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-6-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-01-14 11:01:58 +01:00
Markus Armbruster
a9f1dd7ee0 qapi: Proper intermediate representation for modules
Modules are represented only by their names so far.  Introduce class
QAPISchemaModule.  So far, it merely wraps the name.  The next patch
will put it to more interesting use.

Once again, arrays spice up the patch a bit.  For any other type,
@info points to the definition, which lets us map from @info to
module.  For arrays, there is no definition, and @info points to the
first use instead.  We have to use the element type's module instead,
which is only available after .check().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-5-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-01-14 11:01:58 +01:00
Markus Armbruster
00ca24ff9e qapi: Generate command registration stuff into separate files
Having to include qapi-commands.h just for qmp_init_marshal() is
suboptimal.  Generate it into separate files.  This lets
monitor/misc.c, qga/main.c, and the generated qapi-commands-FOO.h
include less.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-4-armbru@redhat.com>
[Typos in docs/devel/qapi-code-gen.txt fixed]
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-01-14 11:01:58 +01:00
Markus Armbruster
7e9c1707e1 qapi: Tweak "command returns a nice type" check for clarity
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-2-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-01-14 11:01:58 +01:00
Markus Armbruster
e151941d1b qapi: Check feature documentation against the schema
Commit f3ed93d545 "qapi: Allow documentation for features" neglected
to check documentation against the schema.  Fix that: check them the
same way we check arguments.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-20-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
e4def78755 qapi: Polish reporting of bogus member documentation
Improve error messages from

    the following documented members are not in the declaration: a
    the following documented members are not in the declaration: aa, bb

to the more concise

    documented member 'a' does not exist
    documented members 'aa', 'bb' do not exist

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-19-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
b3cdff10e5 qapi: Lift features into QAPISchemaEntity
Commit 6a8c0b5102 "qapi: Add feature flags to struct types" added
features to QAPISchemaObjectType.  Commit a95daa5093 "qapi: Add
feature flags to commands in qapi" added them to QAPISchemaCommand,
duplicating the code.  Tolerable, but the duplication will only get
worse as we add features to more definitions.

To de-duplicate, lift features from QAPISchemaObjectType and
QAPISchemaCommand into QAPISchemaEntity.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-18-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
ad1ecfc679 qapi: Fold normalize_enum() into check_enum()
check_features() is always called together with normalize_features().
Fold the latter into the former.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-17-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
2ce51ef68f qapi: Fold normalize_features() into check_features()
check_features() is always called together with normalize_features():
the former in check_struct() and check_command(), the latter in their
caller check_exprs().  Fold the latter into the former.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-16-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
c145bfda18 qapi: Fold normalize_if() into check_if()
check_if() is always called together with normalize_if().  Fold the
latter into the former.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-15-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
1192a4862b qapi: Eliminate .check_doc() overrides
All sub-classes of QAPISchemaEntity now override .check_doc() the same
way, except for QAPISchemaType and and QAPISchemaArrayType.

Put the overrides' code in QAPISchemaEntity.check_doc(), and drop the
overrides.  QAPISchemaType doesn't care because it's abstract.
QAPISchemaArrayType doesn't care because its .doc is always None.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-14-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
a710e1c8c3 qapi: Simplify ._make_implicit_object_type()
All callers now pass doc=None.  Drop the argument.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-13-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
bf83f04e13 qapi: Fix doc comment checking for commands and events
When a command's 'data' is an object, its doc comment describes the
arguments defined there.  When 'data' names a type, the doc comment
does not describe arguments.  Instead, the doc generator inserts a
pointer to the named type.

An event's doc comment works the same.

We don't actually check doc comments for commands and events.
Instead, QAPISchema._def_command() forwards the doc comment to the
implicit argument type, where it gets checked.  Works because the
check only cares for the implicit argument type's members.

Not only is this needlessly hard to understand, it actually falls
apart in two cases:

* When 'data' is empty, there is nothing to forward to, and the doc
  comment remains unchecked.  Demonstrated by test doc-bad-event-arg.

* When 'data' names a type, we can't forward, as the type has its own
  doc comment.  The command or event's doc comment remains unchecked.
  Demonstrated by test doc-bad-boxed-command-arg.

The forwarding goes back to commit 069fb5b250 "qapi: Prepare for
requiring more complete documentation", put to use in commit
816a57cd6e "qapi: Fix detection of bogus member documentation".  That
fix was incomplete.

To fix this, make QAPISchemaCommand and QAPISchemaEvent check doc
comments, and drop the forwarding of doc comments to implicit argument
types.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-12-armbru@redhat.com>
2019-10-29 07:35:16 +01:00