Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
modules" switched QAPISchema.visit() from
for entity in self._entity_list:
effectively to
for mod in self._module_dict.values():
for entity in mod._entity_list:
Visits in the same order as long as .values() is in insertion order.
That's the case only for Python 3.6 and later. Before, it's in some
arbitrary order, which results in broken generated code.
Fix by making self._module_dict an OrderedDict rather than a dict.
Fixes: 3e7fb5811b
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Thomas Huth <thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: BALATON Zoltan <balaton@eik.bme.hu>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20200116202558.31473-1-armbru@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
When a sub-module doesn't contain any definitions, we don't generate
code for it, but we do generate the #include.
We generate code only for modules that get visited.
QAPISchema.visit() visits only modules that have definitions. It can
visit modules multiple times.
Clean this up as follows. Collect entities in their QAPISchemaModule.
Have QAPISchema.visit() call QAPISchemaModule.visit() for each module.
Have QAPISchemaModule.visit() call .visit_module() for itself, and
QAPISchemaEntity.visit() for each of its entities. This way, we visit
each module exactly once.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-6-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Modules are represented only by their names so far. Introduce class
QAPISchemaModule. So far, it merely wraps the name. The next patch
will put it to more interesting use.
Once again, arrays spice up the patch a bit. For any other type,
@info points to the definition, which lets us map from @info to
module. For arrays, there is no definition, and @info points to the
first use instead. We have to use the element type's module instead,
which is only available after .check().
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-5-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-2-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Commit f3ed93d545 "qapi: Allow documentation for features" neglected
to check documentation against the schema. Fix that: check them the
same way we check arguments.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-20-armbru@redhat.com>
Commit 6a8c0b5102 "qapi: Add feature flags to struct types" added
features to QAPISchemaObjectType. Commit a95daa5093 "qapi: Add
feature flags to commands in qapi" added them to QAPISchemaCommand,
duplicating the code. Tolerable, but the duplication will only get
worse as we add features to more definitions.
To de-duplicate, lift features from QAPISchemaObjectType and
QAPISchemaCommand into QAPISchemaEntity.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-18-armbru@redhat.com>
All sub-classes of QAPISchemaEntity now override .check_doc() the same
way, except for QAPISchemaType and and QAPISchemaArrayType.
Put the overrides' code in QAPISchemaEntity.check_doc(), and drop the
overrides. QAPISchemaType doesn't care because it's abstract.
QAPISchemaArrayType doesn't care because its .doc is always None.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-14-armbru@redhat.com>
All callers now pass doc=None. Drop the argument.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-13-armbru@redhat.com>
When a command's 'data' is an object, its doc comment describes the
arguments defined there. When 'data' names a type, the doc comment
does not describe arguments. Instead, the doc generator inserts a
pointer to the named type.
An event's doc comment works the same.
We don't actually check doc comments for commands and events.
Instead, QAPISchema._def_command() forwards the doc comment to the
implicit argument type, where it gets checked. Works because the
check only cares for the implicit argument type's members.
Not only is this needlessly hard to understand, it actually falls
apart in two cases:
* When 'data' is empty, there is nothing to forward to, and the doc
comment remains unchecked. Demonstrated by test doc-bad-event-arg.
* When 'data' names a type, we can't forward, as the type has its own
doc comment. The command or event's doc comment remains unchecked.
Demonstrated by test doc-bad-boxed-command-arg.
The forwarding goes back to commit 069fb5b250 "qapi: Prepare for
requiring more complete documentation", put to use in commit
816a57cd6e "qapi: Fix detection of bogus member documentation". That
fix was incomplete.
To fix this, make QAPISchemaCommand and QAPISchemaEvent check doc
comments, and drop the forwarding of doc comments to implicit argument
types.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-12-armbru@redhat.com>
An object type's doc comment describes the type's members, less the
ones defined in a named base type. Cases:
* Struct: the members are defined in 'data' and inherited from 'base'.
Since the base type cannot be implicit, the doc comment describes
just 'data'.
* Simple union: the only member is the implicit tag member @type, and
the doc comment describes it.
* Flat union with implicit base type: the members are defined in
'base', and the doc comment describes it.
* Flat union with named base type: the members are inherited from
'base'. The doc comment describes no members.
Before we can check a doc comment with .check_doc(), we need
.connect_doc() connect each of its "argument sections" to the member
it documents.
For structs and simple unions, this is straightforward: the members in
question are in .local_members, and .connect_doc() connects them.
For flat unions with a named base type, it's trivial: .local_members
is empty, and .connect_doc() does nothing.
For flat unions with an implicit base type, it's tricky. We have
QAPISchema._make_implicit_object_type() forward the union's doc
comment to the implicit base type, so that the base type's
.connect_doc() connects the members. The union's .connect_doc() does
nothing, as .local_members is empty.
Dirt effect: we check the doc comment twice, once for the union type,
and once for the implicit base type.
This is needlessly brittle and hard to understand. Clean up as
follows. Make the union's .connect_doc() connect an implicit base's
members itself. Do not forward the union's doc comment to its
implicit base type.
Requires extending .connect_doc() so it can work with a doc comment
other than self.doc. Add an optional argument for that.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-11-armbru@redhat.com>
Enumeration type documentation comments are not checked, as
demonstrated by test doc-bad-enum-member. This is because we neglect
to call self.doc.check() for enumeration types. Messed up in
816a57cd6e "qapi: Fix detection of bogus member documentation". Fix
it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-10-armbru@redhat.com>
Splitting documentation checking off the .check() methods makes them a
bit more focused, which is welcome, as some of them are pretty big.
It also prepares the ground for the following commits.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-9-armbru@redhat.com>
Similarly to features for struct types introduce the feature flags also
for commands. This will allow notifying management layers of fixes and
compatible changes in the behaviour of a command which may not be
detectable any other way.
The changes were heavily inspired by commit 6a8c0b5102.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191018081454.21369-3-armbru@redhat.com>
The QAPI code generator clocks in at some 3100 SLOC in 8 source files.
Almost 60% of the code is in qapi/common.py. Split it into more
focused modules:
* Move QAPISchemaPragma and QAPISourceInfo to qapi/source.py.
* Move QAPIError and its sub-classes to qapi/error.py.
* Move QAPISchemaParser and QAPIDoc to parser.py. Use the opportunity
to put QAPISchemaParser first.
* Move check_expr() & friends to qapi/expr.py. Use the opportunity to
put the code into a more sensible order.
* Move QAPISchema & friends to qapi/schema.py
* Move QAPIGen and its sub-classes, ifcontext,
QAPISchemaModularCVisitor, and QAPISchemaModularCVisitor to qapi/gen.py
* Delete camel_case(), it's unused since commit e98859a9b9 "qapi:
Clean up after recent conversions to QAPISchemaVisitor"
A number of helper functions remain in qapi/common.py. I considered
moving the code generator helpers to qapi/gen.py, but decided not to.
Perhaps we should rewrite them as methods of QAPIGen some day.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-7-armbru@redhat.com>
[Add "# -*- coding: utf-8 -*-" lines]
Markus Armbruster
Peter Krempa