2011-07-19 21:50:39 +02:00
|
|
|
#
|
|
|
|
# QAPI helper library
|
|
|
|
#
|
|
|
|
# Copyright IBM, Corp. 2011
|
2016-01-29 14:48:41 +01:00
|
|
|
# Copyright (c) 2013-2016 Red Hat Inc.
|
2011-07-19 21:50:39 +02:00
|
|
|
#
|
|
|
|
# Authors:
|
|
|
|
# Anthony Liguori <aliguori@us.ibm.com>
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
# Markus Armbruster <armbru@redhat.com>
|
2011-07-19 21:50:39 +02:00
|
|
|
#
|
2014-03-01 08:40:34 +01:00
|
|
|
# This work is licensed under the terms of the GNU GPL, version 2.
|
|
|
|
# See the COPYING file in the top-level directory.
|
2011-07-19 21:50:39 +02:00
|
|
|
|
2014-05-07 20:46:15 +02:00
|
|
|
import re
|
2011-07-19 21:50:39 +02:00
|
|
|
from ordereddict import OrderedDict
|
2015-04-02 14:46:39 +02:00
|
|
|
import errno
|
2015-04-02 13:12:21 +02:00
|
|
|
import getopt
|
2014-05-02 15:52:35 +02:00
|
|
|
import os
|
2013-07-27 17:41:56 +02:00
|
|
|
import sys
|
2015-05-14 14:50:47 +02:00
|
|
|
import string
|
2011-07-19 21:50:39 +02:00
|
|
|
|
2015-05-04 17:05:00 +02:00
|
|
|
builtin_types = {
|
2013-07-08 16:14:21 +02:00
|
|
|
'str': 'QTYPE_QSTRING',
|
|
|
|
'int': 'QTYPE_QINT',
|
|
|
|
'number': 'QTYPE_QFLOAT',
|
|
|
|
'bool': 'QTYPE_QBOOL',
|
|
|
|
'int8': 'QTYPE_QINT',
|
|
|
|
'int16': 'QTYPE_QINT',
|
|
|
|
'int32': 'QTYPE_QINT',
|
|
|
|
'int64': 'QTYPE_QINT',
|
|
|
|
'uint8': 'QTYPE_QINT',
|
|
|
|
'uint16': 'QTYPE_QINT',
|
|
|
|
'uint32': 'QTYPE_QINT',
|
|
|
|
'uint64': 'QTYPE_QINT',
|
2015-05-04 17:05:01 +02:00
|
|
|
'size': 'QTYPE_QINT',
|
2015-12-02 06:20:46 +01:00
|
|
|
'any': None, # any QType possible, actually
|
qapi: Convert QType into QAPI built-in enum type
What's more meta than using qapi to define qapi? :)
Convert QType into a full-fledged[*] builtin qapi enum type, so
that a subsequent patch can then use it as the discriminator
type of qapi alternate types. Fortunately, the judicious use of
'prefix' in the qapi definition avoids churn to the spelling of
the enum constants.
To avoid circular definitions, we have to flip the order of
inclusion between "qobject.h" vs. "qapi-types.h". Back in commit
28770e0, we had the latter include the former, so that we could
use 'QObject *' for our implementation of 'any'. But that usage
also works with only a forward declaration, whereas the
definition of QObject requires QType to be a complete type.
[*] The type has to be builtin, rather than declared in
qapi/common.json, because we want to use it for alternates even
when common.json is not included. But since it is the first
builtin enum type, we have to add special cases to qapi-types
and qapi-visit to only emit definitions once, even when two
qapi files are being compiled into the same binary (the way we
already handled builtin list types like 'intList'). We may
need to revisit how multiple qapi files share common types,
but that's a project for another day.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-4-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:47 +01:00
|
|
|
'QType': 'QTYPE_QSTRING',
|
2013-07-08 16:14:21 +02:00
|
|
|
}
|
|
|
|
|
2015-05-04 17:05:23 +02:00
|
|
|
# Whitelist of commands allowed to return a non-dictionary
|
|
|
|
returns_whitelist = [
|
|
|
|
# From QMP:
|
|
|
|
'human-monitor-command',
|
2015-09-16 13:06:25 +02:00
|
|
|
'qom-get',
|
2015-05-04 17:05:23 +02:00
|
|
|
'query-migrate-cache-size',
|
|
|
|
'query-tpm-models',
|
|
|
|
'query-tpm-types',
|
|
|
|
'ringbuf-read',
|
|
|
|
|
|
|
|
# From QGA:
|
|
|
|
'guest-file-open',
|
|
|
|
'guest-fsfreeze-freeze',
|
|
|
|
'guest-fsfreeze-freeze-list',
|
|
|
|
'guest-fsfreeze-status',
|
|
|
|
'guest-fsfreeze-thaw',
|
|
|
|
'guest-get-time',
|
|
|
|
'guest-set-vcpus',
|
|
|
|
'guest-sync',
|
|
|
|
'guest-sync-delimited',
|
|
|
|
]
|
|
|
|
|
qapi: Enforce (or whitelist) case conventions on qapi members
We document that members of enums and objects should be
'lower-case', although we were not enforcing it. We have to
whitelist a few pre-existing entities that violate the norms.
Add three new tests to expose the new error message, each of
which first uses the whitelisted name 'UuidInfo' to prove the
whitelist works, then triggers the failure (this is the same
pattern used in the existing returns-whitelist.json test).
Note that by adding this check, we have effectively forbidden
an entity with a case-insensitive clash of member names, for
any entity that is not on the whitelist (although there is
still the possibility to clash via '-' vs. '_').
Not done here: a future patch should also add naming convention
support and whitelist exceptions for command, event, and type
names.
The additions to QAPISchemaMember.check_clash() check whether
info['name'] is in the whitelist (the top-most entity name at
the point 'info' tracks), rather than self.owner (the type,
possibly implicit, that directly owns the member), because it
is easier to maintain the whitelist by the names actually in
the user's .json file, rather than worrying about the names
of implicit types.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-14-git-send-email-eblake@redhat.com>
[Simplified a bit as per discussion with Eric]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:57 +01:00
|
|
|
# Whitelist of entities allowed to violate case conventions
|
|
|
|
case_whitelist = [
|
|
|
|
# From QMP:
|
|
|
|
'ACPISlotType', # DIMM, visible through query-acpi-ospm-status
|
|
|
|
'CpuInfoBase', # CPU, visible through query-cpu
|
|
|
|
'CpuInfoMIPS', # PC, visible through query-cpu
|
|
|
|
'CpuInfoTricore', # PC, visible through query-cpu
|
|
|
|
'InputAxis', # TODO: drop when x-input-send-event is fixed
|
|
|
|
'InputButton', # TODO: drop when x-input-send-event is fixed
|
|
|
|
'QapiErrorClass', # all members, visible through errors
|
|
|
|
'UuidInfo', # UUID, visible through query-uuid
|
|
|
|
'X86CPURegister32', # all members, visible indirectly through qom-get
|
|
|
|
]
|
|
|
|
|
2015-05-04 17:05:17 +02:00
|
|
|
enum_types = []
|
|
|
|
struct_types = []
|
|
|
|
union_types = []
|
|
|
|
events = []
|
|
|
|
all_names = {}
|
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
#
|
|
|
|
# Parsing the schema into expressions
|
|
|
|
#
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2014-05-07 20:46:15 +02:00
|
|
|
def error_path(parent):
|
|
|
|
res = ""
|
|
|
|
while parent:
|
|
|
|
res = ("In file included from %s:%d:\n" % (parent['file'],
|
|
|
|
parent['line'])) + res
|
|
|
|
parent = parent['parent']
|
|
|
|
return res
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2013-07-27 17:41:56 +02:00
|
|
|
class QAPISchemaError(Exception):
|
|
|
|
def __init__(self, schema, msg):
|
2015-09-30 00:21:01 +02:00
|
|
|
Exception.__init__(self)
|
2015-06-09 16:22:45 +02:00
|
|
|
self.fname = schema.fname
|
2013-07-27 17:41:56 +02:00
|
|
|
self.msg = msg
|
2014-03-05 03:44:33 +01:00
|
|
|
self.col = 1
|
|
|
|
self.line = schema.line
|
|
|
|
for ch in schema.src[schema.line_pos:schema.pos]:
|
|
|
|
if ch == '\t':
|
2013-07-27 17:41:56 +02:00
|
|
|
self.col = (self.col + 7) % 8 + 1
|
|
|
|
else:
|
|
|
|
self.col += 1
|
2015-06-09 16:22:45 +02:00
|
|
|
self.info = schema.incl_info
|
2013-07-27 17:41:56 +02:00
|
|
|
|
|
|
|
def __str__(self):
|
2014-05-07 20:46:15 +02:00
|
|
|
return error_path(self.info) + \
|
2015-06-09 16:22:45 +02:00
|
|
|
"%s:%d:%d: %s" % (self.fname, self.line, self.col, self.msg)
|
2013-07-27 17:41:56 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2014-03-05 03:44:34 +01:00
|
|
|
class QAPIExprError(Exception):
|
|
|
|
def __init__(self, expr_info, msg):
|
2015-09-30 00:21:01 +02:00
|
|
|
Exception.__init__(self)
|
2015-10-13 06:22:22 +02:00
|
|
|
assert expr_info
|
2014-05-07 20:46:15 +02:00
|
|
|
self.info = expr_info
|
2014-03-05 03:44:34 +01:00
|
|
|
self.msg = msg
|
|
|
|
|
|
|
|
def __str__(self):
|
2014-05-07 20:46:15 +02:00
|
|
|
return error_path(self.info['parent']) + \
|
|
|
|
"%s:%d: %s" % (self.info['file'], self.info['line'], self.msg)
|
2014-03-05 03:44:34 +01:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-09-16 13:06:04 +02:00
|
|
|
class QAPISchemaParser(object):
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
def __init__(self, fp, previously_included=[], incl_info=None):
|
2015-06-09 16:22:45 +02:00
|
|
|
abs_fname = os.path.abspath(fp.name)
|
2015-06-09 18:32:29 +02:00
|
|
|
fname = fp.name
|
2015-06-09 16:22:45 +02:00
|
|
|
self.fname = fname
|
|
|
|
previously_included.append(abs_fname)
|
|
|
|
self.incl_info = incl_info
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
self.src = fp.read()
|
|
|
|
if self.src == '' or self.src[-1] != '\n':
|
|
|
|
self.src += '\n'
|
|
|
|
self.cursor = 0
|
2014-03-05 03:44:33 +01:00
|
|
|
self.line = 1
|
|
|
|
self.line_pos = 0
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
self.exprs = []
|
|
|
|
self.accept()
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
while self.tok is not None:
|
2015-06-09 16:22:45 +02:00
|
|
|
expr_info = {'file': fname, 'line': self.line,
|
|
|
|
'parent': self.incl_info}
|
2014-05-07 20:46:15 +02:00
|
|
|
expr = self.get_expr(False)
|
|
|
|
if isinstance(expr, dict) and "include" in expr:
|
|
|
|
if len(expr) != 1:
|
2015-09-30 00:21:02 +02:00
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"Invalid 'include' directive")
|
2014-05-07 20:46:15 +02:00
|
|
|
include = expr["include"]
|
|
|
|
if not isinstance(include, str):
|
|
|
|
raise QAPIExprError(expr_info,
|
2015-09-30 00:21:00 +02:00
|
|
|
"Value of 'include' must be a string")
|
2015-06-09 16:22:45 +02:00
|
|
|
incl_abs_fname = os.path.join(os.path.dirname(abs_fname),
|
|
|
|
include)
|
2015-06-09 16:54:09 +02:00
|
|
|
# catch inclusion cycle
|
|
|
|
inf = expr_info
|
|
|
|
while inf:
|
|
|
|
if incl_abs_fname == os.path.abspath(inf['file']):
|
2014-08-27 13:08:51 +02:00
|
|
|
raise QAPIExprError(expr_info, "Inclusion loop for %s"
|
|
|
|
% include)
|
2015-06-09 16:54:09 +02:00
|
|
|
inf = inf['parent']
|
2014-05-16 12:51:56 +02:00
|
|
|
# skip multiple include of the same file
|
2015-06-09 16:22:45 +02:00
|
|
|
if incl_abs_fname in previously_included:
|
2014-05-16 12:51:56 +02:00
|
|
|
continue
|
2014-05-07 20:46:15 +02:00
|
|
|
try:
|
2015-06-09 16:22:45 +02:00
|
|
|
fobj = open(incl_abs_fname, 'r')
|
2015-12-18 08:52:41 +01:00
|
|
|
except IOError as e:
|
2014-05-07 20:46:15 +02:00
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
'%s: %s' % (e.strerror, include))
|
2015-09-16 13:06:04 +02:00
|
|
|
exprs_include = QAPISchemaParser(fobj, previously_included,
|
|
|
|
expr_info)
|
2014-05-07 20:46:15 +02:00
|
|
|
self.exprs.extend(exprs_include.exprs)
|
|
|
|
else:
|
|
|
|
expr_elem = {'expr': expr,
|
|
|
|
'info': expr_info}
|
|
|
|
self.exprs.append(expr_elem)
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
|
|
|
|
def accept(self):
|
|
|
|
while True:
|
|
|
|
self.tok = self.src[self.cursor]
|
2013-07-27 17:41:56 +02:00
|
|
|
self.pos = self.cursor
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
self.cursor += 1
|
|
|
|
self.val = None
|
|
|
|
|
2013-07-27 17:42:01 +02:00
|
|
|
if self.tok == '#':
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
self.cursor = self.src.find('\n', self.cursor)
|
2015-10-26 23:34:41 +01:00
|
|
|
elif self.tok in "{}:,[]":
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
return
|
|
|
|
elif self.tok == "'":
|
|
|
|
string = ''
|
|
|
|
esc = False
|
|
|
|
while True:
|
|
|
|
ch = self.src[self.cursor]
|
|
|
|
self.cursor += 1
|
|
|
|
if ch == '\n':
|
2013-07-27 17:41:56 +02:00
|
|
|
raise QAPISchemaError(self,
|
|
|
|
'Missing terminating "\'"')
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
if esc:
|
qapi: Support (subset of) \u escapes in strings
The handling of \ inside QAPI strings was less than ideal, and
really only worked JSON's \/, \\, \", and our extension of \'
(an obvious extension, when you realize we use '' instead of ""
for strings). For other things, like '\n', it resulted in a
literal 'n' instead of a newline.
Of course, at the moment, we really have no use for escaped
characters, as QAPI has to map to C identifiers, and we currently
support ASCII only for that. But down the road, we may add
support for default values for string parameters to a command
or struct; if that happens, it would be nice to correctly support
all JSON escape sequences, such as \n or \uXXXX. This gets us
closer, by supporting Unicode escapes in the ASCII range.
Since JSON does not require \OCTAL or \xXX escapes, and our QMP
implementation does not understand them either, I intentionally
reject it here, but it would be an easy addition if we desired it.
Likewise, intentionally refusing the NUL byte means we don't have
to worry about C strings being shorter than the qapi input.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-04 17:05:36 +02:00
|
|
|
if ch == 'b':
|
|
|
|
string += '\b'
|
|
|
|
elif ch == 'f':
|
|
|
|
string += '\f'
|
|
|
|
elif ch == 'n':
|
|
|
|
string += '\n'
|
|
|
|
elif ch == 'r':
|
|
|
|
string += '\r'
|
|
|
|
elif ch == 't':
|
|
|
|
string += '\t'
|
|
|
|
elif ch == 'u':
|
|
|
|
value = 0
|
2015-09-30 00:21:02 +02:00
|
|
|
for _ in range(0, 4):
|
qapi: Support (subset of) \u escapes in strings
The handling of \ inside QAPI strings was less than ideal, and
really only worked JSON's \/, \\, \", and our extension of \'
(an obvious extension, when you realize we use '' instead of ""
for strings). For other things, like '\n', it resulted in a
literal 'n' instead of a newline.
Of course, at the moment, we really have no use for escaped
characters, as QAPI has to map to C identifiers, and we currently
support ASCII only for that. But down the road, we may add
support for default values for string parameters to a command
or struct; if that happens, it would be nice to correctly support
all JSON escape sequences, such as \n or \uXXXX. This gets us
closer, by supporting Unicode escapes in the ASCII range.
Since JSON does not require \OCTAL or \xXX escapes, and our QMP
implementation does not understand them either, I intentionally
reject it here, but it would be an easy addition if we desired it.
Likewise, intentionally refusing the NUL byte means we don't have
to worry about C strings being shorter than the qapi input.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-04 17:05:36 +02:00
|
|
|
ch = self.src[self.cursor]
|
|
|
|
self.cursor += 1
|
|
|
|
if ch not in "0123456789abcdefABCDEF":
|
|
|
|
raise QAPISchemaError(self,
|
|
|
|
'\\u escape needs 4 '
|
|
|
|
'hex digits')
|
|
|
|
value = (value << 4) + int(ch, 16)
|
|
|
|
# If Python 2 and 3 didn't disagree so much on
|
|
|
|
# how to handle Unicode, then we could allow
|
|
|
|
# Unicode string defaults. But most of QAPI is
|
|
|
|
# ASCII-only, so we aren't losing much for now.
|
|
|
|
if not value or value > 0x7f:
|
|
|
|
raise QAPISchemaError(self,
|
|
|
|
'For now, \\u escape '
|
|
|
|
'only supports non-zero '
|
|
|
|
'values up to \\u007f')
|
|
|
|
string += chr(value)
|
|
|
|
elif ch in "\\/'\"":
|
|
|
|
string += ch
|
|
|
|
else:
|
|
|
|
raise QAPISchemaError(self,
|
2015-09-30 00:21:02 +02:00
|
|
|
"Unknown escape \\%s" % ch)
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
esc = False
|
|
|
|
elif ch == "\\":
|
|
|
|
esc = True
|
|
|
|
elif ch == "'":
|
|
|
|
self.val = string
|
|
|
|
return
|
|
|
|
else:
|
|
|
|
string += ch
|
2015-06-10 08:24:58 +02:00
|
|
|
elif self.src.startswith("true", self.pos):
|
|
|
|
self.val = True
|
|
|
|
self.cursor += 3
|
|
|
|
return
|
|
|
|
elif self.src.startswith("false", self.pos):
|
|
|
|
self.val = False
|
|
|
|
self.cursor += 4
|
|
|
|
return
|
|
|
|
elif self.src.startswith("null", self.pos):
|
|
|
|
self.val = None
|
|
|
|
self.cursor += 3
|
|
|
|
return
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
elif self.tok == '\n':
|
|
|
|
if self.cursor == len(self.src):
|
|
|
|
self.tok = None
|
|
|
|
return
|
2014-03-05 03:44:33 +01:00
|
|
|
self.line += 1
|
|
|
|
self.line_pos = self.cursor
|
2013-07-27 17:41:57 +02:00
|
|
|
elif not self.tok.isspace():
|
|
|
|
raise QAPISchemaError(self, 'Stray "%s"' % self.tok)
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
|
|
|
|
def get_members(self):
|
|
|
|
expr = OrderedDict()
|
2013-07-27 17:41:58 +02:00
|
|
|
if self.tok == '}':
|
|
|
|
self.accept()
|
|
|
|
return expr
|
|
|
|
if self.tok != "'":
|
|
|
|
raise QAPISchemaError(self, 'Expected string or "}"')
|
|
|
|
while True:
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
key = self.val
|
|
|
|
self.accept()
|
2013-07-27 17:41:58 +02:00
|
|
|
if self.tok != ':':
|
|
|
|
raise QAPISchemaError(self, 'Expected ":"')
|
|
|
|
self.accept()
|
2014-03-05 03:44:32 +01:00
|
|
|
if key in expr:
|
|
|
|
raise QAPISchemaError(self, 'Duplicate key "%s"' % key)
|
2013-07-27 17:41:59 +02:00
|
|
|
expr[key] = self.get_expr(True)
|
2013-07-27 17:41:58 +02:00
|
|
|
if self.tok == '}':
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
self.accept()
|
2013-07-27 17:41:58 +02:00
|
|
|
return expr
|
|
|
|
if self.tok != ',':
|
|
|
|
raise QAPISchemaError(self, 'Expected "," or "}"')
|
|
|
|
self.accept()
|
|
|
|
if self.tok != "'":
|
|
|
|
raise QAPISchemaError(self, 'Expected string')
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
|
|
|
|
def get_values(self):
|
|
|
|
expr = []
|
2013-07-27 17:41:58 +02:00
|
|
|
if self.tok == ']':
|
|
|
|
self.accept()
|
|
|
|
return expr
|
2015-09-30 00:21:02 +02:00
|
|
|
if self.tok not in "{['tfn":
|
qapi: Allow true, false and null in schema json
In the near term, we will use it for a sensible-looking
'gen':false inside command declarations, instead of the
current ugly 'gen':'no'.
In the long term, it will allow conversion from shorthand
with defaults mentioned only in side-band documentation:
'data':{'*flag':'bool', '*string':'str'}
into an explicit default value documentation, as in:
'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
'string':{'type':'str', 'optional':true, 'default':null}}
We still don't parse integer values (also necessary before
we can allow explicit defaults), but that can come in a later
series.
Update the testsuite to match an improved error message.
Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-04 17:05:18 +02:00
|
|
|
raise QAPISchemaError(self, 'Expected "{", "[", "]", string, '
|
|
|
|
'boolean or "null"')
|
2013-07-27 17:41:58 +02:00
|
|
|
while True:
|
2013-07-27 17:41:59 +02:00
|
|
|
expr.append(self.get_expr(True))
|
2013-07-27 17:41:58 +02:00
|
|
|
if self.tok == ']':
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
self.accept()
|
2013-07-27 17:41:58 +02:00
|
|
|
return expr
|
|
|
|
if self.tok != ',':
|
|
|
|
raise QAPISchemaError(self, 'Expected "," or "]"')
|
|
|
|
self.accept()
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
|
2013-07-27 17:41:59 +02:00
|
|
|
def get_expr(self, nested):
|
|
|
|
if self.tok != '{' and not nested:
|
|
|
|
raise QAPISchemaError(self, 'Expected "{"')
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
if self.tok == '{':
|
|
|
|
self.accept()
|
|
|
|
expr = self.get_members()
|
|
|
|
elif self.tok == '[':
|
|
|
|
self.accept()
|
|
|
|
expr = self.get_values()
|
qapi: Allow true, false and null in schema json
In the near term, we will use it for a sensible-looking
'gen':false inside command declarations, instead of the
current ugly 'gen':'no'.
In the long term, it will allow conversion from shorthand
with defaults mentioned only in side-band documentation:
'data':{'*flag':'bool', '*string':'str'}
into an explicit default value documentation, as in:
'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
'string':{'type':'str', 'optional':true, 'default':null}}
We still don't parse integer values (also necessary before
we can allow explicit defaults), but that can come in a later
series.
Update the testsuite to match an improved error message.
Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-04 17:05:18 +02:00
|
|
|
elif self.tok in "'tfn":
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
expr = self.val
|
|
|
|
self.accept()
|
2013-07-27 17:41:58 +02:00
|
|
|
else:
|
|
|
|
raise QAPISchemaError(self, 'Expected "{", "[" or string')
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
return expr
|
2013-07-01 16:31:50 +02:00
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
#
|
|
|
|
# Semantic analysis of schema expressions
|
2015-09-16 13:06:05 +02:00
|
|
|
# TODO fold into QAPISchema
|
|
|
|
# TODO catching name collisions in generated code would be nice
|
2015-06-10 10:04:36 +02:00
|
|
|
#
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2014-03-05 03:44:34 +01:00
|
|
|
def find_base_fields(base):
|
|
|
|
base_struct_define = find_struct(base)
|
|
|
|
if not base_struct_define:
|
|
|
|
return None
|
|
|
|
return base_struct_define['data']
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-05-04 17:05:10 +02:00
|
|
|
# Return the qtype of an alternate branch, or None on error.
|
|
|
|
def find_alternate_member_qtype(qapi_type):
|
2015-09-30 00:21:02 +02:00
|
|
|
if qapi_type in builtin_types:
|
2015-05-04 17:05:08 +02:00
|
|
|
return builtin_types[qapi_type]
|
|
|
|
elif find_struct(qapi_type):
|
|
|
|
return "QTYPE_QDICT"
|
|
|
|
elif find_enum(qapi_type):
|
|
|
|
return "QTYPE_QSTRING"
|
2015-05-04 17:05:10 +02:00
|
|
|
elif find_union(qapi_type):
|
|
|
|
return "QTYPE_QDICT"
|
2015-05-04 17:05:08 +02:00
|
|
|
return None
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2014-03-07 02:08:56 +01:00
|
|
|
# Return the discriminator enum define if discriminator is specified as an
|
|
|
|
# enum type, otherwise return None.
|
|
|
|
def discriminator_find_enum_define(expr):
|
|
|
|
base = expr.get('base')
|
|
|
|
discriminator = expr.get('discriminator')
|
|
|
|
|
|
|
|
if not (discriminator and base):
|
|
|
|
return None
|
|
|
|
|
|
|
|
base_fields = find_base_fields(base)
|
|
|
|
if not base_fields:
|
|
|
|
return None
|
|
|
|
|
|
|
|
discriminator_type = base_fields.get(discriminator)
|
|
|
|
if not discriminator_type:
|
|
|
|
return None
|
|
|
|
|
|
|
|
return find_enum(discriminator_type)
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-11-18 09:52:56 +01:00
|
|
|
# Names must be letters, numbers, -, and _. They must start with letter,
|
|
|
|
# except for downstream extensions which must start with __RFQDN_.
|
|
|
|
# Dots are only valid in the downstream extension prefix.
|
|
|
|
valid_name = re.compile('^(__[a-zA-Z0-9.-]+_)?'
|
|
|
|
'[a-zA-Z][a-zA-Z0-9_-]*$')
|
2015-09-30 00:21:02 +02:00
|
|
|
|
|
|
|
|
|
|
|
def check_name(expr_info, source, name, allow_optional=False,
|
|
|
|
enum_member=False):
|
2015-05-04 17:05:22 +02:00
|
|
|
global valid_name
|
|
|
|
membername = name
|
|
|
|
|
|
|
|
if not isinstance(name, str):
|
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"%s requires a string name" % source)
|
|
|
|
if name.startswith('*'):
|
|
|
|
membername = name[1:]
|
|
|
|
if not allow_optional:
|
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"%s does not allow optional name '%s'"
|
|
|
|
% (source, name))
|
|
|
|
# Enum members can start with a digit, because the generated C
|
|
|
|
# code always prefixes it with the enum name
|
2015-11-18 09:52:56 +01:00
|
|
|
if enum_member and membername[0].isdigit():
|
|
|
|
membername = 'D' + membername
|
qapi: Reserve 'q_*' and 'has_*' member names
c_name() produces names starting with 'q_' when protecting a
dictionary member name that would fail to directly compile, but
in doing so can cause clashes with any member name already
beginning with 'q-' or 'q_'. Likewise, we create a C name 'has_'
for any optional member that can clash with any member name
beginning with 'has-' or 'has_'.
Technically, rather than blindly reserving the namespace,
we could try to complain about user names only when an actual
collision occurs, or even teach c_name() how to munge names
to avoid collisions. But it is not trivial, especially when
collisions can occur across multiple types (such as via
inheritance or flat unions). Besides, no existing .json
files are trying to use these names. So it's easier to just
outright forbid the potential for collision. We can always
relax things in the future if a real need arises for QMP to
express member names that have been forbidden here.
'has_' only has to be reserved for struct/union member names,
while 'q_' is reserved everywhere (matching the fact that
only members can be optional, while we use c_name() for munging
both members and entities). Note that we could relax 'q_'
restrictions on entities independently from member names; for
example, c_name('qmp_' + 'unix') would result in a different
function name than our current 'qmp_' + c_name('unix').
Update and add tests to cover the new error messages.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1445898903-12082-6-git-send-email-eblake@redhat.com>
[Consistently pass protect=False to c_name(); commit message tweaked
slightly]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-26 23:34:44 +01:00
|
|
|
# Reserve the entire 'q_' namespace for c_name()
|
|
|
|
if not valid_name.match(membername) or \
|
|
|
|
c_name(membername, False).startswith('q_'):
|
2015-05-04 17:05:22 +02:00
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"%s uses invalid name '%s'" % (source, name))
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
|
|
|
def add_name(name, info, meta, implicit=False):
|
2015-06-10 10:04:36 +02:00
|
|
|
global all_names
|
|
|
|
check_name(info, "'%s'" % meta, name)
|
2015-07-31 11:33:52 +02:00
|
|
|
# FIXME should reject names that differ only in '_' vs. '.'
|
|
|
|
# vs. '-', because they're liable to clash in generated C.
|
2015-06-10 10:04:36 +02:00
|
|
|
if name in all_names:
|
|
|
|
raise QAPIExprError(info,
|
|
|
|
"%s '%s' is already defined"
|
|
|
|
% (all_names[name], name))
|
2015-10-26 23:34:43 +01:00
|
|
|
if not implicit and (name.endswith('Kind') or name.endswith('List')):
|
2015-06-10 10:04:36 +02:00
|
|
|
raise QAPIExprError(info,
|
2015-10-26 23:34:43 +01:00
|
|
|
"%s '%s' should not end in '%s'"
|
|
|
|
% (meta, name, name[-4:]))
|
2015-06-10 10:04:36 +02:00
|
|
|
all_names[name] = meta
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
def add_struct(definition, info):
|
|
|
|
global struct_types
|
|
|
|
name = definition['struct']
|
|
|
|
add_name(name, info, 'struct')
|
|
|
|
struct_types.append(definition)
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
def find_struct(name):
|
|
|
|
global struct_types
|
|
|
|
for struct in struct_types:
|
|
|
|
if struct['struct'] == name:
|
|
|
|
return struct
|
|
|
|
return None
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
def add_union(definition, info):
|
|
|
|
global union_types
|
|
|
|
name = definition['union']
|
|
|
|
add_name(name, info, 'union')
|
|
|
|
union_types.append(definition)
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
def find_union(name):
|
|
|
|
global union_types
|
|
|
|
for union in union_types:
|
|
|
|
if union['union'] == name:
|
|
|
|
return union
|
|
|
|
return None
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
|
|
|
def add_enum(name, info, enum_values=None, implicit=False):
|
2015-06-10 10:04:36 +02:00
|
|
|
global enum_types
|
|
|
|
add_name(name, info, 'enum', implicit)
|
|
|
|
enum_types.append({"enum_name": name, "enum_values": enum_values})
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
def find_enum(name):
|
|
|
|
global enum_types
|
|
|
|
for enum in enum_types:
|
|
|
|
if enum['enum_name'] == name:
|
|
|
|
return enum
|
|
|
|
return None
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
def is_enum(name):
|
2015-09-30 00:21:02 +02:00
|
|
|
return find_enum(name) is not None
|
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
def check_type(expr_info, source, value, allow_array=False,
|
|
|
|
allow_dict=False, allow_optional=False,
|
|
|
|
allow_metas=[]):
|
2015-05-04 17:05:21 +02:00
|
|
|
global all_names
|
|
|
|
|
|
|
|
if value is None:
|
|
|
|
return
|
|
|
|
|
|
|
|
# Check if array type for value is okay
|
|
|
|
if isinstance(value, list):
|
|
|
|
if not allow_array:
|
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"%s cannot be an array" % source)
|
|
|
|
if len(value) != 1 or not isinstance(value[0], str):
|
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"%s: array type must contain single type name"
|
|
|
|
% source)
|
|
|
|
value = value[0]
|
|
|
|
|
|
|
|
# Check if type name for value is okay
|
|
|
|
if isinstance(value, str):
|
2015-09-30 00:21:02 +02:00
|
|
|
if value not in all_names:
|
2015-05-04 17:05:21 +02:00
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"%s uses unknown type '%s'"
|
qapi: Simplify error reporting for array types
check_type() first checks and peels off the array type, then checks
the element type. For two out of four error messages, it takes pains
to report errors for "array of T" instead of just T. Odd. Let's
examine the errors.
* Unknown element type, e.g.
tests/qapi-schema/args-array-unknown.json:
Member 'array' of 'data' for command 'oops' uses unknown type
'array of NoSuchType'
To make sense of this, you need to know that 'array of NoSuchType'
refers to '[NoSuchType]'. Easy enough. However, simply reporting
Member 'array' of 'data' for command 'oops' uses unknown type
'NoSuchType'
is at least as easy to understand.
* Element type's meta-type is inadmissible, e.g.
tests/qapi-schema/returns-whitelist.json:
'returns' for command 'no-way-this-will-get-whitelisted' cannot
use built-in type 'array of int'
'array of int' is technically not a built-in type, but that's
pedantry. However, simply reporting
'returns' for command 'no-way-this-will-get-whitelisted' cannot
use built-in type 'int'
avoids the issue, and is at least as easy to understand.
* The remaining two errors are unreachable, because the array checking
ensures that value is a string.
Thus, reporting some errors for "array of T" instead of just T works,
but doesn't really improve things. Drop it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-08-31 13:54:39 +02:00
|
|
|
% (source, value))
|
2015-05-04 17:05:21 +02:00
|
|
|
if not all_names[value] in allow_metas:
|
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"%s cannot use %s type '%s'"
|
qapi: Simplify error reporting for array types
check_type() first checks and peels off the array type, then checks
the element type. For two out of four error messages, it takes pains
to report errors for "array of T" instead of just T. Odd. Let's
examine the errors.
* Unknown element type, e.g.
tests/qapi-schema/args-array-unknown.json:
Member 'array' of 'data' for command 'oops' uses unknown type
'array of NoSuchType'
To make sense of this, you need to know that 'array of NoSuchType'
refers to '[NoSuchType]'. Easy enough. However, simply reporting
Member 'array' of 'data' for command 'oops' uses unknown type
'NoSuchType'
is at least as easy to understand.
* Element type's meta-type is inadmissible, e.g.
tests/qapi-schema/returns-whitelist.json:
'returns' for command 'no-way-this-will-get-whitelisted' cannot
use built-in type 'array of int'
'array of int' is technically not a built-in type, but that's
pedantry. However, simply reporting
'returns' for command 'no-way-this-will-get-whitelisted' cannot
use built-in type 'int'
avoids the issue, and is at least as easy to understand.
* The remaining two errors are unreachable, because the array checking
ensures that value is a string.
Thus, reporting some errors for "array of T" instead of just T works,
but doesn't really improve things. Drop it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-08-31 13:54:39 +02:00
|
|
|
% (source, all_names[value], value))
|
2015-05-04 17:05:21 +02:00
|
|
|
return
|
|
|
|
|
|
|
|
if not allow_dict:
|
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"%s should be a type name" % source)
|
2015-08-31 17:28:52 +02:00
|
|
|
|
|
|
|
if not isinstance(value, OrderedDict):
|
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"%s should be a dictionary or type name" % source)
|
|
|
|
|
|
|
|
# value is a dictionary, check that each member is okay
|
2015-05-04 17:05:21 +02:00
|
|
|
for (key, arg) in value.items():
|
2015-05-04 17:05:22 +02:00
|
|
|
check_name(expr_info, "Member of %s" % source, key,
|
|
|
|
allow_optional=allow_optional)
|
2015-10-26 23:35:02 +01:00
|
|
|
if c_name(key, False) == 'u' or c_name(key, False).startswith('has_'):
|
qapi: Reserve 'q_*' and 'has_*' member names
c_name() produces names starting with 'q_' when protecting a
dictionary member name that would fail to directly compile, but
in doing so can cause clashes with any member name already
beginning with 'q-' or 'q_'. Likewise, we create a C name 'has_'
for any optional member that can clash with any member name
beginning with 'has-' or 'has_'.
Technically, rather than blindly reserving the namespace,
we could try to complain about user names only when an actual
collision occurs, or even teach c_name() how to munge names
to avoid collisions. But it is not trivial, especially when
collisions can occur across multiple types (such as via
inheritance or flat unions). Besides, no existing .json
files are trying to use these names. So it's easier to just
outright forbid the potential for collision. We can always
relax things in the future if a real need arises for QMP to
express member names that have been forbidden here.
'has_' only has to be reserved for struct/union member names,
while 'q_' is reserved everywhere (matching the fact that
only members can be optional, while we use c_name() for munging
both members and entities). Note that we could relax 'q_'
restrictions on entities independently from member names; for
example, c_name('qmp_' + 'unix') would result in a different
function name than our current 'qmp_' + c_name('unix').
Update and add tests to cover the new error messages.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1445898903-12082-6-git-send-email-eblake@redhat.com>
[Consistently pass protect=False to c_name(); commit message tweaked
slightly]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-26 23:34:44 +01:00
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"Member of %s uses reserved name '%s'"
|
|
|
|
% (source, key))
|
2015-05-04 17:05:33 +02:00
|
|
|
# Todo: allow dictionaries to represent default values of
|
|
|
|
# an optional argument.
|
2015-05-04 17:05:21 +02:00
|
|
|
check_type(expr_info, "Member '%s' of %s" % (key, source), arg,
|
2015-09-16 13:06:27 +02:00
|
|
|
allow_array=True,
|
2015-05-04 17:05:21 +02:00
|
|
|
allow_metas=['built-in', 'union', 'alternate', 'struct',
|
2015-05-04 17:05:33 +02:00
|
|
|
'enum'])
|
2015-05-04 17:05:21 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-05-04 17:05:21 +02:00
|
|
|
def check_command(expr, expr_info):
|
|
|
|
name = expr['command']
|
2015-05-04 17:05:24 +02:00
|
|
|
|
2015-05-04 17:05:21 +02:00
|
|
|
check_type(expr_info, "'data' for command '%s'" % name,
|
2015-05-04 17:05:22 +02:00
|
|
|
expr.get('data'), allow_dict=True, allow_optional=True,
|
2015-09-16 13:06:27 +02:00
|
|
|
allow_metas=['struct'])
|
2015-05-04 17:05:23 +02:00
|
|
|
returns_meta = ['union', 'struct']
|
|
|
|
if name in returns_whitelist:
|
|
|
|
returns_meta += ['built-in', 'alternate', 'enum']
|
2015-05-04 17:05:21 +02:00
|
|
|
check_type(expr_info, "'returns' for command '%s'" % name,
|
qapi: Command returning anonymous type doesn't work, outlaw
Reproducer: with
{ 'command': 'user_def_cmd4', 'returns': { 'a': 'int' } }
added to qapi-schema-test.json, qapi-commands.py dies when it tries to
generate the command handler function
Traceback (most recent call last):
File "/work/armbru/qemu/scripts/qapi-commands.py", line 359, in <module>
ret = generate_command_decl(cmd['command'], arglist, ret_type) + "\n"
File "/work/armbru/qemu/scripts/qapi-commands.py", line 29, in generate_command_decl
ret_type=c_type(ret_type), name=c_name(name),
File "/work/armbru/qemu/scripts/qapi.py", line 927, in c_type
assert isinstance(value, str) and value != ""
AssertionError
because the return type doesn't exist.
Simply outlaw this usage, and drop or dumb down test cases accordingly.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-07-31 17:59:38 +02:00
|
|
|
expr.get('returns'), allow_array=True,
|
2015-09-16 13:06:27 +02:00
|
|
|
allow_optional=True, allow_metas=returns_meta)
|
2015-05-04 17:05:21 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2014-06-18 08:43:28 +02:00
|
|
|
def check_event(expr, expr_info):
|
2015-05-04 17:05:17 +02:00
|
|
|
global events
|
|
|
|
name = expr['event']
|
|
|
|
|
|
|
|
events.append(name)
|
2015-05-04 17:05:21 +02:00
|
|
|
check_type(expr_info, "'data' for event '%s'" % name,
|
2015-05-04 17:05:22 +02:00
|
|
|
expr.get('data'), allow_dict=True, allow_optional=True,
|
2015-07-01 10:12:24 +02:00
|
|
|
allow_metas=['struct'])
|
2014-06-18 08:43:28 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2014-03-05 03:44:34 +01:00
|
|
|
def check_union(expr, expr_info):
|
|
|
|
name = expr['union']
|
|
|
|
base = expr.get('base')
|
|
|
|
discriminator = expr.get('discriminator')
|
|
|
|
members = expr['data']
|
|
|
|
|
2015-05-04 17:05:10 +02:00
|
|
|
# Two types of unions, determined by discriminator.
|
|
|
|
|
|
|
|
# With no discriminator it is a simple union.
|
|
|
|
if discriminator is None:
|
2014-03-05 03:44:34 +01:00
|
|
|
enum_define = None
|
2015-09-30 00:21:02 +02:00
|
|
|
allow_metas = ['built-in', 'union', 'alternate', 'struct', 'enum']
|
2015-05-04 17:05:08 +02:00
|
|
|
if base is not None:
|
|
|
|
raise QAPIExprError(expr_info,
|
2015-05-04 17:05:10 +02:00
|
|
|
"Simple union '%s' must not have a base"
|
2015-05-04 17:05:08 +02:00
|
|
|
% name)
|
2014-03-05 03:44:34 +01:00
|
|
|
|
|
|
|
# Else, it's a flat union.
|
|
|
|
else:
|
2015-05-04 17:05:08 +02:00
|
|
|
# The object must have a string member 'base'.
|
2015-09-30 00:21:07 +02:00
|
|
|
check_type(expr_info, "'base' for union '%s'" % name,
|
|
|
|
base, allow_metas=['struct'])
|
|
|
|
if not base:
|
2014-03-05 03:44:34 +01:00
|
|
|
raise QAPIExprError(expr_info,
|
2015-09-30 00:21:07 +02:00
|
|
|
"Flat union '%s' must have a base"
|
2014-03-05 03:44:34 +01:00
|
|
|
% name)
|
2015-05-04 17:05:08 +02:00
|
|
|
base_fields = find_base_fields(base)
|
2015-09-30 00:21:07 +02:00
|
|
|
assert base_fields
|
2015-05-04 17:05:08 +02:00
|
|
|
|
2015-05-04 17:05:22 +02:00
|
|
|
# The value of member 'discriminator' must name a non-optional
|
2015-05-04 17:05:25 +02:00
|
|
|
# member of the base struct.
|
2015-05-04 17:05:22 +02:00
|
|
|
check_name(expr_info, "Discriminator of flat union '%s'" % name,
|
|
|
|
discriminator)
|
2014-03-05 03:44:34 +01:00
|
|
|
discriminator_type = base_fields.get(discriminator)
|
|
|
|
if not discriminator_type:
|
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"Discriminator '%s' is not a member of base "
|
2015-05-04 17:05:25 +02:00
|
|
|
"struct '%s'"
|
2014-03-05 03:44:34 +01:00
|
|
|
% (discriminator, base))
|
|
|
|
enum_define = find_enum(discriminator_type)
|
2015-09-30 00:21:02 +02:00
|
|
|
allow_metas = ['struct']
|
2014-03-05 03:44:39 +01:00
|
|
|
# Do not allow string discriminator
|
|
|
|
if not enum_define:
|
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"Discriminator '%s' must be of enumeration "
|
|
|
|
"type" % discriminator)
|
2014-03-05 03:44:34 +01:00
|
|
|
|
2016-02-18 07:48:16 +01:00
|
|
|
# Check every branch; don't allow an empty union
|
|
|
|
if len(members) == 0:
|
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"Union '%s' cannot have empty 'data'" % name)
|
2014-03-05 03:44:34 +01:00
|
|
|
for (key, value) in members.items():
|
2015-05-04 17:05:22 +02:00
|
|
|
check_name(expr_info, "Member of union '%s'" % name, key)
|
|
|
|
|
2015-12-02 06:20:58 +01:00
|
|
|
# Each value must name a known type
|
2015-05-04 17:05:21 +02:00
|
|
|
check_type(expr_info, "Member '%s' of union '%s'" % (key, name),
|
2015-06-10 13:07:43 +02:00
|
|
|
value, allow_array=not base, allow_metas=allow_metas)
|
2015-05-04 17:05:21 +02:00
|
|
|
|
2015-05-04 17:05:08 +02:00
|
|
|
# If the discriminator names an enum type, then all members
|
2015-11-18 09:52:49 +01:00
|
|
|
# of 'data' must also be members of the enum type.
|
2015-05-04 17:05:08 +02:00
|
|
|
if enum_define:
|
2015-09-30 00:21:02 +02:00
|
|
|
if key not in enum_define['enum_values']:
|
2015-05-04 17:05:08 +02:00
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"Discriminator value '%s' is not found in "
|
|
|
|
"enum '%s'" %
|
|
|
|
(key, enum_define["enum_name"]))
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-05-04 17:05:10 +02:00
|
|
|
def check_alternate(expr, expr_info):
|
2015-05-04 17:05:13 +02:00
|
|
|
name = expr['alternate']
|
2015-05-04 17:05:10 +02:00
|
|
|
members = expr['data']
|
|
|
|
types_seen = {}
|
|
|
|
|
2016-02-18 07:48:16 +01:00
|
|
|
# Check every branch; require at least two branches
|
|
|
|
if len(members) < 2:
|
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"Alternate '%s' should have at least two branches "
|
|
|
|
"in 'data'" % name)
|
2015-05-04 17:05:10 +02:00
|
|
|
for (key, value) in members.items():
|
2015-05-04 17:05:22 +02:00
|
|
|
check_name(expr_info, "Member of alternate '%s'" % name, key)
|
|
|
|
|
2015-05-04 17:05:10 +02:00
|
|
|
# Ensure alternates have no type conflicts.
|
2015-05-04 17:05:21 +02:00
|
|
|
check_type(expr_info, "Member '%s' of alternate '%s'" % (key, name),
|
|
|
|
value,
|
|
|
|
allow_metas=['built-in', 'union', 'struct', 'enum'])
|
2015-05-04 17:05:10 +02:00
|
|
|
qtype = find_alternate_member_qtype(value)
|
2016-02-18 07:48:17 +01:00
|
|
|
if not qtype:
|
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"Alternate '%s' member '%s' cannot use "
|
|
|
|
"type '%s'" % (name, key, value))
|
2015-05-04 17:05:10 +02:00
|
|
|
if qtype in types_seen:
|
|
|
|
raise QAPIExprError(expr_info,
|
2015-05-04 17:05:13 +02:00
|
|
|
"Alternate '%s' member '%s' can't "
|
2015-05-04 17:05:10 +02:00
|
|
|
"be distinguished from member '%s'"
|
|
|
|
% (name, key, types_seen[qtype]))
|
|
|
|
types_seen[qtype] = key
|
2014-03-05 03:44:34 +01:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-05-04 17:05:04 +02:00
|
|
|
def check_enum(expr, expr_info):
|
|
|
|
name = expr['enum']
|
|
|
|
members = expr.get('data')
|
2015-08-26 15:21:20 +02:00
|
|
|
prefix = expr.get('prefix')
|
2015-05-04 17:05:04 +02:00
|
|
|
|
|
|
|
if not isinstance(members, list):
|
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"Enum '%s' requires an array for 'data'" % name)
|
2015-08-26 15:21:20 +02:00
|
|
|
if prefix is not None and not isinstance(prefix, str):
|
|
|
|
raise QAPIExprError(expr_info,
|
|
|
|
"Enum '%s' requires a string for 'prefix'" % name)
|
2015-05-04 17:05:04 +02:00
|
|
|
for member in members:
|
2015-09-30 00:21:02 +02:00
|
|
|
check_name(expr_info, "Member of enum '%s'" % name, member,
|
2015-05-04 17:05:22 +02:00
|
|
|
enum_member=True)
|
2015-05-04 17:05:04 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-05-04 17:05:21 +02:00
|
|
|
def check_struct(expr, expr_info):
|
2015-05-04 17:05:25 +02:00
|
|
|
name = expr['struct']
|
2015-05-04 17:05:21 +02:00
|
|
|
members = expr['data']
|
|
|
|
|
2015-05-04 17:05:25 +02:00
|
|
|
check_type(expr_info, "'data' for struct '%s'" % name, members,
|
2015-05-04 17:05:22 +02:00
|
|
|
allow_dict=True, allow_optional=True)
|
2015-05-04 17:05:25 +02:00
|
|
|
check_type(expr_info, "'base' for struct '%s'" % name, expr.get('base'),
|
2015-05-04 17:05:21 +02:00
|
|
|
allow_metas=['struct'])
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-05-04 17:05:15 +02:00
|
|
|
def check_keys(expr_elem, meta, required, optional=[]):
|
|
|
|
expr = expr_elem['expr']
|
|
|
|
info = expr_elem['info']
|
|
|
|
name = expr[meta]
|
|
|
|
if not isinstance(name, str):
|
|
|
|
raise QAPIExprError(info,
|
|
|
|
"'%s' key must have a string value" % meta)
|
2015-09-30 00:21:02 +02:00
|
|
|
required = required + [meta]
|
2015-05-04 17:05:15 +02:00
|
|
|
for (key, value) in expr.items():
|
2015-09-30 00:21:02 +02:00
|
|
|
if key not in required and key not in optional:
|
2015-05-04 17:05:15 +02:00
|
|
|
raise QAPIExprError(info,
|
|
|
|
"Unknown key '%s' in %s '%s'"
|
|
|
|
% (key, meta, name))
|
2015-09-30 00:21:02 +02:00
|
|
|
if (key == 'gen' or key == 'success-response') and value is not False:
|
2015-05-04 17:05:24 +02:00
|
|
|
raise QAPIExprError(info,
|
|
|
|
"'%s' of %s '%s' should only use false value"
|
|
|
|
% (key, meta, name))
|
2015-05-04 17:05:15 +02:00
|
|
|
for key in required:
|
2015-09-30 00:21:02 +02:00
|
|
|
if key not in expr:
|
2015-05-04 17:05:15 +02:00
|
|
|
raise QAPIExprError(info,
|
|
|
|
"Key '%s' is missing from %s '%s'"
|
|
|
|
% (key, meta, name))
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-06-10 08:55:21 +02:00
|
|
|
def check_exprs(exprs):
|
2015-05-04 17:05:17 +02:00
|
|
|
global all_names
|
|
|
|
|
2015-06-10 08:55:21 +02:00
|
|
|
# Learn the types and check for valid expression keys
|
|
|
|
for builtin in builtin_types.keys():
|
|
|
|
all_names[builtin] = 'built-in'
|
|
|
|
for expr_elem in exprs:
|
|
|
|
expr = expr_elem['expr']
|
|
|
|
info = expr_elem['info']
|
2015-09-30 00:21:02 +02:00
|
|
|
if 'enum' in expr:
|
2015-08-26 15:21:20 +02:00
|
|
|
check_keys(expr_elem, 'enum', ['data'], ['prefix'])
|
2015-06-10 08:55:21 +02:00
|
|
|
add_enum(expr['enum'], info, expr['data'])
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'union' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_keys(expr_elem, 'union', ['data'],
|
|
|
|
['base', 'discriminator'])
|
|
|
|
add_union(expr, info)
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'alternate' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_keys(expr_elem, 'alternate', ['data'])
|
|
|
|
add_name(expr['alternate'], info, 'alternate')
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'struct' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_keys(expr_elem, 'struct', ['data'], ['base'])
|
|
|
|
add_struct(expr, info)
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'command' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_keys(expr_elem, 'command', [],
|
|
|
|
['data', 'returns', 'gen', 'success-response'])
|
|
|
|
add_name(expr['command'], info, 'command')
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'event' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_keys(expr_elem, 'event', [], ['data'])
|
|
|
|
add_name(expr['event'], info, 'event')
|
|
|
|
else:
|
|
|
|
raise QAPIExprError(expr_elem['info'],
|
|
|
|
"Expression is missing metatype")
|
2013-07-27 17:41:56 +02:00
|
|
|
|
2015-06-10 08:55:21 +02:00
|
|
|
# Try again for hidden UnionKind enum
|
|
|
|
for expr_elem in exprs:
|
|
|
|
expr = expr_elem['expr']
|
2015-09-30 00:21:02 +02:00
|
|
|
if 'union' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
if not discriminator_find_enum_define(expr):
|
|
|
|
add_enum('%sKind' % expr['union'], expr_elem['info'],
|
2015-05-04 17:05:17 +02:00
|
|
|
implicit=True)
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'alternate' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
add_enum('%sKind' % expr['alternate'], expr_elem['info'],
|
|
|
|
implicit=True)
|
|
|
|
|
|
|
|
# Validate that exprs make sense
|
|
|
|
for expr_elem in exprs:
|
|
|
|
expr = expr_elem['expr']
|
|
|
|
info = expr_elem['info']
|
2015-05-04 17:05:09 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
if 'enum' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_enum(expr, info)
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'union' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_union(expr, info)
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'alternate' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_alternate(expr, info)
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'struct' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_struct(expr, info)
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'command' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_command(expr, info)
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'event' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_event(expr, info)
|
|
|
|
else:
|
|
|
|
assert False, 'unexpected meta type'
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
return exprs
|
|
|
|
|
|
|
|
|
|
|
|
#
|
|
|
|
# Schema compiler frontend
|
|
|
|
#
|
|
|
|
|
|
|
|
class QAPISchemaEntity(object):
|
|
|
|
def __init__(self, name, info):
|
|
|
|
assert isinstance(name, str)
|
|
|
|
self.name = name
|
2015-10-13 06:22:32 +02:00
|
|
|
# For explicitly defined entities, info points to the (explicit)
|
|
|
|
# definition. For builtins (and their arrays), info is None.
|
|
|
|
# For implicitly defined entities, info points to a place that
|
|
|
|
# triggered the implicit definition (there may be more than one
|
|
|
|
# such place).
|
2015-09-16 13:06:05 +02:00
|
|
|
self.info = info
|
|
|
|
|
2015-09-16 13:06:06 +02:00
|
|
|
def c_name(self):
|
|
|
|
return c_name(self.name)
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
def check(self, schema):
|
|
|
|
pass
|
|
|
|
|
2015-10-13 06:22:27 +02:00
|
|
|
def is_implicit(self):
|
|
|
|
return not self.info
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
|
|
|
pass
|
|
|
|
|
|
|
|
|
|
|
|
class QAPISchemaVisitor(object):
|
|
|
|
def visit_begin(self, schema):
|
|
|
|
pass
|
|
|
|
|
|
|
|
def visit_end(self):
|
|
|
|
pass
|
|
|
|
|
2015-10-13 06:22:21 +02:00
|
|
|
def visit_needed(self, entity):
|
|
|
|
# Default to visiting everything
|
|
|
|
return True
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit_builtin_type(self, name, info, json_type):
|
|
|
|
pass
|
|
|
|
|
|
|
|
def visit_enum_type(self, name, info, values, prefix):
|
|
|
|
pass
|
|
|
|
|
|
|
|
def visit_array_type(self, name, info, element_type):
|
|
|
|
pass
|
|
|
|
|
|
|
|
def visit_object_type(self, name, info, base, members, variants):
|
|
|
|
pass
|
|
|
|
|
qapi: New QMP command query-qmp-schema for QMP introspection
qapi/introspect.json defines the introspection schema. It's designed
for QMP introspection, but should do for similar uses, such as QGA.
The introspection schema does not reflect all the rules and
restrictions that apply to QAPI schemata. A valid QAPI schema has an
introspection value conforming to the introspection schema, but the
converse is not true.
Introspection lowers away a number of schema details, and makes
implicit things explicit:
* The built-in types are declared with their JSON type.
All integer types are mapped to 'int', because how many bits we use
internally is an implementation detail. It could be pressed into
external interface service as very approximate range information,
but that's a bad idea. If we need range information, we better do
it properly.
* Implicit type definitions are made explicit, and given
auto-generated names:
- Array types, named by appending "List" to the name of their
element type, like in generated C.
- The enumeration types implicitly defined by simple union types,
named by appending "Kind" to the name of their simple union type,
like in generated C.
- Types that don't occur in generated C. Their names start with ':'
so they don't clash with the user's names.
* All type references are by name.
* The struct and union types are generalized into an object type.
* Base types are flattened.
* Commands take a single argument and return a single result.
Dictionary argument or list result is an implicit type definition.
The empty object type is used when a command takes no arguments or
produces no results.
The argument is always of object type, but the introspection schema
doesn't reflect that.
The 'gen': false directive is omitted as implementation detail.
The 'success-response' directive is omitted as well for now, even
though it's not an implementation detail, because it's not used by
QMP.
* Events carry a single data value.
Implicit type definition and empty object type use, just like for
commands.
The value is of object type, but the introspection schema doesn't
reflect that.
* Types not used by commands or events are omitted.
Indirect use counts as use.
* Optional members have a default, which can only be null right now
Instead of a mandatory "optional" flag, we have an optional default.
No default means mandatory, default null means optional without
default value. Non-null is available for optional with default
(possible future extension).
* Clients should *not* look up types by name, because type names are
not ABI. Look up the command or event you're interested in, then
follow the references.
TODO Should we hide the type names to eliminate the temptation?
New generator scripts/qapi-introspect.py computes an introspection
value for its input, and generates a C variable holding it.
It can generate awfully long lines. Marked TODO.
A new test-qmp-input-visitor test case feeds its result for both
tests/qapi-schema/qapi-schema-test.json and qapi-schema.json to a
QmpInputVisitor to verify it actually conforms to the schema.
New QMP command query-qmp-schema takes its return value from that
variable. Its reply is some 85KiBytes for me right now.
If this turns out to be too much, we have a couple of options:
* We can use shorter names in the JSON. Not the QMP style.
* Optionally return the sub-schema for commands and events given as
arguments.
Right now qmp_query_schema() sends the string literal computed by
qmp-introspect.py. To compute sub-schema at run time, we'd have to
duplicate parts of qapi-introspect.py in C. Unattractive.
* Let clients cache the output of query-qmp-schema.
It changes only on QEMU upgrades, i.e. rarely. Provide a command
query-qmp-schema-hash. Clients can have a cache indexed by hash,
and re-query the schema only when they don't have it cached. Even
simpler: put the hash in the QMP greeting.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-16 13:06:28 +02:00
|
|
|
def visit_object_type_flat(self, name, info, members, variants):
|
|
|
|
pass
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit_alternate_type(self, name, info, variants):
|
|
|
|
pass
|
|
|
|
|
|
|
|
def visit_command(self, name, info, arg_type, ret_type,
|
|
|
|
gen, success_response):
|
|
|
|
pass
|
|
|
|
|
|
|
|
def visit_event(self, name, info, arg_type):
|
|
|
|
pass
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchemaType(QAPISchemaEntity):
|
qapi: Don't box struct branch of alternate
There's no reason to do two malloc's for an alternate type visiting
a QAPI struct; let's just inline the struct directly as the C union
branch of the struct.
Surprisingly, no clients were actually using the struct member prior
to this patch outside of the testsuite; an earlier patch in the series
added some testsuite coverage to make the effect of this patch more
obvious.
In qapi.py, c_type() gains a new is_unboxed flag to control when we
are emitting a C struct unboxed within the context of an outer
struct (different from our other two modes of usage with no flags
for normal local variable declarations, and with is_param for adding
'const' in a parameter list). I don't know if there is any more
pythonic way of collapsing the two flags into a single parameter,
as we never have a caller setting both flags at once.
Ultimately, we want to also unbox branches for QAPI unions, but as
that touches a lot more client code, it is better as separate
patches. But since unions and alternates share gen_variants(), I
had to hack in a way to test if we are visiting an alternate type
for setting the is_unboxed flag: look for a non-object branch.
This works because alternates have at least two branches, with at
most one object branch, while unions have only object branches.
The hack will go away in a later patch.
The generated code difference to qapi-types.h is relatively small:
| struct BlockdevRef {
| QType type;
| union { /* union tag is @type */
| void *data;
|- BlockdevOptions *definition;
|+ BlockdevOptions definition;
| char *reference;
| } u;
| };
The corresponding spot in qapi-visit.c calls visit_type_FOO(), which
first calls visit_start_struct() to allocate or deallocate the member
and handle a layer of {} from the JSON stream, then visits the
members. To peel off the indirection and the memory management that
comes with it, we inline this call, then suppress allocation /
deallocation by passing NULL to visit_start_struct(), and adjust the
member visit:
| switch ((*obj)->type) {
| case QTYPE_QDICT:
|- visit_type_BlockdevOptions(v, name, &(*obj)->u.definition, &err);
|+ visit_start_struct(v, name, NULL, 0, &err);
|+ if (err) {
|+ break;
|+ }
|+ visit_type_BlockdevOptions_fields(v, &(*obj)->u.definition, &err);
|+ error_propagate(errp, err);
|+ err = NULL;
|+ visit_end_struct(v, &err);
| break;
| case QTYPE_QSTRING:
| visit_type_str(v, name, &(*obj)->u.reference, &err);
The visit of non-object fields is unchanged.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1455778109-6278-13-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-02-18 07:48:26 +01:00
|
|
|
def c_type(self, is_param=False, is_unboxed=False):
|
2015-09-16 13:06:06 +02:00
|
|
|
return c_name(self.name) + pointer_suffix
|
|
|
|
|
|
|
|
def c_null(self):
|
|
|
|
return 'NULL'
|
|
|
|
|
|
|
|
def json_type(self):
|
|
|
|
pass
|
|
|
|
|
|
|
|
def alternate_qtype(self):
|
|
|
|
json2qtype = {
|
|
|
|
'string': 'QTYPE_QSTRING',
|
|
|
|
'number': 'QTYPE_QFLOAT',
|
|
|
|
'int': 'QTYPE_QINT',
|
|
|
|
'boolean': 'QTYPE_QBOOL',
|
|
|
|
'object': 'QTYPE_QDICT'
|
|
|
|
}
|
|
|
|
return json2qtype.get(self.json_type())
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
|
|
|
|
class QAPISchemaBuiltinType(QAPISchemaType):
|
2015-09-16 13:06:06 +02:00
|
|
|
def __init__(self, name, json_type, c_type, c_null):
|
2015-09-16 13:06:05 +02:00
|
|
|
QAPISchemaType.__init__(self, name, None)
|
2015-09-16 13:06:06 +02:00
|
|
|
assert not c_type or isinstance(c_type, str)
|
|
|
|
assert json_type in ('string', 'number', 'int', 'boolean', 'null',
|
|
|
|
'value')
|
|
|
|
self._json_type_name = json_type
|
|
|
|
self._c_type_name = c_type
|
|
|
|
self._c_null_val = c_null
|
|
|
|
|
|
|
|
def c_name(self):
|
|
|
|
return self.name
|
|
|
|
|
qapi: Don't box struct branch of alternate
There's no reason to do two malloc's for an alternate type visiting
a QAPI struct; let's just inline the struct directly as the C union
branch of the struct.
Surprisingly, no clients were actually using the struct member prior
to this patch outside of the testsuite; an earlier patch in the series
added some testsuite coverage to make the effect of this patch more
obvious.
In qapi.py, c_type() gains a new is_unboxed flag to control when we
are emitting a C struct unboxed within the context of an outer
struct (different from our other two modes of usage with no flags
for normal local variable declarations, and with is_param for adding
'const' in a parameter list). I don't know if there is any more
pythonic way of collapsing the two flags into a single parameter,
as we never have a caller setting both flags at once.
Ultimately, we want to also unbox branches for QAPI unions, but as
that touches a lot more client code, it is better as separate
patches. But since unions and alternates share gen_variants(), I
had to hack in a way to test if we are visiting an alternate type
for setting the is_unboxed flag: look for a non-object branch.
This works because alternates have at least two branches, with at
most one object branch, while unions have only object branches.
The hack will go away in a later patch.
The generated code difference to qapi-types.h is relatively small:
| struct BlockdevRef {
| QType type;
| union { /* union tag is @type */
| void *data;
|- BlockdevOptions *definition;
|+ BlockdevOptions definition;
| char *reference;
| } u;
| };
The corresponding spot in qapi-visit.c calls visit_type_FOO(), which
first calls visit_start_struct() to allocate or deallocate the member
and handle a layer of {} from the JSON stream, then visits the
members. To peel off the indirection and the memory management that
comes with it, we inline this call, then suppress allocation /
deallocation by passing NULL to visit_start_struct(), and adjust the
member visit:
| switch ((*obj)->type) {
| case QTYPE_QDICT:
|- visit_type_BlockdevOptions(v, name, &(*obj)->u.definition, &err);
|+ visit_start_struct(v, name, NULL, 0, &err);
|+ if (err) {
|+ break;
|+ }
|+ visit_type_BlockdevOptions_fields(v, &(*obj)->u.definition, &err);
|+ error_propagate(errp, err);
|+ err = NULL;
|+ visit_end_struct(v, &err);
| break;
| case QTYPE_QSTRING:
| visit_type_str(v, name, &(*obj)->u.reference, &err);
The visit of non-object fields is unchanged.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1455778109-6278-13-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-02-18 07:48:26 +01:00
|
|
|
def c_type(self, is_param=False, is_unboxed=False):
|
2015-09-16 13:06:06 +02:00
|
|
|
if is_param and self.name == 'str':
|
|
|
|
return 'const ' + self._c_type_name
|
|
|
|
return self._c_type_name
|
|
|
|
|
|
|
|
def c_null(self):
|
|
|
|
return self._c_null_val
|
|
|
|
|
|
|
|
def json_type(self):
|
|
|
|
return self._json_type_name
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
|
|
|
visitor.visit_builtin_type(self.name, self.info, self.json_type())
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchemaEnumType(QAPISchemaType):
|
|
|
|
def __init__(self, name, info, values, prefix):
|
|
|
|
QAPISchemaType.__init__(self, name, info)
|
|
|
|
for v in values:
|
2015-12-02 06:20:55 +01:00
|
|
|
assert isinstance(v, QAPISchemaMember)
|
|
|
|
v.set_owner(name)
|
2015-09-16 13:06:05 +02:00
|
|
|
assert prefix is None or isinstance(prefix, str)
|
|
|
|
self.values = values
|
|
|
|
self.prefix = prefix
|
|
|
|
|
|
|
|
def check(self, schema):
|
2015-12-02 06:20:55 +01:00
|
|
|
seen = {}
|
|
|
|
for v in self.values:
|
|
|
|
v.check_clash(self.info, seen)
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def is_implicit(self):
|
|
|
|
# See QAPISchema._make_implicit_enum_type()
|
2015-10-26 23:34:41 +01:00
|
|
|
return self.name.endswith('Kind')
|
2015-10-13 06:22:32 +02:00
|
|
|
|
qapi: Don't box struct branch of alternate
There's no reason to do two malloc's for an alternate type visiting
a QAPI struct; let's just inline the struct directly as the C union
branch of the struct.
Surprisingly, no clients were actually using the struct member prior
to this patch outside of the testsuite; an earlier patch in the series
added some testsuite coverage to make the effect of this patch more
obvious.
In qapi.py, c_type() gains a new is_unboxed flag to control when we
are emitting a C struct unboxed within the context of an outer
struct (different from our other two modes of usage with no flags
for normal local variable declarations, and with is_param for adding
'const' in a parameter list). I don't know if there is any more
pythonic way of collapsing the two flags into a single parameter,
as we never have a caller setting both flags at once.
Ultimately, we want to also unbox branches for QAPI unions, but as
that touches a lot more client code, it is better as separate
patches. But since unions and alternates share gen_variants(), I
had to hack in a way to test if we are visiting an alternate type
for setting the is_unboxed flag: look for a non-object branch.
This works because alternates have at least two branches, with at
most one object branch, while unions have only object branches.
The hack will go away in a later patch.
The generated code difference to qapi-types.h is relatively small:
| struct BlockdevRef {
| QType type;
| union { /* union tag is @type */
| void *data;
|- BlockdevOptions *definition;
|+ BlockdevOptions definition;
| char *reference;
| } u;
| };
The corresponding spot in qapi-visit.c calls visit_type_FOO(), which
first calls visit_start_struct() to allocate or deallocate the member
and handle a layer of {} from the JSON stream, then visits the
members. To peel off the indirection and the memory management that
comes with it, we inline this call, then suppress allocation /
deallocation by passing NULL to visit_start_struct(), and adjust the
member visit:
| switch ((*obj)->type) {
| case QTYPE_QDICT:
|- visit_type_BlockdevOptions(v, name, &(*obj)->u.definition, &err);
|+ visit_start_struct(v, name, NULL, 0, &err);
|+ if (err) {
|+ break;
|+ }
|+ visit_type_BlockdevOptions_fields(v, &(*obj)->u.definition, &err);
|+ error_propagate(errp, err);
|+ err = NULL;
|+ visit_end_struct(v, &err);
| break;
| case QTYPE_QSTRING:
| visit_type_str(v, name, &(*obj)->u.reference, &err);
The visit of non-object fields is unchanged.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1455778109-6278-13-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-02-18 07:48:26 +01:00
|
|
|
def c_type(self, is_param=False, is_unboxed=False):
|
2015-09-16 13:06:06 +02:00
|
|
|
return c_name(self.name)
|
|
|
|
|
2015-12-02 06:20:55 +01:00
|
|
|
def member_names(self):
|
|
|
|
return [v.name for v in self.values]
|
|
|
|
|
2015-09-16 13:06:06 +02:00
|
|
|
def c_null(self):
|
2015-12-02 06:20:55 +01:00
|
|
|
return c_enum_const(self.name, (self.member_names() + ['_MAX'])[0],
|
2015-09-16 13:06:06 +02:00
|
|
|
self.prefix)
|
|
|
|
|
|
|
|
def json_type(self):
|
|
|
|
return 'string'
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
|
|
|
visitor.visit_enum_type(self.name, self.info,
|
2015-12-02 06:20:55 +01:00
|
|
|
self.member_names(), self.prefix)
|
2015-09-16 13:06:07 +02:00
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchemaArrayType(QAPISchemaType):
|
|
|
|
def __init__(self, name, info, element_type):
|
|
|
|
QAPISchemaType.__init__(self, name, info)
|
|
|
|
assert isinstance(element_type, str)
|
|
|
|
self._element_type_name = element_type
|
|
|
|
self.element_type = None
|
|
|
|
|
|
|
|
def check(self, schema):
|
|
|
|
self.element_type = schema.lookup_type(self._element_type_name)
|
|
|
|
assert self.element_type
|
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def is_implicit(self):
|
|
|
|
return True
|
|
|
|
|
2015-09-16 13:06:06 +02:00
|
|
|
def json_type(self):
|
|
|
|
return 'array'
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
|
|
|
visitor.visit_array_type(self.name, self.info, self.element_type)
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchemaObjectType(QAPISchemaType):
|
|
|
|
def __init__(self, name, info, base, local_members, variants):
|
qapi: Track simple union tag in object.local_members
We were previously creating all unions with an empty list for
local_members. However, it will make it easier to unify struct
and union generation if we include the generated tag member in
local_members. That way, we can have a common code pattern:
visit the base (if any), visit the local members (if any), visit
the variants (if any). The local_members of a flat union
remains empty (because the discriminator is already visited as
part of the base). Then, by visiting tag_member.check() during
AlternateType.check(), we no longer need to call it during
Variants.check().
The various front end entities now exist as follows:
struct: optional base, optional local_members, no variants
simple union: no base, one-element local_members, variants with tag_member
from local_members
flat union: base, no local_members, variants with tag_member from base
alternate: no base, no local_members, variants
With the new local members, we require a bit of finesse to
avoid assertions in the clients.
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-2-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:36 +01:00
|
|
|
# struct has local_members, optional base, and no variants
|
|
|
|
# flat union has base, variants, and no local_members
|
|
|
|
# simple union has local_members, variants, and no base
|
2015-09-16 13:06:05 +02:00
|
|
|
QAPISchemaType.__init__(self, name, info)
|
|
|
|
assert base is None or isinstance(base, str)
|
|
|
|
for m in local_members:
|
|
|
|
assert isinstance(m, QAPISchemaObjectTypeMember)
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
m.set_owner(name)
|
|
|
|
if variants is not None:
|
|
|
|
assert isinstance(variants, QAPISchemaObjectTypeVariants)
|
|
|
|
variants.set_owner(name)
|
2015-09-16 13:06:05 +02:00
|
|
|
self._base_name = base
|
|
|
|
self.base = None
|
|
|
|
self.local_members = local_members
|
|
|
|
self.variants = variants
|
|
|
|
self.members = None
|
|
|
|
|
|
|
|
def check(self, schema):
|
qapi: Detect base class loops
It should be fairly obvious that qapi base classes need to
form an acyclic graph, since QMP cannot specify the same
key more than once, while base classes are included as flat
members alongside other members added by the child. But the
old check_member_clash() parser function was not prepared to
check for this, and entered an infinite recursion (at least
until Python gives up, complaining about nesting too deep).
Now that check_member_clash() has been recently removed,
attempts at self-inheritance trigger an assertion failure
introduced by commit ac88219a. The obvious fix is to turn
the assertion into a conditional.
This patch includes both the tests (base-cycle-direct and
base-cycle-indirect) and the fix, since the .err file output
for the unfixed case is not useful (particularly when it was
warning about unbounded recursion, as that limit may be
platform-specific).
We don't need to worry about cycles in flat unions (neither
the base type nor the type of a variant can be a union) nor
in alternates (alternate branches cannot themselves be an
alternate). But if we later allow a union type as a variant,
we will still be okay, as QAPISchemaObjectTypeVariants.check()
triggers the same QAPISchemaObjectType.check() that will
detect any loops.
Likewise, we need not worry about the case of diamond
inheritance where the same class is used for a flat union base
class and one of its variants; either both uses will introduce
a collision in trying to insert the same member name twice, or
the shared type is empty and changes nothing.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-16-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:59 +01:00
|
|
|
if self.members is False: # check for cycles
|
|
|
|
raise QAPIExprError(self.info,
|
|
|
|
"Object %s contains itself" % self.name)
|
2015-09-16 13:06:05 +02:00
|
|
|
if self.members:
|
|
|
|
return
|
|
|
|
self.members = False # mark as being checked
|
2015-11-18 09:52:43 +01:00
|
|
|
seen = OrderedDict()
|
2015-09-16 13:06:05 +02:00
|
|
|
if self._base_name:
|
|
|
|
self.base = schema.lookup_type(self._base_name)
|
|
|
|
assert isinstance(self.base, QAPISchemaObjectType)
|
|
|
|
self.base.check(schema)
|
2015-11-18 09:52:51 +01:00
|
|
|
self.base.check_clash(schema, self.info, seen)
|
2015-09-16 13:06:05 +02:00
|
|
|
for m in self.local_members:
|
2015-11-18 09:52:40 +01:00
|
|
|
m.check(schema)
|
2015-11-18 09:52:51 +01:00
|
|
|
m.check_clash(self.info, seen)
|
2015-11-18 09:52:45 +01:00
|
|
|
self.members = seen.values()
|
2015-09-16 13:06:05 +02:00
|
|
|
if self.variants:
|
2015-11-18 09:52:41 +01:00
|
|
|
self.variants.check(schema, seen)
|
2015-11-18 09:52:45 +01:00
|
|
|
assert self.variants.tag_member in self.members
|
2015-11-18 09:52:51 +01:00
|
|
|
self.variants.check_clash(schema, self.info, seen)
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-11-18 09:52:51 +01:00
|
|
|
# Check that the members of this type do not cause duplicate JSON fields,
|
|
|
|
# and update seen to track the members seen so far. Report any errors
|
|
|
|
# on behalf of info, which is not necessarily self.info
|
|
|
|
def check_clash(self, schema, info, seen):
|
2015-11-18 09:52:47 +01:00
|
|
|
assert not self.variants # not implemented
|
|
|
|
for m in self.members:
|
2015-11-18 09:52:51 +01:00
|
|
|
m.check_clash(info, seen)
|
2015-11-18 09:52:47 +01:00
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def is_implicit(self):
|
|
|
|
# See QAPISchema._make_implicit_object_type()
|
|
|
|
return self.name[0] == ':'
|
|
|
|
|
2015-09-16 13:06:06 +02:00
|
|
|
def c_name(self):
|
2015-10-13 06:22:27 +02:00
|
|
|
assert not self.is_implicit()
|
2015-09-16 13:06:06 +02:00
|
|
|
return QAPISchemaType.c_name(self)
|
|
|
|
|
qapi: Don't box struct branch of alternate
There's no reason to do two malloc's for an alternate type visiting
a QAPI struct; let's just inline the struct directly as the C union
branch of the struct.
Surprisingly, no clients were actually using the struct member prior
to this patch outside of the testsuite; an earlier patch in the series
added some testsuite coverage to make the effect of this patch more
obvious.
In qapi.py, c_type() gains a new is_unboxed flag to control when we
are emitting a C struct unboxed within the context of an outer
struct (different from our other two modes of usage with no flags
for normal local variable declarations, and with is_param for adding
'const' in a parameter list). I don't know if there is any more
pythonic way of collapsing the two flags into a single parameter,
as we never have a caller setting both flags at once.
Ultimately, we want to also unbox branches for QAPI unions, but as
that touches a lot more client code, it is better as separate
patches. But since unions and alternates share gen_variants(), I
had to hack in a way to test if we are visiting an alternate type
for setting the is_unboxed flag: look for a non-object branch.
This works because alternates have at least two branches, with at
most one object branch, while unions have only object branches.
The hack will go away in a later patch.
The generated code difference to qapi-types.h is relatively small:
| struct BlockdevRef {
| QType type;
| union { /* union tag is @type */
| void *data;
|- BlockdevOptions *definition;
|+ BlockdevOptions definition;
| char *reference;
| } u;
| };
The corresponding spot in qapi-visit.c calls visit_type_FOO(), which
first calls visit_start_struct() to allocate or deallocate the member
and handle a layer of {} from the JSON stream, then visits the
members. To peel off the indirection and the memory management that
comes with it, we inline this call, then suppress allocation /
deallocation by passing NULL to visit_start_struct(), and adjust the
member visit:
| switch ((*obj)->type) {
| case QTYPE_QDICT:
|- visit_type_BlockdevOptions(v, name, &(*obj)->u.definition, &err);
|+ visit_start_struct(v, name, NULL, 0, &err);
|+ if (err) {
|+ break;
|+ }
|+ visit_type_BlockdevOptions_fields(v, &(*obj)->u.definition, &err);
|+ error_propagate(errp, err);
|+ err = NULL;
|+ visit_end_struct(v, &err);
| break;
| case QTYPE_QSTRING:
| visit_type_str(v, name, &(*obj)->u.reference, &err);
The visit of non-object fields is unchanged.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1455778109-6278-13-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-02-18 07:48:26 +01:00
|
|
|
def c_type(self, is_param=False, is_unboxed=False):
|
2015-10-13 06:22:27 +02:00
|
|
|
assert not self.is_implicit()
|
qapi: Don't box struct branch of alternate
There's no reason to do two malloc's for an alternate type visiting
a QAPI struct; let's just inline the struct directly as the C union
branch of the struct.
Surprisingly, no clients were actually using the struct member prior
to this patch outside of the testsuite; an earlier patch in the series
added some testsuite coverage to make the effect of this patch more
obvious.
In qapi.py, c_type() gains a new is_unboxed flag to control when we
are emitting a C struct unboxed within the context of an outer
struct (different from our other two modes of usage with no flags
for normal local variable declarations, and with is_param for adding
'const' in a parameter list). I don't know if there is any more
pythonic way of collapsing the two flags into a single parameter,
as we never have a caller setting both flags at once.
Ultimately, we want to also unbox branches for QAPI unions, but as
that touches a lot more client code, it is better as separate
patches. But since unions and alternates share gen_variants(), I
had to hack in a way to test if we are visiting an alternate type
for setting the is_unboxed flag: look for a non-object branch.
This works because alternates have at least two branches, with at
most one object branch, while unions have only object branches.
The hack will go away in a later patch.
The generated code difference to qapi-types.h is relatively small:
| struct BlockdevRef {
| QType type;
| union { /* union tag is @type */
| void *data;
|- BlockdevOptions *definition;
|+ BlockdevOptions definition;
| char *reference;
| } u;
| };
The corresponding spot in qapi-visit.c calls visit_type_FOO(), which
first calls visit_start_struct() to allocate or deallocate the member
and handle a layer of {} from the JSON stream, then visits the
members. To peel off the indirection and the memory management that
comes with it, we inline this call, then suppress allocation /
deallocation by passing NULL to visit_start_struct(), and adjust the
member visit:
| switch ((*obj)->type) {
| case QTYPE_QDICT:
|- visit_type_BlockdevOptions(v, name, &(*obj)->u.definition, &err);
|+ visit_start_struct(v, name, NULL, 0, &err);
|+ if (err) {
|+ break;
|+ }
|+ visit_type_BlockdevOptions_fields(v, &(*obj)->u.definition, &err);
|+ error_propagate(errp, err);
|+ err = NULL;
|+ visit_end_struct(v, &err);
| break;
| case QTYPE_QSTRING:
| visit_type_str(v, name, &(*obj)->u.reference, &err);
The visit of non-object fields is unchanged.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1455778109-6278-13-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-02-18 07:48:26 +01:00
|
|
|
if is_unboxed:
|
|
|
|
return c_name(self.name)
|
|
|
|
return c_name(self.name) + pointer_suffix
|
2015-09-16 13:06:06 +02:00
|
|
|
|
|
|
|
def json_type(self):
|
|
|
|
return 'object'
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
|
|
|
visitor.visit_object_type(self.name, self.info,
|
|
|
|
self.base, self.local_members, self.variants)
|
qapi: New QMP command query-qmp-schema for QMP introspection
qapi/introspect.json defines the introspection schema. It's designed
for QMP introspection, but should do for similar uses, such as QGA.
The introspection schema does not reflect all the rules and
restrictions that apply to QAPI schemata. A valid QAPI schema has an
introspection value conforming to the introspection schema, but the
converse is not true.
Introspection lowers away a number of schema details, and makes
implicit things explicit:
* The built-in types are declared with their JSON type.
All integer types are mapped to 'int', because how many bits we use
internally is an implementation detail. It could be pressed into
external interface service as very approximate range information,
but that's a bad idea. If we need range information, we better do
it properly.
* Implicit type definitions are made explicit, and given
auto-generated names:
- Array types, named by appending "List" to the name of their
element type, like in generated C.
- The enumeration types implicitly defined by simple union types,
named by appending "Kind" to the name of their simple union type,
like in generated C.
- Types that don't occur in generated C. Their names start with ':'
so they don't clash with the user's names.
* All type references are by name.
* The struct and union types are generalized into an object type.
* Base types are flattened.
* Commands take a single argument and return a single result.
Dictionary argument or list result is an implicit type definition.
The empty object type is used when a command takes no arguments or
produces no results.
The argument is always of object type, but the introspection schema
doesn't reflect that.
The 'gen': false directive is omitted as implementation detail.
The 'success-response' directive is omitted as well for now, even
though it's not an implementation detail, because it's not used by
QMP.
* Events carry a single data value.
Implicit type definition and empty object type use, just like for
commands.
The value is of object type, but the introspection schema doesn't
reflect that.
* Types not used by commands or events are omitted.
Indirect use counts as use.
* Optional members have a default, which can only be null right now
Instead of a mandatory "optional" flag, we have an optional default.
No default means mandatory, default null means optional without
default value. Non-null is available for optional with default
(possible future extension).
* Clients should *not* look up types by name, because type names are
not ABI. Look up the command or event you're interested in, then
follow the references.
TODO Should we hide the type names to eliminate the temptation?
New generator scripts/qapi-introspect.py computes an introspection
value for its input, and generates a C variable holding it.
It can generate awfully long lines. Marked TODO.
A new test-qmp-input-visitor test case feeds its result for both
tests/qapi-schema/qapi-schema-test.json and qapi-schema.json to a
QmpInputVisitor to verify it actually conforms to the schema.
New QMP command query-qmp-schema takes its return value from that
variable. Its reply is some 85KiBytes for me right now.
If this turns out to be too much, we have a couple of options:
* We can use shorter names in the JSON. Not the QMP style.
* Optionally return the sub-schema for commands and events given as
arguments.
Right now qmp_query_schema() sends the string literal computed by
qmp-introspect.py. To compute sub-schema at run time, we'd have to
duplicate parts of qapi-introspect.py in C. Unattractive.
* Let clients cache the output of query-qmp-schema.
It changes only on QEMU upgrades, i.e. rarely. Provide a command
query-qmp-schema-hash. Clients can have a cache indexed by hash,
and re-query the schema only when they don't have it cached. Even
simpler: put the hash in the QMP greeting.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-16 13:06:28 +02:00
|
|
|
visitor.visit_object_type_flat(self.name, self.info,
|
|
|
|
self.members, self.variants)
|
2015-09-16 13:06:07 +02:00
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-12-02 06:20:54 +01:00
|
|
|
class QAPISchemaMember(object):
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
role = 'member'
|
|
|
|
|
2015-12-02 06:20:54 +01:00
|
|
|
def __init__(self, name):
|
2015-09-16 13:06:05 +02:00
|
|
|
assert isinstance(name, str)
|
|
|
|
self.name = name
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
self.owner = None
|
|
|
|
|
|
|
|
def set_owner(self, name):
|
|
|
|
assert not self.owner
|
|
|
|
self.owner = name
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-11-18 09:52:51 +01:00
|
|
|
def check_clash(self, info, seen):
|
|
|
|
cname = c_name(self.name)
|
qapi: Enforce (or whitelist) case conventions on qapi members
We document that members of enums and objects should be
'lower-case', although we were not enforcing it. We have to
whitelist a few pre-existing entities that violate the norms.
Add three new tests to expose the new error message, each of
which first uses the whitelisted name 'UuidInfo' to prove the
whitelist works, then triggers the failure (this is the same
pattern used in the existing returns-whitelist.json test).
Note that by adding this check, we have effectively forbidden
an entity with a case-insensitive clash of member names, for
any entity that is not on the whitelist (although there is
still the possibility to clash via '-' vs. '_').
Not done here: a future patch should also add naming convention
support and whitelist exceptions for command, event, and type
names.
The additions to QAPISchemaMember.check_clash() check whether
info['name'] is in the whitelist (the top-most entity name at
the point 'info' tracks), rather than self.owner (the type,
possibly implicit, that directly owns the member), because it
is easier to maintain the whitelist by the names actually in
the user's .json file, rather than worrying about the names
of implicit types.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-14-git-send-email-eblake@redhat.com>
[Simplified a bit as per discussion with Eric]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:57 +01:00
|
|
|
if cname.lower() != cname and self.owner not in case_whitelist:
|
|
|
|
raise QAPIExprError(info,
|
|
|
|
"%s should not use uppercase" % self.describe())
|
2015-11-18 09:52:51 +01:00
|
|
|
if cname in seen:
|
|
|
|
raise QAPIExprError(info,
|
|
|
|
"%s collides with %s"
|
|
|
|
% (self.describe(), seen[cname].describe()))
|
|
|
|
seen[cname] = self
|
2015-11-18 09:52:44 +01:00
|
|
|
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
def _pretty_owner(self):
|
|
|
|
owner = self.owner
|
|
|
|
if owner.startswith(':obj-'):
|
|
|
|
# See QAPISchema._make_implicit_object_type() - reverse the
|
|
|
|
# mapping there to create a nice human-readable description
|
|
|
|
owner = owner[5:]
|
|
|
|
if owner.endswith('-arg'):
|
|
|
|
return '(parameter of %s)' % owner[:-4]
|
|
|
|
else:
|
|
|
|
assert owner.endswith('-wrapper')
|
|
|
|
# Unreachable and not implemented
|
|
|
|
assert False
|
2015-12-02 06:20:55 +01:00
|
|
|
if owner.endswith('Kind'):
|
|
|
|
# See QAPISchema._make_implicit_enum_type()
|
|
|
|
return '(branch of %s)' % owner[:-4]
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
return '(%s of %s)' % (self.role, owner)
|
|
|
|
|
|
|
|
def describe(self):
|
|
|
|
return "'%s' %s" % (self.name, self._pretty_owner())
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-12-02 06:20:54 +01:00
|
|
|
class QAPISchemaObjectTypeMember(QAPISchemaMember):
|
|
|
|
def __init__(self, name, typ, optional):
|
|
|
|
QAPISchemaMember.__init__(self, name)
|
|
|
|
assert isinstance(typ, str)
|
|
|
|
assert isinstance(optional, bool)
|
|
|
|
self._type_name = typ
|
|
|
|
self.type = None
|
|
|
|
self.optional = optional
|
|
|
|
|
|
|
|
def check(self, schema):
|
|
|
|
assert self.owner
|
|
|
|
self.type = schema.lookup_type(self._type_name)
|
|
|
|
assert self.type
|
|
|
|
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
class QAPISchemaObjectTypeVariants(object):
|
2015-10-13 06:22:29 +02:00
|
|
|
def __init__(self, tag_name, tag_member, variants):
|
|
|
|
# Flat unions pass tag_name but not tag_member.
|
|
|
|
# Simple unions and alternates pass tag_member but not tag_name.
|
|
|
|
# After check(), tag_member is always set, and tag_name remains
|
|
|
|
# a reliable witness of being used by a flat union.
|
|
|
|
assert bool(tag_member) != bool(tag_name)
|
|
|
|
assert (isinstance(tag_name, str) or
|
|
|
|
isinstance(tag_member, QAPISchemaObjectTypeMember))
|
2016-02-18 07:48:16 +01:00
|
|
|
assert len(variants) > 0
|
2015-09-16 13:06:05 +02:00
|
|
|
for v in variants:
|
|
|
|
assert isinstance(v, QAPISchemaObjectTypeVariant)
|
|
|
|
self.tag_name = tag_name
|
2015-10-13 06:22:29 +02:00
|
|
|
self.tag_member = tag_member
|
2015-09-16 13:06:05 +02:00
|
|
|
self.variants = variants
|
|
|
|
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
def set_owner(self, name):
|
|
|
|
for v in self.variants:
|
|
|
|
v.set_owner(name)
|
|
|
|
|
2015-11-18 09:52:41 +01:00
|
|
|
def check(self, schema, seen):
|
2015-11-18 09:52:45 +01:00
|
|
|
if not self.tag_member: # flat union
|
2015-11-18 09:52:51 +01:00
|
|
|
self.tag_member = seen[c_name(self.tag_name)]
|
|
|
|
assert self.tag_name == self.tag_member.name
|
2015-09-16 13:06:05 +02:00
|
|
|
assert isinstance(self.tag_member.type, QAPISchemaEnumType)
|
|
|
|
for v in self.variants:
|
2015-11-18 09:52:48 +01:00
|
|
|
v.check(schema)
|
qapi: Simplify visiting of alternate types
Previously, working with alternates required two lookup arrays
and some indirection: for type Foo, we created Foo_qtypes[]
which maps each qtype to a value of the generated FooKind enum,
then look up that value in FooKind_lookup[] like we do for other
union types.
This has a couple of subtle bugs. First, the generator was
creating a call with a parameter '(int *) &(*obj)->type' where
type is an enum type; this is unsafe if the compiler chooses
to store the enum type in a different size than int, where
assigning through the wrong size pointer can corrupt data or
cause a SIGBUS.
Related bug, not not fixed in this patch: qapi-visit.py's
gen_visit_enum() generates a cast of its enum * argument to
int *. Marked FIXME.
Second, since the values of the FooKind enum start at zero, all
entries of the Foo_qtypes[] array that were not explicitly
initialized will map to the same branch of the union as the
first member of the alternate, rather than triggering a desired
failure in visit_get_next_type(). Fortunately, the bug seldom
bites; the very next thing the input visitor does is try to
parse the incoming JSON with the wrong parser, which normally
fails; the output visitor is not used with a C struct in that
state, and the dealloc visitor has nothing to clean up (so
there is no leak).
However, the second bug IS observable in one case: parsing an
integer causes unusual behavior in an alternate that contains
at least a 'number' member but no 'int' member, because the
'number' parser accepts QTYPE_QINT in addition to the expected
QTYPE_QFLOAT (that is, since 'int' is not a member, the type
QTYPE_QINT accidentally maps to FooKind 0; if this enum value
is the 'number' branch the integer parses successfully, but if
the 'number' branch is not first, some other branch tries to
parse the integer and rejects it). A later patch will worry
about fixing alternates to always parse all inputs that a
non-alternate 'number' would accept, for now this is still
marked FIXME in the updated test-qmp-input-visitor.c, to
merely point out that new undesired behavior of 'ans' matches
the existing undesired behavior of 'asn'.
This patch fixes the default-initialization bug by deleting the
indirection, and modifying get_next_type() to directly assign a
QTypeCode parameter. This in turn fixes the type-casting bug,
as we are no longer casting a pointer to enum to a questionable
size. There is no longer a need to generate an implicit FooKind
enum associated with the alternate type (since the QMP wire
format never uses the stringized counterparts of the C union
member names). Since the updated visit_get_next_type() does not
know which qtypes are expected, the generated visitor is
modified to generate an error statement if an unexpected type is
encountered.
Callers now have to know the QTYPE_* mapping when looking at the
discriminator; but so far, only the testsuite was even using the
C struct of an alternate types. I considered the possibility of
keeping the internal enum FooKind, but initialized differently
than most generated arrays, as in:
typedef enum FooKind {
FOO_KIND_A = QTYPE_QDICT,
FOO_KIND_B = QTYPE_QINT,
} FooKind;
to create nicer aliases for knowing when to use foo->a or foo->b
when inspecting foo->type; but it turned out to add too much
complexity, especially without a client.
There is a user-visible side effect to this change, but I
consider it to be an improvement. Previously,
the invalid QMP command:
{"execute":"blockdev-add", "arguments":{"options":
{"driver":"raw", "id":"a", "file":true}}}
failed with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: QDict"}}
(visit_get_next_type() succeeded, and the error comes from the
visit_type_BlockdevOptions() expecting {}; there is no mention of
the fact that a string would also work). Now it fails with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
(the error when the next type doesn't match any expected types for
the overall alternate).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-5-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:48 +01:00
|
|
|
# Union names must match enum values; alternate names are
|
|
|
|
# checked separately. Use 'seen' to tell the two apart.
|
|
|
|
if seen:
|
2015-12-02 06:20:55 +01:00
|
|
|
assert v.name in self.tag_member.type.member_names()
|
qapi: Simplify visiting of alternate types
Previously, working with alternates required two lookup arrays
and some indirection: for type Foo, we created Foo_qtypes[]
which maps each qtype to a value of the generated FooKind enum,
then look up that value in FooKind_lookup[] like we do for other
union types.
This has a couple of subtle bugs. First, the generator was
creating a call with a parameter '(int *) &(*obj)->type' where
type is an enum type; this is unsafe if the compiler chooses
to store the enum type in a different size than int, where
assigning through the wrong size pointer can corrupt data or
cause a SIGBUS.
Related bug, not not fixed in this patch: qapi-visit.py's
gen_visit_enum() generates a cast of its enum * argument to
int *. Marked FIXME.
Second, since the values of the FooKind enum start at zero, all
entries of the Foo_qtypes[] array that were not explicitly
initialized will map to the same branch of the union as the
first member of the alternate, rather than triggering a desired
failure in visit_get_next_type(). Fortunately, the bug seldom
bites; the very next thing the input visitor does is try to
parse the incoming JSON with the wrong parser, which normally
fails; the output visitor is not used with a C struct in that
state, and the dealloc visitor has nothing to clean up (so
there is no leak).
However, the second bug IS observable in one case: parsing an
integer causes unusual behavior in an alternate that contains
at least a 'number' member but no 'int' member, because the
'number' parser accepts QTYPE_QINT in addition to the expected
QTYPE_QFLOAT (that is, since 'int' is not a member, the type
QTYPE_QINT accidentally maps to FooKind 0; if this enum value
is the 'number' branch the integer parses successfully, but if
the 'number' branch is not first, some other branch tries to
parse the integer and rejects it). A later patch will worry
about fixing alternates to always parse all inputs that a
non-alternate 'number' would accept, for now this is still
marked FIXME in the updated test-qmp-input-visitor.c, to
merely point out that new undesired behavior of 'ans' matches
the existing undesired behavior of 'asn'.
This patch fixes the default-initialization bug by deleting the
indirection, and modifying get_next_type() to directly assign a
QTypeCode parameter. This in turn fixes the type-casting bug,
as we are no longer casting a pointer to enum to a questionable
size. There is no longer a need to generate an implicit FooKind
enum associated with the alternate type (since the QMP wire
format never uses the stringized counterparts of the C union
member names). Since the updated visit_get_next_type() does not
know which qtypes are expected, the generated visitor is
modified to generate an error statement if an unexpected type is
encountered.
Callers now have to know the QTYPE_* mapping when looking at the
discriminator; but so far, only the testsuite was even using the
C struct of an alternate types. I considered the possibility of
keeping the internal enum FooKind, but initialized differently
than most generated arrays, as in:
typedef enum FooKind {
FOO_KIND_A = QTYPE_QDICT,
FOO_KIND_B = QTYPE_QINT,
} FooKind;
to create nicer aliases for knowing when to use foo->a or foo->b
when inspecting foo->type; but it turned out to add too much
complexity, especially without a client.
There is a user-visible side effect to this change, but I
consider it to be an improvement. Previously,
the invalid QMP command:
{"execute":"blockdev-add", "arguments":{"options":
{"driver":"raw", "id":"a", "file":true}}}
failed with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: QDict"}}
(visit_get_next_type() succeeded, and the error comes from the
visit_type_BlockdevOptions() expecting {}; there is no mention of
the fact that a string would also work). Now it fails with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
(the error when the next type doesn't match any expected types for
the overall alternate).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-5-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:48 +01:00
|
|
|
assert isinstance(v.type, QAPISchemaObjectType)
|
qapi: Check for QAPI collisions involving variant members
Right now, our ad hoc parser ensures that we cannot have a
flat union that introduces any members that would clash with
non-variant members inherited from the union's base type (see
flat-union-clash-member.json). We want QAPISchemaObjectType.check()
to make the same check, so we can later reduce some of the ad
hoc checks.
We already have a map 'seen' of all non-variant members. We
still need to check for collisions between each variant type's
members and the non-variant ones.
To know the variant type's members, we need to call
variant.type.check(). This also detects when a type contains
itself in a variant, exactly like the existing base.check()
detects when a type contains itself as a base. (Except that
we currently forbid anything but a struct as the type of a
variant, so we can't actually trigger this type of loop yet.)
Slight complication: an alternate's variant can have arbitrary
type, but only an object type's check() may be called outside
QAPISchema.check(). We could either skip the call for variants
of alternates, or skip it for non-object types. For now, do
the latter, because it's easier.
Then we call each variant member's check_clash() with the
appropriate 'seen' map. Since members of different variants
can't clash, we have to clone a fresh seen for each variant.
Wrap this in a new helper method
QAPISchemaObjectTypeVariants.check_clash().
Note that cloning 'seen' inside .check_clash() resembles
the one we just removed from .check() in 'qapi: Drop
obsolete tag value collision assertions'; the difference here is
that we are now checking for clashes among the qapi members of
the variant type, rather than for a single clash with the variant
tag name itself.
Note that, by construction, collisions can't actually happen for
simple unions: each variant's type is a wrapper with a single
member 'data', which will never collide with the only non-variant
member 'type'.
For alternates, there's nothing for a variant object type's
members to clash with, and therefore no need to call the new
variants.check_clash().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-12-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:46 +01:00
|
|
|
v.type.check(schema)
|
|
|
|
|
2015-11-18 09:52:51 +01:00
|
|
|
def check_clash(self, schema, info, seen):
|
qapi: Check for QAPI collisions involving variant members
Right now, our ad hoc parser ensures that we cannot have a
flat union that introduces any members that would clash with
non-variant members inherited from the union's base type (see
flat-union-clash-member.json). We want QAPISchemaObjectType.check()
to make the same check, so we can later reduce some of the ad
hoc checks.
We already have a map 'seen' of all non-variant members. We
still need to check for collisions between each variant type's
members and the non-variant ones.
To know the variant type's members, we need to call
variant.type.check(). This also detects when a type contains
itself in a variant, exactly like the existing base.check()
detects when a type contains itself as a base. (Except that
we currently forbid anything but a struct as the type of a
variant, so we can't actually trigger this type of loop yet.)
Slight complication: an alternate's variant can have arbitrary
type, but only an object type's check() may be called outside
QAPISchema.check(). We could either skip the call for variants
of alternates, or skip it for non-object types. For now, do
the latter, because it's easier.
Then we call each variant member's check_clash() with the
appropriate 'seen' map. Since members of different variants
can't clash, we have to clone a fresh seen for each variant.
Wrap this in a new helper method
QAPISchemaObjectTypeVariants.check_clash().
Note that cloning 'seen' inside .check_clash() resembles
the one we just removed from .check() in 'qapi: Drop
obsolete tag value collision assertions'; the difference here is
that we are now checking for clashes among the qapi members of
the variant type, rather than for a single clash with the variant
tag name itself.
Note that, by construction, collisions can't actually happen for
simple unions: each variant's type is a wrapper with a single
member 'data', which will never collide with the only non-variant
member 'type'.
For alternates, there's nothing for a variant object type's
members to clash with, and therefore no need to call the new
variants.check_clash().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-12-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:46 +01:00
|
|
|
for v in self.variants:
|
|
|
|
# Reset seen map for each variant, since qapi names from one
|
|
|
|
# branch do not affect another branch
|
|
|
|
assert isinstance(v.type, QAPISchemaObjectType)
|
2015-11-18 09:52:51 +01:00
|
|
|
v.type.check_clash(schema, info, dict(seen))
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
role = 'branch'
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
def __init__(self, name, typ):
|
|
|
|
QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
|
|
|
|
|
qapi-types: Convert to QAPISchemaVisitor, fixing flat unions
Fixes flat unions to get the base's base members. Test case is from
commit 2fc0043, in qapi-schema-test.json:
{ 'union': 'UserDefFlatUnion',
'base': 'UserDefUnionBase',
'discriminator': 'enum1',
'data': { 'value1' : 'UserDefA',
'value2' : 'UserDefB',
'value3' : 'UserDefB' } }
{ 'struct': 'UserDefUnionBase',
'base': 'UserDefZero',
'data': { 'string': 'str', 'enum1': 'EnumOne' } }
{ 'struct': 'UserDefZero',
'data': { 'integer': 'int' } }
Patch's effect on UserDefFlatUnion:
struct UserDefFlatUnion {
/* Members inherited from UserDefUnionBase: */
+ int64_t integer;
char *string;
EnumOne enum1;
/* Own members: */
union { /* union tag is @enum1 */
void *data;
UserDefA *value1;
UserDefB *value2;
UserDefB *value3;
};
};
Flat union visitors remain broken. They'll be fixed next.
Code is generated in a different order now, but that doesn't matter.
The two guards QAPI_TYPES_BUILTIN_STRUCT_DECL and
QAPI_TYPES_BUILTIN_CLEANUP_DECL are replaced by just
QAPI_TYPES_BUILTIN.
Two ugly special cases for simple unions now stand out like sore
thumbs:
1. The type tag is named 'type' everywhere, except in generated C,
where it's 'kind'.
2. QAPISchema lowers simple unions to semantically equivalent flat
unions. However, the C generated for a simple unions differs from
the C generated for its equivalent flat union, and we therefore
need special code to preserve that pointless difference for now.
Mark both TODO.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-16 13:06:09 +02:00
|
|
|
# This function exists to support ugly simple union special cases
|
|
|
|
# TODO get rid of them, and drop the function
|
|
|
|
def simple_union_type(self):
|
2015-10-13 06:22:27 +02:00
|
|
|
if (self.type.is_implicit() and
|
|
|
|
isinstance(self.type, QAPISchemaObjectType)):
|
qapi-types: Convert to QAPISchemaVisitor, fixing flat unions
Fixes flat unions to get the base's base members. Test case is from
commit 2fc0043, in qapi-schema-test.json:
{ 'union': 'UserDefFlatUnion',
'base': 'UserDefUnionBase',
'discriminator': 'enum1',
'data': { 'value1' : 'UserDefA',
'value2' : 'UserDefB',
'value3' : 'UserDefB' } }
{ 'struct': 'UserDefUnionBase',
'base': 'UserDefZero',
'data': { 'string': 'str', 'enum1': 'EnumOne' } }
{ 'struct': 'UserDefZero',
'data': { 'integer': 'int' } }
Patch's effect on UserDefFlatUnion:
struct UserDefFlatUnion {
/* Members inherited from UserDefUnionBase: */
+ int64_t integer;
char *string;
EnumOne enum1;
/* Own members: */
union { /* union tag is @enum1 */
void *data;
UserDefA *value1;
UserDefB *value2;
UserDefB *value3;
};
};
Flat union visitors remain broken. They'll be fixed next.
Code is generated in a different order now, but that doesn't matter.
The two guards QAPI_TYPES_BUILTIN_STRUCT_DECL and
QAPI_TYPES_BUILTIN_CLEANUP_DECL are replaced by just
QAPI_TYPES_BUILTIN.
Two ugly special cases for simple unions now stand out like sore
thumbs:
1. The type tag is named 'type' everywhere, except in generated C,
where it's 'kind'.
2. QAPISchema lowers simple unions to semantically equivalent flat
unions. However, the C generated for a simple unions differs from
the C generated for its equivalent flat union, and we therefore
need special code to preserve that pointless difference for now.
Mark both TODO.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-16 13:06:09 +02:00
|
|
|
assert len(self.type.members) == 1
|
|
|
|
assert not self.type.variants
|
|
|
|
return self.type.members[0].type
|
|
|
|
return None
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchemaAlternateType(QAPISchemaType):
|
|
|
|
def __init__(self, name, info, variants):
|
|
|
|
QAPISchemaType.__init__(self, name, info)
|
|
|
|
assert isinstance(variants, QAPISchemaObjectTypeVariants)
|
|
|
|
assert not variants.tag_name
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
variants.set_owner(name)
|
|
|
|
variants.tag_member.set_owner(self.name)
|
2015-09-16 13:06:05 +02:00
|
|
|
self.variants = variants
|
|
|
|
|
|
|
|
def check(self, schema):
|
2015-11-18 09:52:40 +01:00
|
|
|
self.variants.tag_member.check(schema)
|
qapi: Check for QAPI collisions involving variant members
Right now, our ad hoc parser ensures that we cannot have a
flat union that introduces any members that would clash with
non-variant members inherited from the union's base type (see
flat-union-clash-member.json). We want QAPISchemaObjectType.check()
to make the same check, so we can later reduce some of the ad
hoc checks.
We already have a map 'seen' of all non-variant members. We
still need to check for collisions between each variant type's
members and the non-variant ones.
To know the variant type's members, we need to call
variant.type.check(). This also detects when a type contains
itself in a variant, exactly like the existing base.check()
detects when a type contains itself as a base. (Except that
we currently forbid anything but a struct as the type of a
variant, so we can't actually trigger this type of loop yet.)
Slight complication: an alternate's variant can have arbitrary
type, but only an object type's check() may be called outside
QAPISchema.check(). We could either skip the call for variants
of alternates, or skip it for non-object types. For now, do
the latter, because it's easier.
Then we call each variant member's check_clash() with the
appropriate 'seen' map. Since members of different variants
can't clash, we have to clone a fresh seen for each variant.
Wrap this in a new helper method
QAPISchemaObjectTypeVariants.check_clash().
Note that cloning 'seen' inside .check_clash() resembles
the one we just removed from .check() in 'qapi: Drop
obsolete tag value collision assertions'; the difference here is
that we are now checking for clashes among the qapi members of
the variant type, rather than for a single clash with the variant
tag name itself.
Note that, by construction, collisions can't actually happen for
simple unions: each variant's type is a wrapper with a single
member 'data', which will never collide with the only non-variant
member 'type'.
For alternates, there's nothing for a variant object type's
members to clash with, and therefore no need to call the new
variants.check_clash().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-12-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:46 +01:00
|
|
|
# Not calling self.variants.check_clash(), because there's nothing
|
|
|
|
# to clash with
|
2015-11-18 09:52:41 +01:00
|
|
|
self.variants.check(schema, {})
|
qapi: Simplify visiting of alternate types
Previously, working with alternates required two lookup arrays
and some indirection: for type Foo, we created Foo_qtypes[]
which maps each qtype to a value of the generated FooKind enum,
then look up that value in FooKind_lookup[] like we do for other
union types.
This has a couple of subtle bugs. First, the generator was
creating a call with a parameter '(int *) &(*obj)->type' where
type is an enum type; this is unsafe if the compiler chooses
to store the enum type in a different size than int, where
assigning through the wrong size pointer can corrupt data or
cause a SIGBUS.
Related bug, not not fixed in this patch: qapi-visit.py's
gen_visit_enum() generates a cast of its enum * argument to
int *. Marked FIXME.
Second, since the values of the FooKind enum start at zero, all
entries of the Foo_qtypes[] array that were not explicitly
initialized will map to the same branch of the union as the
first member of the alternate, rather than triggering a desired
failure in visit_get_next_type(). Fortunately, the bug seldom
bites; the very next thing the input visitor does is try to
parse the incoming JSON with the wrong parser, which normally
fails; the output visitor is not used with a C struct in that
state, and the dealloc visitor has nothing to clean up (so
there is no leak).
However, the second bug IS observable in one case: parsing an
integer causes unusual behavior in an alternate that contains
at least a 'number' member but no 'int' member, because the
'number' parser accepts QTYPE_QINT in addition to the expected
QTYPE_QFLOAT (that is, since 'int' is not a member, the type
QTYPE_QINT accidentally maps to FooKind 0; if this enum value
is the 'number' branch the integer parses successfully, but if
the 'number' branch is not first, some other branch tries to
parse the integer and rejects it). A later patch will worry
about fixing alternates to always parse all inputs that a
non-alternate 'number' would accept, for now this is still
marked FIXME in the updated test-qmp-input-visitor.c, to
merely point out that new undesired behavior of 'ans' matches
the existing undesired behavior of 'asn'.
This patch fixes the default-initialization bug by deleting the
indirection, and modifying get_next_type() to directly assign a
QTypeCode parameter. This in turn fixes the type-casting bug,
as we are no longer casting a pointer to enum to a questionable
size. There is no longer a need to generate an implicit FooKind
enum associated with the alternate type (since the QMP wire
format never uses the stringized counterparts of the C union
member names). Since the updated visit_get_next_type() does not
know which qtypes are expected, the generated visitor is
modified to generate an error statement if an unexpected type is
encountered.
Callers now have to know the QTYPE_* mapping when looking at the
discriminator; but so far, only the testsuite was even using the
C struct of an alternate types. I considered the possibility of
keeping the internal enum FooKind, but initialized differently
than most generated arrays, as in:
typedef enum FooKind {
FOO_KIND_A = QTYPE_QDICT,
FOO_KIND_B = QTYPE_QINT,
} FooKind;
to create nicer aliases for knowing when to use foo->a or foo->b
when inspecting foo->type; but it turned out to add too much
complexity, especially without a client.
There is a user-visible side effect to this change, but I
consider it to be an improvement. Previously,
the invalid QMP command:
{"execute":"blockdev-add", "arguments":{"options":
{"driver":"raw", "id":"a", "file":true}}}
failed with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: QDict"}}
(visit_get_next_type() succeeded, and the error comes from the
visit_type_BlockdevOptions() expecting {}; there is no mention of
the fact that a string would also work). Now it fails with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
(the error when the next type doesn't match any expected types for
the overall alternate).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-5-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:48 +01:00
|
|
|
# Alternate branch names have no relation to the tag enum values;
|
|
|
|
# so we have to check for potential name collisions ourselves.
|
|
|
|
seen = {}
|
|
|
|
for v in self.variants.variants:
|
|
|
|
v.check_clash(self.info, seen)
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-09-16 13:06:06 +02:00
|
|
|
def json_type(self):
|
|
|
|
return 'value'
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
|
|
|
visitor.visit_alternate_type(self.name, self.info, self.variants)
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchemaCommand(QAPISchemaEntity):
|
|
|
|
def __init__(self, name, info, arg_type, ret_type, gen, success_response):
|
|
|
|
QAPISchemaEntity.__init__(self, name, info)
|
|
|
|
assert not arg_type or isinstance(arg_type, str)
|
|
|
|
assert not ret_type or isinstance(ret_type, str)
|
|
|
|
self._arg_type_name = arg_type
|
|
|
|
self.arg_type = None
|
|
|
|
self._ret_type_name = ret_type
|
|
|
|
self.ret_type = None
|
|
|
|
self.gen = gen
|
|
|
|
self.success_response = success_response
|
|
|
|
|
|
|
|
def check(self, schema):
|
|
|
|
if self._arg_type_name:
|
|
|
|
self.arg_type = schema.lookup_type(self._arg_type_name)
|
|
|
|
assert isinstance(self.arg_type, QAPISchemaObjectType)
|
|
|
|
assert not self.arg_type.variants # not implemented
|
|
|
|
if self._ret_type_name:
|
|
|
|
self.ret_type = schema.lookup_type(self._ret_type_name)
|
|
|
|
assert isinstance(self.ret_type, QAPISchemaType)
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
|
|
|
visitor.visit_command(self.name, self.info,
|
|
|
|
self.arg_type, self.ret_type,
|
|
|
|
self.gen, self.success_response)
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchemaEvent(QAPISchemaEntity):
|
|
|
|
def __init__(self, name, info, arg_type):
|
|
|
|
QAPISchemaEntity.__init__(self, name, info)
|
|
|
|
assert not arg_type or isinstance(arg_type, str)
|
|
|
|
self._arg_type_name = arg_type
|
|
|
|
self.arg_type = None
|
|
|
|
|
|
|
|
def check(self, schema):
|
|
|
|
if self._arg_type_name:
|
|
|
|
self.arg_type = schema.lookup_type(self._arg_type_name)
|
|
|
|
assert isinstance(self.arg_type, QAPISchemaObjectType)
|
|
|
|
assert not self.arg_type.variants # not implemented
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
|
|
|
visitor.visit_event(self.name, self.info, self.arg_type)
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchema(object):
|
|
|
|
def __init__(self, fname):
|
|
|
|
try:
|
|
|
|
self.exprs = check_exprs(QAPISchemaParser(open(fname, "r")).exprs)
|
2015-10-13 06:22:22 +02:00
|
|
|
self._entity_dict = {}
|
2015-10-13 06:22:32 +02:00
|
|
|
self._predefining = True
|
2015-10-13 06:22:22 +02:00
|
|
|
self._def_predefineds()
|
2015-10-13 06:22:32 +02:00
|
|
|
self._predefining = False
|
2015-10-13 06:22:22 +02:00
|
|
|
self._def_exprs()
|
|
|
|
self.check()
|
2015-12-18 08:52:41 +01:00
|
|
|
except (QAPISchemaError, QAPIExprError) as err:
|
2015-09-16 13:06:05 +02:00
|
|
|
print >>sys.stderr, err
|
|
|
|
exit(1)
|
|
|
|
|
|
|
|
def _def_entity(self, ent):
|
2015-10-13 06:22:32 +02:00
|
|
|
# Only the predefined types are allowed to not have info
|
|
|
|
assert ent.info or self._predefining
|
2015-09-16 13:06:05 +02:00
|
|
|
assert ent.name not in self._entity_dict
|
|
|
|
self._entity_dict[ent.name] = ent
|
|
|
|
|
|
|
|
def lookup_entity(self, name, typ=None):
|
|
|
|
ent = self._entity_dict.get(name)
|
|
|
|
if typ and not isinstance(ent, typ):
|
|
|
|
return None
|
|
|
|
return ent
|
|
|
|
|
|
|
|
def lookup_type(self, name):
|
|
|
|
return self.lookup_entity(name, QAPISchemaType)
|
|
|
|
|
2015-09-16 13:06:06 +02:00
|
|
|
def _def_builtin_type(self, name, json_type, c_type, c_null):
|
|
|
|
self._def_entity(QAPISchemaBuiltinType(name, json_type,
|
|
|
|
c_type, c_null))
|
qapi: Lazy creation of array types
Commit ac88219a had several TODO markers about whether we needed
to automatically create the corresponding array type alongside
any other type. It turns out that most of the time, we don't!
There are a few exceptions: 1) We have a few situations where we
use an array type in internal code but do not expose that type
through QMP; fix it by declaring a dummy type that forces the
generator to see that we want to use the array type.
2) The builtin arrays (such as intList for QAPI ['int']) must
always be generated, because of the way our QAPI_TYPES_BUILTIN
compile guard works: we have situations (at the very least
tests/test-qmp-output-visitor.c) that include both top-level
"qapi-types.h" (via "error.h") and a secondary
"test-qapi-types.h". If we were to only emit the builtin types
when used locally, then the first .h file would not include all
types, but the second .h does not declare anything at all because
the first .h set QAPI_TYPES_BUILTIN, and we would end up with
compilation error due to things like unknown type 'int8List'.
Actually, we may need to revisit how we do type guards, and
change from a single QAPI_TYPES_BUILTIN over to a different
usage pattern that does one #ifdef per qapi type - right now,
the only types that are declared multiple times between two qapi
.json files for inclusion by a single .c file happen to be the
builtin arrays. But now that we have QAPI 'include' statements,
it is logical to assume that we will soon reach a point where
we want to reuse non-builtin types (yes, I'm thinking about what
it will take to add introspection to QGA, where we will want to
reuse the SchemaInfo type and friends). One #ifdef per type
will help ensure that generating the same qapi type into more
than one qapi-types.h won't cause collisions when both are
included in the same .c file; but we also have to solve how to
avoid creating duplicate qapi-types.c entry points. So that
is a problem left for another day.
Generated code for qapi-types and qapi-visit is drastically
reduced; less than a third of the arrays that were blindly
created were actually needed (a quick grep shows we dropped
from 219 to 69 *List types), and the .o files lost more than
30% of their bulk. [For best results, diff the generated
files with 'git diff --patience --no-index pre post'.]
Interestingly, the introspection output is unchanged - this is
because we already cull all types that are not indirectly
reachable from a command or event, so introspection was already
using only a subset of array types. The subset of types
introspected is now a much larger percentage of the overall set
of array types emitted in qapi-types.h (since the larger set
shrunk), but still not 100% (evidence that the array types
emitted for our new Dummy structs, and the new struct itself,
don't affect QMP).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1444710158-8723-9-git-send-email-eblake@redhat.com>
[Moved array info tracking to a later patch]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-13 06:22:28 +02:00
|
|
|
# TODO As long as we have QAPI_TYPES_BUILTIN to share multiple
|
|
|
|
# qapi-types.h from a single .c, all arrays of builtins must be
|
|
|
|
# declared in the first file whether or not they are used. Nicer
|
|
|
|
# would be to use lazy instantiation, while figuring out how to
|
|
|
|
# avoid compilation issues with multiple qapi-types.h.
|
2015-10-13 06:22:32 +02:00
|
|
|
self._make_array_type(name, None)
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
def _def_predefineds(self):
|
2015-09-16 13:06:06 +02:00
|
|
|
for t in [('str', 'string', 'char' + pointer_suffix, 'NULL'),
|
|
|
|
('number', 'number', 'double', '0'),
|
|
|
|
('int', 'int', 'int64_t', '0'),
|
|
|
|
('int8', 'int', 'int8_t', '0'),
|
|
|
|
('int16', 'int', 'int16_t', '0'),
|
|
|
|
('int32', 'int', 'int32_t', '0'),
|
|
|
|
('int64', 'int', 'int64_t', '0'),
|
|
|
|
('uint8', 'int', 'uint8_t', '0'),
|
|
|
|
('uint16', 'int', 'uint16_t', '0'),
|
|
|
|
('uint32', 'int', 'uint32_t', '0'),
|
|
|
|
('uint64', 'int', 'uint64_t', '0'),
|
|
|
|
('size', 'int', 'uint64_t', '0'),
|
|
|
|
('bool', 'boolean', 'bool', 'false'),
|
2015-09-16 13:06:24 +02:00
|
|
|
('any', 'value', 'QObject' + pointer_suffix, 'NULL')]:
|
2015-09-16 13:06:06 +02:00
|
|
|
self._def_builtin_type(*t)
|
qapi: New QMP command query-qmp-schema for QMP introspection
qapi/introspect.json defines the introspection schema. It's designed
for QMP introspection, but should do for similar uses, such as QGA.
The introspection schema does not reflect all the rules and
restrictions that apply to QAPI schemata. A valid QAPI schema has an
introspection value conforming to the introspection schema, but the
converse is not true.
Introspection lowers away a number of schema details, and makes
implicit things explicit:
* The built-in types are declared with their JSON type.
All integer types are mapped to 'int', because how many bits we use
internally is an implementation detail. It could be pressed into
external interface service as very approximate range information,
but that's a bad idea. If we need range information, we better do
it properly.
* Implicit type definitions are made explicit, and given
auto-generated names:
- Array types, named by appending "List" to the name of their
element type, like in generated C.
- The enumeration types implicitly defined by simple union types,
named by appending "Kind" to the name of their simple union type,
like in generated C.
- Types that don't occur in generated C. Their names start with ':'
so they don't clash with the user's names.
* All type references are by name.
* The struct and union types are generalized into an object type.
* Base types are flattened.
* Commands take a single argument and return a single result.
Dictionary argument or list result is an implicit type definition.
The empty object type is used when a command takes no arguments or
produces no results.
The argument is always of object type, but the introspection schema
doesn't reflect that.
The 'gen': false directive is omitted as implementation detail.
The 'success-response' directive is omitted as well for now, even
though it's not an implementation detail, because it's not used by
QMP.
* Events carry a single data value.
Implicit type definition and empty object type use, just like for
commands.
The value is of object type, but the introspection schema doesn't
reflect that.
* Types not used by commands or events are omitted.
Indirect use counts as use.
* Optional members have a default, which can only be null right now
Instead of a mandatory "optional" flag, we have an optional default.
No default means mandatory, default null means optional without
default value. Non-null is available for optional with default
(possible future extension).
* Clients should *not* look up types by name, because type names are
not ABI. Look up the command or event you're interested in, then
follow the references.
TODO Should we hide the type names to eliminate the temptation?
New generator scripts/qapi-introspect.py computes an introspection
value for its input, and generates a C variable holding it.
It can generate awfully long lines. Marked TODO.
A new test-qmp-input-visitor test case feeds its result for both
tests/qapi-schema/qapi-schema-test.json and qapi-schema.json to a
QmpInputVisitor to verify it actually conforms to the schema.
New QMP command query-qmp-schema takes its return value from that
variable. Its reply is some 85KiBytes for me right now.
If this turns out to be too much, we have a couple of options:
* We can use shorter names in the JSON. Not the QMP style.
* Optionally return the sub-schema for commands and events given as
arguments.
Right now qmp_query_schema() sends the string literal computed by
qmp-introspect.py. To compute sub-schema at run time, we'd have to
duplicate parts of qapi-introspect.py in C. Unattractive.
* Let clients cache the output of query-qmp-schema.
It changes only on QEMU upgrades, i.e. rarely. Provide a command
query-qmp-schema-hash. Clients can have a cache indexed by hash,
and re-query the schema only when they don't have it cached. Even
simpler: put the hash in the QMP greeting.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-16 13:06:28 +02:00
|
|
|
self.the_empty_object_type = QAPISchemaObjectType(':empty', None, None,
|
|
|
|
[], None)
|
|
|
|
self._def_entity(self.the_empty_object_type)
|
2015-12-02 06:20:55 +01:00
|
|
|
qtype_values = self._make_enum_members(['none', 'qnull', 'qint',
|
|
|
|
'qstring', 'qdict', 'qlist',
|
|
|
|
'qfloat', 'qbool'])
|
|
|
|
self._def_entity(QAPISchemaEnumType('QType', None, qtype_values,
|
qapi: Convert QType into QAPI built-in enum type
What's more meta than using qapi to define qapi? :)
Convert QType into a full-fledged[*] builtin qapi enum type, so
that a subsequent patch can then use it as the discriminator
type of qapi alternate types. Fortunately, the judicious use of
'prefix' in the qapi definition avoids churn to the spelling of
the enum constants.
To avoid circular definitions, we have to flip the order of
inclusion between "qobject.h" vs. "qapi-types.h". Back in commit
28770e0, we had the latter include the former, so that we could
use 'QObject *' for our implementation of 'any'. But that usage
also works with only a forward declaration, whereas the
definition of QObject requires QType to be a complete type.
[*] The type has to be builtin, rather than declared in
qapi/common.json, because we want to use it for alternates even
when common.json is not included. But since it is the first
builtin enum type, we have to add special cases to qapi-types
and qapi-visit to only emit definitions once, even when two
qapi files are being compiled into the same binary (the way we
already handled builtin list types like 'intList'). We may
need to revisit how multiple qapi files share common types,
but that's a project for another day.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-4-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:47 +01:00
|
|
|
'QTYPE'))
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-12-02 06:20:55 +01:00
|
|
|
def _make_enum_members(self, values):
|
|
|
|
return [QAPISchemaMember(v) for v in values]
|
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def _make_implicit_enum_type(self, name, info, values):
|
2015-12-02 06:20:55 +01:00
|
|
|
# See also QAPISchemaObjectTypeMember._pretty_owner()
|
2015-10-13 06:22:27 +02:00
|
|
|
name = name + 'Kind' # Use namespace reserved by add_name()
|
2015-12-02 06:20:55 +01:00
|
|
|
self._def_entity(QAPISchemaEnumType(
|
|
|
|
name, info, self._make_enum_members(values), None))
|
2015-09-16 13:06:05 +02:00
|
|
|
return name
|
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def _make_array_type(self, element_type, info):
|
2015-10-26 23:34:43 +01:00
|
|
|
name = element_type + 'List' # Use namespace reserved by add_name()
|
2015-09-16 13:06:05 +02:00
|
|
|
if not self.lookup_type(name):
|
2015-10-13 06:22:32 +02:00
|
|
|
self._def_entity(QAPISchemaArrayType(name, info, element_type))
|
2015-09-16 13:06:05 +02:00
|
|
|
return name
|
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def _make_implicit_object_type(self, name, info, role, members):
|
2015-09-16 13:06:05 +02:00
|
|
|
if not members:
|
|
|
|
return None
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
# See also QAPISchemaObjectTypeMember._pretty_owner()
|
2015-09-16 13:06:05 +02:00
|
|
|
name = ':obj-%s-%s' % (name, role)
|
|
|
|
if not self.lookup_entity(name, QAPISchemaObjectType):
|
2015-10-13 06:22:32 +02:00
|
|
|
self._def_entity(QAPISchemaObjectType(name, info, None,
|
2015-09-16 13:06:05 +02:00
|
|
|
members, None))
|
|
|
|
return name
|
|
|
|
|
|
|
|
def _def_enum_type(self, expr, info):
|
|
|
|
name = expr['enum']
|
|
|
|
data = expr['data']
|
|
|
|
prefix = expr.get('prefix')
|
2015-12-02 06:20:55 +01:00
|
|
|
self._def_entity(QAPISchemaEnumType(
|
|
|
|
name, info, self._make_enum_members(data), prefix))
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def _make_member(self, name, typ, info):
|
2015-09-16 13:06:05 +02:00
|
|
|
optional = False
|
|
|
|
if name.startswith('*'):
|
|
|
|
name = name[1:]
|
|
|
|
optional = True
|
|
|
|
if isinstance(typ, list):
|
|
|
|
assert len(typ) == 1
|
2015-10-13 06:22:32 +02:00
|
|
|
typ = self._make_array_type(typ[0], info)
|
2015-09-16 13:06:05 +02:00
|
|
|
return QAPISchemaObjectTypeMember(name, typ, optional)
|
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def _make_members(self, data, info):
|
|
|
|
return [self._make_member(key, value, info)
|
2015-09-16 13:06:05 +02:00
|
|
|
for (key, value) in data.iteritems()]
|
|
|
|
|
|
|
|
def _def_struct_type(self, expr, info):
|
|
|
|
name = expr['struct']
|
|
|
|
base = expr.get('base')
|
|
|
|
data = expr['data']
|
|
|
|
self._def_entity(QAPISchemaObjectType(name, info, base,
|
2015-10-13 06:22:32 +02:00
|
|
|
self._make_members(data, info),
|
2015-09-16 13:06:05 +02:00
|
|
|
None))
|
|
|
|
|
|
|
|
def _make_variant(self, case, typ):
|
|
|
|
return QAPISchemaObjectTypeVariant(case, typ)
|
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def _make_simple_variant(self, case, typ, info):
|
2015-09-16 13:06:05 +02:00
|
|
|
if isinstance(typ, list):
|
|
|
|
assert len(typ) == 1
|
2015-10-13 06:22:32 +02:00
|
|
|
typ = self._make_array_type(typ[0], info)
|
|
|
|
typ = self._make_implicit_object_type(
|
|
|
|
typ, info, 'wrapper', [self._make_member('data', typ, info)])
|
2015-09-16 13:06:05 +02:00
|
|
|
return QAPISchemaObjectTypeVariant(case, typ)
|
|
|
|
|
|
|
|
def _def_union_type(self, expr, info):
|
|
|
|
name = expr['union']
|
|
|
|
data = expr['data']
|
|
|
|
base = expr.get('base')
|
|
|
|
tag_name = expr.get('discriminator')
|
2015-10-13 06:22:29 +02:00
|
|
|
tag_member = None
|
2015-09-16 13:06:05 +02:00
|
|
|
if tag_name:
|
|
|
|
variants = [self._make_variant(key, value)
|
|
|
|
for (key, value) in data.iteritems()]
|
qapi: Track simple union tag in object.local_members
We were previously creating all unions with an empty list for
local_members. However, it will make it easier to unify struct
and union generation if we include the generated tag member in
local_members. That way, we can have a common code pattern:
visit the base (if any), visit the local members (if any), visit
the variants (if any). The local_members of a flat union
remains empty (because the discriminator is already visited as
part of the base). Then, by visiting tag_member.check() during
AlternateType.check(), we no longer need to call it during
Variants.check().
The various front end entities now exist as follows:
struct: optional base, optional local_members, no variants
simple union: no base, one-element local_members, variants with tag_member
from local_members
flat union: base, no local_members, variants with tag_member from base
alternate: no base, no local_members, variants
With the new local members, we require a bit of finesse to
avoid assertions in the clients.
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-2-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:36 +01:00
|
|
|
members = []
|
2015-09-16 13:06:05 +02:00
|
|
|
else:
|
2015-10-13 06:22:32 +02:00
|
|
|
variants = [self._make_simple_variant(key, value, info)
|
2015-09-16 13:06:05 +02:00
|
|
|
for (key, value) in data.iteritems()]
|
2015-12-02 06:20:50 +01:00
|
|
|
typ = self._make_implicit_enum_type(name, info,
|
|
|
|
[v.name for v in variants])
|
|
|
|
tag_member = QAPISchemaObjectTypeMember('type', typ, False)
|
qapi: Track simple union tag in object.local_members
We were previously creating all unions with an empty list for
local_members. However, it will make it easier to unify struct
and union generation if we include the generated tag member in
local_members. That way, we can have a common code pattern:
visit the base (if any), visit the local members (if any), visit
the variants (if any). The local_members of a flat union
remains empty (because the discriminator is already visited as
part of the base). Then, by visiting tag_member.check() during
AlternateType.check(), we no longer need to call it during
Variants.check().
The various front end entities now exist as follows:
struct: optional base, optional local_members, no variants
simple union: no base, one-element local_members, variants with tag_member
from local_members
flat union: base, no local_members, variants with tag_member from base
alternate: no base, no local_members, variants
With the new local members, we require a bit of finesse to
avoid assertions in the clients.
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-2-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:36 +01:00
|
|
|
members = [tag_member]
|
2015-09-16 13:06:05 +02:00
|
|
|
self._def_entity(
|
qapi: Track simple union tag in object.local_members
We were previously creating all unions with an empty list for
local_members. However, it will make it easier to unify struct
and union generation if we include the generated tag member in
local_members. That way, we can have a common code pattern:
visit the base (if any), visit the local members (if any), visit
the variants (if any). The local_members of a flat union
remains empty (because the discriminator is already visited as
part of the base). Then, by visiting tag_member.check() during
AlternateType.check(), we no longer need to call it during
Variants.check().
The various front end entities now exist as follows:
struct: optional base, optional local_members, no variants
simple union: no base, one-element local_members, variants with tag_member
from local_members
flat union: base, no local_members, variants with tag_member from base
alternate: no base, no local_members, variants
With the new local members, we require a bit of finesse to
avoid assertions in the clients.
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-2-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:36 +01:00
|
|
|
QAPISchemaObjectType(name, info, base, members,
|
2015-09-16 13:06:05 +02:00
|
|
|
QAPISchemaObjectTypeVariants(tag_name,
|
2015-10-13 06:22:29 +02:00
|
|
|
tag_member,
|
2015-09-16 13:06:05 +02:00
|
|
|
variants)))
|
|
|
|
|
|
|
|
def _def_alternate_type(self, expr, info):
|
|
|
|
name = expr['alternate']
|
|
|
|
data = expr['data']
|
|
|
|
variants = [self._make_variant(key, value)
|
|
|
|
for (key, value) in data.iteritems()]
|
qapi: Simplify visiting of alternate types
Previously, working with alternates required two lookup arrays
and some indirection: for type Foo, we created Foo_qtypes[]
which maps each qtype to a value of the generated FooKind enum,
then look up that value in FooKind_lookup[] like we do for other
union types.
This has a couple of subtle bugs. First, the generator was
creating a call with a parameter '(int *) &(*obj)->type' where
type is an enum type; this is unsafe if the compiler chooses
to store the enum type in a different size than int, where
assigning through the wrong size pointer can corrupt data or
cause a SIGBUS.
Related bug, not not fixed in this patch: qapi-visit.py's
gen_visit_enum() generates a cast of its enum * argument to
int *. Marked FIXME.
Second, since the values of the FooKind enum start at zero, all
entries of the Foo_qtypes[] array that were not explicitly
initialized will map to the same branch of the union as the
first member of the alternate, rather than triggering a desired
failure in visit_get_next_type(). Fortunately, the bug seldom
bites; the very next thing the input visitor does is try to
parse the incoming JSON with the wrong parser, which normally
fails; the output visitor is not used with a C struct in that
state, and the dealloc visitor has nothing to clean up (so
there is no leak).
However, the second bug IS observable in one case: parsing an
integer causes unusual behavior in an alternate that contains
at least a 'number' member but no 'int' member, because the
'number' parser accepts QTYPE_QINT in addition to the expected
QTYPE_QFLOAT (that is, since 'int' is not a member, the type
QTYPE_QINT accidentally maps to FooKind 0; if this enum value
is the 'number' branch the integer parses successfully, but if
the 'number' branch is not first, some other branch tries to
parse the integer and rejects it). A later patch will worry
about fixing alternates to always parse all inputs that a
non-alternate 'number' would accept, for now this is still
marked FIXME in the updated test-qmp-input-visitor.c, to
merely point out that new undesired behavior of 'ans' matches
the existing undesired behavior of 'asn'.
This patch fixes the default-initialization bug by deleting the
indirection, and modifying get_next_type() to directly assign a
QTypeCode parameter. This in turn fixes the type-casting bug,
as we are no longer casting a pointer to enum to a questionable
size. There is no longer a need to generate an implicit FooKind
enum associated with the alternate type (since the QMP wire
format never uses the stringized counterparts of the C union
member names). Since the updated visit_get_next_type() does not
know which qtypes are expected, the generated visitor is
modified to generate an error statement if an unexpected type is
encountered.
Callers now have to know the QTYPE_* mapping when looking at the
discriminator; but so far, only the testsuite was even using the
C struct of an alternate types. I considered the possibility of
keeping the internal enum FooKind, but initialized differently
than most generated arrays, as in:
typedef enum FooKind {
FOO_KIND_A = QTYPE_QDICT,
FOO_KIND_B = QTYPE_QINT,
} FooKind;
to create nicer aliases for knowing when to use foo->a or foo->b
when inspecting foo->type; but it turned out to add too much
complexity, especially without a client.
There is a user-visible side effect to this change, but I
consider it to be an improvement. Previously,
the invalid QMP command:
{"execute":"blockdev-add", "arguments":{"options":
{"driver":"raw", "id":"a", "file":true}}}
failed with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: QDict"}}
(visit_get_next_type() succeeded, and the error comes from the
visit_type_BlockdevOptions() expecting {}; there is no mention of
the fact that a string would also work). Now it fails with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
(the error when the next type doesn't match any expected types for
the overall alternate).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-5-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:48 +01:00
|
|
|
tag_member = QAPISchemaObjectTypeMember('type', 'QType', False)
|
2015-09-16 13:06:05 +02:00
|
|
|
self._def_entity(
|
|
|
|
QAPISchemaAlternateType(name, info,
|
|
|
|
QAPISchemaObjectTypeVariants(None,
|
2015-10-13 06:22:29 +02:00
|
|
|
tag_member,
|
2015-09-16 13:06:05 +02:00
|
|
|
variants)))
|
|
|
|
|
|
|
|
def _def_command(self, expr, info):
|
|
|
|
name = expr['command']
|
|
|
|
data = expr.get('data')
|
|
|
|
rets = expr.get('returns')
|
|
|
|
gen = expr.get('gen', True)
|
|
|
|
success_response = expr.get('success-response', True)
|
|
|
|
if isinstance(data, OrderedDict):
|
2015-10-13 06:22:32 +02:00
|
|
|
data = self._make_implicit_object_type(
|
|
|
|
name, info, 'arg', self._make_members(data, info))
|
2015-09-16 13:06:05 +02:00
|
|
|
if isinstance(rets, list):
|
|
|
|
assert len(rets) == 1
|
2015-10-13 06:22:32 +02:00
|
|
|
rets = self._make_array_type(rets[0], info)
|
2015-09-16 13:06:05 +02:00
|
|
|
self._def_entity(QAPISchemaCommand(name, info, data, rets, gen,
|
|
|
|
success_response))
|
|
|
|
|
|
|
|
def _def_event(self, expr, info):
|
|
|
|
name = expr['event']
|
|
|
|
data = expr.get('data')
|
|
|
|
if isinstance(data, OrderedDict):
|
2015-10-13 06:22:32 +02:00
|
|
|
data = self._make_implicit_object_type(
|
|
|
|
name, info, 'arg', self._make_members(data, info))
|
2015-09-16 13:06:05 +02:00
|
|
|
self._def_entity(QAPISchemaEvent(name, info, data))
|
|
|
|
|
|
|
|
def _def_exprs(self):
|
|
|
|
for expr_elem in self.exprs:
|
|
|
|
expr = expr_elem['expr']
|
|
|
|
info = expr_elem['info']
|
|
|
|
if 'enum' in expr:
|
|
|
|
self._def_enum_type(expr, info)
|
|
|
|
elif 'struct' in expr:
|
|
|
|
self._def_struct_type(expr, info)
|
|
|
|
elif 'union' in expr:
|
|
|
|
self._def_union_type(expr, info)
|
|
|
|
elif 'alternate' in expr:
|
|
|
|
self._def_alternate_type(expr, info)
|
|
|
|
elif 'command' in expr:
|
|
|
|
self._def_command(expr, info)
|
|
|
|
elif 'event' in expr:
|
|
|
|
self._def_event(expr, info)
|
|
|
|
else:
|
|
|
|
assert False
|
|
|
|
|
|
|
|
def check(self):
|
|
|
|
for ent in self._entity_dict.values():
|
|
|
|
ent.check(self)
|
2015-06-10 08:55:21 +02:00
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
2015-10-13 06:22:21 +02:00
|
|
|
visitor.visit_begin(self)
|
|
|
|
for (name, entity) in sorted(self._entity_dict.items()):
|
|
|
|
if visitor.visit_needed(entity):
|
|
|
|
entity.visit(visitor)
|
2015-09-16 13:06:07 +02:00
|
|
|
visitor.visit_end()
|
|
|
|
|
2014-03-05 03:44:34 +01:00
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
#
|
|
|
|
# Code generation helpers
|
|
|
|
#
|
|
|
|
|
2011-07-19 21:50:39 +02:00
|
|
|
def camel_case(name):
|
|
|
|
new_name = ''
|
|
|
|
first = True
|
|
|
|
for ch in name:
|
|
|
|
if ch in ['_', '-']:
|
|
|
|
first = True
|
|
|
|
elif first:
|
|
|
|
new_name += ch.upper()
|
|
|
|
first = False
|
|
|
|
else:
|
|
|
|
new_name += ch.lower()
|
|
|
|
return new_name
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-05-14 14:50:53 +02:00
|
|
|
# ENUMName -> ENUM_NAME, EnumName1 -> ENUM_NAME1
|
|
|
|
# ENUM_NAME -> ENUM_NAME, ENUM_NAME1 -> ENUM_NAME1, ENUM_Name2 -> ENUM_NAME2
|
|
|
|
# ENUM24_Name -> ENUM24_NAME
|
|
|
|
def camel_to_upper(value):
|
|
|
|
c_fun_str = c_name(value, False)
|
|
|
|
if value.isupper():
|
|
|
|
return c_fun_str
|
|
|
|
|
|
|
|
new_name = ''
|
|
|
|
l = len(c_fun_str)
|
|
|
|
for i in range(l):
|
|
|
|
c = c_fun_str[i]
|
|
|
|
# When c is upper and no "_" appears before, do more checks
|
|
|
|
if c.isupper() and (i > 0) and c_fun_str[i - 1] != "_":
|
2015-09-30 00:21:02 +02:00
|
|
|
if i < l - 1 and c_fun_str[i + 1].islower():
|
|
|
|
new_name += '_'
|
|
|
|
elif c_fun_str[i - 1].isdigit():
|
2015-05-14 14:50:53 +02:00
|
|
|
new_name += '_'
|
|
|
|
new_name += c
|
|
|
|
return new_name.lstrip('_').upper()
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-08-26 15:21:20 +02:00
|
|
|
def c_enum_const(type_name, const_name, prefix=None):
|
|
|
|
if prefix is not None:
|
|
|
|
type_name = prefix
|
qapi: Change munging of CamelCase enum values
When munging enum values, the fact that we were passing the entire
prefix + value through camel_to_upper() meant that enum values
spelled with CamelCase could be turned into CAMEL_CASE. However,
this provides a potential collision (both OneTwo and One-Two would
munge into ONE_TWO) for enum types, when the same two names are
valid side-by-side as QAPI member names. By changing the generation
of enum constants to always be prefix + '_' + c_name(value,
False).upper(), and ensuring that there are no case collisions (in
the next patches), we no longer have to worry about names that
would be distinct as QAPI members but collide as variant tag names,
without having to think about what munging the heuristics in
camel_to_upper() will actually perform on an enum value.
Making the change will affect enums that did not follow coding
conventions, using 'CamelCase' rather than desired 'lower-case'.
Thankfully, there are only two culprits: InputButton and ErrorClass.
We already tweaked ErrorClass to make it an alias of QapiErrorClass,
where only the alias needs changing rather than the whole tree. So
the bulk of this change is modifying INPUT_BUTTON_WHEEL_UP to the
new INPUT_BUTTON_WHEELUP (and likewise for WHEELDOWN). That part
of this commit may later need reverting if we rename the enum
constants from 'WheelUp' to 'wheel-up' as part of moving
x-input-send-event to a stable interface; but at least we have
documentation bread crumbs in place to remind us (commit 513e7cd),
and it matches the fact that SDL constants are also spelled
SDL_BUTTON_WHEELUP.
Suggested by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-27-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:53:01 +01:00
|
|
|
return camel_to_upper(type_name) + '_' + c_name(const_name, False).upper()
|
2015-05-14 14:50:53 +02:00
|
|
|
|
2015-05-14 14:50:48 +02:00
|
|
|
c_name_trans = string.maketrans('.-', '__')
|
2015-05-14 14:50:47 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
qapi: Make c_type() consistently convert qapi names
Continuing the string of cleanups for supporting downstream names
containing '.', this patch focuses on ensuring c_type() can
handle a downstream name. This patch alone does not fix the
places where generator output should be calling this function
but was open-coding things instead, but it gets us a step closer.
In particular, the changes to c_list_type() and type_name() mean
that type_name(FOO) now handles the case when FOO contains '.',
'-', or is a ticklish identifier other than a builtin (builtins
are exempted because ['int'] must remain mapped to 'intList' and
not 'q_intList'). Meanwhile, ['unix'] now maps to 'q_unixList'
rather than 'unixList', to match the fact that 'unix' is ticklish;
however, our naming conventions state that complex types should
start with a capital, so no type name following conventions will
ever have the 'q_' prepended.
Likewise, changes to c_type() mean that c_type(FOO) properly
handles an enum or complex type FOO with '.' or '-' in the
name, or is a ticklish identifier (again, a ticklish identifier
as a type name violates conventions).
Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-14 14:50:55 +02:00
|
|
|
# Map @name to a valid C identifier.
|
|
|
|
# If @protect, avoid returning certain ticklish identifiers (like
|
|
|
|
# C keywords) by prepending "q_".
|
|
|
|
#
|
|
|
|
# Used for converting 'name' from a 'name':'type' qapi definition
|
|
|
|
# into a generated struct member, as well as converting type names
|
|
|
|
# into substrings of a generated C function name.
|
|
|
|
# '__a.b_c' -> '__a_b_c', 'x-foo' -> 'x_foo'
|
|
|
|
# protect=True: 'int' -> 'q_int'; protect=False: 'int' -> 'int'
|
2015-05-14 14:50:48 +02:00
|
|
|
def c_name(name, protect=True):
|
2012-07-30 17:46:55 +02:00
|
|
|
# ANSI X3J11/88-090, 3.1.1
|
|
|
|
c89_words = set(['auto', 'break', 'case', 'char', 'const', 'continue',
|
2015-09-30 00:21:02 +02:00
|
|
|
'default', 'do', 'double', 'else', 'enum', 'extern',
|
|
|
|
'float', 'for', 'goto', 'if', 'int', 'long', 'register',
|
|
|
|
'return', 'short', 'signed', 'sizeof', 'static',
|
|
|
|
'struct', 'switch', 'typedef', 'union', 'unsigned',
|
|
|
|
'void', 'volatile', 'while'])
|
2012-07-30 17:46:55 +02:00
|
|
|
# ISO/IEC 9899:1999, 6.4.1
|
|
|
|
c99_words = set(['inline', 'restrict', '_Bool', '_Complex', '_Imaginary'])
|
|
|
|
# ISO/IEC 9899:2011, 6.4.1
|
2015-09-30 00:21:02 +02:00
|
|
|
c11_words = set(['_Alignas', '_Alignof', '_Atomic', '_Generic',
|
|
|
|
'_Noreturn', '_Static_assert', '_Thread_local'])
|
2012-07-30 17:46:55 +02:00
|
|
|
# GCC http://gcc.gnu.org/onlinedocs/gcc-4.7.1/gcc/C-Extensions.html
|
|
|
|
# excluding _.*
|
|
|
|
gcc_words = set(['asm', 'typeof'])
|
2013-08-07 17:39:43 +02:00
|
|
|
# C++ ISO/IEC 14882:2003 2.11
|
|
|
|
cpp_words = set(['bool', 'catch', 'class', 'const_cast', 'delete',
|
|
|
|
'dynamic_cast', 'explicit', 'false', 'friend', 'mutable',
|
|
|
|
'namespace', 'new', 'operator', 'private', 'protected',
|
|
|
|
'public', 'reinterpret_cast', 'static_cast', 'template',
|
|
|
|
'this', 'throw', 'true', 'try', 'typeid', 'typename',
|
|
|
|
'using', 'virtual', 'wchar_t',
|
|
|
|
# alternative representations
|
|
|
|
'and', 'and_eq', 'bitand', 'bitor', 'compl', 'not',
|
|
|
|
'not_eq', 'or', 'or_eq', 'xor', 'xor_eq'])
|
2012-09-19 16:31:07 +02:00
|
|
|
# namespace pollution:
|
2016-02-02 15:51:41 +01:00
|
|
|
polluted_words = set(['unix', 'errno', 'mips', 'sparc'])
|
2015-11-18 09:52:52 +01:00
|
|
|
name = name.translate(c_name_trans)
|
2015-09-30 00:21:02 +02:00
|
|
|
if protect and (name in c89_words | c99_words | c11_words | gcc_words
|
|
|
|
| cpp_words | polluted_words):
|
2012-07-30 17:46:55 +02:00
|
|
|
return "q_" + name
|
2015-11-18 09:52:52 +01:00
|
|
|
return name
|
2011-07-19 21:50:39 +02:00
|
|
|
|
2014-06-10 13:25:53 +02:00
|
|
|
eatspace = '\033EATSPACE.'
|
2015-05-14 14:50:54 +02:00
|
|
|
pointer_suffix = ' *' + eatspace
|
2014-06-10 13:25:53 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2011-07-19 21:50:39 +02:00
|
|
|
def genindent(count):
|
|
|
|
ret = ""
|
2015-09-30 00:21:02 +02:00
|
|
|
for _ in range(count):
|
2011-07-19 21:50:39 +02:00
|
|
|
ret += " "
|
|
|
|
return ret
|
|
|
|
|
|
|
|
indent_level = 0
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2011-07-19 21:50:39 +02:00
|
|
|
def push_indent(indent_amount=4):
|
|
|
|
global indent_level
|
|
|
|
indent_level += indent_amount
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2011-07-19 21:50:39 +02:00
|
|
|
def pop_indent(indent_amount=4):
|
|
|
|
global indent_level
|
|
|
|
indent_level -= indent_amount
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-06-24 19:27:32 +02:00
|
|
|
# Generate @code with @kwds interpolated.
|
|
|
|
# Obey indent_level, and strip eatspace.
|
2011-07-19 21:50:39 +02:00
|
|
|
def cgen(code, **kwds):
|
2015-06-24 19:27:32 +02:00
|
|
|
raw = code % kwds
|
|
|
|
if indent_level:
|
|
|
|
indent = genindent(indent_level)
|
2015-09-07 17:45:55 +02:00
|
|
|
# re.subn() lacks flags support before Python 2.7, use re.compile()
|
|
|
|
raw = re.subn(re.compile("^.", re.MULTILINE),
|
|
|
|
indent + r'\g<0>', raw)
|
2015-06-24 19:27:32 +02:00
|
|
|
raw = raw[0]
|
|
|
|
return re.sub(re.escape(eatspace) + ' *', '', raw)
|
2011-07-19 21:50:39 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2011-07-19 21:50:39 +02:00
|
|
|
def mcgen(code, **kwds):
|
2015-06-24 19:27:32 +02:00
|
|
|
if code[0] == '\n':
|
|
|
|
code = code[1:]
|
|
|
|
return cgen(code, **kwds)
|
2011-07-19 21:50:39 +02:00
|
|
|
|
|
|
|
|
|
|
|
def guardname(filename):
|
2015-06-27 07:27:21 +02:00
|
|
|
return c_name(filename, protect=False).upper()
|
2013-05-11 00:46:00 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2013-05-11 00:46:00 +02:00
|
|
|
def guardstart(name):
|
|
|
|
return mcgen('''
|
|
|
|
|
|
|
|
#ifndef %(name)s
|
|
|
|
#define %(name)s
|
|
|
|
|
|
|
|
''',
|
|
|
|
name=guardname(name))
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2013-05-11 00:46:00 +02:00
|
|
|
def guardend(name):
|
|
|
|
return mcgen('''
|
|
|
|
|
|
|
|
#endif /* %(name)s */
|
|
|
|
|
|
|
|
''',
|
|
|
|
name=guardname(name))
|
2015-04-02 13:12:21 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-09-16 13:06:16 +02:00
|
|
|
def gen_enum_lookup(name, values, prefix=None):
|
2015-09-16 13:06:12 +02:00
|
|
|
ret = mcgen('''
|
|
|
|
|
2015-09-16 13:06:16 +02:00
|
|
|
const char *const %(c_name)s_lookup[] = {
|
2015-09-16 13:06:12 +02:00
|
|
|
''',
|
2015-09-16 13:06:16 +02:00
|
|
|
c_name=c_name(name))
|
2015-09-16 13:06:12 +02:00
|
|
|
for value in values:
|
|
|
|
index = c_enum_const(name, value, prefix)
|
|
|
|
ret += mcgen('''
|
|
|
|
[%(index)s] = "%(value)s",
|
|
|
|
''',
|
2015-09-16 13:06:16 +02:00
|
|
|
index=index, value=value)
|
2015-09-16 13:06:12 +02:00
|
|
|
|
qapi: Don't let implicit enum MAX member collide
Now that we guarantee the user doesn't have any enum values
beginning with a single underscore, we can use that for our
own purposes. Renaming ENUM_MAX to ENUM__MAX makes it obvious
that the sentinel is generated.
This patch was mostly generated by applying a temporary patch:
|diff --git a/scripts/qapi.py b/scripts/qapi.py
|index e6d014b..b862ec9 100644
|--- a/scripts/qapi.py
|+++ b/scripts/qapi.py
|@@ -1570,6 +1570,7 @@ const char *const %(c_name)s_lookup[] = {
| max_index = c_enum_const(name, 'MAX', prefix)
| ret += mcgen('''
| [%(max_index)s] = NULL,
|+// %(max_index)s
| };
| ''',
| max_index=max_index)
then running:
$ cat qapi-{types,event}.c tests/test-qapi-types.c |
sed -n 's,^// \(.*\)MAX,s|\1MAX|\1_MAX|g,p' > list
$ git grep -l _MAX | xargs sed -i -f list
The only things not generated are the changes in scripts/qapi.py.
Rejecting enum members named 'MAX' is now useless, and will be dropped
in the next patch.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-23-git-send-email-eblake@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
[Rebased to current master, commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:57 +01:00
|
|
|
max_index = c_enum_const(name, '_MAX', prefix)
|
2015-09-16 13:06:12 +02:00
|
|
|
ret += mcgen('''
|
|
|
|
[%(max_index)s] = NULL,
|
|
|
|
};
|
|
|
|
''',
|
2015-09-16 13:06:16 +02:00
|
|
|
max_index=max_index)
|
2015-09-16 13:06:12 +02:00
|
|
|
return ret
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-09-16 13:06:16 +02:00
|
|
|
def gen_enum(name, values, prefix=None):
|
|
|
|
# append automatically generated _MAX value
|
qapi: Don't let implicit enum MAX member collide
Now that we guarantee the user doesn't have any enum values
beginning with a single underscore, we can use that for our
own purposes. Renaming ENUM_MAX to ENUM__MAX makes it obvious
that the sentinel is generated.
This patch was mostly generated by applying a temporary patch:
|diff --git a/scripts/qapi.py b/scripts/qapi.py
|index e6d014b..b862ec9 100644
|--- a/scripts/qapi.py
|+++ b/scripts/qapi.py
|@@ -1570,6 +1570,7 @@ const char *const %(c_name)s_lookup[] = {
| max_index = c_enum_const(name, 'MAX', prefix)
| ret += mcgen('''
| [%(max_index)s] = NULL,
|+// %(max_index)s
| };
| ''',
| max_index=max_index)
then running:
$ cat qapi-{types,event}.c tests/test-qapi-types.c |
sed -n 's,^// \(.*\)MAX,s|\1MAX|\1_MAX|g,p' > list
$ git grep -l _MAX | xargs sed -i -f list
The only things not generated are the changes in scripts/qapi.py.
Rejecting enum members named 'MAX' is now useless, and will be dropped
in the next patch.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-23-git-send-email-eblake@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
[Rebased to current master, commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:57 +01:00
|
|
|
enum_values = values + ['_MAX']
|
2015-09-16 13:06:12 +02:00
|
|
|
|
2015-09-16 13:06:16 +02:00
|
|
|
ret = mcgen('''
|
2015-09-16 13:06:12 +02:00
|
|
|
|
2015-09-16 13:06:16 +02:00
|
|
|
typedef enum %(c_name)s {
|
2015-09-16 13:06:12 +02:00
|
|
|
''',
|
2015-09-16 13:06:16 +02:00
|
|
|
c_name=c_name(name))
|
2015-09-16 13:06:12 +02:00
|
|
|
|
|
|
|
i = 0
|
|
|
|
for value in enum_values:
|
2015-09-16 13:06:16 +02:00
|
|
|
ret += mcgen('''
|
|
|
|
%(c_enum)s = %(i)d,
|
2015-09-16 13:06:12 +02:00
|
|
|
''',
|
2015-09-16 13:06:16 +02:00
|
|
|
c_enum=c_enum_const(name, value, prefix),
|
2015-09-16 13:06:12 +02:00
|
|
|
i=i)
|
|
|
|
i += 1
|
|
|
|
|
2015-09-16 13:06:16 +02:00
|
|
|
ret += mcgen('''
|
|
|
|
} %(c_name)s;
|
2015-09-16 13:06:12 +02:00
|
|
|
''',
|
2015-09-16 13:06:16 +02:00
|
|
|
c_name=c_name(name))
|
|
|
|
|
|
|
|
ret += mcgen('''
|
2015-09-16 13:06:12 +02:00
|
|
|
|
2015-09-16 13:06:16 +02:00
|
|
|
extern const char *const %(c_name)s_lookup[];
|
|
|
|
''',
|
|
|
|
c_name=c_name(name))
|
|
|
|
return ret
|
2015-09-16 13:06:12 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-09-16 13:06:20 +02:00
|
|
|
def gen_params(arg_type, extra):
|
|
|
|
if not arg_type:
|
|
|
|
return extra
|
|
|
|
assert not arg_type.variants
|
|
|
|
ret = ''
|
|
|
|
sep = ''
|
|
|
|
for memb in arg_type.members:
|
|
|
|
ret += sep
|
|
|
|
sep = ', '
|
|
|
|
if memb.optional:
|
|
|
|
ret += 'bool has_%s, ' % c_name(memb.name)
|
|
|
|
ret += '%s %s' % (memb.type.c_type(is_param=True), c_name(memb.name))
|
|
|
|
if extra:
|
|
|
|
ret += sep + extra
|
|
|
|
return ret
|
|
|
|
|
2015-09-30 00:21:13 +02:00
|
|
|
|
2015-09-30 00:21:15 +02:00
|
|
|
def gen_err_check(label='out', skiperr=False):
|
|
|
|
if skiperr:
|
2015-09-30 00:21:13 +02:00
|
|
|
return ''
|
|
|
|
return mcgen('''
|
2015-09-30 00:21:15 +02:00
|
|
|
if (err) {
|
2015-09-30 00:21:13 +02:00
|
|
|
goto %(label)s;
|
|
|
|
}
|
|
|
|
''',
|
2015-09-30 00:21:15 +02:00
|
|
|
label=label)
|
2015-09-30 00:21:13 +02:00
|
|
|
|
|
|
|
|
qapi: Improve generated event use of qapi visitor
All other successful clients of visit_start_struct() were paired
with an unconditional visit_end_struct(); but the generated
code for events was relying on qmp_output_visitor_cleanup() to
work on an incomplete visit. Alter the code to guarantee that
the struct is completed, which will make a future patch to
split visit_end_struct() easier to reason about. While at it,
drop some assertions and comments that are not present in other
uses of the qmp output visitor, and pass NULL rather than "" as
the 'kind' parameter (matching most other uses where obj is NULL).
The changes to the generated code look like:
| qmp = qmp_event_build_dict("DEVICE_TRAY_MOVED");
|
| qov = qmp_output_visitor_new();
|- g_assert(qov);
|-
| v = qmp_output_get_visitor(qov);
|- g_assert(v);
|
|- /* Fake visit, as if all members are under a structure */
|- visit_start_struct(v, NULL, "", "DEVICE_TRAY_MOVED", 0, &err);
|+ visit_start_struct(v, NULL, NULL, "DEVICE_TRAY_MOVED", 0, &err);
| if (err) {
| goto out;
| }
| visit_type_str(v, (char **)&device, "device", &err);
| if (err) {
|- goto out;
|+ goto out_obj;
| }
| visit_type_bool(v, &tray_open, "tray-open", &err);
| if (err) {
|- goto out;
|+ goto out_obj;
| }
|- visit_end_struct(v, &err);
|+out_obj:
|+ visit_end_struct(v, err ? NULL : &err);
| if (err) {
| goto out;
| }
|
| obj = qmp_output_get_qobject(qov);
|- g_assert(obj != NULL);
|+ g_assert(obj);
|
| qdict_put_obj(qmp, "data", obj);
| emit(QAPI_EVENT_DEVICE_TRAY_MOVED, qmp, &err);
Note that the 'goto out_obj' with no intervening code before the
label, as well as the construct of 'err ? NULL : &err', are both
a bit unusual but also temporary; they get fixed in a later patch
that splits visit_end_struct() to drop its errp parameter by moving
some checking before the label. But until that time, this was the
simplest way to avoid the appearance of passing a possibly-set
error to visit_end_struct(), even though actual code inspection
shows that visit_end_struct() for a QMP output visitor will never
set an error.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1454075341-13658-11-git-send-email-eblake@redhat.com>
[Commit message's code diff tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-01-29 14:48:46 +01:00
|
|
|
def gen_visit_fields(members, prefix='', need_cast=False, skiperr=False,
|
|
|
|
label='out'):
|
qapi: Share gen_visit_fields()
Consolidate the code between visit, command marshalling, and
event generation that iterates over the members of a struct.
It reduces code duplication in the generator, so that a future
patch can reduce the size of generated code while touching only
one instead of three locations.
There are no changes to the generated marshal code.
The visitor code becomes slightly more verbose, but remains
semantically equivalent, and is actually easier to read as
it follows a more common idiom:
| visit_optional(v, &(*obj)->has_device, "device", &err);
|- if (!err && (*obj)->has_device) {
|- visit_type_str(v, &(*obj)->device, "device", &err);
|- }
| if (err) {
| goto out;
| }
|+ if ((*obj)->has_device) {
|+ visit_type_str(v, &(*obj)->device, "device", &err);
|+ if (err) {
|+ goto out;
|+ }
|+ }
The event code becomes slightly more verbose, but this is
arguably a bug fix: although the visitors are not well
documented, use of an optional member should not be attempted
unless guarded by a prior call to visit_optional(). Works only
because the output qmp visitor has a no-op visit_optional():
|+ visit_optional(v, &has_offset, "offset", &err);
|+ if (err) {
|+ goto out;
|+ }
| if (has_offset) {
| visit_type_int(v, &offset, "offset", &err);
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1443565276-4535-17-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-09-30 00:21:14 +02:00
|
|
|
ret = ''
|
2015-09-30 00:21:15 +02:00
|
|
|
if skiperr:
|
qapi: Share gen_visit_fields()
Consolidate the code between visit, command marshalling, and
event generation that iterates over the members of a struct.
It reduces code duplication in the generator, so that a future
patch can reduce the size of generated code while touching only
one instead of three locations.
There are no changes to the generated marshal code.
The visitor code becomes slightly more verbose, but remains
semantically equivalent, and is actually easier to read as
it follows a more common idiom:
| visit_optional(v, &(*obj)->has_device, "device", &err);
|- if (!err && (*obj)->has_device) {
|- visit_type_str(v, &(*obj)->device, "device", &err);
|- }
| if (err) {
| goto out;
| }
|+ if ((*obj)->has_device) {
|+ visit_type_str(v, &(*obj)->device, "device", &err);
|+ if (err) {
|+ goto out;
|+ }
|+ }
The event code becomes slightly more verbose, but this is
arguably a bug fix: although the visitors are not well
documented, use of an optional member should not be attempted
unless guarded by a prior call to visit_optional(). Works only
because the output qmp visitor has a no-op visit_optional():
|+ visit_optional(v, &has_offset, "offset", &err);
|+ if (err) {
|+ goto out;
|+ }
| if (has_offset) {
| visit_type_int(v, &offset, "offset", &err);
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1443565276-4535-17-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-09-30 00:21:14 +02:00
|
|
|
errparg = 'NULL'
|
2015-09-30 00:21:15 +02:00
|
|
|
else:
|
|
|
|
errparg = '&err'
|
qapi: Share gen_visit_fields()
Consolidate the code between visit, command marshalling, and
event generation that iterates over the members of a struct.
It reduces code duplication in the generator, so that a future
patch can reduce the size of generated code while touching only
one instead of three locations.
There are no changes to the generated marshal code.
The visitor code becomes slightly more verbose, but remains
semantically equivalent, and is actually easier to read as
it follows a more common idiom:
| visit_optional(v, &(*obj)->has_device, "device", &err);
|- if (!err && (*obj)->has_device) {
|- visit_type_str(v, &(*obj)->device, "device", &err);
|- }
| if (err) {
| goto out;
| }
|+ if ((*obj)->has_device) {
|+ visit_type_str(v, &(*obj)->device, "device", &err);
|+ if (err) {
|+ goto out;
|+ }
|+ }
The event code becomes slightly more verbose, but this is
arguably a bug fix: although the visitors are not well
documented, use of an optional member should not be attempted
unless guarded by a prior call to visit_optional(). Works only
because the output qmp visitor has a no-op visit_optional():
|+ visit_optional(v, &has_offset, "offset", &err);
|+ if (err) {
|+ goto out;
|+ }
| if (has_offset) {
| visit_type_int(v, &offset, "offset", &err);
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1443565276-4535-17-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-09-30 00:21:14 +02:00
|
|
|
|
|
|
|
for memb in members:
|
|
|
|
if memb.optional:
|
|
|
|
ret += mcgen('''
|
qapi: Swap visit_* arguments for consistent 'name' placement
JSON uses "name":value, but many of our visitor interfaces were
called with visit_type_FOO(v, &value, name, errp). This can be
a bit confusing to have to mentally swap the parameter order to
match JSON order. It's particularly bad for visit_start_struct(),
where the 'name' parameter is smack in the middle of the
otherwise-related group of 'obj, kind, size' parameters! It's
time to do a global swap of the parameter ordering, so that the
'name' parameter is always immediately after the Visitor argument.
Additional reason in favor of the swap: the existing include/qjson.h
prefers listing 'name' first in json_prop_*(), and I have plans to
unify that file with the qapi visitors; listing 'name' first in
qapi will minimize churn to the (admittedly few) qjson.h clients.
Later patches will then fix docs, object.h, visitor-impl.h, and
those clients to match.
Done by first patching scripts/qapi*.py by hand to make generated
files do what I want, then by running the following Coccinelle
script to affect the rest of the code base:
$ spatch --sp-file script `git grep -l '\bvisit_' -- '**/*.[ch]'`
I then had to apply some touchups (Coccinelle insisted on TAB
indentation in visitor.h, and botched the signature of
visit_type_enum() by rewriting 'const char *const strings[]' to
the syntactically invalid 'const char*const[] strings'). The
movement of parameters is sufficient to provoke compiler errors
if any callers were missed.
// Part 1: Swap declaration order
@@
type TV, TErr, TObj, T1, T2;
identifier OBJ, ARG1, ARG2;
@@
void visit_start_struct
-(TV v, TObj OBJ, T1 ARG1, const char *name, T2 ARG2, TErr errp)
+(TV v, const char *name, TObj OBJ, T1 ARG1, T2 ARG2, TErr errp)
{ ... }
@@
type bool, TV, T1;
identifier ARG1;
@@
bool visit_optional
-(TV v, T1 ARG1, const char *name)
+(TV v, const char *name, T1 ARG1)
{ ... }
@@
type TV, TErr, TObj, T1;
identifier OBJ, ARG1;
@@
void visit_get_next_type
-(TV v, TObj OBJ, T1 ARG1, const char *name, TErr errp)
+(TV v, const char *name, TObj OBJ, T1 ARG1, TErr errp)
{ ... }
@@
type TV, TErr, TObj, T1, T2;
identifier OBJ, ARG1, ARG2;
@@
void visit_type_enum
-(TV v, TObj OBJ, T1 ARG1, T2 ARG2, const char *name, TErr errp)
+(TV v, const char *name, TObj OBJ, T1 ARG1, T2 ARG2, TErr errp)
{ ... }
@@
type TV, TErr, TObj;
identifier OBJ;
identifier VISIT_TYPE =~ "^visit_type_";
@@
void VISIT_TYPE
-(TV v, TObj OBJ, const char *name, TErr errp)
+(TV v, const char *name, TObj OBJ, TErr errp)
{ ... }
// Part 2: swap caller order
@@
expression V, NAME, OBJ, ARG1, ARG2, ERR;
identifier VISIT_TYPE =~ "^visit_type_";
@@
(
-visit_start_struct(V, OBJ, ARG1, NAME, ARG2, ERR)
+visit_start_struct(V, NAME, OBJ, ARG1, ARG2, ERR)
|
-visit_optional(V, ARG1, NAME)
+visit_optional(V, NAME, ARG1)
|
-visit_get_next_type(V, OBJ, ARG1, NAME, ERR)
+visit_get_next_type(V, NAME, OBJ, ARG1, ERR)
|
-visit_type_enum(V, OBJ, ARG1, ARG2, NAME, ERR)
+visit_type_enum(V, NAME, OBJ, ARG1, ARG2, ERR)
|
-VISIT_TYPE(V, OBJ, NAME, ERR)
+VISIT_TYPE(V, NAME, OBJ, ERR)
)
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <1454075341-13658-19-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-01-29 14:48:54 +01:00
|
|
|
if (visit_optional(v, "%(name)s", &%(prefix)shas_%(c_name)s)) {
|
qapi: Share gen_visit_fields()
Consolidate the code between visit, command marshalling, and
event generation that iterates over the members of a struct.
It reduces code duplication in the generator, so that a future
patch can reduce the size of generated code while touching only
one instead of three locations.
There are no changes to the generated marshal code.
The visitor code becomes slightly more verbose, but remains
semantically equivalent, and is actually easier to read as
it follows a more common idiom:
| visit_optional(v, &(*obj)->has_device, "device", &err);
|- if (!err && (*obj)->has_device) {
|- visit_type_str(v, &(*obj)->device, "device", &err);
|- }
| if (err) {
| goto out;
| }
|+ if ((*obj)->has_device) {
|+ visit_type_str(v, &(*obj)->device, "device", &err);
|+ if (err) {
|+ goto out;
|+ }
|+ }
The event code becomes slightly more verbose, but this is
arguably a bug fix: although the visitors are not well
documented, use of an optional member should not be attempted
unless guarded by a prior call to visit_optional(). Works only
because the output qmp visitor has a no-op visit_optional():
|+ visit_optional(v, &has_offset, "offset", &err);
|+ if (err) {
|+ goto out;
|+ }
| if (has_offset) {
| visit_type_int(v, &offset, "offset", &err);
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1443565276-4535-17-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-09-30 00:21:14 +02:00
|
|
|
''',
|
|
|
|
prefix=prefix, c_name=c_name(memb.name),
|
2016-01-29 14:48:41 +01:00
|
|
|
name=memb.name)
|
qapi: Share gen_visit_fields()
Consolidate the code between visit, command marshalling, and
event generation that iterates over the members of a struct.
It reduces code duplication in the generator, so that a future
patch can reduce the size of generated code while touching only
one instead of three locations.
There are no changes to the generated marshal code.
The visitor code becomes slightly more verbose, but remains
semantically equivalent, and is actually easier to read as
it follows a more common idiom:
| visit_optional(v, &(*obj)->has_device, "device", &err);
|- if (!err && (*obj)->has_device) {
|- visit_type_str(v, &(*obj)->device, "device", &err);
|- }
| if (err) {
| goto out;
| }
|+ if ((*obj)->has_device) {
|+ visit_type_str(v, &(*obj)->device, "device", &err);
|+ if (err) {
|+ goto out;
|+ }
|+ }
The event code becomes slightly more verbose, but this is
arguably a bug fix: although the visitors are not well
documented, use of an optional member should not be attempted
unless guarded by a prior call to visit_optional(). Works only
because the output qmp visitor has a no-op visit_optional():
|+ visit_optional(v, &has_offset, "offset", &err);
|+ if (err) {
|+ goto out;
|+ }
| if (has_offset) {
| visit_type_int(v, &offset, "offset", &err);
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1443565276-4535-17-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-09-30 00:21:14 +02:00
|
|
|
push_indent()
|
|
|
|
|
|
|
|
# Ugly: sometimes we need to cast away const
|
|
|
|
if need_cast and memb.type.name == 'str':
|
|
|
|
cast = '(char **)'
|
|
|
|
else:
|
|
|
|
cast = ''
|
|
|
|
|
|
|
|
ret += mcgen('''
|
qapi: Swap visit_* arguments for consistent 'name' placement
JSON uses "name":value, but many of our visitor interfaces were
called with visit_type_FOO(v, &value, name, errp). This can be
a bit confusing to have to mentally swap the parameter order to
match JSON order. It's particularly bad for visit_start_struct(),
where the 'name' parameter is smack in the middle of the
otherwise-related group of 'obj, kind, size' parameters! It's
time to do a global swap of the parameter ordering, so that the
'name' parameter is always immediately after the Visitor argument.
Additional reason in favor of the swap: the existing include/qjson.h
prefers listing 'name' first in json_prop_*(), and I have plans to
unify that file with the qapi visitors; listing 'name' first in
qapi will minimize churn to the (admittedly few) qjson.h clients.
Later patches will then fix docs, object.h, visitor-impl.h, and
those clients to match.
Done by first patching scripts/qapi*.py by hand to make generated
files do what I want, then by running the following Coccinelle
script to affect the rest of the code base:
$ spatch --sp-file script `git grep -l '\bvisit_' -- '**/*.[ch]'`
I then had to apply some touchups (Coccinelle insisted on TAB
indentation in visitor.h, and botched the signature of
visit_type_enum() by rewriting 'const char *const strings[]' to
the syntactically invalid 'const char*const[] strings'). The
movement of parameters is sufficient to provoke compiler errors
if any callers were missed.
// Part 1: Swap declaration order
@@
type TV, TErr, TObj, T1, T2;
identifier OBJ, ARG1, ARG2;
@@
void visit_start_struct
-(TV v, TObj OBJ, T1 ARG1, const char *name, T2 ARG2, TErr errp)
+(TV v, const char *name, TObj OBJ, T1 ARG1, T2 ARG2, TErr errp)
{ ... }
@@
type bool, TV, T1;
identifier ARG1;
@@
bool visit_optional
-(TV v, T1 ARG1, const char *name)
+(TV v, const char *name, T1 ARG1)
{ ... }
@@
type TV, TErr, TObj, T1;
identifier OBJ, ARG1;
@@
void visit_get_next_type
-(TV v, TObj OBJ, T1 ARG1, const char *name, TErr errp)
+(TV v, const char *name, TObj OBJ, T1 ARG1, TErr errp)
{ ... }
@@
type TV, TErr, TObj, T1, T2;
identifier OBJ, ARG1, ARG2;
@@
void visit_type_enum
-(TV v, TObj OBJ, T1 ARG1, T2 ARG2, const char *name, TErr errp)
+(TV v, const char *name, TObj OBJ, T1 ARG1, T2 ARG2, TErr errp)
{ ... }
@@
type TV, TErr, TObj;
identifier OBJ;
identifier VISIT_TYPE =~ "^visit_type_";
@@
void VISIT_TYPE
-(TV v, TObj OBJ, const char *name, TErr errp)
+(TV v, const char *name, TObj OBJ, TErr errp)
{ ... }
// Part 2: swap caller order
@@
expression V, NAME, OBJ, ARG1, ARG2, ERR;
identifier VISIT_TYPE =~ "^visit_type_";
@@
(
-visit_start_struct(V, OBJ, ARG1, NAME, ARG2, ERR)
+visit_start_struct(V, NAME, OBJ, ARG1, ARG2, ERR)
|
-visit_optional(V, ARG1, NAME)
+visit_optional(V, NAME, ARG1)
|
-visit_get_next_type(V, OBJ, ARG1, NAME, ERR)
+visit_get_next_type(V, NAME, OBJ, ARG1, ERR)
|
-visit_type_enum(V, OBJ, ARG1, ARG2, NAME, ERR)
+visit_type_enum(V, NAME, OBJ, ARG1, ARG2, ERR)
|
-VISIT_TYPE(V, OBJ, NAME, ERR)
+VISIT_TYPE(V, NAME, OBJ, ERR)
)
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <1454075341-13658-19-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-01-29 14:48:54 +01:00
|
|
|
visit_type_%(c_type)s(v, "%(name)s", %(cast)s&%(prefix)s%(c_name)s, %(errp)s);
|
qapi: Share gen_visit_fields()
Consolidate the code between visit, command marshalling, and
event generation that iterates over the members of a struct.
It reduces code duplication in the generator, so that a future
patch can reduce the size of generated code while touching only
one instead of three locations.
There are no changes to the generated marshal code.
The visitor code becomes slightly more verbose, but remains
semantically equivalent, and is actually easier to read as
it follows a more common idiom:
| visit_optional(v, &(*obj)->has_device, "device", &err);
|- if (!err && (*obj)->has_device) {
|- visit_type_str(v, &(*obj)->device, "device", &err);
|- }
| if (err) {
| goto out;
| }
|+ if ((*obj)->has_device) {
|+ visit_type_str(v, &(*obj)->device, "device", &err);
|+ if (err) {
|+ goto out;
|+ }
|+ }
The event code becomes slightly more verbose, but this is
arguably a bug fix: although the visitors are not well
documented, use of an optional member should not be attempted
unless guarded by a prior call to visit_optional(). Works only
because the output qmp visitor has a no-op visit_optional():
|+ visit_optional(v, &has_offset, "offset", &err);
|+ if (err) {
|+ goto out;
|+ }
| if (has_offset) {
| visit_type_int(v, &offset, "offset", &err);
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1443565276-4535-17-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-09-30 00:21:14 +02:00
|
|
|
''',
|
|
|
|
c_type=memb.type.c_name(), prefix=prefix, cast=cast,
|
|
|
|
c_name=c_name(memb.name), name=memb.name,
|
|
|
|
errp=errparg)
|
qapi: Improve generated event use of qapi visitor
All other successful clients of visit_start_struct() were paired
with an unconditional visit_end_struct(); but the generated
code for events was relying on qmp_output_visitor_cleanup() to
work on an incomplete visit. Alter the code to guarantee that
the struct is completed, which will make a future patch to
split visit_end_struct() easier to reason about. While at it,
drop some assertions and comments that are not present in other
uses of the qmp output visitor, and pass NULL rather than "" as
the 'kind' parameter (matching most other uses where obj is NULL).
The changes to the generated code look like:
| qmp = qmp_event_build_dict("DEVICE_TRAY_MOVED");
|
| qov = qmp_output_visitor_new();
|- g_assert(qov);
|-
| v = qmp_output_get_visitor(qov);
|- g_assert(v);
|
|- /* Fake visit, as if all members are under a structure */
|- visit_start_struct(v, NULL, "", "DEVICE_TRAY_MOVED", 0, &err);
|+ visit_start_struct(v, NULL, NULL, "DEVICE_TRAY_MOVED", 0, &err);
| if (err) {
| goto out;
| }
| visit_type_str(v, (char **)&device, "device", &err);
| if (err) {
|- goto out;
|+ goto out_obj;
| }
| visit_type_bool(v, &tray_open, "tray-open", &err);
| if (err) {
|- goto out;
|+ goto out_obj;
| }
|- visit_end_struct(v, &err);
|+out_obj:
|+ visit_end_struct(v, err ? NULL : &err);
| if (err) {
| goto out;
| }
|
| obj = qmp_output_get_qobject(qov);
|- g_assert(obj != NULL);
|+ g_assert(obj);
|
| qdict_put_obj(qmp, "data", obj);
| emit(QAPI_EVENT_DEVICE_TRAY_MOVED, qmp, &err);
Note that the 'goto out_obj' with no intervening code before the
label, as well as the construct of 'err ? NULL : &err', are both
a bit unusual but also temporary; they get fixed in a later patch
that splits visit_end_struct() to drop its errp parameter by moving
some checking before the label. But until that time, this was the
simplest way to avoid the appearance of passing a possibly-set
error to visit_end_struct(), even though actual code inspection
shows that visit_end_struct() for a QMP output visitor will never
set an error.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1454075341-13658-11-git-send-email-eblake@redhat.com>
[Commit message's code diff tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-01-29 14:48:46 +01:00
|
|
|
ret += gen_err_check(skiperr=skiperr, label=label)
|
qapi: Share gen_visit_fields()
Consolidate the code between visit, command marshalling, and
event generation that iterates over the members of a struct.
It reduces code duplication in the generator, so that a future
patch can reduce the size of generated code while touching only
one instead of three locations.
There are no changes to the generated marshal code.
The visitor code becomes slightly more verbose, but remains
semantically equivalent, and is actually easier to read as
it follows a more common idiom:
| visit_optional(v, &(*obj)->has_device, "device", &err);
|- if (!err && (*obj)->has_device) {
|- visit_type_str(v, &(*obj)->device, "device", &err);
|- }
| if (err) {
| goto out;
| }
|+ if ((*obj)->has_device) {
|+ visit_type_str(v, &(*obj)->device, "device", &err);
|+ if (err) {
|+ goto out;
|+ }
|+ }
The event code becomes slightly more verbose, but this is
arguably a bug fix: although the visitors are not well
documented, use of an optional member should not be attempted
unless guarded by a prior call to visit_optional(). Works only
because the output qmp visitor has a no-op visit_optional():
|+ visit_optional(v, &has_offset, "offset", &err);
|+ if (err) {
|+ goto out;
|+ }
| if (has_offset) {
| visit_type_int(v, &offset, "offset", &err);
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1443565276-4535-17-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-09-30 00:21:14 +02:00
|
|
|
|
|
|
|
if memb.optional:
|
|
|
|
pop_indent()
|
|
|
|
ret += mcgen('''
|
|
|
|
}
|
|
|
|
''')
|
|
|
|
return ret
|
|
|
|
|
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
#
|
|
|
|
# Common command line parsing
|
|
|
|
#
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
|
|
|
def parse_command_line(extra_options="", extra_long_options=[]):
|
2015-04-02 13:12:21 +02:00
|
|
|
|
|
|
|
try:
|
|
|
|
opts, args = getopt.gnu_getopt(sys.argv[1:],
|
2015-04-02 13:32:16 +02:00
|
|
|
"chp:o:" + extra_options,
|
2015-04-02 13:12:21 +02:00
|
|
|
["source", "header", "prefix=",
|
2015-04-02 13:32:16 +02:00
|
|
|
"output-dir="] + extra_long_options)
|
2015-12-18 08:52:41 +01:00
|
|
|
except getopt.GetoptError as err:
|
2015-04-02 13:17:34 +02:00
|
|
|
print >>sys.stderr, "%s: %s" % (sys.argv[0], str(err))
|
2015-04-02 13:12:21 +02:00
|
|
|
sys.exit(1)
|
|
|
|
|
|
|
|
output_dir = ""
|
|
|
|
prefix = ""
|
|
|
|
do_c = False
|
|
|
|
do_h = False
|
|
|
|
extra_opts = []
|
|
|
|
|
|
|
|
for oa in opts:
|
|
|
|
o, a = oa
|
|
|
|
if o in ("-p", "--prefix"):
|
2015-07-01 13:13:54 +02:00
|
|
|
match = re.match('([A-Za-z_.-][A-Za-z0-9_.-]*)?', a)
|
|
|
|
if match.end() != len(a):
|
|
|
|
print >>sys.stderr, \
|
|
|
|
"%s: 'funny character '%s' in argument of --prefix" \
|
|
|
|
% (sys.argv[0], a[match.end()])
|
|
|
|
sys.exit(1)
|
2015-04-02 13:12:21 +02:00
|
|
|
prefix = a
|
|
|
|
elif o in ("-o", "--output-dir"):
|
|
|
|
output_dir = a + "/"
|
|
|
|
elif o in ("-c", "--source"):
|
|
|
|
do_c = True
|
|
|
|
elif o in ("-h", "--header"):
|
|
|
|
do_h = True
|
|
|
|
else:
|
|
|
|
extra_opts.append(oa)
|
|
|
|
|
|
|
|
if not do_c and not do_h:
|
|
|
|
do_c = True
|
|
|
|
do_h = True
|
|
|
|
|
2015-04-02 13:32:16 +02:00
|
|
|
if len(args) != 1:
|
|
|
|
print >>sys.stderr, "%s: need exactly one argument" % sys.argv[0]
|
2015-04-02 13:17:34 +02:00
|
|
|
sys.exit(1)
|
2015-06-09 16:22:45 +02:00
|
|
|
fname = args[0]
|
2015-04-02 13:17:34 +02:00
|
|
|
|
2015-06-09 16:22:45 +02:00
|
|
|
return (fname, output_dir, do_c, do_h, prefix, extra_opts)
|
2015-04-02 14:46:39 +02:00
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
#
|
|
|
|
# Generate output files with boilerplate
|
|
|
|
#
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-04-02 14:46:39 +02:00
|
|
|
def open_output(output_dir, do_c, do_h, prefix, c_file, h_file,
|
|
|
|
c_comment, h_comment):
|
2015-06-27 07:27:21 +02:00
|
|
|
guard = guardname(prefix + h_file)
|
2015-04-02 14:46:39 +02:00
|
|
|
c_file = output_dir + prefix + c_file
|
|
|
|
h_file = output_dir + prefix + h_file
|
|
|
|
|
2015-09-03 10:24:25 +02:00
|
|
|
if output_dir:
|
|
|
|
try:
|
|
|
|
os.makedirs(output_dir)
|
2015-12-18 08:52:41 +01:00
|
|
|
except os.error as e:
|
2015-09-03 10:24:25 +02:00
|
|
|
if e.errno != errno.EEXIST:
|
|
|
|
raise
|
2015-04-02 14:46:39 +02:00
|
|
|
|
|
|
|
def maybe_open(really, name, opt):
|
|
|
|
if really:
|
|
|
|
return open(name, opt)
|
|
|
|
else:
|
|
|
|
import StringIO
|
|
|
|
return StringIO.StringIO()
|
|
|
|
|
|
|
|
fdef = maybe_open(do_c, c_file, 'w')
|
|
|
|
fdecl = maybe_open(do_h, h_file, 'w')
|
|
|
|
|
|
|
|
fdef.write(mcgen('''
|
|
|
|
/* AUTOMATICALLY GENERATED, DO NOT MODIFY */
|
|
|
|
%(comment)s
|
|
|
|
''',
|
2015-09-30 00:21:02 +02:00
|
|
|
comment=c_comment))
|
2015-04-02 14:46:39 +02:00
|
|
|
|
|
|
|
fdecl.write(mcgen('''
|
|
|
|
/* AUTOMATICALLY GENERATED, DO NOT MODIFY */
|
|
|
|
%(comment)s
|
|
|
|
#ifndef %(guard)s
|
|
|
|
#define %(guard)s
|
|
|
|
|
|
|
|
''',
|
2015-09-30 00:21:02 +02:00
|
|
|
comment=h_comment, guard=guard))
|
2015-04-02 14:46:39 +02:00
|
|
|
|
|
|
|
return (fdef, fdecl)
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-04-02 14:46:39 +02:00
|
|
|
def close_output(fdef, fdecl):
|
|
|
|
fdecl.write('''
|
|
|
|
#endif
|
|
|
|
''')
|
|
|
|
fdecl.close()
|
|
|
|
fdef.close()
|