Commit Graph

487 Commits

Author SHA1 Message Date
Markus Armbruster
1a9a507b2e qapi-introspect: Hide type names
To eliminate the temptation for clients to look up types by name
(which are not ABI), replace all type names by meaningless strings.

Reduces output of query-schema by 13 out of 85KiB.

As a debugging aid, provide option -u to suppress the hiding.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1442401589-24189-27-git-send-email-armbru@redhat.com>
2015-09-21 09:56:49 +02:00
Markus Armbruster
39a1815816 qapi: New QMP command query-qmp-schema for QMP introspection
qapi/introspect.json defines the introspection schema.  It's designed
for QMP introspection, but should do for similar uses, such as QGA.

The introspection schema does not reflect all the rules and
restrictions that apply to QAPI schemata.  A valid QAPI schema has an
introspection value conforming to the introspection schema, but the
converse is not true.

Introspection lowers away a number of schema details, and makes
implicit things explicit:

* The built-in types are declared with their JSON type.

  All integer types are mapped to 'int', because how many bits we use
  internally is an implementation detail.  It could be pressed into
  external interface service as very approximate range information,
  but that's a bad idea.  If we need range information, we better do
  it properly.

* Implicit type definitions are made explicit, and given
  auto-generated names:

  - Array types, named by appending "List" to the name of their
    element type, like in generated C.

  - The enumeration types implicitly defined by simple union types,
    named by appending "Kind" to the name of their simple union type,
    like in generated C.

  - Types that don't occur in generated C.  Their names start with ':'
    so they don't clash with the user's names.

* All type references are by name.

* The struct and union types are generalized into an object type.

* Base types are flattened.

* Commands take a single argument and return a single result.

  Dictionary argument or list result is an implicit type definition.

  The empty object type is used when a command takes no arguments or
  produces no results.

  The argument is always of object type, but the introspection schema
  doesn't reflect that.

  The 'gen': false directive is omitted as implementation detail.

  The 'success-response' directive is omitted as well for now, even
  though it's not an implementation detail, because it's not used by
  QMP.

* Events carry a single data value.

  Implicit type definition and empty object type use, just like for
  commands.

  The value is of object type, but the introspection schema doesn't
  reflect that.

* Types not used by commands or events are omitted.

  Indirect use counts as use.

* Optional members have a default, which can only be null right now

  Instead of a mandatory "optional" flag, we have an optional default.
  No default means mandatory, default null means optional without
  default value.  Non-null is available for optional with default
  (possible future extension).

* Clients should *not* look up types by name, because type names are
  not ABI.  Look up the command or event you're interested in, then
  follow the references.

  TODO Should we hide the type names to eliminate the temptation?

New generator scripts/qapi-introspect.py computes an introspection
value for its input, and generates a C variable holding it.

It can generate awfully long lines.  Marked TODO.

A new test-qmp-input-visitor test case feeds its result for both
tests/qapi-schema/qapi-schema-test.json and qapi-schema.json to a
QmpInputVisitor to verify it actually conforms to the schema.

New QMP command query-qmp-schema takes its return value from that
variable.  Its reply is some 85KiBytes for me right now.

If this turns out to be too much, we have a couple of options:

* We can use shorter names in the JSON.  Not the QMP style.

* Optionally return the sub-schema for commands and events given as
  arguments.

  Right now qmp_query_schema() sends the string literal computed by
  qmp-introspect.py.  To compute sub-schema at run time, we'd have to
  duplicate parts of qapi-introspect.py in C.  Unattractive.

* Let clients cache the output of query-qmp-schema.

  It changes only on QEMU upgrades, i.e. rarely.  Provide a command
  query-qmp-schema-hash.  Clients can have a cache indexed by hash,
  and re-query the schema only when they don't have it cached.  Even
  simpler: put the hash in the QMP greeting.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-21 09:56:49 +02:00
Markus Armbruster
2d21291ae6 qapi: Pseudo-type '**' is now unused, drop it
'gen': false needs to stay for now, because netdev_add is still using
it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1442401589-24189-25-git-send-email-armbru@redhat.com>
2015-09-21 09:56:49 +02:00
Markus Armbruster
6eb3937e9b qom: Don't use 'gen': false for qom-get, qom-set, object-add
With the previous commit, the generated marshalers just work, and save
us a bit of handwritten code.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1442401589-24189-23-git-send-email-armbru@redhat.com>
2015-09-21 09:56:49 +02:00
Markus Armbruster
28770e057f qapi: Introduce a first class 'any' type
It's first class, because unlike '**', it actually works, i.e. doesn't
require 'gen': false.

'**' will go away next.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
2015-09-21 09:56:49 +02:00
Markus Armbruster
56d92b003a qapi-commands: De-duplicate output marshaling functions
gen_marshal_output() uses its parameter name only for name of the
generated function.  Name it after the type being marshaled instead of
its caller, and drop duplicates.

Saves 7 copies of qmp_marshal_output_int() in qemu-ga, and one copy of
qmp_marshal_output_str() in qemu-system-*.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1442401589-24189-19-git-send-email-armbru@redhat.com>
2015-09-21 09:56:48 +02:00
Markus Armbruster
03b4367a55 qapi: De-duplicate parameter list generation
Generated qapi-event.[ch] lose line breaks.  No change otherwise.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1442401589-24189-18-git-send-email-armbru@redhat.com>
2015-09-21 09:56:48 +02:00
Markus Armbruster
7fad30f06e qapi: Rename qmp_marshal_input_FOO() to qmp_marshal_FOO()
These functions marshal both input and output.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1442401589-24189-17-git-send-email-armbru@redhat.com>
2015-09-21 09:56:48 +02:00
Markus Armbruster
f15380190a qapi-commands: Rearrange code
Rename gen_marshal_input() to gen_marshal(), because the generated
function marshals both arguments and results.

Rename gen_visitor_input_containers_decl() to gen_marshal_vars(), and
move the other variable declarations there, too.

Rename gen_visitor_input_block() to gen_marshal_input_visit(), and
rearrange its code slightly.

Rename gen_marshal_input_decl() to gen_marshal_proto(), because the
result isn't a full declaration, unlike gen_command_decl()'s.

New gen_marshal_decl() actually returns a full declaration.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1442401589-24189-16-git-send-email-armbru@redhat.com>
2015-09-21 09:56:48 +02:00
Markus Armbruster
60f8546acd qapi-visit: Rearrange code a bit
Move gen_visit_decl() to a better place.  Inline
generate_visit_struct_body().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1442401589-24189-15-git-send-email-armbru@redhat.com>
2015-09-21 09:56:48 +02:00
Markus Armbruster
e98859a9b9 qapi: Clean up after recent conversions to QAPISchemaVisitor
Generate just 'FOO' instead of 'struct FOO' when possible.

Drop helper functions that are now unused.

Make pep8 and pylint reasonably happy.

Rename generate_FOO() functions to gen_FOO() for consistency.

Use more consistent and sensible variable names.

Consistently use c_ for mapping keys when their value is a C
identifier or type.

Simplify gen_enum() and gen_visit_union()

Consistently use single quotes for C text string literals.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1442401589-24189-14-git-send-email-armbru@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-21 09:56:48 +02:00
Markus Armbruster
5710153e73 qapi: Replace dirty is_c_ptr() by method c_null()
is_c_ptr() looks whether the end of the C text for the type looks like
a pointer.  Works, but is fragile.

We now have a better tool: use QAPISchemaType method c_null().  The
initializers for non-pointers become prettier: 0, false or the
enumeration constant with the value 0 instead of {0}.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1442401589-24189-13-git-send-email-armbru@redhat.com>
2015-09-21 09:56:48 +02:00
Markus Armbruster
05f43a9608 qapi-event: Convert to QAPISchemaVisitor, fixing data with base
Fixes events whose data is struct with base to include the struct's
base members.  Test case is qapi-schema-test.json's event
__org.qemu_x-command:

    { 'event': '__ORG.QEMU_X-EVENT', 'data': '__org.qemu_x-Struct' }

    { 'struct': '__org.qemu_x-Struct', 'base': '__org.qemu_x-Base',
      'data': { '__org.qemu_x-member2': 'str' } }

    { 'struct': '__org.qemu_x-Base',
      'data': { '__org.qemu_x-member1': '__org.qemu_x-Enum' } }

Patch's effect on generated qapi_event_send___org_qemu_x_event():

    -void qapi_event_send___org_qemu_x_event(const char *__org_qemu_x_member2,
    +void qapi_event_send___org_qemu_x_event(__org_qemu_x_Enum __org_qemu_x_member1,
    +                                        const char *__org_qemu_x_member2,
                                             Error **errp)
     {
         QDict *qmp;
    @@ -224,6 +225,10 @@ void qapi_event_send___org_qemu_x_event(
             goto clean;
         }

    +    visit_type___org_qemu_x_Enum(v, &__org_qemu_x_member1, "__org.qemu_x-member1", &local_err);
    +    if (local_err) {
    +        goto clean;
    +    }
         visit_type_str(v, (char **)&__org_qemu_x_member2, "__org.qemu_x-member2", &local_err);
         if (local_err) {
             goto clean;

Code is generated in a different order now, but that doesn't matter.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
2015-09-21 09:56:48 +02:00
Markus Armbruster
7b24626cd0 qapi-event: Eliminate global variable event_enum_value
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1442401589-24189-11-git-send-email-armbru@redhat.com>
2015-09-21 09:56:48 +02:00
Markus Armbruster
efd2eaa6c2 qapi: De-duplicate enum code generation
Duplicated in commit 21cd70d.  Yes, we can't import qapi-types, but
that's no excuse.  Move the helpers from qapi-types.py to qapi.py, and
replace the duplicates in qapi-event.py.

The generated event enumeration type's lookup table becomes
const-correct (see commit 2e4450f), and uses explicit indexes instead
of relying on order (see commit 912ae9c).

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1442401589-24189-10-git-send-email-armbru@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-21 09:56:47 +02:00
Markus Armbruster
ee44602857 qapi-commands: Convert to QAPISchemaVisitor
Output unchanged apart from reordering and white-space.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1442401589-24189-9-git-send-email-armbru@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
2015-09-21 09:56:47 +02:00
Markus Armbruster
441cbac0c7 qapi-visit: Convert to QAPISchemaVisitor, fixing bugs
Fixes flat unions to visit the base's base members (the previous
commit merely added them to the struct).  Same test case.

Patch's effect on visit_type_UserDefFlatUnion():

     static void visit_type_UserDefFlatUnion_fields(Visitor *m, UserDefFlatUnion **obj, Error **errp)
     {
         Error *err = NULL;

    +    visit_type_int(m, &(*obj)->integer, "integer", &err);
    +    if (err) {
    +        goto out;
    +    }
         visit_type_str(m, &(*obj)->string, "string", &err);
         if (err) {
             goto out;

Test cases updated for the bug fix.

Fixes alternates to generate a visitor for their implicit enumeration
type.  None of them are currently used, obviously.  Example:
block-core.json's BlockdevRef now generates
visit_type_BlockdevRefKind().

Code is generated in a different order now, and therefore has got a
few new forward declarations.  Doesn't matter.

The guard QAPI_VISIT_BUILTIN_VISITOR_DECL is renamed to
QAPI_VISIT_BUILTIN.

The previous commit's two ugly special cases exist here, too.  Mark
both TODO.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-21 09:56:40 +02:00
Markus Armbruster
2b162ccbe8 qapi-types: Convert to QAPISchemaVisitor, fixing flat unions
Fixes flat unions to get the base's base members.  Test case is from
commit 2fc0043, in qapi-schema-test.json:

    { 'union': 'UserDefFlatUnion',
      'base': 'UserDefUnionBase',
      'discriminator': 'enum1',
      'data': { 'value1' : 'UserDefA',
                'value2' : 'UserDefB',
                'value3' : 'UserDefB' } }

    { 'struct': 'UserDefUnionBase',
      'base': 'UserDefZero',
      'data': { 'string': 'str', 'enum1': 'EnumOne' } }

    { 'struct': 'UserDefZero',
      'data': { 'integer': 'int' } }

Patch's effect on UserDefFlatUnion:

     struct UserDefFlatUnion {
         /* Members inherited from UserDefUnionBase: */
    +    int64_t integer;
         char *string;
         EnumOne enum1;
         /* Own members: */
         union { /* union tag is @enum1 */
             void *data;
             UserDefA *value1;
             UserDefB *value2;
             UserDefB *value3;
         };
     };

Flat union visitors remain broken.  They'll be fixed next.

Code is generated in a different order now, but that doesn't matter.

The two guards QAPI_TYPES_BUILTIN_STRUCT_DECL and
QAPI_TYPES_BUILTIN_CLEANUP_DECL are replaced by just
QAPI_TYPES_BUILTIN.

Two ugly special cases for simple unions now stand out like sore
thumbs:

1. The type tag is named 'type' everywhere, except in generated C,
   where it's 'kind'.

2. QAPISchema lowers simple unions to semantically equivalent flat
   unions.  However, the C generated for a simple unions differs from
   the C generated for its equivalent flat union, and we therefore
   need special code to preserve that pointless difference for now.

Mark both TODO.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-21 09:53:16 +02:00
Markus Armbruster
3f7dc21bee qapi: New QAPISchemaVisitor
The visitor will help keeping the code generation code simple and
reasonably separated from QAPISchema details.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1442401589-24189-5-git-send-email-armbru@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-21 09:32:42 +02:00
Markus Armbruster
f51d8c3db1 qapi: QAPISchema code generation helper methods
New methods c_name(), c_type(), c_null(), json_type(),
alternate_qtype().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1442401589-24189-4-git-send-email-armbru@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-21 09:32:05 +02:00
Markus Armbruster
ac88219a6c qapi: New QAPISchema intermediate reperesentation
The QAPI code generators work with a syntax tree (nested dictionaries)
plus a few symbol tables (also dictionaries) on the side.

They have clearly outgrown these simple data structures.  There's lots
of rummaging around in dictionaries, and information is recomputed on
the fly.  For the work I'm going to do, I want more clearly defined
and more convenient interfaces.

Going forward, I also want less coupling between the back-ends and the
syntax tree, to make messing with the syntax easier.

Create a bunch of classes to represent QAPI schemata.

Have the QAPISchema initializer call the parser, then walk the syntax
tree to create the new internal representation, and finally perform
semantic analysis.

Shortcut: the semantic analysis still relies on existing check_exprs()
to do the actual semantic checking.  All this code needs to move into
the classes.  Mark as TODO.

Simple unions are lowered to flat unions.  Flat unions and structs are
represented as a more general object type.

Catching name collisions in generated code would be nice.  Mark as
TODO.

We generate array types eagerly, even though most of them aren't used.
Mark as TODO.

Nothing uses the new intermediate representation just yet, thus no
change to generated files.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-21 09:31:29 +02:00
Markus Armbruster
a4bcb2080d qapi: Rename class QAPISchema to QAPISchemaParser
I want to name a new class QAPISchema.

While there, make it a new-style class.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1442401589-24189-2-git-send-email-armbru@redhat.com>
2015-09-21 09:11:42 +02:00
Peter Maydell
1c9f03b81c * Linux header update and cleanup
* Support for HyperV crash report
 * Cleanup of target-specific HMP commands
 * Multiarch batch
 * Checkpatch fix for Perl 5.22
 * NBD fix
 * Revert incorrect commit 5243722376
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQEcBAABCAAGBQJV+Y0VAAoJEL/70l94x66DppAH/393irRPGYJiZqClXDmPZymd
 Ilv+mKQIA+QSGiMaVoq0POWqvSa6oPOIOAK8BexhHDWPnxDSeAU9JzjrkwQILjD3
 A5kFsR1mhw/oV8aZCsE926SUoY90VH7QP2r0iGdur1VV9DG7CUE8q95umJuU1FCo
 hvGI2HjlAPvxSrO+Y5vu4ZNOa5XBiUoLodT0j+jNZfoNMx3irAqkJLPqTUuPT8np
 jBeu1MtMe3wNjDBm3WMVZwkNEZ3YtuhRkBeggXqDIl1Jm9lq4Eitk7rtETXm6yHz
 Vjp9uAF4en/72gXbELRXp+aWvESEABcouLZrdPV/yjOmQD2oTFC7sZrm1c5rZPw=
 =MkK8
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* Linux header update and cleanup
* Support for HyperV crash report
* Cleanup of target-specific HMP commands
* Multiarch batch
* Checkpatch fix for Perl 5.22
* NBD fix
* Revert incorrect commit 5243722376

# gpg: Signature made Wed 16 Sep 2015 16:39:01 BST using RSA key ID 78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"

* remotes/bonzini/tags/for-upstream: (24 commits)
  nbd: release exp->blk after all clients are closed
  checkpatch: Escape left braces in regex
  monitor: uninclude cpu_ldst
  include/exec: Move cputlb exec.c defs out
  cputlb: Change tlb_set_dirty() arg to cpu
  cputlb: move CPU_LOOP() for tlb_reset() to exec.c
  translate: move real_host_page setting to -common
  tcg: Move tci_tb_ptr to -common
  tcg: split tcg_op_defs to -common
  translate-all: Move tcg_handle_interrupt() to -common
  cpu-exec: Migrate some generic fns to cpu-exec-common
  qemu-char: Use g_new() & friends where that makes obvious sense
  monitor: added generation of documentation for hmp-commands-info.hx
  hmp-commands.hx: fix end of table info
  monitor: remove target-specific code from monitor.c
  hmp-commands-info: move info_cmds content out of monitor.c
  i386/kvm: Hyper-v crash msrs set/get'ers and migration
  kvm: Add kvm system event crash handler
  cpu: Add crash_occurred flag into CPUState
  target-i386: move asm-x86/hyperv.h to standard-headers
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-09-16 18:06:54 +01:00
Fam Zheng
04f2562f8e checkpatch: Escape left braces in regex
Latest perl now deprecates "{" literal in regex and print warnings like
"unescaped left brace in regex is deprecated".  Add escape to keep it
happy.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <1441969656-2640-1-git-send-email-famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-09-16 17:33:33 +02:00
Paolo Bonzini
73aa529a48 target-i386: move asm-x86/hyperv.h to standard-headers
The Hyper-V definitions are an industry standard and can be used
from code that is not KVM-specific.

Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-09-16 17:33:32 +02:00
Paolo Bonzini
eddb4de3cc update-linux-headers: copy standard-headers files one by one
cp_virtio is called for both the asm-s390/ and linux/ directories,
so it looks for pci_regs.h and input.h files in asm-s390/ too.  This
makes little sense.  In the next patch we will have the opposite
problem; we want to add asm-x86/hyperv.h, and there's also a
linux/hyperv.h file with unwanted dependencies on additional Linux
uapi headers.  We do not want to copy linux/hyperv.h.

The solution is to make cp_virtio (now renamed to cp_portable) copy
one file only, instead of using the "find" command, and call it multiple
times.  The new function is really just a reindentation of the old one.

Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-09-16 17:33:32 +02:00
Paolo Bonzini
120758fba4 update Linux headers to 4.3-rc1
The update to 4.2 was reviewed by Michael S. Tsirkin and Cornelia
Huck.  The further update to 4.3-rc1 only touches KVM files.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-09-16 17:33:32 +02:00
Daniel P. Berrange
351d36e454 qapi: allow override of default enum prefix naming
The camel_to_upper() method applies some heuristics to turn
a mixed case type name into an all-uppercase name. This is
used for example, to generate enum constant name prefixes.

The heuristics don't also generate a satisfactory name
though. eg

  { 'enum': 'QCryptoTLSCredsEndpoint',
    'data': ['client', 'server']}

Results in Q_CRYPTOTLS_CREDS_ENDPOINT_CLIENT. This has
an undesirable _ after the initial Q and is missing an
_ between the CRYPTO & TLS strings.

Rather than try to add more and more heuristics to try
to cope with this, simply allow the QAPI schema to
specify the desired enum constant prefix explicitly.

eg

  { 'enum': 'QCryptoTLSCredsEndpoint',
    'prefix': 'QCRYPTO_TLS_CREDS_ENDPOINT',
    'data': ['client', 'server']}

Now gives the QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT name.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-09-15 10:59:28 +01:00
Markus Armbruster
2752e5bedb qapi: Fix cgen() for Python older than 2.7
A feature new in Python 2.7 crept into commit 77e703b: re.subn()'s
fifth argument.  Avoid that, use re.compile().

Reported-by: Laurent Desnogues <laurent.desnogues@gmail.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Laurent Desnogues <laurent.desnogues@gmail.com>
Message-id: 1441640755-23902-1-git-send-email-armbru@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-09-14 18:02:59 +01:00
Peter Maydell
a2aa09e181 * Support for jemalloc
* qemu_mutex_lock_iothread "No such process" fix
 * cutils: qemu_strto* wrappers
 * iohandler.c simplification
 * Many other fixes and misc patches.
 
 And some MTTCG work (with Emilio's fixes squashed):
 * Signal-free TCG kick
 * Removing spinlock in favor of QemuMutex
 * User-mode emulation multi-threading fixes/docs
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQEcBAABCAAGBQJV8Tk7AAoJEL/70l94x66Ds3QH/3bi0RRR2NtKIXAQrGo5tfuD
 NPMu1K5Hy+/26AC6mEVNRh4kh7dPH5E4NnDGbxet1+osvmpjxAjc2JrxEybhHD0j
 fkpzqynuBN6cA2Gu5GUNoKzxxTmi2RrEYigWDZqCftRXBeO2Hsr1etxJh9UoZw5H
 dgpU3j/n0Q8s08jUJ1o789knZI/ckwL4oXK4u2KhSC7ZTCWhJT7Qr7c0JmiKReaF
 JEYAsKkQhICVKRVmC8NxML8U58O8maBjQ62UN6nQpVaQd0Yo/6cstFTZsRrHMHL3
 7A2Tyg862cMvp+1DOX3Bk02yXA+nxnzLF8kUe0rYo6llqDBDStzqyn1j9R0qeqA=
 =nB06
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* Support for jemalloc
* qemu_mutex_lock_iothread "No such process" fix
* cutils: qemu_strto* wrappers
* iohandler.c simplification
* Many other fixes and misc patches.

And some MTTCG work (with Emilio's fixes squashed):
* Signal-free TCG kick
* Removing spinlock in favor of QemuMutex
* User-mode emulation multi-threading fixes/docs

# gpg: Signature made Thu 10 Sep 2015 09:03:07 BST using RSA key ID 78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"

* remotes/bonzini/tags/for-upstream: (44 commits)
  cutils: work around platform differences in strto{l,ul,ll,ull}
  cpu-exec: fix lock hierarchy for user-mode emulation
  exec: make mmap_lock/mmap_unlock globally available
  tcg: comment on which functions have to be called with mmap_lock held
  tcg: add memory barriers in page_find_alloc accesses
  remove unused spinlock.
  replace spinlock by QemuMutex.
  cpus: remove tcg_halt_cond and tcg_cpu_thread globals
  cpus: protect work list with work_mutex
  scripts/dump-guest-memory.py: fix after RAMBlock change
  configure: Add support for jemalloc
  add macro file for coccinelle
  configure: factor out adding disas configure
  vhost-scsi: fix wrong vhost-scsi firmware path
  checkpatch: remove tests that are not relevant outside the kernel
  checkpatch: adapt some tests to QEMU
  CODING_STYLE: update mixed declaration rules
  qmp: Add example usage of strto*l() qemu wrapper
  cutils: Add qemu_strtoull() wrapper
  cutils: Add qemu_strtoll() wrapper
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-09-14 16:13:16 +01:00
Peter Maydell
30c38c90bd scripts/qemu-gdb: Add brief comment describing usage
Add a brief comment describing how to use the debug support
from GDB.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1439574392-4403-5-git-send-email-peter.maydell@linaro.org
2015-09-11 17:14:50 +01:00
Peter Maydell
5e3c72d41e scripts/qemu-gdb: Silently pass through SIGUSR1
SIGUSR1 is QEMU's IPI signal, and it gets sent a lot, so is
best silently passed through to the guest without stopping.
Make qemu-gdb.py do this bit of configuration for the user.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1439574392-4403-4-git-send-email-peter.maydell@linaro.org
2015-09-11 17:14:50 +01:00
Peter Maydell
191590f09d scripts/qemu-gdb: Split CoroutineCommand into its own file
Split the implementation of CoroutineCommand into its own file.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1439574392-4403-3-git-send-email-peter.maydell@linaro.org
2015-09-11 17:14:49 +01:00
Peter Maydell
93b1b365dc scripts/qemu-gdb: Split MtreeCommand into its own module
As we add more commands to our Python gdb debugging support, it's
going to get unwieldy to have everything in a single file. Split
the implementation of the 'mtree' command from qemu-gdb.py into
its own module.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1439574392-4403-2-git-send-email-peter.maydell@linaro.org
2015-09-11 17:14:49 +01:00
Michael S. Tsirkin
0c71d41e2a scripts/dump-guest-memory.py: fix after RAMBlock change
commit 9b8424d573
    "exec: split length -> used_length/max_length"
changed field names in struct RAMBlock

It turns out that scripts/dump-guest-memory.py was
poking at this field, update it accordingly.

Cc: qemu-stable@nongnu.org
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Message-Id: <1440666378-3152-1-git-send-email-mst@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-09-09 15:34:55 +02:00
Paolo Bonzini
3f7a899ff4 add macro file for coccinelle
Coccinelle chokes on some idioms from compiler.h and queue.h.
Extract those in a macro file, to be used with "--macro-file
scripts/cocci-macro-file.h".

Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-09-09 15:34:55 +02:00
Paolo Bonzini
f1e155bbf8 checkpatch: remove tests that are not relevant outside the kernel
Fully removing Sparse support requires more invasive changes.  Only
remove the really kernel-specific parts such as address space names.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-09-09 15:34:55 +02:00
Paolo Bonzini
71c47b01ca checkpatch: adapt some tests to QEMU
Mostly change severity levels, but some tests can also be adjusted to refer
to QEMU APIs or data structures.

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-09-09 15:34:55 +02:00
Peter Maydell
b597aa037d Monitor patches
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJV6YMbAAoJEDhwtADrkYZTmqUP/0PioppNHxOM+ah+9SoNBq+K
 jf6JLfxUkazPamtI1V3BYEhFgOMs+frQ/VG9mJjWdHeARHYePJce7arrUYV71L2+
 Z9hWBM9I71axRG449ox8qwk2NZRNwGPoF4gQeoOYUaiDsQIQVpX69lIiF2XkJ1+c
 v521rLmfprHcJW9Ggzpds45FLUv8t3tvNobuBZO2XAs2oXdesBgIIp2HjL8onFlu
 r1LLnZWxdROe9HM9HXvjWC5aWmgRK0kgAoaEWo9cQj5jMGBNYQDKH56eGx03IZkF
 29+AZ0ACI3b1OwlK7DK/YAYR78LR9z+TA+qf42+SPWLapImmH8qwrTN0vZpTs6GP
 caFGwklGyD6sQHiO5fmkSdxrES/W1ndshf8PfulgHQ4Wjb1rIcgCxl1gcH+zyT1G
 uBHJ7tozpdYZ78T60wY8dj46AidPX9Ru7+Y3D8fTncGuHJa/LacPp2AehA0lEbCK
 DGG5XFOJKIHBy0AXFPIyJ6pNPzU93jXGFf6pI5c15nMoCkh2wiNPWRM3ZXTrgWWc
 l7GZdwTewcm45UZVaITec+IJca7EjQIBvfUxs8yCaeBe6aEJQKw/P9lgvZPzt8Tk
 ozyCkQQIlnRupDb18p+MRD1/U2SZSjrTrlY/qNm/uBCyStjWKsyYFYvlcPFWSlOs
 BqE2BpLiUR2XDIuqe6E4
 =6HLt
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-monitor-2015-09-04' into staging

Monitor patches

# gpg: Signature made Fri 04 Sep 2015 12:40:11 BST using RSA key ID EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>"

* remotes/armbru/tags/pull-monitor-2015-09-04:
  hmp: add info iothreads command
  qmp-shell: add documentation

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-09-04 17:37:50 +01:00
Markus Armbruster
c4f498fe85 qapi: Generators crash when --output-dir isn't given, fix
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-04 15:47:16 +02:00
Markus Armbruster
eddf817bd8 qapi: Simplify error reporting for array types
check_type() first checks and peels off the array type, then checks
the element type.  For two out of four error messages, it takes pains
to report errors for "array of T" instead of just T.  Odd.  Let's
examine the errors.

* Unknown element type, e.g.
  tests/qapi-schema/args-array-unknown.json:

      Member 'array' of 'data' for command 'oops' uses unknown type
      'array of NoSuchType'

  To make sense of this, you need to know that 'array of NoSuchType'
  refers to '[NoSuchType]'.  Easy enough.  However, simply reporting

      Member 'array' of 'data' for command 'oops' uses unknown type
      'NoSuchType'

  is at least as easy to understand.

* Element type's meta-type is inadmissible, e.g.
  tests/qapi-schema/returns-whitelist.json:

      'returns' for command 'no-way-this-will-get-whitelisted' cannot
      use built-in type 'array of int'

  'array of int' is technically not a built-in type, but that's
  pedantry.  However, simply reporting

      'returns' for command 'no-way-this-will-get-whitelisted' cannot
      use built-in type 'int'

  avoids the issue, and is at least as easy to understand.

* The remaining two errors are unreachable, because the array checking
  ensures that value is a string.

Thus, reporting some errors for "array of T" instead of just T works,
but doesn't really improve things.  Drop it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-04 15:47:16 +02:00
Markus Armbruster
c6b71e5ae7 qapi: Fix errors for non-string, non-dictionary members
Fixes the errors demonstrated by the previous commit.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-04 15:47:16 +02:00
Markus Armbruster
65fbe12545 qapi: Drop one of two "simple union must not have base" checks
The first check ensures the second one can't trigger.  Drop the first
one, because the second one is in a more logical place, and emits a
nicer error message.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-04 15:47:16 +02:00
Markus Armbruster
3a864e7c52 qapi: Generated code cleanup
Clean up white-space, brace placement, and superfluous #ifdef
QAPI_TYPES_BUILTIN_CLEANUP_DEF.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-04 15:47:16 +02:00
Markus Armbruster
3f99144cd9 qapi-commands: Drop useless initialization
In generated command handlers, the assignment to retval dominates its
only use.  Therefore, its initialization is useless.  Drop it.

Suggested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-04 15:47:15 +02:00
Markus Armbruster
1f9a7a1a58 qapi-commands: Don't feed output of mcgen() to mcgen() again
Multiple passes through mcgen() is prone to produce unwanted blank
lines, which we then combat by sprinkling .rstrip() on top.  Just
don't do it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-04 15:47:15 +02:00
Markus Armbruster
e02bca281c qapi-commands: Inline gen_marshal_output_call()
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-04 15:47:15 +02:00
Markus Armbruster
8102307f51 qapi-commands: Fix gen_err_check(e) for e and e != 'local_err'
gen_err_check() hard-codes 'local_err' instead of substituting the
argument.  Currently harmless, since all callers pass either None or
'local_err'.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-04 15:47:15 +02:00
Markus Armbruster
9b090d42ae qapi: Command returning anonymous type doesn't work, outlaw
Reproducer: with

    { 'command': 'user_def_cmd4', 'returns': { 'a': 'int' } }

added to qapi-schema-test.json, qapi-commands.py dies when it tries to
generate the command handler function

    Traceback (most recent call last):
      File "/work/armbru/qemu/scripts/qapi-commands.py", line 359, in <module>
        ret = generate_command_decl(cmd['command'], arglist, ret_type) + "\n"
      File "/work/armbru/qemu/scripts/qapi-commands.py", line 29, in generate_command_decl
        ret_type=c_type(ret_type), name=c_name(name),
      File "/work/armbru/qemu/scripts/qapi.py", line 927, in c_type
        assert isinstance(value, str) and value != ""
    AssertionError

because the return type doesn't exist.

Simply outlaw this usage, and drop or dumb down test cases accordingly.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-04 15:47:15 +02:00
Markus Armbruster
315932b5ed qapi: Fix to reject union command and event arguments
A command's or event's 'data' must be a struct type, given either as a
dictionary, or as struct type name.

Commit dd883c6 tightened the checking there, but not enough: we still
accept 'union'.  Fix to reject it.

We may want to support union types there, but we'll have to extend
qapi-commands.py and qapi-events.py for it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-04 15:47:15 +02:00