Commit Graph

1833 Commits

Author SHA1 Message Date
Corey Minyard 24f976d30a ipmi: Add tests
Test the KCS interface with a local BMC and a BT interface with an
external BMC.

Signed-off-by: Corey Minyard <cminyard@mvista.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2015-12-22 18:39:19 +02:00
Peter Maydell c688084506 Merge QCryptoSecret object support
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABCAAGBQJWdDmJAAoJEL6G67QVEE/fJ4EP+gNC4ErBDpbg+I4RhLHv/FsF
 i2iEYkmBfFzUmiB8iSFlJY12XJ/CPnbrWks0WNHIoarUgtGuvfqH91KGkERxJQFE
 TkD65EOhNHWlP1zaI5r5ZMizhOdO6EPa0pbS/QH/UCy5qwu5IbG5EesOw00d7nL8
 5gT39ehXAFljutRygWsa1JyOkDB04WNehVsly/l2t9v16aQUDJOC5mWVuoInrDEo
 ye1VG2Cx7Y1/FRo3fFCDwzYD+8jgxIBAu8Igwjk/95VbfBVl769PBAilQRc5zMHt
 //eMNdul6GooVKmu/K1JWkmjIZJFUiboEMgPJElWV1y8bWmhh++4J6EVb55owoDk
 VRv84cqiaYErVb+56gaImr92GSKezll0APWz6YlDsFZgPClCPnUjDSr39+23t1h4
 LprirtbkAjw73T92kuQ7kzbXElWm7rSfcx5u1/S6YPP+EDzZpW9+h62lKGGnuS2M
 bzwFOOmWHe1MhbRSh+BOzGBf1wWhMSCKgLAOmPuRQ8slS91vfE66bIlqpIKBGgfn
 42t0wZCEW8bqIe8xry5pC5UoDfm3cVDhgGHGyMLWWDMez0qDchaAkWNkIDtc8Juv
 a1WqE/0lP/sVb36yLVANvt1/Qvpg6M3JwMbTjVaJl2eTDDtwho4PK+Chxx0a8BGl
 Z6oGj1rmvCDD2Dsi/EXI
 =g+9G
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/berrange/tags/pull-qcrypto-secrets-base-2015-12-18-1' into staging

Merge QCryptoSecret object support

# gpg: Signature made Fri 18 Dec 2015 16:51:21 GMT using RSA key ID 15104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>"
# gpg:                 aka "Daniel P. Berrange <berrange@redhat.com>"

* remotes/berrange/tags/pull-qcrypto-secrets-base-2015-12-18-1:
  crypto: add support for loading encrypted x509 keys
  crypto: add QCryptoSecret object class for password/key handling
  qga: convert to use error checked base64 decode
  qemu-char: convert to use error checked base64 decode
  util: add base64 decoding function

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-12-18 17:04:15 +00:00
Daniel P. Berrange ac1d887849 crypto: add QCryptoSecret object class for password/key handling
Introduce a new QCryptoSecret object class which will be used
for providing passwords and keys to other objects which need
sensitive credentials.

The new object can provide secret values directly as properties,
or indirectly via a file. The latter includes support for file
descriptor passing syntax on UNIX platforms. Ordinarily passing
secret values directly as properties is insecure, since they
are visible in process listings, or in log files showing the
CLI args / QMP commands. It is possible to use AES-256-CBC to
encrypt the secret values though, in which case all that is
visible is the ciphertext.  For ad hoc developer testing though,
it is fine to provide the secrets directly without encryption
so this is not explicitly forbidden.

The anticipated scenario is that libvirtd will create a random
master key per QEMU instance (eg /var/run/libvirt/qemu/$VMNAME.key)
and will use that key to encrypt all passwords it provides to
QEMU via '-object secret,....'.  This avoids the need for libvirt
(or other mgmt apps) to worry about file descriptor passing.

It also makes life easier for people who are scripting the
management of QEMU, for whom FD passing is significantly more
complex.

Providing data inline (insecure, only for ad hoc dev testing)

  $QEMU -object secret,id=sec0,data=letmein

Providing data indirectly in raw format

  printf "letmein" > mypasswd.txt
  $QEMU -object secret,id=sec0,file=mypasswd.txt

Providing data indirectly in base64 format

  $QEMU -object secret,id=sec0,file=mykey.b64,format=base64

Providing data with encryption

  $QEMU -object secret,id=master0,file=mykey.b64,format=base64 \
        -object secret,id=sec0,data=[base64 ciphertext],\
	           keyid=master0,iv=[base64 IV],format=base64

Note that 'format' here refers to the format of the ciphertext
data. The decrypted data must always be in raw byte format.

More examples are shown in the updated docs.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-12-18 16:25:08 +00:00
Daniel P. Berrange 89bc0b6cae util: add base64 decoding function
The standard glib provided g_base64_decode doesn't provide any
kind of sensible error checking on its input. Add a QEMU custom
wrapper qbase64_decode which can be used with untrustworthy
input that can contain invalid base64 characters, embedded
NUL characters, or not be NUL terminated at all.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-12-18 16:25:08 +00:00
John Snow a5002d5302 block/qapi: allow best-effort query
For more complex BDS trees that can be created under normal circumstances,
we lose the ability to issue query commands because of our inability to
re-construct the absolute filename.

Instead, omit this field when it is a problem and present as much information
as we can.

This will change the expected output in iotest 110, where we will now see a
json filename and the lack of an absolute filename instead of an error.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1450122916-4706-6-git-send-email-jsnow@redhat.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2015-12-18 14:36:17 +01:00
John Snow 12dcb1c018 block/qapi: always report full_backing_filename
Always report full_backing_filename, even if it's the same as
backing_filename. In the next patch, full_backing_filename may be
omitted if it cannot be generated instead of allowing e.g. drive_query
to abort if it runs into this scenario.

The presence or absence of the "full" field becomes useful information.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 1450122916-4706-3-git-send-email-jsnow@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2015-12-18 14:36:17 +01:00
Bo Tu a41aa71c15 qemu-iotests: s390x: fix test 068
Now, s390-virtio-ccw is default machine and s390-ccw.img is default boot
loader. If the s390-virtio-ccw machine finds no device to load from and
errors out, then emits a panic and exits the vm. This breaks test cases
068 for s390x.
Adding the parameter of "-no-shutdown" for s390-ccw-virtio will pause VM
before shutdown.

Acked-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Sascha Silbe <silbe@linux.vnet.ibm.com>
Signed-off-by: Bo Tu <tubo@linux.vnet.ibm.com>
Message-id: 1449136891-26850-4-git-send-email-tubo@linux.vnet.ibm.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2015-12-18 14:36:17 +01:00
Bo Tu 289f3ebae8 qemu-iotests: s390x: fix test 051
The tests for ide device should only be tested for the pc
platform.
Set device_id to "drive0", and replace every "-drive file..."
by "-drive file=...,if=none,id=$device_id", then x86 and s390x
can get the common output in the test of "Snapshot mode".
Warning message expected for s390x when drive without device.
A x86 platform specific output file is also needed.

Reviewed-by: Sascha Silbe <silbe@linux.vnet.ibm.com>
Signed-off-by: Bo Tu <tubo@linux.vnet.ibm.com>
Message-id: 1449136891-26850-3-git-send-email-tubo@linux.vnet.ibm.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2015-12-18 14:36:17 +01:00
Bo Tu 8a7607c2e2 qemu-iotests: refine common.config
Replacing awk with sed, then it's easier to read.
Replacing "[ ! -z "$default_alias_machine" ]" with
"[[ $default_alias_machine ]]", then it's slightly shorter.

Reviewed-by: Max Reitz <mreitz@redhat.com>
Suggested-By: Sascha Silbe <silbe@linux.vnet.ibm.com>
Reviewed-by: Sascha Silbe <silbe@linux.vnet.ibm.com>
Reviewed-by: Eric Blake   <eblake@redhat.com>
Signed-off-by: Bo Tu <tubo@linux.vnet.ibm.com>
Message-id: 1449136891-26850-2-git-send-email-tubo@linux.vnet.ibm.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2015-12-18 14:36:17 +01:00
Fam Zheng 8382ba6153 iotests: Update comments for bdrv_swap() in 094
Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-12-18 14:34:44 +01:00
Max Reitz e9dbdc5e46 iotests: Extend test 112 for qemu-img amend
Add tests for conversion between different refcount widths.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-12-18 14:34:43 +01:00
Max Reitz 29d72431ef qcow2: Use error_report() in qcow2_amend_options()
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-12-18 14:34:43 +01:00
Kevin Wolf 0a8111e0fb qemu-iotests: Test reopen with node-name/driver options
'node-name' and 'driver' should not be changed during a reopen
operation. It is, however, valid to specify them with the same value as
they already had.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2015-12-18 14:34:43 +01:00
Kevin Wolf 8f7acbe6ea qemu-iotests: Test cache mode option inheritance
This is doing a more complete test on setting cache modes both while
opening an image (i.e. in a -drive command line) and in reopen
situations. It checks that reopen can specify options for child nodes
and that cache modes are correctly inherited from parent nodes where
they are not specified.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2015-12-18 14:34:43 +01:00
Kevin Wolf 768ee459f5 qemu-iotests: Try setting cache mode for children
This is a basic test for specifying cache modes for child nodes on the
command line. It doesn't take much time and works without O_DIRECT
support.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2015-12-18 14:34:43 +01:00
Kevin Wolf fc17c25931 qemu-iotests: Remove cache mode test without medium
Specifying the cache mode for a driver without a medium is not a useful
thing to do: As long as there is no medium, the cache mode doesn't make
a difference, and once the 'change' command is used to insert a medium,
it ignores the old cache mode and makes the new medium use
cache=writethrough.

Later patches will make it an error to specify the cache mode for an
empty drive. Remove the corresponding test case.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2015-12-18 14:34:43 +01:00
Kevin Wolf 39c4ae941e blockdev: Set 'format' indicates non-empty drive
Creating an empty drive while specifying 'format' doesn't make sense.
The specified format driver would simply be ignored.

Make a set 'format' option an indication that a non-empty drive should
be created. This makes 'format' consistent with 'driver' and allows
using it with a block driver that doesn't need any other options (like
null-co/null-aio).

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2015-12-18 14:34:43 +01:00
John Snow 35cea22373 iotests: 124: don't reopen qcow2
Don't create two interfaces to the same drive in the recently moved
failure test.

Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-12-18 14:34:42 +01:00
John Snow ce2cbc4910 iotests: 124: move incremental failure test
Code motion only, in preparation for adjusting
the setUp procedure for this test.

Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-12-18 14:34:42 +01:00
John Snow 1b19bb9d17 iotests: 124: Split into two test classes
Split it into an abstract test class and an implementation class.

The split is primarily to facilitate more flexible setUp variations
for other kinds of tests without having to rewrite or shuffle around
all of these helpers.

See the following two patches for more of the "why."

Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-12-18 14:34:42 +01:00
Daniel P. Berrange d98e4eb7de io: add QIOChannelBuffer class
Add a QIOChannel subclass that is capable of performing I/O
to/from a memory buffer. This implementation does not attempt
to support concurrent readers & writers. It is designed for
serialized access where by a single thread at a time may write
data, seek and then read data back out.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-12-18 12:18:31 +00:00
Daniel P. Berrange 195e14d026 io: add QIOChannelCommand class
Add a QIOChannel subclass that is capable of performing I/O
to/from a separate process, via a pair of pipes. The command
can be used for unidirectional or bi-directional I/O.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-12-18 12:18:31 +00:00
Daniel P. Berrange ed8ee42c40 io: add QIOChannelTLS class
Add a QIOChannel subclass that can run the TLS protocol over
the top of another QIOChannel instance. The object provides a
simplified API to perform the handshake when starting the TLS
session. The layering of TLS over the underlying channel does
not have to be setup immediately. It is possible to take an
existing QIOChannel that has done some handshake and then swap
in the QIOChannelTLS layer. This allows for use with protocols
which start TLS right away, and those which start plain text
and then negotiate TLS.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-12-18 12:18:31 +00:00
Daniel P. Berrange d6e48869a4 io: add QIOChannelFile class
Add a QIOChannel subclass that is capable of operating on things
that are files, such as plain files, pipes, character/block
devices, but notably not sockets.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-12-18 12:18:31 +00:00
Daniel P. Berrange 559607ea17 io: add QIOChannelSocket class
Implement a QIOChannel subclass that supports sockets I/O.
The implementation is able to manage a single socket file
descriptor, whether a TCP/UNIX listener, TCP/UNIX connection,
or a UDP datagram. It provides APIs which can listen and
connect either asynchronously or synchronously. Since there
is no asynchronous DNS lookup API available, it uses the
QIOTask helper for spawning a background thread to ensure
non-blocking operation.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-12-18 12:18:31 +00:00
Daniel P. Berrange b02db2d920 io: add QIOTask class for async operations
A number of I/O operations need to be performed asynchronously
to avoid blocking the main loop. The caller of such APIs need
to provide a callback to be invoked on completion/error and
need access to the error, if any. The small QIOTask provides
a simple framework for dealing with such probes. The API
docs inline provide an outline of how this is to be used.

Some functions don't have the ability to run asynchronously
(eg getaddrinfo always blocks), so to facilitate their use,
the task class provides a mechanism to run a blocking
function in a thread, while triggering the completion
callback in the main event loop thread. This easily allows
any synchronous function to be made asynchronous, albeit
at the cost of spawning a thread.

In this series, the QIOTask class will be used for things like
the TLS handshake, the websockets handshake and TCP connect()
progress.

The concept of QIOTask is inspired by the GAsyncResult
interface / GTask class in the GIO libraries. The min
version requirements on glib don't allow those to be
used from QEMU, so QIOTask provides a facsimilie which
can be easily switched to GTask in the future if the
min version is increased.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-12-18 12:18:30 +00:00
Alex Bennée 261f4d6d3e tests/guest-debug: introduce basic gdbstub tests
The aim of these tests is to combine with an appropriate kernel
image (with symbol-file vmlinux) and check it behaves as it should.
Given a kernel it checks:

  - single step
  - software breakpoint
  - hardware breakpoint
  - access, read and write watchpoints

On success it returns 0 to the calling process.

I've not plumbed this into the "make check" logic though as we need a
solution for providing non-host binaries to the tests. However the test
is structured to work with pretty much any Linux kernel image as it
uses the basic kernel_init code which is common across architectures.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 1449599553-24713-7-git-send-email-alex.bennee@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-12-17 13:37:15 +00:00
Eric Blake bac5429ccb qapi: Detect base class loops
It should be fairly obvious that qapi base classes need to
form an acyclic graph, since QMP cannot specify the same
key more than once, while base classes are included as flat
members alongside other members added by the child.  But the
old check_member_clash() parser function was not prepared to
check for this, and entered an infinite recursion (at least
until Python gives up, complaining about nesting too deep).

Now that check_member_clash() has been recently removed,
attempts at self-inheritance trigger an assertion failure
introduced by commit ac88219a.  The obvious fix is to turn
the assertion into a conditional.

This patch includes both the tests (base-cycle-direct and
base-cycle-indirect) and the fix, since the .err file output
for the unfixed case is not useful (particularly when it was
warning about unbounded recursion, as that limit may be
platform-specific).

We don't need to worry about cycles in flat unions (neither
the base type nor the type of a variant can be a union) nor
in alternates (alternate branches cannot themselves be an
alternate).  But if we later allow a union type as a variant,
we will still be okay, as QAPISchemaObjectTypeVariants.check()
triggers the same QAPISchemaObjectType.check() that will
detect any loops.

Likewise, we need not worry about the case of diamond
inheritance where the same class is used for a flat union base
class and one of its variants; either both uses will introduce
a collision in trying to insert the same member name twice, or
the shared type is empty and changes nothing.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-16-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-17 08:21:29 +01:00
Eric Blake 01cfbaa4c3 qapi: Move duplicate collision checks to schema check()
With the recent commit 'qapi: Detect collisions in C member
names', we have two different locations for detecting clashes -
one at parse time, and another at QAPISchema*.check() time.
Remove all of the ad hoc parser checks, and delete associated
code (for example, the global check_member_clash() method is
no longer needed).

Testing this showed that the test union-bad-branch wasn't adding
much: union-clash-branches also exposes the error message when
branches collide, and we've recently fixed things to avoid an
implicit collision with max.  Likewise, the error for
enum-clash-member changes to report our new detection of
upper case in a value name, unless we modify the test to use
all lower case.

The wording of several error messages has changed, but the
change is generally an improvement rather than a regression.

No change to generated code.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-15-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-17 08:21:29 +01:00
Eric Blake 893e1f2c51 qapi: Enforce (or whitelist) case conventions on qapi members
We document that members of enums and objects should be
'lower-case', although we were not enforcing it.  We have to
whitelist a few pre-existing entities that violate the norms.
Add three new tests to expose the new error message, each of
which first uses the whitelisted name 'UuidInfo' to prove the
whitelist works, then triggers the failure (this is the same
pattern used in the existing returns-whitelist.json test).

Note that by adding this check, we have effectively forbidden
an entity with a case-insensitive clash of member names, for
any entity that is not on the whitelist (although there is
still the possibility to clash via '-' vs. '_').

Not done here: a future patch should also add naming convention
support and whitelist exceptions for command, event, and type
names.

The additions to QAPISchemaMember.check_clash() check whether
info['name'] is in the whitelist (the top-most entity name at
the point 'info' tracks), rather than self.owner (the type,
possibly implicit, that directly owns the member), because it
is easier to maintain the whitelist by the names actually in
the user's .json file, rather than worrying about the names
of implicit types.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-14-git-send-email-eblake@redhat.com>
[Simplified a bit as per discussion with Eric]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-17 08:21:29 +01:00
Eric Blake d00341af38 qapi: Fix alternates that accept 'number' but not 'int'
The QMP input visitor allows integral values to be assigned by
promotion to a QTYPE_QFLOAT.  However, when parsing an alternate,
we did not take this into account, such that an alternate that
accepts 'number' and some other type, but not 'int', would reject
integral values.

With this patch, we now have the following desirable table:

    alternate has      case selected for
    'int'  'number'    QTYPE_QINT  QTYPE_QFLOAT
      no        no     error       error
      no       yes     'number'    'number'
     yes        no     'int'       error
     yes       yes     'int'       'number'

While it is unlikely that we will ever use 'number' in an
alternate other than in the testsuite, it never hurts to be
more precise in what we allow.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-8-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-17 08:21:28 +01:00
Eric Blake 0426d53c65 qapi: Simplify visiting of alternate types
Previously, working with alternates required two lookup arrays
and some indirection: for type Foo, we created Foo_qtypes[]
which maps each qtype to a value of the generated FooKind enum,
then look up that value in FooKind_lookup[] like we do for other
union types.

This has a couple of subtle bugs.  First, the generator was
creating a call with a parameter '(int *) &(*obj)->type' where
type is an enum type; this is unsafe if the compiler chooses
to store the enum type in a different size than int, where
assigning through the wrong size pointer can corrupt data or
cause a SIGBUS.

Related bug, not not fixed in this patch: qapi-visit.py's
gen_visit_enum() generates a cast of its enum * argument to
int *. Marked FIXME.

Second, since the values of the FooKind enum start at zero, all
entries of the Foo_qtypes[] array that were not explicitly
initialized will map to the same branch of the union as the
first member of the alternate, rather than triggering a desired
failure in visit_get_next_type().  Fortunately, the bug seldom
bites; the very next thing the input visitor does is try to
parse the incoming JSON with the wrong parser, which normally
fails; the output visitor is not used with a C struct in that
state, and the dealloc visitor has nothing to clean up (so
there is no leak).

However, the second bug IS observable in one case: parsing an
integer causes unusual behavior in an alternate that contains
at least a 'number' member but no 'int' member, because the
'number' parser accepts QTYPE_QINT in addition to the expected
QTYPE_QFLOAT (that is, since 'int' is not a member, the type
QTYPE_QINT accidentally maps to FooKind 0; if this enum value
is the 'number' branch the integer parses successfully, but if
the 'number' branch is not first, some other branch tries to
parse the integer and rejects it).  A later patch will worry
about fixing alternates to always parse all inputs that a
non-alternate 'number' would accept, for now this is still
marked FIXME in the updated test-qmp-input-visitor.c, to
merely point out that new undesired behavior of 'ans' matches
the existing undesired behavior of 'asn'.

This patch fixes the default-initialization bug by deleting the
indirection, and modifying get_next_type() to directly assign a
QTypeCode parameter.  This in turn fixes the type-casting bug,
as we are no longer casting a pointer to enum to a questionable
size. There is no longer a need to generate an implicit FooKind
enum associated with the alternate type (since the QMP wire
format never uses the stringized counterparts of the C union
member names).  Since the updated visit_get_next_type() does not
know which qtypes are expected, the generated visitor is
modified to generate an error statement if an unexpected type is
encountered.

Callers now have to know the QTYPE_* mapping when looking at the
discriminator; but so far, only the testsuite was even using the
C struct of an alternate types.  I considered the possibility of
keeping the internal enum FooKind, but initialized differently
than most generated arrays, as in:
  typedef enum FooKind {
      FOO_KIND_A = QTYPE_QDICT,
      FOO_KIND_B = QTYPE_QINT,
  } FooKind;
to create nicer aliases for knowing when to use foo->a or foo->b
when inspecting foo->type; but it turned out to add too much
complexity, especially without a client.

There is a user-visible side effect to this change, but I
consider it to be an improvement. Previously,
the invalid QMP command:
  {"execute":"blockdev-add", "arguments":{"options":
    {"driver":"raw", "id":"a", "file":true}}}
failed with:
  {"error": {"class": "GenericError",
    "desc": "Invalid parameter type for 'file', expected: QDict"}}
(visit_get_next_type() succeeded, and the error comes from the
visit_type_BlockdevOptions() expecting {}; there is no mention of
the fact that a string would also work).  Now it fails with:
  {"error": {"class": "GenericError",
    "desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
(the error when the next type doesn't match any expected types for
the overall alternate).

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-5-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-17 08:21:28 +01:00
Eric Blake 7264f5c50c qapi: Convert QType into QAPI built-in enum type
What's more meta than using qapi to define qapi? :)

Convert QType into a full-fledged[*] builtin qapi enum type, so
that a subsequent patch can then use it as the discriminator
type of qapi alternate types.  Fortunately, the judicious use of
'prefix' in the qapi definition avoids churn to the spelling of
the enum constants.

To avoid circular definitions, we have to flip the order of
inclusion between "qobject.h" vs. "qapi-types.h".  Back in commit
28770e0, we had the latter include the former, so that we could
use 'QObject *' for our implementation of 'any'.  But that usage
also works with only a forward declaration, whereas the
definition of QObject requires QType to be a complete type.

[*] The type has to be builtin, rather than declared in
qapi/common.json, because we want to use it for alternates even
when common.json is not included. But since it is the first
builtin enum type, we have to add special cases to qapi-types
and qapi-visit to only emit definitions once, even when two
qapi files are being compiled into the same binary (the way we
already handled builtin list types like 'intList').  We may
need to revisit how multiple qapi files share common types,
but that's a project for another day.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-4-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-17 08:21:28 +01:00
Eric Blake 04e0639d4e qapi: Remove obsolete tests for MAX collision
Now that we no longer collide with an implicit _MAX enum member,
we no longer need to reject it in the ad hoc parser, and can
remove several tests that are no longer needed.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-24-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-17 08:21:28 +01:00
Eric Blake 7fb1cf1606 qapi: Don't let implicit enum MAX member collide
Now that we guarantee the user doesn't have any enum values
beginning with a single underscore, we can use that for our
own purposes.  Renaming ENUM_MAX to ENUM__MAX makes it obvious
that the sentinel is generated.

This patch was mostly generated by applying a temporary patch:

|diff --git a/scripts/qapi.py b/scripts/qapi.py
|index e6d014b..b862ec9 100644
|--- a/scripts/qapi.py
|+++ b/scripts/qapi.py
|@@ -1570,6 +1570,7 @@ const char *const %(c_name)s_lookup[] = {
|     max_index = c_enum_const(name, 'MAX', prefix)
|     ret += mcgen('''
|     [%(max_index)s] = NULL,
|+// %(max_index)s
| };
| ''',
|                max_index=max_index)

then running:

$ cat qapi-{types,event}.c tests/test-qapi-types.c |
    sed -n 's,^// \(.*\)MAX,s|\1MAX|\1_MAX|g,p' > list
$ git grep -l _MAX | xargs sed -i -f list

The only things not generated are the changes in scripts/qapi.py.

Rejecting enum members named 'MAX' is now useless, and will be dropped
in the next patch.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-23-git-send-email-eblake@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
[Rebased to current master, commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-17 08:21:28 +01:00
Eric Blake 59a92feedc qapi: Tighten the regex on valid names
We already documented that qapi names should match specific
patterns (such as starting with a letter unless it was an enum
value or a downstream extension).  Tighten that from a suggestion
into a hard requirement, which frees up names beginning with a
single underscore for qapi internal usage.

The tighter regex doesn't forbid everything insane that a user
could provide (for example, a user could name a type 'Foo-lookup'
to collide with the generated 'Foo_lookup[]' for an enum 'Foo'),
but does a good job at protecting the most obvious uses, and
also happens to reserve single leading underscore for later use.

The handling of enum values starting with a digit is tricky:
commit 9fb081e introduced a subtle bug by using c_name() on
a munged value, which would allow an enum to include the
member 'q-int' in spite of our reservation.  Furthermore,
munging with a leading '_' would fail our tighter regex.  So
fix it by only munging for leading digits (which are never
ticklish in c_name()) and by using a different prefix (I
picked 'D', although any letter should do).

Add new tests, reserved-member-underscore and reserved-enum-q,
to demonstrate the tighter checking.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-22-git-send-email-eblake@redhat.com>
Message-Id: <1447883135-18020-1-git-send-email-eblake@redhat.com>
[Eric's fixup squashed in]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-17 08:21:27 +01:00
Eric Blake 5be5b7764f blkdebug: Avoid '.' in enum values
Our qapi conventions document that '.' should only be used in
the prefix of downstream names.  BlkdebugEvent was a lone
exception to this.  Changing this is not backwards compatible
to the 'blockdev-add' QMP command; however, that command is
not yet fully stable.  It can also be argued that the testsuite
is the biggest user of blkdebug, and that any other user can
be taught to deal with the change by paying attention to
introspection results.

Done with:

$ for str in \
     l1_grow.{alloc,write,activate}_table \
     l2_alloc.{cow_read,write} \
     refblock_alloc.{hookup,write,write_blocks,write_table,switch_table} \
     pwritev_rmw.{head,after_head,tail,after_tail}; do
   str1=$(echo "$str" | sed 's/\./\\./')
   str2=$(echo "$str" | sed 's/\./_/')
   git grep -l "$str1" | xargs -r sed -i "s/$str1/$str2/g"
 done

followed by a manual touchup to test 77 to keep the test working.

Reported-by: Markus Armbruster <armbru@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-21-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-17 08:21:27 +01:00
Eric Blake c43567c120 qapi: Fix c_name() munging
The method c_name() is supposed to do two different actions: munge
'-' into '_', and add a 'q_' prefix to ticklish names.  But it did
these steps out of order, making it possible to submit input that
is not ticklish until after munging, where the output then lacked
the desired prefix.

The failure is exposed easily if you have a compiler that recognizes
C11 keywords, and try to name a member '_Thread-local', as it would
result in trying to compile the declaration 'uint64_t _Thread_local;'
which is not valid.  However, this name violates our conventions
(ultimately, want to enforce that no qapi names start with single
underscore), so the test is slightly weaker by instead testing
'wchar-t'; the declaration 'uint64_t wchar_t;' is valid in C (where
wchar_t is only a typedef) but would fail with a C++ compiler (where
it is a keyword).

Fix things by reversing the order of actions within c_name().

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-18-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-17 08:21:27 +01:00
Eric Blake 27b60ab93b qapi: Detect collisions in C member names
Detect attempts to declare two object members that would result
in the same C member name, by keying the 'seen' dictionary off
of the C name rather than the qapi name.  It also requires passing
info through the check_clash() methods.

This addresses a TODO and fixes the previously-broken
args-name-clash test.  The resulting error message demonstrates
the utility of the .describe() method added previously.  No change
to generated code.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-17-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-17 08:21:27 +01:00
Eric Blake 61a946611b qapi: Remove outdated tests related to QMP/branch collisions
Now that branches are in a separate C namespace, we can remove
the restrictions in the parser that claim a branch name would
collide with QMP, and delete the negative tests that are no
longer problematic.  A separate patch can then add positive
tests to qapi-schema-test to test that any corner cases will
compile correctly.

This reverts the scripts/qapi.py portion of commit 7b2a5c2,
now that the assertions that it plugged are no longer possible.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-15-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-17 08:21:27 +01:00
Eric Blake da34a9bd99 qapi: Track simple union tag in object.local_members
We were previously creating all unions with an empty list for
local_members.  However, it will make it easier to unify struct
and union generation if we include the generated tag member in
local_members.  That way, we can have a common code pattern:
visit the base (if any), visit the local members (if any), visit
the variants (if any).  The local_members of a flat union
remains empty (because the discriminator is already visited as
part of the base).  Then, by visiting tag_member.check() during
AlternateType.check(), we no longer need to call it during
Variants.check().

The various front end entities now exist as follows:
struct: optional base, optional local_members, no variants
simple union: no base, one-element local_members, variants with tag_member
  from local_members
flat union: base, no local_members, variants with tag_member from base
alternate: no base, no local_members, variants

With the new local members, we require a bit of finesse to
avoid assertions in the clients.

No change to generated code.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-2-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-17 08:21:26 +01:00
Max Reitz 6e0abc251d blockdev: Mark {insert, remove}-medium experimental
While in the long term we want throttling to be its own block filter
BDS, in the short term we want it to be part of the BB instead of a BDS;
even in the long term we may want legacy throttling to be automatically
tied to the BB.

blockdev-insert-medium and blockdev-remove-medium do not retain
throttling information in the BB (deliberately so). Therefore, using
them means tying this information to a BDS, which would break the model
described above. (The same applies to other flags such as
detect_zeroes.) We probably want to move this information to the BB or
its own filter BDS before blockdev-{insert,remove}-medium can be
considered completely stable.

Therefore, mark these functions experimental for the time being.

Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 1449847385-13986-2-git-send-email-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
[PMM: fixed format nit (underlining) in qmp-commands.hx]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-12-11 15:39:29 +00:00
Marc-André Lureau 0d2cd785ef qom-test: Fix qmp() leaks
Before this patch ASAN reported:
SUMMARY: AddressSanitizer: 677165875 byte(s) leaked in 1272437 allocation(s)

After this patch:
SUMMARY: AddressSanitizer: 465 byte(s) leaked in 32 allocation(s)

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <1448551895-871-1-git-send-email-marcandre.lureau@redhat.com>
[Straightforwardly rebased onto the previous patch]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Andreas Färber <afaerber@suse.de>
2015-12-04 18:29:31 +01:00
Markus Armbruster 041088c719 tests: Use proper functions types instead of void (*fn)
We have several function parameters declared as void (*fn).  This is
just a stupid way to write void *, and the only purpose writing it
like that could serve is obscuring the sin of bypassing the type
system without need.

The original sin is commit 49ee359: its qtest_add_func() is a wrapper
for g_test_add_func().  Fix the parameter type to match
g_test_add_func()'s.  This uncovers type errors in ide-test.c; fix
them.

Commit 7949c0e faithfully repeated the sin for qtest_add_data_func().
Fix it the same way, along with a harmless type error uncovered in
vhost-user-test.c.

Commit 063c23d repeated it for qtest_add_abrt_handler().  The screwy
parameter gets assigned to GHook member func, so change its type to
match.  Requires wrapping kill_qemu() to keep the type checker happy.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[AF/armbru: Inline GTestFunc/GTestDataFunc typedef for old GLib]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Andreas Färber <afaerber@suse.de>
2015-12-04 18:25:42 +01:00
Andreas Färber b5e62af8aa tests: Fix check-report-qtest-% target
Commit e253c28 ("tests: Fix how qom-test is run") introduced
$(qtest-generic-y) and used it for check-qtest-% target, but did not
update check-report-qtest-%. This causes check-report-qtest-aarch64.xml
target to fail with a gtester usage error for lack of test arguments.

Fix this by adding $(qtest-generic-y) in check-report-qtest-%.
Also add it in check-clean target, spotted by Markus.

Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Andreas Färber <afaerber@suse.de>
2015-12-03 20:07:05 +01:00
Peter Maydell efdeb96c5a -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
 
 iQEcBAABAgAGBQJWX8xEAAoJEJykq7OBq3PIY+YH/0fhpy2S0G2LtfjYX3522Q81
 3SsIC+4934+SkGLkQflnNsy8HPKvqQndkV+5FEqbzUTwJ1kjixyKapfpLyA0tvbm
 +uxvC1Mn91nVfqlfh3zwGOqprcEwPvXtfIyeOlfeq+6m72fDLWUakIwzAfNGWeV8
 REc3j2yTEw2esRPKau5kP1q3taN7w6UvIx9I8g1/cbnq89ca0ici/+AhBz/XGGb1
 gTlxtBoVWfH+k0kO2rqhZt+RLi7u22cmtPOywOTuqIt3HEYgFdaaf6S0UcZ+mgno
 CNf0hhbHuMhBCgWvXcGXDssFrI2GoSl8hEuTBbLHyOSFSHZ8pfRLFmoFuZzXW0c=
 =TtA6
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

# gpg: Signature made Thu 03 Dec 2015 04:59:48 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"

* remotes/stefanha/tags/block-pull-request:
  iotests: Add regresion test case for write notifier assertion failure
  iotests: Add "add_drive_raw" method
  block: Don't wait serialising for non-COR read requests
  iothread: include id in thread name

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-12-03 11:08:43 +00:00
Fam Zheng 9cc0f19de2 iotests: Add regresion test case for write notifier assertion failure
The idea is to let the top level bs have a big request alignment with
blkdebug, so that the aio_write request issued from monitor will be
serialised. This tests that QEMU doesn't crash upon the read request
from the backup job's write notifier, which is a very special case of
"reentrant" request.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-id: 1448962590-2842-4-git-send-email-famz@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2015-12-03 11:08:07 +08:00
Fam Zheng 78b666f46b iotests: Add "add_drive_raw" method
This offers full manual control over the "-drive" options.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-id: 1448962590-2842-3-git-send-email-famz@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2015-12-03 11:08:07 +08:00
Victor Kaplansky 6d0b908a62 tests/vhost-user-bridge.c: fix fd leakage
This fixes file descriptor leakage in vhost-user-bridge
application. Whenever a new callfd or kickfd is set, the previous
one should be explicitly closed. File descriptors used to map
guest's memory are closed immediately after mmap call.

Signed-off-by: Victor Kaplansky <victork@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2015-12-02 19:27:26 +02:00
Marc-André Lureau 45ce512670 vhost-user-test: fix crash with glib < 2.36
The prepare callback needs to be implemented with glib < 2.36,
quoting glib documentation:
"Since 2.36 this may be NULL, in which case the effect is as if the
function always returns FALSE with a timeout of -1."

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2015-12-02 16:42:26 +02:00