Commit Graph

1845 Commits

Author SHA1 Message Date
Alex Bennée 93a1100768 docs: move CODING_STYLE into the developer documentation
There is no particular reason to keep this on it's own in the root of
the tree. Move it into the rest of the fine developer manual and fixup
any links to it. The only tweak I've made is to fix the code-block
annotations to mention the language C.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Claudio Fontana <cfontana@suse.de>
Message-Id: <20210223095931.16908-1-alex.bennee@linaro.org>
2021-02-24 11:05:21 +00:00
Peter Maydell e90ef02389 QAPI patches patches for 2021-02-18
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmAutzcSHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZTvBsP/jI3Dh9j8HLLsvauU117LcIGnn5Qwog4
 8v0IrEieaOeMDNDzR8AgSqBLYm0ZQcBoZl7Qkv8XMxYa7iWiJv2uCShRS7MNl5NK
 M8IW0XWVjCJJ7TfFGj5iP/zlnhBMTzp66EDfkdiBWlig7NdZ+TBVp+WiZr8mT7EJ
 MdI2slzmgChDHJ2BLntLDt3w778GUIIXYxNFAhH4woAs6Xq7Z8tq3y+7c9JMXL/1
 B1esVN49h582mcldT5LcMK1AWOjxon+IaKS5KloMcr/uVCqaineSR/rPTXI4k9RL
 KfKeetniOz0ohyD+4dfLYW7OPhjJ3ZiUm6tSP6aVcg8D8qJAEvocw1uZS3iFpR5T
 lqdS6rcaumx3FaPix3MmhbNJLytohiAvvJyHpjLQ32ByhJ2FPzUOLFyRvoeoE+kD
 wJSTLn0iyjIlY3bz4sdA3STWK9TyKJ8enyRWGGZECL0ZJwCUe/Z7KwxZi7Ad4vLl
 j/os65T+XudTEJzk4iKpA03ThvAs82q34mpSkpACfDA9VZ584fRC79WbQoVAGmrk
 3CV4vfBS3pcPz7nCQqauTGWg3iqQT4iKYGdw9YTqA88kIk/swfPiucGrAwI1WPXz
 VReMbz/LBBa8gLb2Sx4MMlRjO/v/WlOyxJ1sfdETmiX+ZjUl8s7+94PwlWI2kT6k
 3r9bl3Tz+QlZ
 =pIh0
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2021-02-18' into staging

QAPI patches patches for 2021-02-18

# gpg: Signature made Thu 18 Feb 2021 18:51:35 GMT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2021-02-18:
  qapi/introspect.py: set _gen_tree's default ifcond argument to ()
  qapi/introspect.py: Type _gen_tree variants as Sequence[str]
  qapi/introspect.py: Update copyright and authors list
  qapi/introspect.py: Add docstrings to _gen_tree and _tree_to_qlit
  qapi/introspect.py: add type hint annotations
  qapi/introspect.py: remove _gen_variants helper
  qapi/introspect.py: improve readability of _tree_to_qlit
  qapi/introspect.py: improve _tree_to_qlit error message
  qapi/introspect.py: create a typed 'Annotated' data strutcure
  qapi/introspect.py: Introduce preliminary tree typing
  qapi/introspect.py: Always define all 'extra' dict keys
  qapi/introspect.py: replace 'extra' dict with 'comment' argument
  qapi/introspect.py: Unify return type of _make_tree()
  qapi/introspect.py: guard against ifcond/comment misuse
  qapi/introspect.py: add _gen_features helper
  qapi/introspect.py: use _make_tree for features nodes
  qapi/introspect.py: assert schema is not None
  qapi: Replace List[str] with Sequence[str] for ifcond

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-19 17:22:42 +00:00
Peter Maydell ce42fe17ad * Always build the container images in the gitlab-CI
* Some other small gitlab-CI improvements
 * Some qtest fixes
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmAvVkwRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbX6JRAAmpDk8N3JP8XxG4ZVBbAquIRr87CgVykO
 Si4bQtoy6UGcht1aFM2QnfoU54lxVBArSjbqsV1soqHjC5mLl1t7FaHS98hOx7t4
 4x3oejxyiXhh9U+6AEzuSAlXLXYUs4eqpQ7f7AHrhqnj/pLea/dQFzal6TT2A8qf
 wYhUyPTVnVKrVPR5b0wlHtLJgpNa4A1McCDsD68NY/5uKTlgfteNln1XKBMJwokL
 LoZm8iFvLEdzPL0ikslrUmbbaYJvKiedSy9ZEYbuFllY2aVpkKPuN49ZWh35IJSA
 GkiXpbirPMIjydGrq68qWd/9p4i4ltX9LfKgpcDua1NptbK2LvYe7FaeyPBYoSFP
 wsXP497vwt9g3llTzbrEHX5ZeAivW/JqemX60pF61p0Lh1UnBA+30gHQVwYgvQZa
 HUCeNMrK3dMljfHOBrWLX84lfzBi3BUQhYVzyx1kTcOAEotgxnuKH7adWAvqXlpE
 x2Ff4hDQj+xL2mbbHWNGRHBS33SNNIKIvH+kMtqzzHteDp6jROw83+HlNMw38Zl5
 s0CwiA5Wk2vAW7gFZ0FTan+a9TxpotHVgRZQZ8TLk15UTFMWoqQfBz6OdsPz38MG
 5PjouX6K4ZR9uHZiOggsjN5XZkmJro0S3Lb9mWVCXLZp0XF0e9lCw/xXiUznsXu2
 QZXcSCmiMkw=
 =SDf2
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2021-02-19' into staging

* Always build the container images in the gitlab-CI
* Some other small gitlab-CI improvements
* Some qtest fixes

# gpg: Signature made Fri 19 Feb 2021 06:10:20 GMT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2021-02-19:
  travis.yml: Limit simultaneous jobs to 3
  gitlab-ci.yml: Run check-tcg with TCI
  tests/qtest/boot-sector: Check that the guest did not panic
  gitlab-ci: Disable vhost-kernel in build-disable job
  scripts/checkpatch: Improve the check for authors mangled by the mailing list
  gitlab-ci: Display Avocado log content when tests timeout
  gitlab: fix inconsistent indentation
  gitlab: add fine grained job deps for all build jobs
  gitlab: always build container images
  tests/qtest/boot-serial-test: Test Virt machine with 'max'

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-19 14:54:52 +00:00
Thomas Huth 7b59d10e23 scripts/checkpatch: Improve the check for authors mangled by the mailing list
There were recently some patches on the list which had their "From:"
line mangled like this:

 From: qemu_oss--- via <qemu-devel@nongnu.org>

Since our test in the checkpatch.pl script did not trigger here, the
patches finally also ended up in a pull request, with the wrong author
set. So let's improve the regular expression to also complain on
these new patterns, too.

Message-Id: <20210216071512.1199827-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-19 06:29:05 +01:00
Eduardo Habkost 1a14d4e16a device-crash-test: Remove problematic language
Replace "whitelist" in the device-crash-test script with
"rule list".

I'm using "rule list" instead of "allow list" or "pass list"
because the list is not used only for expected/allowed errors.
It also contain rules specifying which errors shouldn't be
ignored and/or should be fatal.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210202191207.4103973-1-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2021-02-18 18:34:47 -05:00
John Snow 9b77d94699 qapi/introspect.py: set _gen_tree's default ifcond argument to ()
We don't need to create an empty, mutable list to pass to _gen_tree;
since it is now typed as a Sequence, we can use the empty tuple as a
default and omit the argument.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-19-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:51:14 +01:00
John Snow cea53c31b7 qapi/introspect.py: Type _gen_tree variants as Sequence[str]
Optional[List] is clunky; an empty sequence can more elegantly convey
"no variants". By downgrading "List" to "Sequence", we can also accept
tuples; this is useful for the empty tuple specifically, which we may
use as a default parameter because it is immutable.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-18-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Doc string touched up]
Reviewed-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:51:08 +01:00
John Snow cf26906c4a qapi/introspect.py: Update copyright and authors list
To reflect the work that went into strictly typing introspect.py,
punish myself by claiming credit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-17-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:50:26 +01:00
John Snow 5444dedfc7 qapi/introspect.py: Add docstrings to _gen_tree and _tree_to_qlit
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-16-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Doc string improvements squashed in]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:50:23 +01:00
John Snow 82b52f6b84 qapi/introspect.py: add type hint annotations
NB: The type aliases (SchemaInfo et al) declare intent for some of the
"dictly-typed" objects we pass around in introspect.py. They do not
enforce the shape of those objects, and cannot, until Python 3.7 or
later. (And even then, it may not be "worth it".)

Annotations are also added to the QAPISchemaEntity __init__ method in
schema.py to allow mypy to statically prove the type of typ.name,
needed to prove the return type of
QAPISchemaGenIntrospectVisitor._use_type().

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-15-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Note on QAPISchemaEntity.__init__() squashed into commit message,
Comment wrapped to conform to PEP 8]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow cf5db2142b qapi/introspect.py: remove _gen_variants helper
It is easier to give a name to all of the dictly-typed objects we pass
around in introspect.py by removing this helper, as it does not return
an object that has any knowable type by itself.

Inline it into its only caller instead.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-14-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow c0e8d9f3c1 qapi/introspect.py: improve readability of _tree_to_qlit
Subjective, but I find getting rid of the comprehensions helps. Also,
divide the sections into scalar and non-scalar sections, and remove
old-style string formatting.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-13-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow 2a6c161bbf qapi/introspect.py: improve _tree_to_qlit error message
Trivial; make the error message just a pinch more explicit in case we
trip this by accident in the future.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow 4f7f97a7b3 qapi/introspect.py: create a typed 'Annotated' data strutcure
Presently, we use a tuple to attach a dict containing annotations
(comments and compile-time conditionals) to a tree node. This is
undesirable because dicts are difficult to strongly type; promoting it
to a real class allows us to name the values and types of the
annotations we are expecting.

In terms of typing, the Annotated<T> type serves as a generic container
where the annotated node's type is preserved, allowing for greater
specificity than we'd be able to provide without a generic.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:17 +01:00
John Snow 9db2734613 qapi/introspect.py: Introduce preliminary tree typing
The types will be used in forthcoming patches to add typing. These types
describe the layout and structure of the objects passed to
_tree_to_qlit, but lack the power to describe annotations until the next
commit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-10-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:38:00 +01:00
John Snow d4c5b429fd qapi/introspect.py: Always define all 'extra' dict keys
This mimics how a typed object works, where 'if' and 'comment' are
always set, regardless of if they have a value set or not.

It is safe to do this because of the way that _tree_to_qlit processes
these values (using dict.get with a default of None), resulting in no
change of output from _tree_to_qlit. There are no other users of this
data.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:54 +01:00
John Snow 5f50cede4a qapi/introspect.py: replace 'extra' dict with 'comment' argument
This is only used to pass in a dictionary with a comment already set, so
skip the runaround and just accept the (optional) comment.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:49 +01:00
John Snow 84cf09915f qapi/introspect.py: Unify return type of _make_tree()
Returning two different types conditionally can be complicated to
type. Return one type for consistency.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:41 +01:00
John Snow 055569603d qapi/introspect.py: guard against ifcond/comment misuse
_tree_to_qlit is called recursively on dict values (isolated from their
keys); at such a point in generating output it is too late to apply an
ifcond. Similarly, comments do not necessarily have a "tidy" place they
can be printed in such a circumstance.

Forbid this usage by renaming "suppress_first_indent" to "dict_value" to
emphasize that indents are suppressed only for the benefit of dict
values; then add an assertion assuring we do not pass ifcond/comments
in this case.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Comment wrapped to conform to PEP 8]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:28 +01:00
John Snow 84bece7dd4 qapi/introspect.py: add _gen_features helper
_make_tree might receive a dict (a SchemaInfo object) or some other type
(usually, a string) for its obj parameter. Adding features information
should arguably be performed by the caller at such a time when we know
the type of the object and don't have to re-interrogate it.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:20 +01:00
John Snow d70f5130f6 qapi/introspect.py: use _make_tree for features nodes
At present, we open-code this in _make_tree itself; but if the structure
of the tree changes, this is brittle. Use an explicit recursive call to
_make_tree when appropriate to help keep the interior node typing
consistent.

A consequence of doing this is that the 'ifcond' key of the features
dict will be omitted when ifcond is false-ish, just like it is omitted
in top-level calls to _make_tree. This also increases consistency in our
handling of this property.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-4-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:13 +01:00
John Snow 6b67bcac0f qapi/introspect.py: assert schema is not None
The introspect visitor is stateful, but expects that it will have a
schema to refer to. Add assertions that state this.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:35:11 +01:00
Taylor Simpson 3e7a84eecc Hexagon build infrastructure
Add file to default-configs
Add hexagon to meson.build
Add hexagon to target/meson.build
Add target/hexagon/meson.build
Change scripts/qemu-binfmt-conf.sh

We can build a hexagon-linux-user target and run programs on
the Hexagon scalar core.  With hexagon-linux-clang installed,
"make check-tcg" will pass.

Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Taylor Simpson <tsimpson@quicinc.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <1612763186-18161-35-git-send-email-tsimpson@quicinc.com>
[rth: Use top-level python variable]
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-02-18 08:25:06 -08:00
John Snow 2184bca7b1 qapi: Replace List[str] with Sequence[str] for ifcond
It does happen to be a list (as of now), but we can describe it in more
general terms with no loss in accuracy to allow tuples and other
constructs.

In the future, we can write "ifcond: Sequence[str] = ()" as a default
parameter, which we could not do safely with a Mutable type like a List.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 17:10:29 +01:00
Taylor Simpson d2a56bd242 Hexagon (linux-user/hexagon) Linux user emulation
Implementation of Linux user emulation for Hexagon
Some common files modified in addition to new files in linux-user/hexagon

Acked-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Taylor Simpson <tsimpson@quicinc.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <1612763186-18161-31-git-send-email-tsimpson@quicinc.com>
[rth: Fix termbits.h on review by Laurent]
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-02-18 07:48:22 -08:00
Philippe Mathieu-Daudé 04250c69bc travis.yml: Move gprof/gcov test across to gitlab
Similarly to commit 8cdb2cef3f, move the gprof/gcov test to GitLab.

The coverage-summary.sh script is not Travis-CI specific, make it
generic.

[thuth: Add gcovr and bsdmainutils which are required for the
        coverage-summary.sh script to the ubuntu docker file,
        and use 'check' as test target]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201108204535.2319870-10-philmd@redhat.com>
Message-Id: <20210211045455.456371-2-thuth@redhat.com>
Message-Id: <20210211122750.22645-2-alex.bennee@linaro.org>
2021-02-15 09:38:19 +00:00
Stefan Hajnoczi 7a53cd20e7 get_maintainer: update repo URL to GitLab
qemu.org is running out of bandwidth and the QEMU project is moving
towards a gating CI on GitLab. Use the GitLab repos instead of qemu.org
(they will become mirrors).

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210111115017.156802-7-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-02-09 20:53:56 +00:00
Peter Maydell 41d306ec7d * Fuzzing improvements (Qiuhao, Alexander)
* i386: Fix BMI decoding for instructions with the 0x66 prefix (David)
 * initial attempt at fixing event_notifier emulation (Maxim)
 * i386: PKS emulation, fix for "qemu-system-i386 -cpu host" (myself)
 * meson: RBD test fixes (myself)
 * meson: TCI warnings (Philippe)
 * Leaner build for --disable-guest-agent, --disable-system and
   --disable-tools (Philippe, Stefan)
 * --enable-tcg-interpreter fix (Richard)
 * i386: SVM feature bits (Wei)
 * KVM bugfix (Thomas H.)
 * Add missing MemoryRegionOps callbacks (PJP)
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmAhR4cUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroOviAf/Ymk/KwHZKySVqHOjDZnvP5PXMAru
 p1zlMRLAorXK+CTbshkIliaQyD8ggzT4HCinJ2NisdfTWMmlWbgr8gahNqKyZ5UG
 HlL28va3dvGhelswh/CNso1ZhVb2Q+aAYn/c6LXQva2r0xi26ohJTkIkSCPP/bnI
 +73dGzwAilBOsBVbn4cCm/70XtwDpPkw41IZIDoy/4lhL8ZdpHMz8oOjNIlOdlcU
 aEDfM8vYE4C70OtUlRZ1OwVxzcjS1Bf6dQYcpg5gAKy/jAAqR+v2PStxXiUuj5D3
 cAzd03Goh78Wcre+CbWxDKGcGtiooUT+J09wmvDPYVUHcpQMbumf4MufrQ==
 =INB5
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

* Fuzzing improvements (Qiuhao, Alexander)
* i386: Fix BMI decoding for instructions with the 0x66 prefix (David)
* initial attempt at fixing event_notifier emulation (Maxim)
* i386: PKS emulation, fix for "qemu-system-i386 -cpu host" (myself)
* meson: RBD test fixes (myself)
* meson: TCI warnings (Philippe)
* Leaner build for --disable-guest-agent, --disable-system and
  --disable-tools (Philippe, Stefan)
* --enable-tcg-interpreter fix (Richard)
* i386: SVM feature bits (Wei)
* KVM bugfix (Thomas H.)
* Add missing MemoryRegionOps callbacks (PJP)

# gpg: Signature made Mon 08 Feb 2021 14:15:35 GMT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream: (46 commits)
  target/i386: Expose VMX entry/exit load pkrs control bits
  target/i386: Add support for save/load IA32_PKRS MSR
  imx7-ccm: add digprog mmio write method
  tz-ppc: add dummy read/write methods
  spapr_pci: add spapr msi read method
  nvram: add nrf51_soc flash read method
  prep: add ppc-parity write method
  vfio: add quirk device write method
  pci-host: designware: add pcie-msi read method
  hw/pci-host: add pci-intack write method
  cpu-throttle: Remove timer_mod() from cpu_throttle_set()
  replay: rng-builtin support
  pc-bios/descriptors: fix paths in json files
  replay: fix replay of the interrupts
  accel/kvm/kvm-all: Fix wrong return code handling in dirty log code
  qapi/meson: Restrict UI module to system emulation and tools
  qapi/meson: Restrict system-mode specific modules
  qapi/meson: Remove QMP from user-mode emulation
  qapi/meson: Restrict qdev code to system-mode emulation
  meson: Restrict emulation code
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-09 10:04:51 +00:00
Peter Maydell 2436651b26 Migration pull 2021-02-08
v2
   Dropped vmstate: Fix memory leak in vmstate_handle_alloc
     Broke on Power
   Added migration: only check page size match if RAM postcopy is enabled
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEERfXHG0oMt/uXep+pBRYzHrxb/ecFAmAhIE4ACgkQBRYzHrxb
 /ecPuA/+Pgo++1ZSseJUgbLePwyTVc0jahdcvYEDmLUn8UM6ikBcBXBgUKHdkFW3
 bjSSVgB/xxvXSiafBK4xFNrCqSgqMSr3DJcHmvWgv2wVARcYf6Z26Da53LZq1Qru
 0tvRyb40Od1f9zb8Zj7e2Y3pjQ9ybLLbjfNhgnOBbQivqWkjZI31oV2KUCWY2+eV
 T1BEwr6mgYepqhmeB6OvQZtaQVC5toirS6NajNF4nt0vZEIGIvK6/A9erCVU8Tze
 5ch1J0MUqgc3q6ZSE/I9BHEy6MaL0X8G6H+ezjxdoRQtbt1iM/YqZJCSrXkAxiLC
 ROohryb6qVk26+UYuana79faLwrw359WlkwNEE6SEIRSENu+6p7bgN3LZuCILCO7
 xJEkeTgy6r40IGCkDC9aWa8pyLHpNX9gyLpGBHdIRD6zEOWaKNtzh7E2uo/T0ann
 BpcfgQOsYN25hIHiiXnxozUREbx71VDfMq7GqGB6eC3u2+a3U6jpSJb1nNq5NB89
 FJYLZy5Rbuy7OStMwfMsxRs7E63XvGgnwrN8FczU/pumCPX4lDYIpnocqinUmP8p
 XubRQQVaVDSKIq1mvzw7iR/1NsP9vfYvnrAIv941f38NBmDKqdPuMOXR/qB/Kp2Y
 jB7b1L5/JcXbWsQmK7fda9jmPzFwSO2cTeTiUonk9RfuuDEws0A=
 =4tbe
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20210208a' into staging

Migration pull 2021-02-08

v2
  Dropped vmstate: Fix memory leak in vmstate_handle_alloc
    Broke on Power
  Added migration: only check page size match if RAM postcopy is enabled

# gpg: Signature made Mon 08 Feb 2021 11:28:14 GMT
# gpg:                using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert/tags/pull-migration-20210208a: (27 commits)
  migration: only check page size match if RAM postcopy is enabled
  migration: introduce snapshot-{save, load, delete} QMP commands
  iotests: fix loading of common.config from tests/ subdir
  iotests: add support for capturing and matching QMP events
  migration: introduce a delete_snapshot wrapper
  migration: wire up support for snapshot device selection
  migration: control whether snapshots are ovewritten
  block: rename and alter bdrv_all_find_snapshot semantics
  block: allow specifying name of block device for vmstate storage
  block: add ability to specify list of blockdevs during snapshot
  migration: stop returning errno from load_snapshot()
  migration: Make save_snapshot() return bool, not 0/-1
  block: push error reporting into bdrv_all_*_snapshot functions
  migration: Display the migration blockers
  migration: Add blocker information
  migration: Fix a few absurdly defective error messages
  migration: Fix cache_init()'s "Failed to allocate" error messages
  migration: Clean up signed vs. unsigned XBZRLE cache-size
  migration: Fix migrate-set-parameters argument validation
  migration: introduce 'userfaultfd-wrlat.py' script
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-08 18:23:47 +00:00
Peter Maydell 4f799257b3 QAPI patches patches for 2021-02-08
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmAhQpISHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZTBFYP/jh1x1Isn35q3jwjzKy6N3EYbzmaRKhW
 rc5bFlZLQ95Nso+Q3b8izUSeoGz7t8E+WLZBBxzNoOWBS2uC8HS0jIpUmCJaB+Yu
 aVrG70+5/s/BPTX7iF6LIrcRj3GIQdFEWR5Zn1V4PFk5G7NZFwEiTnViq8pLw87H
 c2ktAorDyH6Zl4DA2YjV0XBLPMclt53sQ/M5b6ythUq6E05bVUkQFiUm3N8joIEX
 2djwumVi/dYpDwR/pVPBjPXcP5gc44y4D9sjJU6I16kyCyKi8Tr9mmEcSJFmR/Y/
 GdSilmtFhxUikAQogZwi/48BhR0GawhgcKZP5IkKNjvfwqE13Vhx8/p0pEubFPUE
 YexvFqkSmBRZ1mT0QpCcG4lp+u6PzhTyc/gbWwPm42/T+x9fG0cDgvDZCPjIRSsg
 LiUmmqrwOXr+Lw6GP1Q0f0KKv5QhCfeq1YcYrEXTQsa1PDT5T8ARBzxR5O1X2UNR
 Xw7j4u9R63p6P5nOk5+wwRLQkUkGl7N0SYqe4thyHUfW4r48V5J9RT7ONKD2CXO9
 Dw/Q3Ga1GmHaydpoZ4Az1k5kyr5dVBbAISRRWRpYA0sneg85o3RU3aTMt3r43pzC
 5pjeSpx6v7K4Y3NJPL1e/j/qJZq10WtEslkF/TKcBS2qLgiKCZ1oXQPORQxxX9uK
 zuK8oPQR7W42
 =uLm+
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2021-02-08' into staging

QAPI patches patches for 2021-02-08

# gpg: Signature made Mon 08 Feb 2021 13:54:26 GMT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2021-02-08:
  qapi: enable strict-optional checks
  qapi: type 'info' as Optional[QAPISourceInfo]
  qapi/gen: Drop support for QAPIGen without a file name
  qapi/commands: Simplify command registry generation
  qapi/gen: Support switching to another module temporarily
  qapi/gen: write _genc/_genh access shims
  qapi: centralize the built-in module name definition
  qapi/gen: Combine ._add_[user|system]_module
  qapi: use './builtin' as the built-in module name
  qapi: use explicitly internal module names
  qapi/gen: Replace ._begin_system_module()
  qapi: centralize is_[user|system|builtin]_module methods
  qapi/gen: inline _wrap_ifcond into end_if()
  qapi/main: handle theoretical None-return from re.match()
  qapi/events: fix visit_event typing
  qapi/commands: assert arg_type is not None

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-08 16:12:21 +00:00
Qiuhao Li 487a1d13ba fuzz: fix wrong index in clear_bits
Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Message-Id: <SYCPR01MB3502E9F6EB06DEDCD484F738FCBA9@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-02-08 14:43:54 +01:00
John Snow c51172667b qapi: enable strict-optional checks
In the modules that we are checking so far, we can be stricter about the
difference between Optional[T] and T types. Enable that check.

Enabling it now will assist review on further typing and cleanup work.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-17-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow 4a82e468e7 qapi: type 'info' as Optional[QAPISourceInfo]
For everything typed so far, type this parameter as
Optional[QAPISourceInfo].

In the most generic case, QAPISchemaEntity's info field may be None to
represent types that come from built-in definitions. Although some
Entity types may not currently have any built-in definitions, it is not
easily possible to constrain the type except on an ad-hoc basis using
assertions.

It's easier and simpler, then, to just say it's always an Optional type.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-16-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster cc0747f6b7 qapi/gen: Drop support for QAPIGen without a file name
The previous commit removed the only user of QAPIGen(None).  Tighten
the type hint.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-15-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster c6cd7e4151 qapi/commands: Simplify command registry generation
QAPISchemaGenCommandVisitor.visit_command() needs to generate the
marshalling function into the current module, and also generate its
registration into the ./init system module.  The latter is done
somewhat awkwardly: .__init__() creates a QAPIGenCCode that will not
be written out, each .visit_command() adds its registration to it, and
.visit_end() copies its contents into the ./init module it creates.

Instead provide the means to temporarily switch to another module.
Create the ./init module in .visit_begin(), and generate its initial
part.  Add registrations to it in .visit_command().  Finish it in
.visit_end().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-14-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster d921d27c1b qapi/gen: Support switching to another module temporarily
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-13-jsnow@redhat.com>
[Commit message tweaked]
2021-02-08 14:15:58 +01:00
John Snow fd9b160384 qapi/gen: write _genc/_genh access shims
Many places assume they can access these fields without checking them
first to ensure they are defined. Eliminating the _genc and _genh fields
and replacing them with functional properties that check for correct
state can ease the typing overhead by eliminating the Optional[T] return
type.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow 39b2d838f1 qapi: centralize the built-in module name definition
Use a constant to make it obvious we're referring to a very specific thing.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster 4ab0ff6da0 qapi/gen: Combine ._add_[user|system]_module
With callers to _add_system_module now explicitly using the './' prefix
to indicate a system module, there is no longer any reason to have
separate interfaces for adding system vs user modules; use a unified
interface that differentiates based on the name.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-10-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow e2bbc4eaa7 qapi: use './builtin' as the built-in module name
Use './builtin' as the built-in module name instead of
None. Clarify the typing that this is now always a string.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow 12893a8ea7 qapi: use explicitly internal module names
QAPISchemaModularCVisitor._add_system_module() prefixes './' to its name
argument to make it a module name.  Pass the module name instead.  This
will allow us to coalesce the methods to add modules later on.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message reworded]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster f3a705928a qapi/gen: Replace ._begin_system_module()
QAPISchemaModularCVisitor._begin_system_module() is actually just for
the builtin module.  Rename it to ._begin_builtin_module() and drop
its useless @name parameter.

Clarify conditionals in visit_module to make this clear.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-7-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow 98967c248c qapi: centralize is_[user|system|builtin]_module methods
Define what a module is and define what kind of a module it is once and
for all, in one place.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow a253b3eb9a qapi/gen: inline _wrap_ifcond into end_if()
We assert _start_if is not None in end_if, but that's opaque to mypy.
By inlining _wrap_ifcond, that constraint becomes provable to mypy.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow ad1218086e qapi/main: handle theoretical None-return from re.match()
Mypy cannot understand that this match can never be None, so help it
along.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-4-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow 3cc01c546b qapi/events: fix visit_event typing
Actually, the arg_type can indeed be Optional.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow ec9697ab3f qapi/commands: assert arg_type is not None
When boxed is True, expr.py asserts that we must have
arguments. Ultimately, this should mean that if boxed is True that
arg_type should be defined. Mypy cannot infer this, and does not support
'stateful' type inference, e.g.:

```
if x:
    assert y is not None

...

if x:
    y.etc()
```

does not work, because mypy does not statefully remember the conditional
assertion in the second block. Help mypy out by creating a new local
that it can track more easily.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
Andrey Gruzdev c7243566d0 migration: introduce 'userfaultfd-wrlat.py' script
Add BCC/eBPF script to analyze userfaultfd write fault latency distribution.

Signed-off-by: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210129101407.103458-6-andrey.gruzdev@virtuozzo.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2021-02-08 11:19:51 +00:00
Alex Bennée 47e3424ac9 scripts/mtest2make.py: export all-%s-targets variable and use it
There are some places where the conditional makefile support is the
simplest solution. Now we don't expose CONFIG_TCG as a variable create
a new one that can be checked for the check-help output.

As check-tcg is a PHONY target we re-use check-softfloat to gate that
as well.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20210202134001.25738-13-alex.bennee@linaro.org>
2021-02-08 10:55:20 +00:00
Volker Rümelin c6e93c9d62 simpletrace: build() missing 2 required positional arguments
Commit 4e66c9ef64 "tracetool: add input filename and line number to
Event" forgot to add a line number and a filename argument at one
build method call site.

Traceback (most recent call last):
  File "./scripts/simpletrace.py", line 261, in <module>
    run(Formatter())
  File "./scripts/simpletrace.py", line 236, in run
    process(events, sys.argv[2], analyzer, read_header=read_header)
  File "./scripts/simpletrace.py", line 177, in process
    dropped_event =
      Event.build("Dropped_Event(uint64_t num_events_dropped)")
TypeError: build() missing 2 required positional arguments:
  'lineno' and 'filename'

Add the missing arguments.

Signed-off-by: Volker Rümelin <vr_qemu@t-online.de>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20210131173415.3392-1-vr_qemu@t-online.de
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-02-01 10:51:00 +00:00