Commit Graph

1324 Commits

Author SHA1 Message Date
Peter Maydell
082c0543ba scripts/make-release: Stop shipping u-boot source as a tarball
In commit d0dead3b6d we changed to shipping the u-boot
sources as a tarball, to work around a problem where they
contained a file and directory that had the same name except
for case, which was preventing QEMU's source tarball being
unpacked on case-insensitive filesystems.

In commit f2a3b549e3 we updated our u-boot blob
and sources to v2019.01, which no longer has this problem,
so we can finally remove our workaround (effectively
reverting d0dead3b6d).

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Message-id: 20190314155628.8822-1-peter.maydell@linaro.org
2019-03-18 15:40:00 +00:00
Peter Maydell
f39901d59f Break out documentation to docs/devel/.
Add support for pattern groups.
 Other misc cleanups for multiple decode functions.
 -----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJch+V5AAoJEGTfOOivfiFfsFsH/1KW6UWAiieZ1+HPYEp24Ku8
 hCNxlfj0iKe1ZEuC8qp2U27GzePi71IlIJ7p5AuAhiTQBBWz8bPzJJUALm3EliaI
 V4/13fLnTYALnPWoUJclU5frdHBhpIWxFUtnLdB50dSU1cTbFFyS+63JsW3wSSXt
 UqntlhSsAmAQr3ULnKufwDZQJgQoft/8G4YzvMOm/7E0ZeV3B9mARAkn6m/30gLx
 nXgLI2OQrA1ATLeTfzNRup9G+YjLx0nW2LRhAseIWcQAW8PyfJsfW6tJeou93+bf
 fK6BkLMgor74QH37Y3u7KVJGJ04u2Gtu0p2JzBA9MU/0l07WihWPA0eJGnP396I=
 =BxBC
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/rth/tags/pull-dt-20190312' into staging

Break out documentation to docs/devel/.
Add support for pattern groups.
Other misc cleanups for multiple decode functions.

# gpg: Signature made Tue 12 Mar 2019 16:59:37 GMT
# gpg:                using RSA key 64DF38E8AF7E215F
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [full]
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A  05C0 64DF 38E8 AF7E 215F

* remotes/rth/tags/pull-dt-20190312:
  decodetree: Properly diagnose fields overflowing an insn
  decodetree: Prefix extract function names with decode_function
  decodetree: Allow +- to begin a number initializing a field
  decodetree: Produce clean output for an empty input file
  decodetree: Add --static-decode option
  test/decode: Add tests for PatternGroups
  decodetree: Allow grouping of overlapping patterns
  decodetree: Do not unconditionaly return from Pattern.output_code
  decodetree: Ensure build_tree does not include values outside insnmask
  decodetree: Document the usefulness of argument sets
  decodetree: Move documentation to docs/devel/decodetree.rst
  MAINTAINERS: Add scripts/decodetree.py to the TCG section

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-03-13 13:09:38 +00:00
Peter Maydell
3f3bbfc7ce - qtest patches
- One SD patch (with Reviewed-by from the maintainer)
 - One license fix patch
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQIcBAABAgAGBQJch3X+AAoJEC7Z13T+cC21gY4P/iaLcbYHWYxghqNooA7W+sAQ
 JRMzYgMZ/7W6P5WrDUhv6wtPMU2BiALEaHwcPQUtASpsfEHnEPZ/9xPbG7aPRXet
 xE9x5Xpuc0CiHOGNBjfzhlRWQgsVuDn7uiv7aj48ZJmXYb4SI/MT2FKGByXHb2ie
 E6E92NdjLvY0qJnm7A4TLMKyPdIKG8zLcqQbPz+fpD3bqI7Do4XtCN11i35yKsy9
 Co2v+mRxUiZho7f5QuKYD/pU9DVaHO1Ra4fAZhE+LajUeFwpEWZPkZameB8Bkyzk
 l3CE2XgYvrTWspb1N1VX5M+3WHyozfwF15lPnkm3ANxX32bg6TNc0JSs0udtr+WW
 RnJLAc1G68qRAor1bE8SxS1yQ8p3VDbiQMVc/ogBxpbJEefyb5n0CcsAgi0pkVNt
 Uf+CMk2qc9sqXsGFBJCsXBTNnb/fJ/MX41s3SJOxesXpsS9uqXJcK1sa4inY7Waw
 366MliFaYYhJyANvzuQVi+onENUq8kw3tCHQb7dieKe7A73f84CqBhtKbn88JUae
 3ZpcqpjVwLMO3gLVsDRUrRJy23RId4FeqJZAhVbC3K+eggTDh2SWQEhgryfvcY/O
 BRc1L3pifblyzxI74wGlgdSpNFqjE2YeTM5HkBiNrIsU0gNmRGoorZJE1TOYp/HY
 /EzPnw2s1YhrEO+RylvR
 =aFrM
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2019-03-12' into staging

- qtest patches
- One SD patch (with Reviewed-by from the maintainer)
- One license fix patch

# gpg: Signature made Tue 12 Mar 2019 09:03:58 GMT
# gpg:                using RSA key 2ED9D774FE702DB5
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2019-03-12:
  scripts/qemugdb: re-license timers.py to GPLv2 or later
  hw/sd/sdhci: Move PCI-related code into a separate file
  ahci-test: Drop dependence on global_qtest
  tests: test-announce-self: fix memory leak

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-03-12 21:06:26 +00:00
Peter Maydell
9d86712365 qmp-shell: fix nested json regression
One small bug fix.
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABCAAGBQJchxZ4AAoJECgHk2+YTcWmFicP/2FldAN4G5RWP17k6EK8QB6Z
 zn2708JmtDrTXiVbkhfG+o7d3QxD5uzPFFvN+NLc/fX+pHC5z5NOpIc/Qbqy323J
 GTuonkoUlIRTvWAd4dGV7Q5ZRaCUQjSVk2Jkx1fbnKNKKRNMNibZTzmGPavF5DSg
 nwDZnibrD/OLwM8GWFgwnbrPJywIzhd6m1jw9Bfou3dQ3HU0B3jJxb1m8ptDEarH
 pzrBg6m345+CKRvSfLfdjTRBUTcjPi6rwcmUR2GZE1rIEh9WbFRyz0WxewKLfNbV
 G0F0TQ50J6RAvfiWuZqIqXsMXYnVs35maiN4n/C+FhU4N0zv1OfCg5f9xeCgQYwQ
 WJvqeQpEkeJVoDY/wypmEhuxE+2ykZh6vkqmMIBUbaC+EGaZmBiEuso921s5Lnn+
 qB5nOkdY8qQ6YNi9sxLrJzF74epJgwsR+an9nb9Gzh3+pNCn8XmX+oSSTR16igmM
 7BHloeENOYiOeT89VM0iOk+oBJfFK+7Ed71/e12B/EmwAIJG3JUW/xIqXuv6l0pC
 t60HSjcA/WXMAjH7UNPG1IvQOne/yA1loLbhVo05bvHMaubKcYp4JJ9Kea8pLuxf
 I57k3OfEF0OwsFOS3G8jVbh6wn5zbx9eelyY9q5ELGRbTwCFfJSUU+PuMJRztlfr
 CXBOEqaKfqaRqw79LlCc
 =ivLG
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' into staging

qmp-shell: fix nested json regression

One small bug fix.

# gpg: Signature made Tue 12 Mar 2019 02:16:24 GMT
# gpg:                using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" [full]
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/python-next-pull-request:
  qmp-shell: fix nested json regression

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-03-12 18:31:56 +00:00
Richard Henderson
2decfc9558 decodetree: Properly diagnose fields overflowing an insn
Previously this would result in an exception for shifting
the field mask by a negative number.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-03-12 09:46:58 -07:00
Richard Henderson
71ecf79bf4 decodetree: Prefix extract function names with decode_function
This makes it easier to name Formats within multiple decode files.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-03-12 09:46:58 -07:00
Richard Henderson
263ac638a7 decodetree: Allow +- to begin a number initializing a field
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-03-12 09:46:58 -07:00
Richard Henderson
82bfac1c06 decodetree: Produce clean output for an empty input file
This is interesting for bisection, where an output file is plumbed,
but does not yet have patterns.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-03-12 09:46:58 -07:00
Richard Henderson
cd3e7fc18d decodetree: Add --static-decode option
Like --decode, but do not drop 'static' qualifier.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-03-12 09:46:58 -07:00
Richard Henderson
0eff2df4a2 decodetree: Allow grouping of overlapping patterns
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-03-12 09:46:58 -07:00
Richard Henderson
eb6b87fac7 decodetree: Do not unconditionaly return from Pattern.output_code
As a consequence, the 'return false' gets pushed up one level.

This will allow us to perform some other action when the
translator returns failure.

Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-03-12 09:46:58 -07:00
Philippe Mathieu-Daudé
9b3186e38f decodetree: Ensure build_tree does not include values outside insnmask
Reproduced with "scripts/decodetree.py /dev/null".

Reviewed-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-03-12 09:46:58 -07:00
Richard Henderson
3fdbf5d679 decodetree: Move documentation to docs/devel/decodetree.rst
One great big block comment isn't the best way to document
the syntax of a language.

Reviewed-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2019-03-12 09:46:58 -07:00
Alex Bennée
390947ed2a scripts/qemugdb: re-license timers.py to GPLv2 or later
I'm the sole author (aside from a one line by Greg fixing encoding)
and I was asked nicely on IRC to bring it into line with the rest of
the files.

Cc: Greg Kurz <groug@kaod.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2019-03-12 08:04:22 +01:00
Paolo Bonzini
328eb60dc1 qemugdb: fix licensing
qemu-gdb.py was committed after 2012-01-13, so the notice about
GPL v2-only contributions does not apply.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-03-11 16:55:52 +01:00
Marc-André Lureau
b35203b227 qmp-shell: fix nested json regression
Commit fcfab7541 ("qmp-shell: learn to send commands with quoted
arguments") introduces the usage of Python 'shlex' to handle quoted
arguments, but it accidentally broke generation of nested JSON
structs.

shlex drops quotes, which breaks parsing of the nested struct.

cmd='blockdev-create job-id="job0 foo" options={"driver":"qcow2","size":16384,"file":{"driver":"file","filename":"foo.qcow2"}}'

shlex.split(cmd)
['blockdev-create',
 'job-id=job0 foo',
 'options={driver:qcow2,size:16384,file:{driver:file,filename:foo.qcow2}}']

Replace with a regexp to split while respecting quoted strings and preserving quotes:

re.findall(r'''(?:[^\s"']|"(?:\\.|[^"])*"|'(?:\\.|[^'])*')+''', cmd)
['blockdev-create',
 'job-id="job0 foo"',
 'options={"driver":"qcow2","size":16384,"file":{"driver":"file","filename":"foo.qcow2"}}']

Fixes: fcfab7541 ("qmp-shell: learn to send commands with quoted arguments")
Reported-by: Kashyap Chamarthy <kchamart@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20190205134926.8312-1-marcandre.lureau@redhat.com>
Tested-by: Kashyap Chamarthy <kchamart@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-03-11 10:22:31 -03:00
Paolo Bonzini
f349474920 minikconfig: implement allnoconfig and defconfig modes
Apart from defconfig (which is a no-op),
allyesconfig/allnoconfig/randcondfig can be implemented simply by ignoring
the RHS of assignments and "default" statements.  The RHS is replaced
respectively by "true", "false" or a random value.

However, allyesconfig and randconfig do not quite work, because all the
files for hw/ARCH/Kconfig are sourced and therefore you could end up
enabling some ARM boards in x86 or things like that.  This is left for
future work, but I am leaving it in to help debugging minikconf itself.

allnoconfig mode is tied to a new configure option, --without-default-devices.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-03-07 21:45:53 +01:00
Paolo Bonzini
e0e312f352 build: switch to Kconfig
The make_device_config.sh script is replaced by minikconf, which
is modified to support the same command line as its predecessor.

The roots of the parsing are default-configs/*.mak, Kconfig.host and
hw/Kconfig.  One difference with make_device_config.sh is that all symbols
have to be defined in a Kconfig file, including those coming from the
configure script.  This is the reason for the Kconfig.host file introduced
in the previous patch. Whenever a file in default-configs/*.mak used
$(...) to refer to a config-host.mak symbol, this is replaced by a
Kconfig dependency; this part must be done already in this patch
for bisectability.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Yang Zhong <yang.zhong@intel.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190123065618.3520-28-yang.zhong@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-03-07 21:45:53 +01:00
Paolo Bonzini
82f5181777 kconfig: introduce kconfig files
The Kconfig files were generated mostly with this script:

  for i in `grep -ho CONFIG_[A-Z0-9_]* default-configs/* | sort -u`; do
    set fnord `git grep -lw $i -- 'hw/*/Makefile.objs' `
    shift
    if test $# = 1; then
      cat >> $(dirname $1)/Kconfig << EOF
config ${i#CONFIG_}
    bool

EOF
      git add $(dirname $1)/Kconfig
    else
      echo $i $*
    fi
  done
  sed -i '$d' hw/*/Kconfig
  for i in hw/*; do
    if test -d $i && ! test -f $i/Kconfig; then
      touch $i/Kconfig
      git add $i/Kconfig
    fi
  done

Whenever a symbol is referenced from multiple subdirectories, the
script prints the list of directories that reference the symbol.
These symbols have to be added manually to the Kconfig files.

Kconfig.host and hw/Kconfig were created manually.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Yang Zhong <yang.zhong@intel.com>
Message-Id: <20190123065618.3520-27-yang.zhong@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-03-07 21:45:53 +01:00
Paolo Bonzini
f7082a9a7c minikconfig: add semantic analysis
There are three parts in the semantic analysis:

1) evaluating expressions.  This is done as a simple visit
of the Expr nodes.

2) ordering clauses.  This is done by constructing a graph of variables.
There is an edge from X to Y if Y depends on X, if X selects Y, or if
X appears in a conditional selection of Y; in other words, if the value
of X can affect the value of Y.  Each clause has a "destination" variable
whose value can be affected by the clause, and clauses will be processed
according to a topological sorting of their destination variables.
Defaults are processed after all other clauses with the same destination.

3) deriving the value of the variables.  This is done by processing
the clauses in the topological order provided by the previous step.
A "depends on" clause will force a variable to False, a "select" clause
will force a variable to True, an assignment will force a variable
to its RHS.  A default will set a variable to its RHS if it has not
been set before.  Because all variables have a default, after visiting
all clauses all variables will have been set.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20190123065618.3520-25-yang.zhong@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-03-07 21:45:53 +01:00
Paolo Bonzini
53167f5626 minikconfig: add AST
Add Python classes that represent the Kconfig abstract syntax tree.
The abstract syntax tree is stored as a list of clauses.  For example:

    config FOO
        depends on BAR
        select BAZ

is represented as three clauses:

    FOO depends on BAR
    FOO default n
    select BAZ if FOO

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20190123065618.3520-24-yang.zhong@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-03-07 21:45:53 +01:00
Paolo Bonzini
d4fdccadba minikconfig: add parser skeleton
This implements a scanner and recursive descent parser for Kconfig-like
configuration files.  The only "action" of the parser is for now to
detect undefined variables and process include files.

The main differences between Kconfig and this are:

* only the "bool" type is supported

* variables can only be defined once

* choices are not supported (but they could be added as syntactic
sugar for multiple Boolean values)

* menus and other graphical concepts (prompts, help text) are not
supported

* assignments ("CONFIG_FOO=y", "CONFIG_FOO=n") are parsed as part
of the Kconfig language, not as a separate file.

The idea was originally by Ákos Kovács, but I could not find his
implementation so I had to redo it.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20190123065618.3520-23-yang.zhong@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-03-07 21:45:53 +01:00
Peter Maydell
6cb4f6db4f Python queue, 2019-02-22
Python:
 * introduce "python" directory with module namespace
 * log QEMU launch command line on qemu.QEMUMachine
 
 Acceptance Tests:
 * initrd 4GiB+ test
 * migration test
 * multi vm support in test class
 * bump Avocado version and drop "🥑 enable"
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABCAAGBQJccE9jAAoJEGV+jTOl8gnzb1cP/j99kGbgfQJA4CftO9eRXdIm
 FKms4Z42n7KPus+/DphgfOXGYaHzPcqJQNguQYHuPlWaM3DWNU0rcFfAi/QdcZC1
 3iYMyQwiRubjnCMN0Ab4k+GhpCPW6fea6GTzyvqha4jNRhCIhx7v54GTDfxWESQp
 nqW40gAONGSG98DdFgubxg1YYqt7zlI9EVogGixe1gO9SVDkMEe7uH8tPCl9mt2m
 VjN7AeP/NTDmidiwu+2LwSpDC0UmpDAsFnxGI6rDcNx8NOnjSHkSHmtxNJ8j2uZz
 9P0ncGui+LfivdQh/yiBgrjTWXEXAx/oHKQCz7r8uJ8f60eYLFtjTHm//2G7lG48
 luLSnNKq/niM4k/vNhBQr0ByqoHHlpmqAjbmYqw7wdvImBbkXN2Gh9kjNs55S8VZ
 Z7wTceC0G7pyM3LCdFnikyCXKoRxLZ3AXQ3YXFN0PgX/IsyHVuBWBGPFkPkLwcRa
 JW3DEmwx/oeTg2MKp7iA3dGTUIarbsjp+R04erMznlLvE+NgmB8ENY8T+qZ6c+NM
 ZNyp1MH2nuTJsYxY3CkVKwPUqNSoaTLkMxvoZW5rKQdtvNinCYZpaeHuBchaHJed
 E63r0+1n9vAMH3PHDrypW5qjcjSDBOHS+8ajhr0jr2r+6grLQKYEP8q+PwubUaMq
 BsS5jOb8gLGC8ESfZxx/
 =dwff
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/cleber/tags/python-next-pull-request' into staging

Python queue, 2019-02-22

Python:
* introduce "python" directory with module namespace
* log QEMU launch command line on qemu.QEMUMachine

Acceptance Tests:
* initrd 4GiB+ test
* migration test
* multi vm support in test class
* bump Avocado version and drop "🥑 enable"

# gpg: Signature made Fri 22 Feb 2019 19:37:07 GMT
# gpg:                using RSA key 657E8D33A5F209F3
# gpg: Good signature from "Cleber Rosa <crosa@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 7ABB 96EB 8B46 B94D 5E0F  E9BB 657E 8D33 A5F2 09F3

* remotes/cleber/tags/python-next-pull-request:
  Acceptance tests: expect boot to extract 2GiB+ initrd with linux-v4.16
  Acceptance tests: use linux-3.6 and set vm memory to 4GiB
  tests.acceptance: adds simple migration test
  tests.acceptance: adds multi vm capability for acceptance tests
  scripts/qemu.py: log QEMU launch command line
  Introduce a Python module structure
  Acceptance tests: drop usage of "🥑 enable"

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-03-07 16:16:02 +00:00
Markus Armbruster
56a4689582 qapi: Fix array first used in a different module
We generally put implicitly defined types in whatever module triggered
their definition.  This is wrong for array types, as the included test
case demonstrates.  Let's have a closer look at it.

Type 'Status' is defined sub-sub-module.json.  Array type ['Status']
occurs in main module qapi-schema-test.json and in
include/sub-module.json.  The main module's use is first, so the array
type gets put into the main module.

The generated C headers define StatusList in qapi-types.h.  But
include/qapi-types-sub-module.h uses it without including
qapi-types.h.  Oops.

To fix that, put the array type into its element type's module.

Now StatusList gets generated into qapi-types-sub-module.h, which all
its users include.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-8-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-03-05 14:43:11 +01:00
Markus Armbruster
709395f8f6 qapi: Fix code generation for sub-modules in other directories
The #include directives to pull in sub-modules use file names relative
to the main module.  Works only when all modules are in the same
directory, or the main module's output directory is in the compiler's
include path.  Use relative file names instead.

The dummy variable we generate to avoid empty .o files has an invalid
name for sub-modules in other directories.  Fix that.

Both messed up in commit 252dc3105f "qapi: Generate separate .h, .c
for each module".  Escaped testing because tests/qapi-schema-test.json
doesn't cover sub-modules in other directories, only
tests/qapi-schema/include-relpath.json does, and we generate and
compile C code only for the former, not the latter.  Fold the latter
into the former.  This would have caught the mistakes fixed in this
commit.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-5-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-03-05 14:43:11 +01:00
Markus Armbruster
dddee4d7ba qapi: Pass file name to QAPIGen constructor instead of methods
Not much of an improvement now, but the next commit will profit.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-4-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-03-05 14:43:11 +01:00
Max Reitz
e35792b6fd iotests: Let 045 be run concurrently
Adding a telnet monitor for no real purpose on a fixed port is not so
great.  Just use a null monitor instead.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190210145736.1486-10-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-02-25 15:11:28 +01:00
Cleber Rosa
8f8fd9edba Introduce a Python module structure
This is a simple move of Python code that wraps common QEMU
functionality, and are used by a number of different tests
and scripts.

By treating that code as a real Python module, we can more easily:
 * reuse code
 * have a proper place for the module's own unittests
 * apply a more consistent style
 * generate documentation

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Caio Carrara <ccarrara@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20190206162901.19082-2-crosa@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2019-02-22 14:07:01 -05:00
Markus Armbruster
093e367951 Revert "qapi-events: add 'if' condition to implicit event enum"
This reverts commit 7bd2634905.

The commit applied the events' conditions to the members of enum
QAPIEvent.  Awkward, because it renders QAPIEvent unusable in
target-independent code as soon as we make an event target-dependent.
Reverting this has the following effects:

* ui/vnc.c can remain target independent.

* monitor_qapi_event_conf[] doesn't have to muck around with #ifdef.

* query-events again doesn't reflect conditionals.  I'm going to
  deprecate it in favor of query-qmp-schema.

Another option would be to split target-dependent parts off enum
QAPIEvent into a target-dependent enum.  Doesn't seem worthwhile right
now.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20190214152251.2073-17-armbru@redhat.com>
2019-02-18 14:44:05 +01:00
Markus Armbruster
5d75648b56 qapi: Generate QAPIEvent stuff into separate files
Having to include qapi-events.h just for QAPIEvent is suboptimal, but
quite tolerable now.  It'll become problematic when we have events
conditional on the target, because then qapi-events.h won't be usable
from target-independent code anymore.  Avoid that by generating it
into separate files.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20190214152251.2073-6-armbru@redhat.com>
2019-02-18 14:44:04 +01:00
Markus Armbruster
c2e196a9b4 qapi: Prepare for system modules other than 'builtin'
The next commit wants to generate qapi-emit-events.{c.h}.  To enable
that, extend QAPISchemaModularCVisitor to support additional "system
modules", i.e. modules that don't correspond to a (user-defined) QAPI
schema module.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20190214152251.2073-5-armbru@redhat.com>
2019-02-18 14:44:04 +01:00
Markus Armbruster
dcac64711e qapi: Clean up modular built-in code generation a bit
We neglect to call .visit_module() for the special module we use for
built-ins.  Harmless, but clean it up anyway.  The
tests/qapi-schema/*.out now show the built-in module as 'module None'.

Subclasses of QAPISchemaModularCVisitor need to ._add_module() this
special module to enable code generation for built-ins.  When this
hasn't been done, QAPISchemaModularCVisitor.visit_module() does
nothing for the special module.  That looks like built-ins could
accidentally be generated into the wrong module when a subclass
neglects to call ._add_module().  Can't happen, because built-ins are
all visited before any other module.  But that's non-obvious.  Switch
off code generation explicitly.

Rename QAPISchemaModularCVisitor._begin_module() to
._begin_user_module().

New QAPISchemaModularCVisitor._is_builtin_module(), for clarity.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20190214152251.2073-4-armbru@redhat.com>
2019-02-18 14:44:04 +01:00
Vladimir Sementsov-Ogievskiy
6eaa20c836 qemugdb/coroutine: fix arch_prctl has unknown return type
qemu coroutine command results in following error output:

Python Exception <class 'gdb.error'> 'arch_prctl' has unknown return
type; cast the call to its declared return type: Error occurred in
Python command: 'arch_prctl' has unknown return type; cast the call to
its declared return type

Fix it by giving it what it wants: arch_prctl return type.

Information on the topic:
   https://sourceware.org/gdb/onlinedocs/gdb/Calling.html

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20190206151425.105871-1-vsementsov@virtuozzo.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2019-02-12 11:49:17 +08:00
Alex Bennée
2d4e4c01b1 scripts/qemu.py: allow arches use KVM for their 32bit cousins
A lot of architectures can run their 32 bit cousins on KVM so the
kvm_available function needs to be a little less restricting when
deciding if KVM is available.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-02-08 17:32:35 +00:00
Philippe Mathieu-Daudé
d3b442654c archive-source.sh: Clone the submodules locally
We cloned the QEMU repository from the local storage. Since the
submodules are also available there, clone them too. This is
quicker and reduce network use.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[AJB: incorporated review suggestions from danpb]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-02-08 17:32:35 +00:00
Peter Maydell
632351e0e1 Pull request
-----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJcWvXyAAoJENro4Ql1lpzlm5UP/2oOYoDKpEe4me+enqMeMVB2
 EgrYpaIG3SO0pxNXlf7iX//w0VNTDTIeXiMTx7e5cQo08oL0fnfL1T+UfnX2bZnC
 uEKGdzI1l/FoZNUy5VoAGidSN6ImyYL+84t8l7AlVWoPTrVvxM6LYD8a4rWQKOQz
 a9i5VbGtdrzw7Bfif9dwqhqNJA7/HgsFBO9hek9mdTY+yUzR3Ba8SdEijZ076M/8
 iRUS9eLGuZwK/cu3ID8GLrpEuN4pfrslRKqpMsW5G0ozcmFUiguPwczdjvOz3Sgy
 NjkaYuFh4vJCdjEU73rAdX/R8Ea/nxLcZD60PYW+kBWgQW4vt+Kxrs6FosYU8IbW
 DPKidpKzLEQYCFJidTdgFXYDeHJtjY2MV4bOHEiqtch2zGDbAc+y0UdE5MrgIXe2
 9BtyZ/6ncgzTvUR8J0xQZV85KSjOwiYYcGmr4hXmYcJW4M9qqG8sft6Hb+rCdzay
 DKKz41OAkI+4wAVCe5kr0NnFNsiO+Pw+lBdjRjQJPh94Yb5cQGqmcha1EEfA2Qjd
 vJ+/NssOOPhxnTkkaRZ6MJsubyyagqbX/Cei4mE29anr8YuT8kJCUkmz/6h85AKK
 xsTbYi+iH0r7fJr7TXB2kOMwSRGzgFEsqJ+rZ0Hfj/Jisj0Yqbd1w7GXxnE52VUc
 +dtBNdMB7l3Os48KQYHn
 =zmV0
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/elmarco/tags/dump-pull-request' into staging

Pull request

# gpg: Signature made Wed 06 Feb 2019 14:57:54 GMT
# gpg:                using RSA key DAE8E10975969CE5
# gpg: Good signature from "Marc-André Lureau <marcandre.lureau@redhat.com>" [full]
# gpg:                 aka "Marc-André Lureau <marcandre.lureau@gmail.com>" [full]
# Primary key fingerprint: 87A9 BD93 3F87 C606 D276  F62D DAE8 E109 7596 9CE5

* remotes/elmarco/tags/dump-pull-request:
  dump: Set correct vaddr for ELF dump

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-02-07 14:20:46 +00:00
Jon Doron
e17bebd049 dump: Set correct vaddr for ELF dump
vaddr needs to be equal to the paddr since the dump file represents the
physical memory image.

Without setting vaddr correctly, GDB would load all the different memory
regions on top of each other to vaddr 0, thus making GDB showing the wrong
memory data for a given address.

Signed-off-by: Jon Doron <arilou@gmail.com>
Message-Id: <20190109082203.27142-1-arilou@gmail.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Tested-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
2019-02-06 15:51:12 +01:00
Peter Maydell
3e29da9fd8 * cpu-exec fixes (Emilio, Laurent)
* TCG bugfix in queue.h (Paolo)
 * high address load for linuxboot (Zhijian)
 * PVH support (Liam, Stefano)
 * misc i386 changes (Paolo, Robert, Doug)
 * configure tweak for openpty (Thomas)
 * elf2dmp port to Windows (Viktor)
 * initial improvements to Makefile infrastructure (Yang + GSoC 2013)
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQEcBAABAgAGBQJcWckyAAoJEL/70l94x66DCU0H/03tjXBR5iVGjBIroSCq7tti
 6+BWvVbDEHQMS9i3BQc6rNgc4ZAyfJ4iO9wQkpx43PltPIG9e6ZiJaCB4F3jmN5f
 3i2LKBXJGFmGNwz8cAq2qpSIBrx7iPeCzbO/BylpwsILfNycb5K35oS7Qr7ezUcj
 xLM5VfW+3TF0SqI0utNHNAlO/xeBOKh+N1Iettqn+L5MAgI9rmnfDkaD3Pmkbw1H
 Iw8yzEypU4Qsqy4zUyb+dppkwSLELOZ24uJVtYnV+HeTwejXD66FMhvFssw0P7kF
 VBK8L6SttYfe9ltUAsXmlLSsnYThCiV0AMclHy8U3mvA47KbBPxTR7u47UDAZSE=
 =2trt
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* cpu-exec fixes (Emilio, Laurent)
* TCG bugfix in queue.h (Paolo)
* high address load for linuxboot (Zhijian)
* PVH support (Liam, Stefano)
* misc i386 changes (Paolo, Robert, Doug)
* configure tweak for openpty (Thomas)
* elf2dmp port to Windows (Viktor)
* initial improvements to Makefile infrastructure (Yang + GSoC 2013)

# gpg: Signature made Tue 05 Feb 2019 17:34:42 GMT
# gpg:                using RSA key BFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream: (76 commits)
  queue: fix QTAILQ_FOREACH_REVERSE_SAFE
  scsi-generic: Convert from DPRINTF() macro to trace events
  scsi-disk: Convert from DPRINTF() macro to trace events
  pc: Use hotplug_handler_(plug|unplug|unplug_request)
  i386: hvf: Fix smp boot hangs
  hw/vfio/Makefile.objs: Create new CONFIG_* variables for VFIO core and PCI
  hw/i2c/Makefile.objs: Create new CONFIG_* variables for EEPROM and ACPI controller
  hw/tricore/Makefile.objs: Create CONFIG_* for tricore
  hw/openrisc/Makefile.objs: Create CONFIG_* for openrisc
  hw/moxie/Makefile.objs: Conditionally build moxie
  hw/hppa/Makefile.objs: Create CONFIG_* for hppa
  hw/cris/Makefile.objs: Create CONFIG_* for cris
  hw/alpha/Makefile.objs: Create CONFIG_* for alpha
  hw/sparc64/Makefile.objs: Create CONFIG_* for sparc64
  hw/riscv/Makefile.objs: Create CONFIG_* for riscv boards
  hw/nios2/Makefile.objs: Conditionally build nios2
  hw/xtensa/Makefile.objs: Build xtensa_sim and xtensa_fpga conditionally
  hw/lm32/Makefile.objs: Conditionally build lm32 and milkmyst
  hw/sparc/Makefile.objs: CONFIG_* for sun4m and leon3 created
  hw/s390/Makefile.objs: Create new CONFIG_* variables for s390x boards and devices
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

# Conflicts:
#	qemu-deprecated.texi
2019-02-05 19:39:22 +00:00
Li Zhijian
06e0259a7c i386: import & use bootparam.h
it's from v4.20-rc5.

CC: Stefano Garzarella <sgarzare@redhat.com>
CC: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-02-05 16:50:18 +01:00
Paolo Bonzini
2a11ee1019 tap: flush STDOUT on newline
This makes it easier to follow what is going on.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-02-05 16:50:16 +01:00
Paolo Bonzini
a0a6ef91a4 scripts/update-linux-headers.sh: adjust for Linux 4.21-rc1 (or 5.0-rc1)
There are three new indirect inclusions: vhost_types.h, which we'll
shortly put to use as a portable header and thus is copied to
standard-headers; and new per-subtarget versions of MIPS unistd.h
and PowerPC unistd.h.

Because vhost.h includes vhost_types.h, we also need a proxy include
from linux/vhost.h to standard-headers.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2019-02-04 00:08:58 -05:00
Alberto Garcia
a5df73baaf qtest.py: Wait for the result of qtest commands
The cmd() method of the QEMUQtestProtocol class sends a qtest command
to QEMU but doesn't wait for the return message ("OK", "FAIL", "ERR").
Because of this, it can return control to the caller before the
command has actually finished.

In cases like clock_step or clock_set this means that cmd() can return
before all the timers triggered by the clock change have been fired.
This can be fixed by making cmd() wait for the output of the qtest
command.

This fixes iotests 093 and 136, which are flaky since commit
8258292e18 when the machine is under heavy workload.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-02-01 13:46:45 +01:00
Peter Maydell
cfe6c54769 Block patches:
- New debugging QMP command to explore block graphs
 - Converted DPRINTF()s to trace events
 - Fixed qemu-io's use of getopt() for systems with optreset
 - Minor NVMe emulation fixes
 - An iotest fix
 -----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJcUkaiAAoJEPQH2wBh1c9AHsEIAIU0+FNjtdz7lNgyeBCSFCFa
 /qWNk4+w6QBfhTTx/N0hGwh5/FvNYQhby8VHtZitE4/QcLbJwHYgWf14pwce3tP3
 3qNB87AdQpKMpbajQM2x2Xy8lnlPeM7fe21Q/12vuX7AlEDT3gH+W9rg94bw2oFN
 r+xBk6H5F2aVElw3CwMM7eary4+dPnnCQwAnoqM+g5hdpL+0scrIyARGw7v0hmSn
 LDWESCM4a55lEYmwj1wS3J3uj6Fj00yzBvcEuCcT1GO+lXlV8/ciO9r2HqxVKwgz
 4GAi/BERoMKjfn+/77/yI5flprPx2voNGgkyBY4C3z9ncnN6u02QBZSusBIWpSg=
 =Kt4r
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/xanclic/tags/pull-block-2019-01-31' into staging

Block patches:
- New debugging QMP command to explore block graphs
- Converted DPRINTF()s to trace events
- Fixed qemu-io's use of getopt() for systems with optreset
- Minor NVMe emulation fixes
- An iotest fix

# gpg: Signature made Thu 31 Jan 2019 00:51:46 GMT
# gpg:                using RSA key F407DB0061D5CF40
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/xanclic/tags/pull-block-2019-01-31:
  iotests: Allow 147 to be run concurrently
  iotests: Bind qemu-nbd to localhost in 147
  iotests.py: Add qemu_nbd_pipe()
  nvme: use pci_dev directly in nvme_realize
  nvme: ensure the num_queues is not zero
  nvme: use TYPE_NVME instead of constant string
  qemu-io: Add generic function for reinitializing optind.
  block/sheepdog: Convert from DPRINTF() macro to trace events
  block/file-posix: Convert from DPRINTF() macro to trace events
  block/curl: Convert from DPRINTF() macro to trace events
  block/ssh: Convert from DPRINTF() macro to trace events
  scripts: add render_block_graph function for QEMUMachine
  qapi: add x-debug-query-block-graph

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-01-31 19:26:09 +00:00
Peter Maydell
460da1005d Pull request
User-visible changes:
  * The new qemu-trace-stap script makes it convenient to collect traces without
    writing SystemTap scripts.  See "man qemu-trace-stap" for details.
 -----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJcURdlAAoJEJykq7OBq3PIPqAH/iSkYDDeWLQy4eqeTPpbsxd4
 U6mUYC/m2g1bevj1TxdFmr2g5LReGTd4w35w6/SUaLMHsu701T7gK+0z1gP2/N/D
 qzJiM9xxF6xYq1P4hWJGf+XsbJ0OVf7oRwn1j8qXVBxjIxERX98z0ZUtbk/aulGi
 wnNXycBufpKGk2PkQC+pBfhU2775dMqpUV49z9mqyVzsiZQlzbx8WMDQj1Ic1fbe
 ZcAvX5D350HJjB3Z+9wJ1V2pC9Gu+z3TIup+YR1Bkch0ywyTCVTqcepoOXwzQamm
 84bifPdObBm7SbbwtrwoVKYLrdIrbb3PTWaOlWVUKruKIIf8hzn5BxC3wChb2Qk=
 =bex6
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stefanha/tags/tracing-pull-request' into staging

Pull request

User-visible changes:
 * The new qemu-trace-stap script makes it convenient to collect traces without
   writing SystemTap scripts.  See "man qemu-trace-stap" for details.

# gpg: Signature made Wed 30 Jan 2019 03:17:57 GMT
# gpg:                using RSA key 9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/tracing-pull-request:
  trace: rerun tracetool after ./configure changes
  trace: improve runstate tracing
  trace: add ability to do simple printf logging via systemtap
  trace: forbid use of %m in trace event format strings
  trace: enforce that every trace-events file has a final newline
  display: ensure qxl log_buf is a nul terminated string

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-01-31 12:03:40 +00:00
Vladimir Sementsov-Ogievskiy
dc2b651a0b scripts: add render_block_graph function for QEMUMachine
Render block nodes graph with help of graphviz. This new function is
for debugging, so there is no sense to put it into qemu.py as a method
of QEMUMachine. Let's instead put it separately.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20181221170909.25584-3-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-01-31 00:38:19 +01:00
Peter Maydell
b94e809d3e checkpatch: Don't emit spurious warnings about block comments
In checkpatch we attempt to check for and warn about
block comments which start with /* or /** followed by a
non-blank. Unfortunately a bug in the regex meant that
we would incorrectly warn about comments starting with
"/**" with no following text:

  git show 9813dc6ac3954d58ba16b3920556f106f97e1c67|./scripts/checkpatch.pl -
  WARNING: Block comments use a leading /* on a separate line
  #34: FILE: tests/libqtest.h:233:
  +/**

The sequence "/\*\*?" was intended to match either "/*" or "/**",
but Perl's semantics for '?' allow it to backtrack and try the
"matches 0 chars" option if the "matches 1 char" choice leads to
a failure of the rest of the regex to match.  Switch to "/\*\*?+"
which uses what perlre(1) calls the "possessive" quantifier form:
this means that if it matches the "/**" string it will not later
backtrack to matching just the "/*" prefix.

The other end of the regex is also wrong: it is attempting
to check for "/* or /** followed by something that isn't
just whitespace", but [ \t]*.+[ \t]* will match on pure
whitespace. This is less significant but means that a line
with just a comment-starter followed by trailing whitespace
will generate an incorrect warning about block comment style
as well as the correct error about trailing whitespace which
a different checkpatch test emits.

Fixes: 8c06fbdf36 ("scripts/checkpatch.pl: Enforce multiline comment syntax")
Reported-by: Thomas Huth <thuth@redhat.com>
Reported-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20190118165050.22270-1-peter.maydell@linaro.org
2019-01-29 11:46:05 +00:00
Paolo Bonzini
651514df88 decodetree: re.fullmatch was added in 3.4
Python 3 versions earlier than 3.4 do not have it, use the
same workaround that is in place for 3.0.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <1548410602-16008-1-git-send-email-pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-01-25 11:41:42 -02:00
Nisarg Shah
55e0a34635 device-crash-test: Python 3 compatibility fix
Restrict whitelist entry stats in debug mode to be sorted only by
"count", since Python 3 does not implicitly support comparing
dictionaries.

Signed-off-by: Nisarg Shah <nshah@disroot.org>
Message-Id: <20190116183358.30287-1-nshah@disroot.org>
[ehabkost: removed 2 unnecessary hunks from patch]
[ehabkost: edited commit message]
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-01-25 11:41:33 -02:00
Peter Maydell
9dd0d8111f QAPI patches for 2019-01-24
-----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJcScrPAAoJEDhwtADrkYZTONEP/jbAXZGv1N4HM/0oTtVrLWCo
 tNshxN6GOmSLov6DXeJoaPQTVmDSBdFQ5tprCdOjeJ3YxCVZtWea5l+IB5UMG/IL
 /wfiOH7ajJV7OxwoA3Ts6T4gX6ypgRm0lss0xQff7/RIbmltounYwvHDHH410K+o
 UVed5sSzTDmmSvpin5GaJBdVhRPIW7iyiuTIVypF2u4On5VqCttx8P5b2AKd5yON
 JbftwsFCjsXq/rzPA7itGqYe/yPhaoMI1mcLQxSKqfnwilpXZriTwm/YE9a1rwYU
 pwNwX/Db9GwOxQ78PZWeP/m+jpe6KHltyZ+H0KZQN5HsIfdqKHpuQ7xibT36hxPk
 ikDKIptbp66nAGKk8PL3i5ANEumLP3OgNTc8aoakqIb5pAu2wnf8lFS340KNmzj3
 xxzv6SEPqYt/ycDCW6HRCkunXaYIU9dsWi8HDNVij+y14IENi0LwzhsrbDlfljTG
 thXXAylwfY5HkFKUWVpVb7sAPRCYyMUNBYbUiwJkwLzF8gkaT0vXsNTTJ+mzYUFk
 SqHHAmUml9Vkz3KCTiDJGrsUMdLxJKHrLjTabk6ACFJ5iur+Cqv8h/nmJjK6JmpU
 9OrTc0CYQnTTPpDVSwDc3VqYKzKRHK/DUkLTTvA05avVuFiUDCJMMfrLJCrtel88
 f3648CQG/utzIr8NruLV
 =Oima
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2019-01-24' into staging

QAPI patches for 2019-01-24

# gpg: Signature made Thu 24 Jan 2019 14:25:19 GMT
# gpg:                using RSA key 3870B400EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>"
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2019-01-24:
  json: Fix % handling when not interpolating
  qmp: Add examples to qom list, get, and set commands
  qapi: Eliminate indirection through qmp_event_get_func_emit()
  qapi: Belatedly update docs for commit 9c2f56e9f9

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-01-25 11:52:12 +00:00
Daniel P. Berrangé
62dd1048c0 trace: add ability to do simple printf logging via systemtap
The dtrace systemtap trace backend for QEMU is very powerful but it is
also somewhat unfriendly to users who aren't familiar with systemtap,
or who don't need its power right now.

  stap -e "....some strange script...."

The 'log' backend for QEMU by comparison is very crude but incredibly
easy to use:

 $ qemu -d trace:qio* ...some args...
 23266@1547735759.137292:qio_channel_socket_new Socket new ioc=0x563a8a39d400
 23266@1547735759.137305:qio_task_new Task new task=0x563a891d0570 source=0x563a8a39d400 func=0x563a86f1e6c0 opaque=0x563a89078000
 23266@1547735759.137326:qio_task_thread_start Task thread start task=0x563a891d0570 worker=0x563a86f1ce50 opaque=0x563a891d9d90
 23273@1547735759.137491:qio_task_thread_run Task thread run task=0x563a891d0570
 23273@1547735759.137503:qio_channel_socket_connect_sync Socket connect sync ioc=0x563a8a39d400 addr=0x563a891d9d90
 23273@1547735759.138108:qio_channel_socket_connect_fail Socket connect fail ioc=0x563a8a39d400

This commit introduces a way to do simple printf style logging of probe
points using systemtap. In particular it creates another set of tapsets,
one per emulator:

  /usr/share/systemtap/tapset/qemu-*-log.stp

These pre-define probe functions which simply call printf() on their
arguments. The printf() format string is taken from the normal
trace-events files, with a little munging to the format specifiers
to cope with systemtap's more restrictive syntax.

With this you can now do

 $ stap -e 'probe qemu.system.x86_64.log.qio*{}'
 22806@1547735341399856820 qio_channel_socket_new Socket new ioc=0x56135d1d7c00
 22806@1547735341399862570 qio_task_new Task new task=0x56135cd66eb0 source=0x56135d1d7c00 func=0x56135af746c0 opaque=0x56135bf06400
 22806@1547735341399865943 qio_task_thread_start Task thread start task=0x56135cd66eb0 worker=0x56135af72e50 opaque=0x56135c071d70
 22806@1547735341399976816 qio_task_thread_run Task thread run task=0x56135cd66eb0

We go one step further though and introduce a 'qemu-trace-stap' tool to
make this even easier

 $ qemu-trace-stap run qemu-system-x86_64 'qio*'
 22806@1547735341399856820 qio_channel_socket_new Socket new ioc=0x56135d1d7c00
 22806@1547735341399862570 qio_task_new Task new task=0x56135cd66eb0 source=0x56135d1d7c00 func=0x56135af746c0 opaque=0x56135bf06400
 22806@1547735341399865943 qio_task_thread_start Task thread start task=0x56135cd66eb0 worker=0x56135af72e50 opaque=0x56135c071d70
 22806@1547735341399976816 qio_task_thread_run Task thread run task=0x56135cd66eb0

This tool is clever in that it will automatically change the
SYSTEMTAP_TAPSET env variable to point to the directory containing the
right set of probes for the QEMU binary path you give it. This is useful
if you have QEMU installed in /usr but are trying to test and trace a
binary in /home/berrange/usr/qemu-git. In that case you'd do

 $ qemu-trace-stap run /home/berrange/usr/qemu-git/bin/qemu-system-x86_64 'qio*'

And it'll make sure /home/berrange/usr/qemu-git/share/systemtap/tapset
is used for the trace session

The 'qemu-trace-stap' script takes a verbose arg so you can understand
what it is running

 $ qemu-trace-stap run /home/berrange/usr/qemu-git/bin/qemu-system-x86_64 'qio*'
 Using tapset dir '/home/berrange/usr/qemu-git/share/systemtap/tapset' for binary '/home/berrange/usr/qemu-git/bin/qemu-system-x86_64'
 Compiling script 'probe qemu.system.x86_64.log.qio* {}'
 Running script, <Ctrl>-c to quit
 ...trace output...

It can enable multiple probes at once

 $ qemu-trace-stap run qemu-system-x86_64 'qio*' 'qcrypto*' 'buffer*'

By default it monitors all existing running processes and all future
launched proceses. This can be restricted to a specific PID using the
--pid arg

 $ qemu-trace-stap run --pid 2532 qemu-system-x86_64 'qio*'

Finally if you can't remember what probes are valid it can tell you

 $ qemu-trace-stap list qemu-system-x86_64
 ahci_check_irq
 ahci_cmd_done
 ahci_dma_prepare_buf
 ahci_dma_prepare_buf_fail
 ahci_dma_rw_buf
 ahci_irq_lower
 ...snip...

Or list just those matching a prefix pattern

 $ qemu-trace-stap list -v qemu-system-x86_64 'qio*'
 Using tapset dir '/home/berrange/usr/qemu-git/share/systemtap/tapset' for binary '/home/berrange/usr/qemu-git/bin/qemu-system-x86_64'
 Listing probes with name 'qemu.system.x86_64.log.qio*'
 qio_channel_command_abort
 qio_channel_command_new_pid
 qio_channel_command_new_spawn
 qio_channel_command_wait
 qio_channel_file_new_fd
 ...snip...

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20190123120016.4538-5-berrange@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2019-01-24 14:16:56 +00:00