Python 3.4 introduced the inheritable attribute for FDs. At the same
time, it changed the default so that all FDs are not inheritable by
default, that only inheritable FDs are inherited to subprocesses, and
only if close_fds is explicitly set to False.
Adhere to this by setting close_fds to False when working with
subprocesses that may want to inherit FDs, and by trying to
set_inheritable() on FDs that we do want to bequeath to them.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20181022135307.14398-7-mreitz@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
(Thank you to Thomas Huth)
v2: fix 32bit build with updated patch (v3) from Philippe Mathieu-Daudé
built in a 32bit debian sid chroot
-----BEGIN PGP SIGNATURE-----
iQIcBAABAgAGBQJb2D8VAAoJEPMMOL0/L748dZEP/11pPehjPPYVxesxM++pFeuf
2EOrLuOTkwlRX23itj2JHv8UTY3YZR9Z8kkF3SWe7qYfp4kB4dTEYjnJY5Im6fWQ
TUbC9D9SivknOOPyQUtGXZQRN8D8m6V4hN2ZcoXC2M48GT23/uqUWBwCKYeHxdLf
iJQFmhwDnXSZr+D0l9mpMK2vBsZ5ywcbne8GufTtrkz7Dq9A0nDWVc/XUEHzzahf
C+6r2fRPjtImxIjhAGQeAEzOk5tYnqK/3kXjy6T4UygvnZw0pkAS1rIb3hvlzm1e
kBlbA+pgL0kKumMmT9LBR4Os4hlL95URUF+BDNGa3EusImSL/wmhsawslQbfxVyv
5at3VKIdvPXr7GQvmhaJ3dllXiQixX7A+axevkwyZkuIcYLnuhvh6bCR3ap+4mq/
GRk4vwXStS6S8rDLAzo4GA4DsE4EDYJSnU13wMEaj1L9sYPVg1224AgCjnlIBbQa
ntGD3lY7+nG5q1BeVfZXmpNZ4+N4TSpu2uEBxNvWY2/YkaouleQXJ8W4eFirB1Eo
G8TN2fbroLcKgxhOlpvgFrfrgs8T5ZprpqQnvpE2h6M2Nu4JWJq4008q3uIPOwTy
o9MrquqOjdG0+OBHr8Ji5HwDKex68NRQhl8BYhqtPhi/+XycDo47YSodNBfw2U/Q
Ec9301/TQjBcvCBLEzrt
=sHPv
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/vivier2/tags/qemu-trivial-for-3.1-pull-request' into staging
QEMU trivial patches collected between June and October 2018
(Thank you to Thomas Huth)
v2: fix 32bit build with updated patch (v3) from Philippe Mathieu-Daudé
built in a 32bit debian sid chroot
# gpg: Signature made Tue 30 Oct 2018 11:23:01 GMT
# gpg: using RSA key F30C38BD3F2FBE3C
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>"
# gpg: aka "Laurent Vivier <laurent@vivier.eu>"
# gpg: aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>"
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F 5173 F30C 38BD 3F2F BE3C
* remotes/vivier2/tags/qemu-trivial-for-3.1-pull-request:
milkymist-minimac2: Use qemu_log_mask(GUEST_ERROR) instead of error_report
ppc: move at24c to its own CONFIG_ symbol
hw/intc/gicv3: Remove useless parenthesis around DIV_ROUND_UP macro
hw/pci-host: Remove useless parenthesis around DIV_ROUND_UP macro
tests/bios-tables-test: Remove an useless cast
xen: Use the PCI_DEVICE macro
qobject: Catch another straggler for use of qdict_put_str()
configure: Support pkg-config for zlib
tests: Fix typos in comments and help message (found by codespell)
cpu.h: fix a typo in comment
linux-user: fix comment s/atomic_write/atomic_set/
qemu-iotests: make 218 executable
scripts/qemu.py: remove trailing quotes on docstring
scripts/decodetree.py: remove unused imports
docs/devel/testing.rst: add missing newlines after code block
qemu-iotests: fix filename containing checks
tests/tcg/README: fix location for lm32 tests
memory.h: fix typos in comments
vga_int: remove unused function protype
configs/alpha: Remove unused CONFIG_PARALLEL_ISA switch
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
The set_console() method is intended to ease higher level use cases
that require a console device.
The amount of intelligence is limited on purpose, requiring either the
device type explicitly, or the existence of a machine (pattern)
definition.
Because of the console device type selection criteria (by machine
type), users should also be able to define that. It'll then be used
for both '-machine' and for the console device type selection.
Users of the set_console() method will certainly be interested in
accessing the console device, and for that a console_socket property
has been added.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20180530184156.15634-5-crosa@redhat.com>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Tests will often need to add extra arguments to QEMU command
line arguments.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20180530184156.15634-3-crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
items() is less efficient on Python 2.x, but makes the code work
on both Python 2 and Python 3.
Cc: Lukáš Doktor <ldoktor@redhat.com>
Cc: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180312185503.5746-2-ehabkost@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
If a VM is launched, files are created and a cleanup is required before
a new launch. This cleanup is executed by shutdown(), so shutdown() must
be called even if the VM is manually terminated (i.e. using kill).
This patch creates a control to make sure launch() will not be executed
again if shutdown() is not called after the previous launch().
Signed-off-by: Amador Pahim <apahim@redhat.com>
Message-Id: <20180122205033.24893-7-apahim@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Now that shutdown() is guaranteed to always execute self._load_io_log()
and self._post_shutdown(), their calls in 'except' became redundant and
we can safely replace it by a call to shutdown().
Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Amador Pahim <apahim@redhat.com>
Message-Id: <20180122205033.24893-6-apahim@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
The 'returncode' Popen attribute is not guaranteed to be updated. It
actually depends on a call to either poll(), wait() or communicate().
On the other hand, poll() will: "Check if child process has terminated.
Set and return returncode attribute."
Let's use the poll() to check whether the process is running and to get
the updated process exit code, when the process is finished.
Reviewed-by: Fam Zheng <famz@redhat.com>
eviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Amador Pahim <apahim@redhat.com>
Message-Id: <20180122205033.24893-5-apahim@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Currently we only cleanup on shutdown() if the VM is running.
To make sure we will always cleanup, this patch makes the
self._load_io_log() and the self._post_shutdown() to
always be called on shutdown(), regardless the VM running state.
Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Amador Pahim <apahim@redhat.com>
Message-Id: <20180122205033.24893-4-apahim@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
This is just a refactor to separate the exception handler from the
actual launch procedure, improving the readability and making future
maintenances in this piece of code easier.
Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Amador Pahim <apahim@redhat.com>
Message-Id: <20180122205033.24893-3-apahim@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
To launch a VM, we need to create basically two files: the monitor
socket (if it's a UNIX socket) and the qemu log file.
For the qemu log file, we currently just open the path, which will
create the file if it does not exist or overwrite the file if it does
exist.
For the monitor socket, if it already exists, we are currently removing
it, even if it's not created by us.
This patch moves to _pre_launch() the responsibility to create a
temporary directory to host the files so we can remove the whole
directory on _post_shutdown().
Signed-off-by: Amador Pahim <apahim@redhat.com>
Message-Id: <20180122205033.24893-2-apahim@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
All scripts that use the QEMUMachine and QEMUQtestMachine classes
(device-crash-test, tests/migration/*, iotests.py, basevm.py)
already configure logging.
The basicConfig() call inside QEMUMachine.__init__() is being
kept just to make sure a script would still work if it didn't
configure logging.
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20171005172013.3098-4-ehabkost@redhat.com>
Reviewed-by: Lukáš Doktor <ldoktor@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Use logging module for the QMP debug messages. The only scripts
that set debug=True are iotests.py and guestperf/engine.py, and
they already call logging.basicConfig() to set up logging.
Scripts that don't configure logging are safe as long as they
don't need debugging output, because debug messages don't trigger
the "No handlers could be found for logger" message from the
Python logging module.
Scripts that already configure logging but don't use debug=True
(e.g. scripts/vm/basevm.py) will get QMP debugging enabled for
free.
Cc: "Alex Bennée" <alex.bennee@linaro.org>
Cc: Fam Zheng <famz@redhat.com>
Cc: "Philippe Mathieu-Daudé" <f4bug@amsat.org>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20171005172013.3098-3-ehabkost@redhat.com>
Reviewed-by: Lukáš Doktor <ldoktor@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Not all scripts using qemu.py configure the Python logging
module, and end up generating a "No handlers could be found for
logger" message instead of actual log messages.
To avoid requiring every script using qemu.py to configure
logging manually, call basicConfig() when creating a QEMUMachine
object. This won't affect scripts that already set up logging,
but will ensure that scripts that don't configure logging keep
working.
Reported-by: Kevin Wolf <kwolf@redhat.com>
Fixes: 4738b0a85a
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20170921162234.847-1-ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Acked-by: Lukáš Doktor <ldoktor@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Python requires parentheses around multiline expression. This fixes the
breakage of all Python-based qemu-iotests cases that was introduced in
commit dab91d9aa0.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20170918052524.4045-1-kwolf@redhat.com
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
When launching a VM, if an exception happens and the VM is not
initiated, it might be useful to see the qemu command line and
the qemu command output.
This patch creates that message. Notice that self._iolog needs to be
cleaned up in the beginning of the launch() to make sure we will not
expose the qemu log from a previous launch if the current one fails.
Signed-off-by: Amador Pahim <apahim@redhat.com>
Message-Id: <20170901112829.2571-6-apahim@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
The current message shows 'self._args', which contains only part of the
options used in the Qemu command line.
This patch makes the qemu full args list an instance variable and then
uses it in the negative exit code message.
Message was moved outside the 'if is_running' block to make sure it will
be logged if the VM finishes before the call to shutdown().
Signed-off-by: Amador Pahim <apahim@redhat.com>
Message-Id: <20170901112829.2571-5-apahim@redhat.com>
[ehabkost: removed superfluous parenthesis]
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
This module should not write directly to stdout/stderr. Instead, it
should either raise exceptions or just log the messages and let the
callers handle them and decide what to do. For example, scripts could
choose to send the log messages stderr or/and write them to a file if
verbose or debugging mode is enabled.
This patch replaces the writes to stderr by an exception in the
send_fd_scm() when _socket_scm_helper is not set or not present. In the
same method, the subprocess Popen will now redirect the stdout/stderr to
logging.debug instead of writing to system stderr. As consequence, since
the Popen.communicate() is now used (in order to get the stdout), the
further call to wait() became redundant and was replaced by
Popen.returncode.
The shutdown() message on negative exit code will now be logged
to logging.warn instead of written to system stderr.
Signed-off-by: Amador Pahim <apahim@redhat.com>
Message-Id: <20170901112829.2571-3-apahim@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
is_running() returns None when called before the first time we
call launch():
>>> import qemu
>>> vm = qemu.QEMUMachine('qemu-system-x86_64')
>>> vm.is_running()
>>>
It should return False instead. This patch fixes that.
For consistence, this patch removes the parenthesis from the
second clause as it's not really needed.
Signed-off-by: Amador Pahim <apahim@redhat.com>
Message-Id: <20170901112829.2571-2-apahim@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
The naked Exception should not be widely used. It makes sense to be a
bit more specific and use better-suited custom exceptions. As a benefit
we can store the full reply in the exception in case someone needs it
when catching the exception.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20170818142613.32394-6-ldoktor@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
The QMP key conversion consist of '_'s to be replaced with '-'s, which
can easily be done by a single `str.replace` method which is faster and
does not require `string` module import.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20170818142613.32394-5-ldoktor@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Let's avoid creating an in-memory list of keys and query for each value
and use `iteritems` which is an iterator of key-value pairs.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20170818142613.32394-4-ldoktor@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
The list object is mutable in python and potentially might modify other
object's arguments when used as default argument. Reproducer:
>>> vm1 = QEMUMachine("qemu")
>>> vm2 = QEMUMachine("qemu")
>>> vm1._wrapper.append("foo")
>>> print vm2._wrapper
['foo']
In this case the `args` is actually copied so it would be safe to keep
it, but it's not a good practice to keep it. The same issue applies in
inherited qtest module.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-Id: <20170818142613.32394-3-ldoktor@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
No actual code changes, just several pylint/style fixes and docstring
clarifications.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20170818142613.32394-2-ldoktor@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
There are a number of ways to ensure that the QEMU process is shut down
when the test ends, including atexit.register(), try: finally:, or
unittest.teardown() methods. All of these require extra code and the
programmer must remember to add vm.shutdown().
A nice solution is context managers:
with VM(binary) as vm:
...
# vm is guaranteed to be shut down here
Cc: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Message-id: 20170824072202.26818-2-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Allow the exit code of QEMU to be queried by scripts.
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20170526181200.17227-3-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Keep the Popen object around to we can query its exit code later.
To keep the existing 'self._popen is None' checks working, add a
is_running() method, that will check if the process is still running.
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20170526181200.17227-2-ehabkost@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
The previous commit refactoring iotests.py:
commit 6661397446
Author: Daniel P. Berrange <berrange@redhat.com>
Date: Wed Jul 20 14:23:10 2016 +0100
scripts: refactor the VM class in iotests for reuse
was not properly tested and included a number of broken
bits.
- The 'event_match' method was not moved into qemu.py
- The 'self._args' list parameter in QEMUMachine needs
to be copied otherwise modifications will affect the
global 'qemu_opts' variable in iotests.py
- The QEMUQtestMachine class methods had inverted
parameter order for the super() calls
- The QEMUQtestMachine class forgot to add
'-machine accel=qtest'
- The QEMUQtestMachine class constructor needs to set
a default 'name' value before using it as it may
be None
- The QEMUQtestMachine class constructor needs to use
named parameters when calling the super constructor
as it is leaving out some positional parameters.
- The 'qemu_prog' variable should be a string not a
list in iotests.py
- The VM classs constructor needs to use named
parameters when calling the super constructor
as it is leaving out some positional parameters.
- The path to the socket-scm-helper needs to be
passed into the QEMUMachine class
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id: 1469549767-27249-1-git-send-email-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
The iotests module has a python class for controlling QEMU
processes. Pull the generic functionality out of this file
and create a scripts/qemu.py module containing a QEMUMachine
class. Put the QTest integration support into a subclass
QEMUQtestMachine.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1469020993-29426-4-git-send-email-berrange@redhat.com>
Signed-off-by: Amit Shah <amit.shah@redhat.com>
