Commit Graph

90618 Commits

Author SHA1 Message Date
Yanan Wang 553dc36b38 qemu-options: Tweak [, maxcpus=cpus] to [, maxcpus=maxcpus]
In qemu-option.hx, there is "-smp [[cpus=]n][,maxcpus=cpus]..." in the
DEF part, and "-smp [[cpus=]n][,maxcpus=maxcpus]..." in the RST part.
Obviously the later is right, let's fix the previous one.

Signed-off-by: Yanan Wang <wangyanan55@huawei.com>
Reviewed-by: Damien Hedde <damien.hedde@greensocs.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Message-Id: <20210928121134.21064-2-wangyanan55@huawei.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2021-10-04 09:47:26 +02:00
Markus Armbruster ba858d1fad qemu-options: -chardev reconnect=seconds duplicated in help, tidy up
Fixes: 5dd1f02b4b
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210928071449.1416022-1-armbru@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2021-10-04 09:47:26 +02:00
Richard Henderson 30bd1db58b * -smp cleanpus (Yanan)
* Hyper-V enlightenment functionality (Vitaly)
 * virtio-mem support in dump, tpm and QMP (David)
 * NetBSD GCC 7.4 compiler support (Nia)
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmFZXqoUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroOD7wf+Nu+ussVGhCGM34Jp8HhpBvp97Xap
 CSz9udJc6uOK8qeZgOj9zqZ8kDKXv1hifuuOnyWnyi+Ojgyi5mKEhmnlZBt7WK8b
 gwqADAgNpBzM8OSK6kT6WjLKods9OkBnDqSKFOakddrLJ1lmfjox1BKfykqtsux3
 sc+eKVFYA9mu5AEiWOs9yDXUUPbdJfTjEI2dHcJco6K+/Ei5++z7hyMjcX/pDj/Q
 Bq9LtrKS6XZahpy6ywsBQVjq1NqN9PXtFJOkUGftEYj/BAyl69Z9nU0g+HUKJOMw
 40VsUpkL4su5rfID0gOGJhT09w5u/Hfyc/4VzCLAJ4NbP76uNcnMVXyiQA==
 =nrp5
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* -smp cleanpus (Yanan)
* Hyper-V enlightenment functionality (Vitaly)
* virtio-mem support in dump, tpm and QMP (David)
* NetBSD GCC 7.4 compiler support (Nia)

# gpg: Signature made Sun 03 Oct 2021 03:41:30 AM EDT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]

* remotes/bonzini/tags/for-upstream:
  softmmu/memory_mapping: optimize for RamDiscardManager sections
  softmmu/memory_mapping: factor out adding physical memory ranges
  softmmu/memory_mapping: never merge ranges accross memory regions
  tpm: mark correct memory region range dirty when clearing RAM
  monitor: Rate-limit MEMORY_DEVICE_SIZE_CHANGE qapi events per device
  qapi: Include qom-path in MEMORY_DEVICE_SIZE_CHANGE qapi events
  virtio-mem-pci: Fix memory leak when creating MEMORY_DEVICE_SIZE_CHANGE event
  configure: Loosen GCC requirement from 7.5.0 to 7.4.0

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-10-03 08:45:19 -04:00
Richard Henderson f50ecf548c QAPI patches patches for 2021-10-02
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmFX8AcSHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZTlowQAIyc/agjl8qkObdNb9F9dDtB/MEnlT8C
 WED8620kxLaaWmltaG+yVAg6x4UoWQKhAedhWkX2cpfm4udTYjxC/ud8AmE+yx3v
 HsWKVutsU5le4XNuj8ZXe9GG71HKw8EXcqLjdFgQ12zCY3uVEVACP4XTo9eIZOG4
 Dubt7JWeWlo68MWFdlWXh24qy5tDoy+I3UN8zWd6p+6KJCrc9qOtjz3/CDWa4Wut
 QZh8tFnxSnmQ5t7OV1vUnZMdNo/9bwMGQ7/J1MxdFLEOAyvWeJWDrB46MCuRkdmE
 D3HOpm4Y1dJY4xHhdtTDYYmptsxyZ7B63J7AUOC7DClgedCE4yioumpkGkMhxncg
 N4LQ/ESYH+vXmhNslRglQ/J8FXmLoGOyGRLHhVWKUw6hbAyD6ba1Bie8x4fK2U7N
 K6S5vq5YsZsqfEy/jADIoUAnTsDxcO9XmQ6Jnu1z0tKEiMH1J0hOwbXl4csFaYeZ
 gfYhKBC10+VE1M90B+ByT4EmG/jCfb2N4HukAaGb1l+2xCDlA3Pk7XSZ3HDItS3K
 qA1CCWuBl4+mmvAEDyoEBoh9gcY7B3Z9pu6GuTXB1H5snci55ltlNnn96zfKexgM
 /nEwKZht+6UYCsh1QE4Is0xPnkCaJySSQDV1vaPn+IU6atD+eicZ7bqr9O4ZP/ih
 vbIqJXGr0rWI
 =IVP/
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2021-10-02' into staging

QAPI patches patches for 2021-10-02

# gpg: Signature made Sat 02 Oct 2021 01:37:11 AM EDT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]

* remotes/armbru/tags/pull-qapi-2021-10-02:
  qapi/parser: enable pylint checks
  qapi/parser: Silence too-few-public-methods warning
  qapi/parser: enable mypy checks
  qapi/parser: Add FIXME for consolidating JSON-related types
  qapi/parser: add type hint annotations (QAPIDoc)
  qapi/parser: add import cycle workaround
  qapi/parser: Introduce NullSection
  qapi/parser: clarify _end_section() logic
  qapi/parser: remove FIXME comment from _append_body_line
  qapi: Add spaces after symbol declaration for consistency
  qapi/parser: fix unused check_args_section arguments
  qapi/gen: use dict.items() to iterate over _modules
  qapi/pylintrc: ignore 'consider-using-f-string' warning

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-10-02 09:03:55 -04:00
David Hildenbrand cb83ba8c1a softmmu/memory_mapping: optimize for RamDiscardManager sections
virtio-mem logically plugs/unplugs memory within a sparse memory region
and notifies via the RamDiscardManager interface when parts become
plugged (populated) or unplugged (discarded).

Currently, we end up (via the two users)
1) zeroing all logically unplugged/discarded memory during TPM resets.
2) reading all logically unplugged/discarded memory when dumping, to
   figure out the content is zero.

1) is always bad, because we assume unplugged memory stays discarded
   (and is already implicitly zero).
2) isn't that bad with anonymous memory, we end up reading the zero
   page (slow and unnecessary, though). However, once we use some
   file-backed memory (future use case), even reading will populate memory.

Let's cut out all parts marked as not-populated (discarded) via the
RamDiscardManager. As virtio-mem is the single user, this now means that
logically unplugged memory ranges will no longer be included in the
dump, which results in smaller dump files and faster dumping.

virtio-mem has a minimum granularity of 1 MiB (and the default is usually
2 MiB). Theoretically, we can see quite some fragmentation, in practice
we won't have it completely fragmented in 1 MiB pieces. Still, we might
end up with many physical ranges.

Both, the ELF format and kdump seem to be ready to support many
individual ranges (e.g., for ELF it seems to be UINT32_MAX, kdump has a
linear bitmap).

Reviewed-by: Peter Xu <peterx@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Claudio Fontana <cfontana@suse.de>
Cc: Thomas Huth <thuth@redhat.com>
Cc: "Alex Bennée" <alex.bennee@linaro.org>
Cc: Peter Xu <peterx@redhat.com>
Cc: Laurent Vivier <lvivier@redhat.com>
Cc: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210727082545.17934-5-david@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-02 08:43:22 +02:00
David Hildenbrand 3513bb1be1 softmmu/memory_mapping: factor out adding physical memory ranges
Let's factor out adding a MemoryRegionSection to the list, to be reused in
RamDiscardManager context next.

Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Claudio Fontana <cfontana@suse.de>
Cc: Thomas Huth <thuth@redhat.com>
Cc: "Alex Bennée" <alex.bennee@linaro.org>
Cc: Peter Xu <peterx@redhat.com>
Cc: Laurent Vivier <lvivier@redhat.com>
Cc: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210727082545.17934-4-david@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-02 08:43:22 +02:00
David Hildenbrand 602f8ea79c softmmu/memory_mapping: never merge ranges accross memory regions
Let's make sure to not merge when different memory regions are involved.
Unlikely, but theoretically possible.

Acked-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Claudio Fontana <cfontana@suse.de>
Cc: Thomas Huth <thuth@redhat.com>
Cc: "Alex Bennée" <alex.bennee@linaro.org>
Cc: Peter Xu <peterx@redhat.com>
Cc: Laurent Vivier <lvivier@redhat.com>
Cc: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210727082545.17934-3-david@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-02 08:43:22 +02:00
David Hildenbrand 45e576c745 tpm: mark correct memory region range dirty when clearing RAM
We might not start at the beginning of the memory region. Let's
calculate the offset into the memory region via the difference in the
host addresses.

Acked-by: Stefan Berger <stefanb@linux.ibm.com>
Fixes: ffab1be706 ("tpm: clear RAM when "memory overwrite" requested")
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Claudio Fontana <cfontana@suse.de>
Cc: Thomas Huth <thuth@redhat.com>
Cc: "Alex Bennée" <alex.bennee@linaro.org>
Cc: Peter Xu <peterx@redhat.com>
Cc: Laurent Vivier <lvivier@redhat.com>
Cc: Stefan Berger <stefanb@linux.vnet.ibm.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210727082545.17934-2-david@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-02 08:43:21 +02:00
David Hildenbrand 77ae2302ae monitor: Rate-limit MEMORY_DEVICE_SIZE_CHANGE qapi events per device
We want to rate-limit MEMORY_DEVICE_SIZE_CHANGE events per device,
otherwise we can lose some events for devices. We can now use the
qom-path to reliably map an event to a device and make rate-limiting
device-aware.

This was noticed by starting a VM with two virtio-mem devices that each
have a requested size > 0. The Linux guest will initialize both devices
in parallel, resulting in losing MEMORY_DEVICE_SIZE_CHANGE events for
one of the devices.

Fixes: 722a3c783e ("virtio-pci: Send qapi events when the virtio-mem size changes")
Suggested-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210929162445.64060-4-david@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-02 08:43:21 +02:00
David Hildenbrand d89dd28f0e qapi: Include qom-path in MEMORY_DEVICE_SIZE_CHANGE qapi events
As we might not always have a device id, it is impossible to always
match MEMORY_DEVICE_SIZE_CHANGE events to an actual device. Let's
include the qom-path in the event, which allows for reliable mapping of
events to devices.

Fixes: 722a3c783e ("virtio-pci: Send qapi events when the virtio-mem size changes")
Suggested-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210929162445.64060-3-david@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-02 08:43:21 +02:00
David Hildenbrand 75b98cb9f6 virtio-mem-pci: Fix memory leak when creating MEMORY_DEVICE_SIZE_CHANGE event
Apparently, we don't have to duplicate the string.

Fixes: 722a3c783e ("virtio-pci: Send qapi events when the virtio-mem size changes")
Cc: qemu-stable@nongnu.org
Signed-off-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210929162445.64060-2-david@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-02 08:43:21 +02:00
nia 3830df5f83 configure: Loosen GCC requirement from 7.5.0 to 7.4.0
As discussed in issue 614, we're shipping GCC 7.4.0 as the
system compiler in NetBSD 9, the most recent stable branch,
and are still actively interested in QEMU on this platform.

The differences between GCC 7.5.0 and 7.4.0 are trivial.

Signed-off-by: Nia Alarie <nia@NetBSD.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <YVcpe79I0rly1HJh@homeworld.netbsd.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-02 08:43:21 +02:00
John Snow d183e0481b qapi/parser: enable pylint checks
Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210930205716.1148693-14-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow 18e3673e0f qapi/parser: Silence too-few-public-methods warning
Eh. Not worth the fuss today. There are bigger fish to fry.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-13-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow 2e28283e41 qapi/parser: enable mypy checks
Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210930205716.1148693-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow 15acf48cfe qapi/parser: Add FIXME for consolidating JSON-related types
The fix for this comment is forthcoming in a future commit, but this
will keep me honest. The linting configuration in ./python/setup.cfg
prohibits 'FIXME' comments. A goal of this long-running series is to
move ./scripts/qapi to ./python/qemu/qapi so that the QAPI generator is
regularly type-checked by GitLab CI.

This comment is a time-bomb to force me to address this issue prior to
that step.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow 5f0d9f3bc7 qapi/parser: add type hint annotations (QAPIDoc)
Annotations do not change runtime behavior.
This commit consists of only annotations.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-10-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow e7ac60fcd0 qapi/parser: add import cycle workaround
Adding static types causes a cycle in the QAPI generator:
[schema -> expr -> parser -> schema]. It exists because the QAPIDoc
class needs the names of types defined by the schema module, but the
schema module needs to import both expr.py/parser.py to do its actual
parsing.

Ultimately, the layering violation is that parser.py should not have any
knowledge of specifics of the Schema. QAPIDoc performs double-duty here
both as a parser *and* as a finalized object that is part of the schema.

In this patch, add the offending type hints alongside the workaround to
avoid the cycle becoming a problem at runtime. See
https://mypy.readthedocs.io/en/latest/runtime_troubles.html#import-cycles
for more information on this workaround technique.

I see three ultimate resolutions here:

(1) Just keep this patch and use the TYPE_CHECKING trick to eliminate
    the cycle which is only present during static analysis.

(2) Don't bother to annotate connect_member() et al, give them 'object'
    or 'Any'. I don't particularly like this, because it diminishes the
    usefulness of type hints for documentation purposes. Still, it's an
    extremely quick fix.

(3) Reimplement doc <--> definition correlation directly in schema.py,
    integrating doc fields directly into QAPISchemaMember and relieving
    the QAPIDoc class of the responsibility. Users of the information
    would instead visit the members first and retrieve their
    documentation instead of the inverse operation -- visiting the
    documentation and retrieving their members.

My preference is (3), but in the short-term (1) is the easiest way to
have my cake (strong type hints) and eat it too (Not have import
cycles). Do (1) for now, but plan for (3).

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow f4c05aaf14 qapi/parser: Introduce NullSection
Here's the weird bit. QAPIDoc generally expects -- virtually everywhere
-- that it will always have a current section. The sole exception to
this is in the case that end_comment() is called, which leaves us with
*no* section. However, in this case, we also don't expect to actually
ever mutate the comment contents ever again.

NullSection is just a Null-object that allows us to maintain the
invariant that we *always* have a current section, enforced by static
typing -- allowing us to type that field as QAPIDoc.Section instead of
the more ambiguous Optional[QAPIDoc.Section].

end_section is renamed to switch_section and now accepts as an argument
the new section to activate, clarifying that no callers ever just
unilaterally end a section; they only do so when starting a new section.

Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210930205716.1148693-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow 1e20a77576 qapi/parser: clarify _end_section() logic
The "if self._section" clause in end_section is mysterious: In which
circumstances might we end a section when we don't have one?

QAPIDoc always expects there to be a "current section", only except
after a call to end_comment(). This actually *shouldn't* ever be 'None',
so let's remove that logic so I don't wonder why it's like this again in
three months.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow cd87c14cde qapi/parser: remove FIXME comment from _append_body_line
True, we do not check the validity of this symbol -- but we don't check
the validity of definition names during parse, either -- that happens
later, during the expr check. I don't want to introduce a dependency on
expr.py:check_name_str here and introduce a cycle.

Instead, rest assured that a documentation block is required for each
definition. This requirement uses the names of each section to ensure
that we fulfilled this requirement.

e.g., let's say that block-core.json has a comment block for
"Snapshot!Info" by accident. We'll see this error message:

In file included from ../../qapi/block.json:8:
../../qapi/block-core.json: In struct 'SnapshotInfo':
../../qapi/block-core.json:38: documentation comment is for 'Snapshot!Info'

That's a pretty decent error message.

Now, let's say that we actually mangle it twice, identically:

../../qapi/block-core.json: In struct 'Snapshot!Info':
../../qapi/block-core.json:38: struct has an invalid name

That's also pretty decent. If we forget to fix it in both places, we'll
just be back to the first error.

Therefore, let's just drop this FIXME and adjust the error message to
not imply a more thorough check than is actually performed.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow a9e2eb06ed qapi: Add spaces after symbol declaration for consistency
Several QGA definitions omit a blank line after the symbol
declaration. This works OK currently, but it's the only place where we
do this. Adjust it for consistency.

Future commits may wind up enforcing this formatting.

Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210930205716.1148693-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:41 +02:00
John Snow 012336a152 qapi/parser: fix unused check_args_section arguments
Pylint informs us we're not using these arguments. Oops, it's
right. Correct the error message and remove the remaining unused
parameter.

Fix test output now that the error message is improved.

Fixes: e151941d1b
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-4-jsnow@redhat.com>
[Commit message formatting tweaked]
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:41 +02:00
John Snow 2adb988ed4 qapi/gen: use dict.items() to iterate over _modules
New pylint warning. I could silence it, but this is the only occurrence
in the entire tree, including everything in iotests/ and python/. Easier
to just change this one instance.

(The warning is emitted in cases where you are fetching the values
anyway, so you may as well just take advantage of the iterator to avoid
redundant lookups.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:41 +02:00
John Snow 1c00917409 qapi/pylintrc: ignore 'consider-using-f-string' warning
Pylint 2.11.x adds this warning. We're not yet ready to pursue that
conversion, so silence it for now.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:41 +02:00
Richard Henderson 5f99210238 * -smp cleanpus
* Hyper-V englightenment functionality
 * Documentation cleanups
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmFXQSQUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroNJUgf/bi96/9B1/6YhZ15l0jzxhtyNaWJ6
 nLtMrAsafbiPM8z3WKQ2BY3hLfRwvPbn8TZkO+mIwvfHP5gQDfYXL8xwW0KmxpPo
 1c8xjV85yqjVMFd2FFvSrLfpUhR8Pduti5zfDLcbFTzjbR08RRRbVjaVNtrm6fpE
 /sfZRl6vC3Znt6e9dM1n5//WDV2yGVEcD9BCbjgnEIf6fBr9pSNAoV/9bd5l+a9M
 kKq+SD3MIozipi96JX02447h1qhfrthAEbYhJc9XBh1Z7Ah6DldHasWiz07+cAp6
 ggwWMJ3p6ss07tgaZ1OlN8ivifRJ1+oFVXZOixAKlFQn6yhXyhZUjEMQdg==
 =3LXK
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* -smp cleanpus
* Hyper-V englightenment functionality
* Documentation cleanups

# gpg: Signature made Fri 01 Oct 2021 01:11:00 PM EDT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]

* remotes/bonzini/tags/for-upstream: (29 commits)
  docs: reorganize testing.rst
  docs: move gcov section at the end of testing.rst
  docs: reorganize tcg-plugins.rst
  docs: reorganize qgraph.rst
  docs: put "make" information together in build-system.rst
  docs: move notes inside the body of the document
  docs: name included files ".rst.inc"
  i386: Change the default Hyper-V version to match WS2016
  i386: Make Hyper-V version id configurable
  i386: Implement pseudo 'hv-avic' ('hv-apicv') enlightenment
  i386: Move HV_APIC_ACCESS_RECOMMENDED bit setting to hyperv_fill_cpuids()
  i386: Support KVM_CAP_HYPERV_ENFORCE_CPUID
  i386: Support KVM_CAP_ENFORCE_PV_FEATURE_CPUID
  machine: Put all sanity-check in the generic SMP parser
  machine: Use g_autoptr in machine_set_smp
  machine: Move smp_prefer_sockets to struct SMPCompatProps
  machine: Remove smp_parse callback from MachineClass
  machine: Make smp_parse generic enough for all arches
  machine: Tweak the order of topology members in struct CpuTopology
  machine: Use ms instead of global current_machine in sanity-check
  ...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-10-01 13:44:36 -04:00
Vitaly Kuznetsov f701ecec2b i386: Change the default Hyper-V version to match WS2016
KVM implements some Hyper-V 2016 functions so providing WS2008R2 version
is somewhat incorrect. While generally guests shouldn't care about it
and always check feature bits, it is known that some tools in Windows
actually check version info.

For compatibility reasons make the change for 6.2 machine types only.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20210902093530.345756-9-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 19:04:45 +02:00
Paolo Bonzini 16e79e1b01 docs: reorganize testing.rst
Clean up the heading levels to use === --- ~~~ ^^^ '''.  Reorganize the
outline for the Avocado part, and always include headings for the
class names.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 19:04:45 +02:00
Vitaly Kuznetsov af7228b88d i386: Make Hyper-V version id configurable
Currently, we hardcode Hyper-V version id (CPUID 0x40000002) to
WS2008R2 and it is known that certain tools in Windows check this. It
seems useful to provide some flexibility by making it possible to change
this info at will. CPUID information is defined in TLFS as:

EAX: Build Number
EBX Bits 31-16: Major Version
    Bits 15-0: Minor Version
ECX Service Pack
EDX Bits 31-24: Service Branch
    Bits 23-0: Service Number

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20210902093530.345756-8-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 19:04:45 +02:00
Paolo Bonzini 9fce360176 docs: move gcov section at the end of testing.rst
gcov testing applies to all tests, not just make check.  Move it
out of the make check section.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 19:04:45 +02:00
Vitaly Kuznetsov e1f9a8e8c9 i386: Implement pseudo 'hv-avic' ('hv-apicv') enlightenment
The enlightenment allows to use Hyper-V SynIC with hardware APICv/AVIC
enabled. Normally, Hyper-V SynIC disables these hardware features and
suggests the guest to use paravirtualized AutoEOI feature. Linux-4.15
gains support for conditional APICv/AVIC disablement, the feature
stays on until the guest tries to use AutoEOI feature with SynIC. With
'HV_DEPRECATING_AEOI_RECOMMENDED' bit exposed, modern enough Windows/
Hyper-V versions should follow the recommendation and not use the
(unwanted) feature.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20210902093530.345756-7-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 19:04:45 +02:00
Paolo Bonzini e9adb4ace2 docs: reorganize tcg-plugins.rst
Clean up the heading levels to use === --- ~~~, and create a new "writing
plugins" section.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 19:04:45 +02:00
Vitaly Kuznetsov 050716292a i386: Move HV_APIC_ACCESS_RECOMMENDED bit setting to hyperv_fill_cpuids()
In preparation to enabling Hyper-V + APICv/AVIC move
HV_APIC_ACCESS_RECOMMENDED setting out of kvm_hyperv_properties[]: the
'real' feature bit for the vAPIC features is HV_APIC_ACCESS_AVAILABLE,
HV_APIC_ACCESS_RECOMMENDED is a recommendation to use the feature which
we may not always want to give.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20210902093530.345756-6-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 19:04:45 +02:00
Paolo Bonzini 768f14f94e docs: reorganize qgraph.rst
Clean up the heading levels to use === --- ~~~, and move the command line
building near to the other execution steps.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 19:04:45 +02:00
Vitaly Kuznetsov 70367f0917 i386: Support KVM_CAP_HYPERV_ENFORCE_CPUID
By default, KVM allows the guest to use all currently supported Hyper-V
enlightenments when Hyper-V CPUID interface was exposed, regardless of if
some features were not announced in guest visible CPUIDs. hv-enforce-cpuid
feature alters this behavior and only allows the guest to use exposed
Hyper-V enlightenments. The feature is supported by Linux >= 5.14 and is
not enabled by default in QEMU.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20210902093530.345756-5-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 19:04:45 +02:00
Paolo Bonzini 8b8939e44f docs: put "make" information together in build-system.rst
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 19:04:45 +02:00
Vitaly Kuznetsov 988f7b8bfe i386: Support KVM_CAP_ENFORCE_PV_FEATURE_CPUID
By default, KVM allows the guest to use all currently supported PV features
even when they were not announced in guest visible CPUIDs. Introduce a new
"kvm-pv-enforce-cpuid" flag to limit the supported feature set to the
exposed features. The feature is supported by Linux >= 5.10 and is not
enabled by default in QEMU.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20210902093530.345756-4-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 19:04:45 +02:00
Paolo Bonzini f9df7aac75 docs: move notes inside the body of the document
Make all documents start with a heading.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 19:04:45 +02:00
Paolo Bonzini bcfdfae78f docs: name included files ".rst.inc"
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 19:04:45 +02:00
Yanan Wang 8bdfec393a machine: Put all sanity-check in the generic SMP parser
Put both sanity-check of the input SMP configuration and sanity-check
of the output SMP configuration uniformly in the generic parser. Then
machine_set_smp() will become cleaner, also all the invalid scenarios
can be tested only by calling the parser.

Signed-off-by: Yanan Wang <wangyanan55@huawei.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Pankaj Gupta <pankaj.gupta@ionos.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210929025816.21076-16-wangyanan55@huawei.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 19:04:31 +02:00
Paolo Bonzini e7f944bb94 machine: Use g_autoptr in machine_set_smp
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 19:04:03 +02:00
Yanan Wang 2b52619994 machine: Move smp_prefer_sockets to struct SMPCompatProps
Now we have a common structure SMPCompatProps used to store information
about SMP compatibility stuff, so we can also move smp_prefer_sockets
there for cleaner code.

No functional change intended.

Signed-off-by: Yanan Wang <wangyanan55@huawei.com>
Acked-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210929025816.21076-15-wangyanan55@huawei.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 15:29:15 +02:00
Yanan Wang 7687b2b3ed machine: Remove smp_parse callback from MachineClass
Now we have a generic smp parser for all arches, and there will
not be any other arch specific ones, so let's remove the callback
from MachineClass and call the parser directly.

Signed-off-by: Yanan Wang <wangyanan55@huawei.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210929025816.21076-14-wangyanan55@huawei.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 15:28:19 +02:00
Yanan Wang e4a97a893b machine: Make smp_parse generic enough for all arches
Currently the only difference between smp_parse and pc_smp_parse
is the support of dies parameter and the related error reporting.
With some arch compat variables like "bool dies_supported", we can
make smp_parse generic enough for all arches and the PC specific
one can be removed.

Making smp_parse() generic enough can reduce code duplication and
ease the code maintenance, and also allows extending the topology
with more arch specific members (e.g., clusters) in the future.

Suggested-by: Andrew Jones <drjones@redhat.com>
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Yanan Wang <wangyanan55@huawei.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210929025816.21076-13-wangyanan55@huawei.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 15:28:19 +02:00
Yanan Wang 003f230e37 machine: Tweak the order of topology members in struct CpuTopology
Now that all the possible topology parameters are integrated in struct
CpuTopology, tweak the order of topology members to be "cpus/sockets/
dies/cores/threads/maxcpus" for readability and consistency. We also
tweak the comment by adding explanation of dies parameter.

Signed-off-by: Yanan Wang <wangyanan55@huawei.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Pankaj Gupta <pankaj.gupta@ionos.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210929025816.21076-12-wangyanan55@huawei.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 15:28:19 +02:00
Yanan Wang 69fc28a78d machine: Use ms instead of global current_machine in sanity-check
In the sanity-check of smp_cpus and max_cpus against mc in function
machine_set_smp(), we are now using ms->smp.max_cpus for the check
but using current_machine->smp.max_cpus in the error message.
Tweak this by uniformly using the local ms.

Signed-off-by: Yanan Wang <wangyanan55@huawei.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Pankaj Gupta <pankaj.gupta@ionos.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210929025816.21076-11-wangyanan55@huawei.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 15:28:19 +02:00
Yanan Wang 4a0af2930a machine: Prefer cores over sockets in smp parsing since 6.2
In the real SMP hardware topology world, it's much more likely that
we have high cores-per-socket counts and few sockets totally. While
the current preference of sockets over cores in smp parsing results
in a virtual cpu topology with low cores-per-sockets counts and a
large number of sockets, which is just contrary to the real world.

Given that it is better to make the virtual cpu topology be more
reflective of the real world and also for the sake of compatibility,
we start to prefer cores over sockets over threads in smp parsing
since machine type 6.2 for different arches.

In this patch, a boolean "smp_prefer_sockets" is added, and we only
enable the old preference on older machines and enable the new one
since type 6.2 for all arches by using the machine compat mechanism.

Suggested-by: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Yanan Wang <wangyanan55@huawei.com>
Acked-by: David Gibson <david@gibson.dropbear.id.au>
Acked-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Pankaj Gupta <pankaj.gupta@ionos.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210929025816.21076-10-wangyanan55@huawei.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 15:28:16 +02:00
Yanan Wang bbb0c0ec6d qtest/numa-test: Use detailed -smp CLIs in test_def_cpu_split
Since commit 80d7835749 (qemu-options: rewrite help for -smp options),
the preference of sockets/cores in -smp parsing is considered liable
to change, and actually we are going to change it in a coming commit.
So it'll be more stable to use detailed -smp CLIs in the testcases
that have strong dependency on the parsing results.

Currently, test_def_cpu_split use "-smp 8" and will get 8 CPU sockets
based on current parsing rule. But if we change to prefer cores over
sockets we will get one CPU socket with 8 cores, and this testcase
will not get expected numa set by default on x86_64 (Ok on aarch64).

So now explicitly use "-smp 8,sockets=8" to avoid affect from parsing
logic change.

Signed-off-by: Yanan Wang <wangyanan55@huawei.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210929025816.21076-9-wangyanan55@huawei.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 15:27:56 +02:00
Yanan Wang afc8e9aaa7 qtest/numa-test: Use detailed -smp CLIs in pc_dynamic_cpu_cfg
Since commit 80d7835749 (qemu-options: rewrite help for -smp options),
the preference of sockets/cores in -smp parsing is considered liable
to change, and actually we are going to change it in a coming commit.
So it'll be more stable to use detailed -smp CLIs in testing if we
have strong dependency on the parsing results.

pc_dynamic_cpu_cfg currently assumes/needs that there will be 2 CPU
sockets with "-smp 2". To avoid breaking the test because of parsing
logic change, now explicitly use "-smp 2,sockets=2".

Signed-off-by: Yanan Wang <wangyanan55@huawei.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210929025816.21076-8-wangyanan55@huawei.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 15:27:56 +02:00
Yanan Wang 52082d3ba4 machine: Improve the error reporting of smp parsing
We have two requirements for a valid SMP configuration:
the product of "sockets * cores * threads" must represent all the
possible cpus, i.e., max_cpus, and then must include the initially
present cpus, i.e., smp_cpus.

So we only need to ensure 1) "sockets * cores * threads == maxcpus"
at first and then ensure 2) "maxcpus >= cpus". With a reasonable
order of the sanity check, we can simplify the error reporting code.
When reporting an error message we also report the exact value of
each topology member to make users easily see what's going on.

Signed-off-by: Yanan Wang <wangyanan55@huawei.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Pankaj Gupta <pankaj.gupta@ionos.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210929025816.21076-7-wangyanan55@huawei.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01 15:27:56 +02:00