454e202d52
A basic wildcard matching is supported in both the monitor command
"trace-event" and the events list file. That means you can enable/disable
the events having a common prefix in a batch. For example, virtio-blk trace
events could be enabled using:
trace-event virtio_blk_* on
Signed-off-by: Mark Wu <wudxw@linux.vnet.ibm.com>
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
237 lines
8.6 KiB
Plaintext
237 lines
8.6 KiB
Plaintext
= Tracing =
|
|
|
|
== Introduction ==
|
|
|
|
This document describes the tracing infrastructure in QEMU and how to use it
|
|
for debugging, profiling, and observing execution.
|
|
|
|
== Quickstart ==
|
|
|
|
1. Build with the 'simple' trace backend:
|
|
|
|
./configure --trace-backend=simple
|
|
make
|
|
|
|
2. Create a file with the events you want to trace:
|
|
|
|
echo bdrv_aio_readv > /tmp/events
|
|
echo bdrv_aio_writev >> /tmp/events
|
|
|
|
3. Run the virtual machine to produce a trace file:
|
|
|
|
qemu -trace events=/tmp/events ... # your normal QEMU invocation
|
|
|
|
4. Pretty-print the binary trace file:
|
|
|
|
./simpletrace.py trace-events trace-*
|
|
|
|
== Trace events ==
|
|
|
|
There is a set of static trace events declared in the "trace-events" source
|
|
file. Each trace event declaration names the event, its arguments, and the
|
|
format string which can be used for pretty-printing:
|
|
|
|
qemu_vmalloc(size_t size, void *ptr) "size %zu ptr %p"
|
|
qemu_vfree(void *ptr) "ptr %p"
|
|
|
|
The "trace-events" file is processed by the "tracetool" script during build to
|
|
generate code for the trace events. Trace events are invoked directly from
|
|
source code like this:
|
|
|
|
#include "trace.h" /* needed for trace event prototype */
|
|
|
|
void *qemu_vmalloc(size_t size)
|
|
{
|
|
void *ptr;
|
|
size_t align = QEMU_VMALLOC_ALIGN;
|
|
|
|
if (size < align) {
|
|
align = getpagesize();
|
|
}
|
|
ptr = qemu_memalign(align, size);
|
|
trace_qemu_vmalloc(size, ptr);
|
|
return ptr;
|
|
}
|
|
|
|
=== Declaring trace events ===
|
|
|
|
The "tracetool" script produces the trace.h header file which is included by
|
|
every source file that uses trace events. Since many source files include
|
|
trace.h, it uses a minimum of types and other header files included to keep the
|
|
namespace clean and compile times and dependencies down.
|
|
|
|
Trace events should use types as follows:
|
|
|
|
* Use stdint.h types for fixed-size types. Most offsets and guest memory
|
|
addresses are best represented with uint32_t or uint64_t. Use fixed-size
|
|
types over primitive types whose size may change depending on the host
|
|
(32-bit versus 64-bit) so trace events don't truncate values or break
|
|
the build.
|
|
|
|
* Use void * for pointers to structs or for arrays. The trace.h header
|
|
cannot include all user-defined struct declarations and it is therefore
|
|
necessary to use void * for pointers to structs.
|
|
|
|
* For everything else, use primitive scalar types (char, int, long) with the
|
|
appropriate signedness.
|
|
|
|
Format strings should reflect the types defined in the trace event. Take
|
|
special care to use PRId64 and PRIu64 for int64_t and uint64_t types,
|
|
respectively. This ensures portability between 32- and 64-bit platforms.
|
|
|
|
=== Hints for adding new trace events ===
|
|
|
|
1. Trace state changes in the code. Interesting points in the code usually
|
|
involve a state change like starting, stopping, allocating, freeing. State
|
|
changes are good trace events because they can be used to understand the
|
|
execution of the system.
|
|
|
|
2. Trace guest operations. Guest I/O accesses like reading device registers
|
|
are good trace events because they can be used to understand guest
|
|
interactions.
|
|
|
|
3. Use correlator fields so the context of an individual line of trace output
|
|
can be understood. For example, trace the pointer returned by malloc and
|
|
used as an argument to free. This way mallocs and frees can be matched up.
|
|
Trace events with no context are not very useful.
|
|
|
|
4. Name trace events after their function. If there are multiple trace events
|
|
in one function, append a unique distinguisher at the end of the name.
|
|
|
|
5. If specific trace events are going to be called a huge number of times, this
|
|
might have a noticeable performance impact even when the trace events are
|
|
programmatically disabled. In this case you should declare the trace event
|
|
with the "disable" property, which will effectively disable it at compile
|
|
time (using the "nop" backend).
|
|
|
|
== Generic interface and monitor commands ==
|
|
|
|
You can programmatically query and control the dynamic state of trace events
|
|
through a backend-agnostic interface:
|
|
|
|
* trace_print_events
|
|
|
|
* trace_event_set_state
|
|
Enables or disables trace events at runtime inside QEMU.
|
|
The function returns "true" if the state of the event has been successfully
|
|
changed, or "false" otherwise:
|
|
|
|
#include "trace/control.h"
|
|
|
|
trace_event_set_state("virtio_irq", true); /* enable */
|
|
[...]
|
|
trace_event_set_state("virtio_irq", false); /* disable */
|
|
|
|
Note that some of the backends do not provide an implementation for this
|
|
interface, in which case QEMU will just print a warning.
|
|
|
|
This functionality is also provided through monitor commands:
|
|
|
|
* info trace-events
|
|
View available trace events and their state. State 1 means enabled, state 0
|
|
means disabled.
|
|
|
|
* trace-event NAME on|off
|
|
Enable/disable a given trace event or a group of events having common prefix
|
|
through wildcard.
|
|
|
|
The "-trace events=<file>" command line argument can be used to enable the
|
|
events listed in <file> from the very beginning of the program. This file must
|
|
contain one event name per line.
|
|
|
|
A basic wildcard matching is supported in both the monitor command "trace
|
|
-event" and the events list file. That means you can enable/disable the events
|
|
having a common prefix in a batch. For example, virtio-blk trace events could
|
|
be enabled using:
|
|
trace-event virtio_blk_* on
|
|
|
|
== Trace backends ==
|
|
|
|
The "tracetool" script automates tedious trace event code generation and also
|
|
keeps the trace event declarations independent of the trace backend. The trace
|
|
events are not tightly coupled to a specific trace backend, such as LTTng or
|
|
SystemTap. Support for trace backends can be added by extending the "tracetool"
|
|
script.
|
|
|
|
The trace backend is chosen at configure time and only one trace backend can
|
|
be built into the binary:
|
|
|
|
./configure --trace-backend=simple
|
|
|
|
For a list of supported trace backends, try ./configure --help or see below.
|
|
|
|
The following subsections describe the supported trace backends.
|
|
|
|
=== Nop ===
|
|
|
|
The "nop" backend generates empty trace event functions so that the compiler
|
|
can optimize out trace events completely. This is the default and imposes no
|
|
performance penalty.
|
|
|
|
Note that regardless of the selected trace backend, events with the "disable"
|
|
property will be generated with the "nop" backend.
|
|
|
|
=== Stderr ===
|
|
|
|
The "stderr" backend sends trace events directly to standard error. This
|
|
effectively turns trace events into debug printfs.
|
|
|
|
This is the simplest backend and can be used together with existing code that
|
|
uses DPRINTF().
|
|
|
|
=== Simpletrace ===
|
|
|
|
The "simple" backend supports common use cases and comes as part of the QEMU
|
|
source tree. It may not be as powerful as platform-specific or third-party
|
|
trace backends but it is portable. This is the recommended trace backend
|
|
unless you have specific needs for more advanced backends.
|
|
|
|
The "simple" backend currently does not capture string arguments, it simply
|
|
records the char* pointer value instead of the string that is pointed to.
|
|
|
|
==== Monitor commands ====
|
|
|
|
* info trace
|
|
Display the contents of trace buffer. This command dumps the trace buffer
|
|
with simple formatting. For full pretty-printing, use the simpletrace.py
|
|
script on a binary trace file.
|
|
|
|
The trace buffer is written into until full. The full trace buffer is
|
|
flushed and emptied. This means the 'info trace' will display few or no
|
|
entries if the buffer has just been flushed.
|
|
|
|
* trace-file on|off|flush|set <path>
|
|
Enable/disable/flush the trace file or set the trace file name.
|
|
|
|
==== Analyzing trace files ====
|
|
|
|
The "simple" backend produces binary trace files that can be formatted with the
|
|
simpletrace.py script. The script takes the "trace-events" file and the binary
|
|
trace:
|
|
|
|
./simpletrace.py trace-events trace-12345
|
|
|
|
You must ensure that the same "trace-events" file was used to build QEMU,
|
|
otherwise trace event declarations may have changed and output will not be
|
|
consistent.
|
|
|
|
=== LTTng Userspace Tracer ===
|
|
|
|
The "ust" backend uses the LTTng Userspace Tracer library. There are no
|
|
monitor commands built into QEMU, instead UST utilities should be used to list,
|
|
enable/disable, and dump traces.
|
|
|
|
=== SystemTap ===
|
|
|
|
The "dtrace" backend uses DTrace sdt probes but has only been tested with
|
|
SystemTap. When SystemTap support is detected a .stp file with wrapper probes
|
|
is generated to make use in scripts more convenient. This step can also be
|
|
performed manually after a build in order to change the binary name in the .stp
|
|
probes:
|
|
|
|
scripts/tracetool --dtrace --stap \
|
|
--binary path/to/qemu-binary \
|
|
--target-type system \
|
|
--target-arch x86_64 \
|
|
<trace-events >qemu.stp
|