Initially should look loosely like the venerable 'strace' tool, but
using the infrastructure in the perf tools to allow tracing extra
targets:
[acme@sandy linux]$ perf trace --hell
Error: unknown option `hell'
usage: perf trace <PID>
-p, --pid <pid> trace events on existing process id
--tid <tid> trace events on existing thread id
--all-cpus system-wide collection from all CPUs
--cpu <cpu> list of cpus to monitor
--no-inherit child tasks do not inherit counters
--mmap-pages <n> number of mmap data pages
--uid <user> user to profile
[acme@sandy linux]$
Those should have the same semantics as when using with 'perf record'.
It gets stuck sometimes, but hey, it works sometimes too!
In time it should support perf.data based workloads, i.e. it should have
a:
-o filename
Command line option that will produce a perf.data file that can then be
used with 'perf trace' or any of the other perf tools (script, report,
etc).
It will also eventually have the set of functionalities described in the
previous 'trace' prototype by Thomas Gleixner:
"Announcing a new utility: 'trace'"
http://lwn.net/Articles/415728/
Also planned is to have some of the features suggested in the comments
of that LWN article.
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Namhyung Kim <namhyung@gmail.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Now we have isolated all ELF-specific stuff, it's possible to build
without libelf. The output binary can do most of jobs but lacks (user
level) symbol information - kernel symbols are still accessable thanks
to the kallsyms.
To build perf without libelf (elfutils), give NO_LIBELF=1 to make.
For now, only 'perf probe' command is removed since it depends on
libelf/libdw heavily.
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1344228082-15569-4-git-send-email-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Merge reason: This is an older commit under testing that was not pushed yet - merge it.
Also fix up the merge in command-list.txt.
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Acked-by: Tom Zanussi <tzanussi@gmail.com>
New commands need to have Documentation and be added to command-list.txt
so that they can appear when 'perf' is called withouth any subcommand:
[root@doppio linux-2.6-tip]# perf
usage: perf [--version] [--help] COMMAND [ARGS]
The most commonly used perf commands are:
annotate Read perf.data (created by perf record) and display annotated code
archive Create archive with object files with build-ids found in perf.data file
bench General framework for benchmark suites
buildid-cache Manage build-id cache.
buildid-list List the buildids in a perf.data file
diff Read two perf.data files and display the differential profile
inject Filter to augment the events stream with additional information
kmem Tool to trace/measure kernel memory(slab) properties
kvm Tool to trace/measure kvm guest os
list List all symbolic event types
lock Analyze lock events
probe Define new dynamic tracepoints
record Run a command and record its profile into perf.data
report Read perf.data (created by perf record) and display the profile
sched Tool to trace/measure scheduler properties (latencies)
stat Run a command and gather performance counter statistics
test Runs sanity tests.
timechart Tool to visualize total system behavior during a workload
top System profiling tool.
trace Read perf.data (created by perf record) and display trace output
See 'perf help COMMAND' for more information on a specific command.
[root@doppio linux-2.6-tip]#
The new 'perf inject' command hadn't so it wasn't appearing on that list.
Also fix the long option, that should have no spaces in it, rename the faulty one
to be '--build-ids', instead of '--inject build-ids'.
Reported-by: Ingo Molnar <mingo@elte.hu>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
First an example with the first internal test:
[acme@doppio linux-2.6-tip]$ perf test
1: vmlinux symtab matches kallsyms: Ok
So it run just one test, that is "vmlinux symtab matches kallsyms", and it was
successful.
If we run it in verbose mode, we'll see details about errors and extra warnings
for non-fatal problems:
[acme@doppio linux-2.6-tip]$ perf test -v
1: vmlinux symtab matches kallsyms:
--- start ---
Looking at the vmlinux_path (5 entries long)
No build_id in vmlinux, ignoring it
No build_id in /boot/vmlinux, ignoring it
No build_id in /boot/vmlinux-2.6.34-rc4-tip+, ignoring it
Using /lib/modules/2.6.34-rc4-tip+/build/vmlinux for symbols
Maps only in vmlinux:
ffffffff81cb81b1-ffffffff81e1149b 0 [kernel].init.text
ffffffff81e1149c-ffffffff9fffffff 0 [kernel].exit.text
ffffffffff600000-ffffffffff6000ff 0 [kernel].vsyscall_0
ffffffffff600100-ffffffffff6003ff 0 [kernel].vsyscall_fn
ffffffffff600400-ffffffffff6007ff 0 [kernel].vsyscall_1
ffffffffff600800-ffffffffffffffff 0 [kernel].vsyscall_2
Maps in vmlinux with a different name in kallsyms:
ffffffffff600000-ffffffffff6000ff 0 [kernel].vsyscall_0 in kallsyms as [kernel].0
ffffffffff600100-ffffffffff6003ff 0 [kernel].vsyscall_fn in kallsyms as:
*ffffffffff600100-ffffffffff60012f 0 [kernel].2
ffffffffff600400-ffffffffff6007ff 0 [kernel].vsyscall_1 in kallsyms as [kernel].6
ffffffffff600800-ffffffffffffffff 0 [kernel].vsyscall_2 in kallsyms as [kernel].8
Maps only in kallsyms:
ffffffffff600130-ffffffffff6003ff 0 [kernel].4
---- end ----
vmlinux symtab matches kallsyms: Ok
[acme@doppio linux-2.6-tip]$
In the above case we only know the name of the non contiguous kernel ranges in
the address space when reading the symbol information from the ELF symtab in
vmlinux.
The /proc/kallsyms file lack this, we only notice they are separate because
there are modules after the kernel and after that more kernel functions, so we
need to have a module rbtree backed by the module .ko path to get symtabs in
the vmlinux case.
The tool uses it to match by address to emit appropriate warning, but don't
considers this fatal.
The .init.text and .exit.text ines, of course, aren't in kallsyms, so I left
these cases just as extra info in verbose mode.
The end of the sections also aren't in kallsyms, so we the symbols layer does
another pass and sets the end addresses as the next map start minus one, which
sometimes pads, causing harmless mismatches.
But at least the symbols match, tested it by copying /proc/kallsyms to
/tmp/kallsyms and doing changes to see if they were detected.
This first test also should serve as a first stab at documenting the
symbol library by providing a self contained example that exercises it
together with comments about what is being done.
More tests to check if actions done on a monitored app, like doing mmaps, etc,
makes the kernel generate the expected events should be added next.
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
I've forgot to add 'perf lock' line to command-list.txt,
so users of perf could not find perf lock when they type 'perf'.
Fixing command-list.txt requires document
(tools/perf/Documentation/perf-lock.txt).
But perf lock is too much "under construction" to write a
stable document, so this is something like pseudo document for now.
And I wrote description of perf lock at help section of
CONFIG_LOCK_STAT, this will navigate users of lock trace events.
Signed-off-by: Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
LKML-Reference: <1265267295-8388-1-git-send-email-mitake@dcl.info.waseda.ac.jp>
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
For now it just has operations to examine a given file, find its
build-id and add or remove it to/from the cache.
Useful, for instance, when adding binaries sent together with a
perf.data file, so that we can add them to the cache and have
the tools find it when resolving symbols.
It'll also manage the size of the cache like 'ccache' does.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1264008525-29025-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
This also makes it appear on the 'perf --help' output, i.e.
util/generate-cmdlist.sh now takes it into account.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1263837559-24168-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
It uses 'perf buildid-list --with-hits' to create a tarball with
what is needed to have in the destination machine ~/.debug
hierarchy to properly decode the perf.data file specified.
Here is an example where a perf.data file collected on a x86-64
machine running Fedora 12 is used and then the data is packaged,
transferred and decoded on a PARISC64 machine running Debian
Testing, 32-bit userspace:
[root@doppio linux-2.6-tip]# uname -a
Linux doppio.ghostprotocols.net 2.6.33-rc4-tip+ #3 SMP Wed Jan 13 11:58:15 BRST 2010 x86_64 x86_64 x86_64 GNU/Linux
[root@doppio linux-2.6-tip]# perf archive
[root@doppio linux-2.6-tip]# ls -la perf.data*
-rw------- 1 root root 737696 2010-01-14 23:36 perf.data
-rw-r--r-- 1 root root 8840025 2010-01-15 12:27 perf.data.tar.bz2
[root@doppio linux-2.6-tip]# scp perf.data.* parisc64:.
Password:
perf.data.tar.bz2 100% 8633KB 1.4MB/s 00:06
[root@doppio linux-2.6-tip]# ssh parisc64
Password:
Linux parisc 2.6.19-g2bbf29ac-dirty #1 Sun Dec 3 17:24:04 BRST 2006 parisc64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Jan 14 11:23:24 2010 from d
parisc:~# uname -a
Linux parisc 2.6.19-g2bbf29ac-dirty #1 Sun Dec 3 17:24:04 BRST 2006 parisc64 GNU/Linux
parisc:~# mkdir .debug
parisc:~# tar xvf perf.data.tar.bz2 -C ~/.debug
tar: Record size = 8 blocks
.build-id/74/f9930ee94475b6b3238caf3725a50d59cb994b
[kernel.kallsyms]/74f9930ee94475b6b3238caf3725a50d59cb994b
.build-id/9f/fdcac0a7935922d1f04b6cc9029dfef0f066ef
lib/modules/2.6.33-rc4-tip+/kernel/arch/x86/crypto/aes-x86_64.ko/9ffdcac0a7935922d1f04b6cc9029dfef0f066ef
.build-id/3a/af89c32ebfc438ff546c93597d41788e3e65f3
lib/modules/2.6.33-rc4-tip+/kernel/drivers/net/wireless/iwlwifi/iwl3945.ko/3aaf89c32ebfc438ff546c93597d41788e3e65f3
.build-id/19/f46033f73e1ec612937189bb118c5daba5a0c8
lib/modules/2.6.33-rc4-tip+/kernel/net/mac80211/mac80211.ko/19f46033f73e1ec612937189bb118c5daba5a0c8
.build-id/17/72f014a7a7272859655acb0c64a20ab20b75ee
lib/modules/2.6.33-rc4-tip+/kernel/drivers/net/e1000e/e1000e.ko/1772f014a7a7272859655acb0c64a20ab20b75ee
.build-id/eb/4ec8fa8b2a5eb18cad173c92f27ed8887ed1c1
lib64/libc-2.10.2.so/eb4ec8fa8b2a5eb18cad173c92f27ed8887ed1c1
.build-id/5c/68f7afeb33309c78037e374b0deee84dd441f6
lib64/libpthread-2.10.2.so/5c68f7afeb33309c78037e374b0deee84dd441f6
.build-id/e9/c9ad5c138ef882e4507d2605645b597da43873
bin/dbus-daemon/e9c9ad5c138ef882e4507d2605645b597da43873
.build-id/bc/da7d09eb6c9ee380dae0ed3d591d4311decc31
lib64/libdbus-1.so.3.4.0/bcda7d09eb6c9ee380dae0ed3d591d4311decc31
.build-id/7c/c449a77f48b85d6088114000e970ced613bed8
usr/lib64/libcrypto.so.0.9.8k/7cc449a77f48b85d6088114000e970ced613bed8
.build-id/fd/d1ccd1ff7917ab020653147ab3bacf0a85b5b9
lib64/libglib-2.0.so.0.2000.5/fdd1ccd1ff7917ab020653147ab3bacf0a85b5b9
.build-id/e4/417ebb8762e5f2eee93c8011a71115ff5edad8
lib64/libgobject-2.0.so.0.2000.5/e4417ebb8762e5f2eee93c8011a71115ff5edad8
.build-id/93/1e49461f6df99104f0febcc52f6fed5e2efce6
usr/sbin/sshd/931e49461f6df99104f0febcc52f6fed5e2efce6
.build-id/da/b5f724c088f89fbd8304da553ed6cb30bbec96
usr/lib64/libgdk-x11-2.0.so.0.1600.6/dab5f724c088f89fbd8304da553ed6cb30bbec96
.build-id/f2/037a091ef36b591187a858d75e203690ea9409
usr/sbin/openvpn/f2037a091ef36b591187a858d75e203690ea9409
.build-id/a8/e4f743b40fb1fd8b85e2f9b88d93b661472b8f
bin/find/a8e4f743b40fb1fd8b85e2f9b88d93b661472b8f
.build-id/81/120aada06e68b1e85882925a0fc6d7345ef59a
home/acme/bin/perf/81120aada06e68b1e85882925a0fc6d7345ef59a
parisc:~# perf report 2> /dev/null | head -25
9.07% find find [.] 0x0000000000fb0e
3.29% perf libc-2.10.2.so [.] __GI_strcmp
3.19% find [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore
2.70% find libc-2.10.2.so [.] __GI_memmove
2.62% perf [kernel.kallsyms] [k] vsnprintf
2.03% find libc-2.10.2.so [.] _int_malloc
2.02% perf [kernel.kallsyms] [k] format_decode
1.70% find [kernel.kallsyms] [k] n_tty_write
1.70% find [kernel.kallsyms] [k] half_md4_transform
1.67% find libc-2.10.2.so [.] _IO_vfprintf_internal
1.66% perf [kernel.kallsyms] [k] audit_free_aux
1.62% swapper [kernel.kallsyms] [k] mwait_idle_with_hints
1.58% find [kernel.kallsyms] [k] __kmalloc
1.35% find [kernel.kallsyms] [k] sched_clock_local
1.35% find [kernel.kallsyms] [k] ext4_check_dir_entry
1.35% find [kernel.kallsyms] [k] ext4_htree_store_dirent
1.35% find [kernel.kallsyms] [k] sys_write
1.35% find [e1000e] [k] e1000_clean
1.35% find [kernel.kallsyms] [k] _atomic_dec_and_lock
1.34% find [kernel.kallsyms] [k] __d_lookup
parisc:~#
Probably the next step is to have 'perf report' notice that there is a
perf.data.tar.bz2 file in the same directory and look if it was already
added to ~/.debug/.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1263568672-30323-2-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
With this we can list the buildids in a perf.data file so that
we can pipe them to other, distro specific tools that from the
buildids can figure out separate packages (foo-debuginfo) where
we can find the matching symtabs so that perf report can do its
job.
E.g:
[acme@doppio linux-2.6-tip]$ perf buildid-list | head -5
8e08b117e5458ad3f85da16d42d0fc5cd21c5869
520c2387a587cc5acfcf881e27dba1caaeab4b1f
ec8dd400904ddfcac8b1c343263a790f977159dc
7caedbca5a6d8ab39a7fe44bd28c07d3e14a3f3f
379bb828fd08859dbea73279f04abefabc95a6a3
[acme@doppio linux-2.6-tip]$ perf buildid-list -v | head -5
8e08b117e5458ad3f85da16d42d0fc5cd21c5869 /sbin/init
520c2387a587cc5acfcf881e27dba1caaeab4b1f /lib64/ld-2.10.1.so
ec8dd400904ddfcac8b1c343263a790f977159dc /lib64/libc-2.10.1.so
7caedbca5a6d8ab39a7fe44bd28c07d3e14a3f3f /sbin/udevd
379bb828fd08859dbea73279f04abefabc95a6a3 /lib64/libdl-2.10.1.so
[acme@doppio linux-2.6-tip]$
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
LKML-Reference: <1258396365-29217-5-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
This patch modifies command-list.txt for the entry of
perf-bench. So perf will show 'bench' in command list.
Example:
% perf
usage: perf [--version] [--help] COMMAND [ARGS]
The most commonly used perf commands are:
annotate Read perf.data (created by perf record) and display annotated code
bench General framework for benchmark suites
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
list List all symbolic event types
probe Define new dynamic tracepoints
record Run a command and record its profile into perf.data
report Read perf.data (created by perf record) and display the profile
sched Tool to trace/measure scheduler properties (latencies)
stat Run a command and gather performance counter statistics
timechart Tool to visualize total system behavior during a workload
top System profiling tool.
trace Read perf.data (created by perf record) and display trace output
See 'perf help COMMAND' for more information on a specific command.
Signed-off-by: Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1257853855-28934-4-git-send-email-mitake@dcl.info.waseda.ac.jp>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
As suggested by Ingo, add a timechart man page help text, as well
as add it to the "perf help" overview.
Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Cc: fweisbec@gmail.com
Cc: peterz@infradead.org
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <20090919133604.3767fa35@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
This turn-key tool allows scheduler measurements to be
conducted and the results be displayed numerically.
First baby step towards that goal: clone the new command off of
perf trace.
Fix a few other details along the way:
- add (minimal) perf trace documentation
- reorder a few places
- list perf trace in the mainporcelain list as well
as it's a very useful utility.
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Several people have suggested that 'perf' has become a full-fledged
tool that should be moved out of Documentation/. Move it to the
(new) tools/ directory.
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
LKML-Reference: <new-submission>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Arnaldo Carvalho de Melo