When we come to emulating Thumb instructions then, to interwork
correctly, the code on in the instruction slot must be invoked with a
function pointer which has the least significant bit set. Rather that
set this by hand in every Thumb emulation function we will add a new
field for this purpose to arch_specific_insn, called insn_fn.
This also enables us to seamlessly share emulation functions between ARM
and Thumb code.
Signed-off-by: Jon Medhurst <tixy@yxit.co.uk>
Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
When a probe fires we must single-step the instruction which was
replaced by a breakpoint. As the steps to do this vary between ARM and
Thumb instructions we need a way to customise single-stepping.
This is done by adding a new hook called insn_singlestep to
arch_specific_insn which is initialised by the instruction decoding
functions.
These single-step hooks must update PC and call the instruction handler.
For Thumb instructions an additional step of updating ITSTATE is needed.
We do this after calling the handler because some handlers will need to
test if they are running in an IT block.
Signed-off-by: Jon Medhurst <tixy@yxit.co.uk>
Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
Now we no longer trigger probes on conditional instructions when the
condition is false, we can make use of conditional instructions as
breakpoints in ARM code to avoid taking unnecessary exceptions.
Note, we can't rely on not getting an exception when the condition check
fails, as that is Implementation Defined on newer ARM architectures. We
therefore still need to perform manual condition checks as well.
Signed-off-by: Jon Medhurst <tixy@yxit.co.uk>
Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
This patch changes the behavior of kprobes on ARM so that:
Kprobes on conditional instructions don't trigger when the
condition is false. For conditional branches, this means that
they don't trigger in the branch not taken case.
Rationale:
When probes are placed onto conditionally executed instructions in a
Thumb IT block, they may not fire if the condition is not met. This
is because we use invalid instructions for breakpoints and "it is
IMPLEMENTATION DEFINED whether the instruction executes as a NOP or
causes an Undefined Instruction exception". Therefore, for consistency,
we will ignore all probes on any conditional instructions when the
condition is false. Alternative solutions seem to be too complex to
implement or inconsistent.
This issue was discussed on linux.arm.kernel in the thread titled
"[RFC] kprobes with thumb2 conditional code" See
http://comments.gmane.org/gmane.linux.linaro.devel/2985
Signed-off-by: Jon Medhurst <tixy@yxit.co.uk>
Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
Extend the breakpoint insertion and catching functions to support Thumb
code.
As breakpoints are no longer of a fixed size, the flush_insns macro
is modified to take a size argument instead of an instruction count.
Note, we need both 16- and 32-bit Thumb breakpoints, because if we
were to use a 16-bit breakpoint to replace a 32-bit instruction which
was in an IT block, and the condition check failed, then the breakpoint
may not fire (it's unpredictable behaviour) and the CPU could then try
and execute the second half of the 32-bit Thumb instruction.
Signed-off-by: Jon Medhurst <tixy@yxit.co.uk>
Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
Extend arch_prepare_kprobe to support probing of Thumb code. For
the actual decoding of Thumb instructions, stub functions are
added which currently just reject the probe.
Signed-off-by: Jon Medhurst <tixy@yxit.co.uk>
Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
Fix up kprobes framework so that it builds and correctly interworks on
Thumb-2 kernels.
Signed-off-by: Jon Medhurst <tixy@yxit.co.uk>
Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
Later, we will be adding a considerable amount of internal
implementation definitions to kprobe header files and it would be good
to have these in local header file along side the source code, rather
than pollute the existing header which is include by all users of
kprobes.
To this end, we add arch/arm/kernel/kprobes.h and move into this the
existing internal defintions from arch/arm/include/asm/kprobes.h
Signed-off-by: Jon Medhurst <tixy@yxit.co.uk>
Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
When a kprobe is placed onto conditionally executed ARM instructions,
many of the emulation routines used to single step them produce corrupt
register results. Rather than fix all of these cases we modify the
framework which calls them to test the relevant condition flags and, if
the test fails, skip calling the emulation code.
Signed-off-by: Jon Medhurst <tixy@yxit.co.uk>
Signed-off-by: Nicolas Pitre <nicolas.pitre@linaro.org>
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
Current implementation of jprobes allocates empty pt_regs from the
stack which is then passed to kprobe_handler() and eventually to
singlestep(). Now when instruction being simulated is STMFD (like
in normal function prologues without CONFIG_FRAME_POINTER), stores
using SP actually write over top of the fabricated pt_regs
structure.
This can be reproduced for example by using LKDTM module:
# modprobe lkdtm
# mount -t debugfs none /sys/kernel/debug
# echo PANIC > /sys/kernel/debug/provoke-crash/INT_HW_IRQ_EN
after this, it fails with corrupted registers (before the requested crash would occur):
lkdtm: Crash point INT_HW_IRQ_EN of type PANIC hit, trigger in 9 rounds
lkdtm: Crash point INT_HW_IRQ_EN of type PANIC hit, trigger in 8 rounds
Internal error: Oops - undefined instruction: 0 [#1]
last sysfs file: /sys/devices/platform/serial8250.0/sleep_timeout
Modules linked in: lkdtm
CPU: 0 Not tainted (2.6.34-rc2 #69)
PC is at irq_desc+0x1638/0xeeb0
LR is at 0x25
pc : [<c050b428>] lr : [<00000025>] psr: c80a0013
sp : ce94bd60 ip : c050b3e8 fp : a0000013
r10: c0aa453c r9 : cf5d4000 r8 : ce9a1822
r7 : c050b424 r6 : 00000025 r5 : c039d8f8 r4 : c050b3e8
r3 : 00000001 r2 : cf4d0440 r1 : c039d8f8 r0 : 00000020
Flags: NZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 10c5387d Table: 8e804019 DAC: 00000015
Process sh (pid: 496, stack limit = 0xce94a2e8)
Stack: (0xce94bd60 to 0xce94c000)
[...]
Code: 000002cd 00000000 00000000 00000001 (dead4ead)
---[ end trace 2b46d5f2b682f370 ]---
Kernel panic - not syncing: Fatal exception in interrupt
This patch allocates enough space (2 * sizeof(struct pt_regs)) from
the stack to prevent such corruption.
Signed-off-by: Mika Westerberg <ext-mika.1.westerberg@nokia.com>
Acked-by: Nicolas Pitre <nico@marvell.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM kprobes use an illegal instruction to trigger kprobes. In the
current implementation, there's a race between the unregistration of a
kprobe and the illegal instruction exception handler if they run at the
same time on different cores.
When reading the value of the undefined instruction, the exception
handler might get the original legal instruction as just patched
concurrently by arch_disarm_kprobe(). When this happen the kprobe
handler won't run, and thus the exception handler will oops because it
believe it just hit an undefined instruction in kernel space.
The following patch synchronizes the code patching in the kprobes
unregistration using stop_machine and thus avoids the above race.
Signed-off-by: Frederic RISS <frederic.riss@gmail.com>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Add kprobe_insn_mutex for protecting kprobe_insn_pages hlist, and remove
kprobe_mutex from architecture dependent code.
This allows us to call arch_remove_kprobe() (and free_insn_slot) while
holding kprobe_mutex.
Signed-off-by: Masami Hiramatsu <mhiramat@redhat.com>
Acked-by: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Cc: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: "Luck, Tony" <tony.luck@intel.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
As mentioned in commit 796969104c,
and because of commit b03a5b7559,
the direct calling of kprobe_trap_handler() can be removed.
Signed-off-by: Nicolas Pitre <nico@marvell.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Currently list of kretprobe instances are stored in kretprobe object (as
used_instances,free_instances) and in kretprobe hash table. We have one
global kretprobe lock to serialise the access to these lists. This causes
only one kretprobe handler to execute at a time. Hence affects system
performance, particularly on SMP systems and when return probe is set on
lot of functions (like on all systemcalls).
Solution proposed here gives fine-grain locks that performs better on SMP
system compared to present kretprobe implementation.
Solution:
1) Instead of having one global lock to protect kretprobe instances
present in kretprobe object and kretprobe hash table. We will have
two locks, one lock for protecting kretprobe hash table and another
lock for kretporbe object.
2) We hold lock present in kretprobe object while we modify kretprobe
instance in kretprobe object and we hold per-hash-list lock while
modifying kretprobe instances present in that hash list. To prevent
deadlock, we never grab a per-hash-list lock while holding a kretprobe
lock.
3) We can remove used_instances from struct kretprobe, as we can
track used instances of kretprobe instances using kretprobe hash
table.
Time duration for kernel compilation ("make -j 8") on a 8-way ppc64 system
with return probes set on all systemcalls looks like this.
cacheline non-cacheline Un-patched kernel
aligned patch aligned patch
===============================================================================
real 9m46.784s 9m54.412s 10m2.450s
user 40m5.715s 40m7.142s 40m4.273s
sys 2m57.754s 2m58.583s 3m17.430s
===========================================================
Time duration for kernel compilation ("make -j 8) on the same system, when
kernel is not probed.
=========================
real 9m26.389s
user 40m8.775s
sys 2m7.283s
=========================
Signed-off-by: Srinivasa DS <srinivasa@in.ibm.com>
Signed-off-by: Jim Keniston <jkenisto@us.ibm.com>
Acked-by: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Cc: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: Masami Hiramatsu <mhiramat@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Follow suit from kprobe implementations on other archs and make kretprobe_trampoline non-static. Ftrace implmentation (more specifically, kernel/trace/trace.c) requires access to it (see-> http://kerneltrap.org/mailarchive/linux-kernel/2008/5/27/1955234).
Signed-off-by: Abhishek Sagar <sagar.abhishek@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
It is more useful to flush the cache with the actual buffer address
rather than the address containing a pointer to the buffer.
Signed-off-by: Nicolas Pitre <nico@marvell.com>
Acked-by: Lennert Buytenhek <buytenh@marvell.com>
The kprobes code is already able to cope with reentrant probes, so its
handler must be called outside of the region protected by undef_lock.
If ever this lock is released when handlers are called then this commit
could be reverted.
Signed-off-by: Nicolas Pitre <nico@marvell.com>
This is a full implementation of Kprobes including Jprobes and
Kretprobes support.
This ARM implementation does not follow the usual kprobes double-
exception model. The traditional model is where the initial kprobes
breakpoint calls kprobe_handler(), which returns from exception to
execute the instruction in its original context, then immediately
re-enters after a second breakpoint (or single-stepping exception)
into post_kprobe_handler(), each time the probe is hit.. The ARM
implementation only executes one kprobes exception per hit, so no
post_kprobe_handler() phase. All side-effects from the kprobe'd
instruction are resolved before returning from the initial exception.
As a result, all instructions are _always_ effectively boosted
regardless of the type of instruction, and even regardless of whether
or not there is a post-handler for the probe.
Signed-off-by: Abhishek Sagar <sagar.abhishek@gmail.com>
Signed-off-by: Quentin Barnes <qbarnes@gmail.com>
Signed-off-by: Nicolas Pitre <nico@marvell.com>
Jon Medhurst