Christoph Lameter demonstrated very poor scalability on the SGI 512-way, with
a many-threaded application which concurrently initializes different parts of
a large anonymous area.
This patch corrects that, by using a separate spinlock per page table page, to
guard the page table entries in that page, instead of using the mm's single
page_table_lock. (But even then, page_table_lock is still used to guard page
table allocation, and anon_vma allocation.)
In this implementation, the spinlock is tucked inside the struct page of the
page table page: with a BUILD_BUG_ON in case it overflows - which it would in
the case of 32-bit PA-RISC with spinlock debugging enabled.
Splitting the lock is not quite for free: another cacheline access. Ideally,
I suppose we would use split ptlock only for multi-threaded processes on
multi-cpu machines; but deciding that dynamically would have its own costs.
So for now enable it by config, at some number of cpus - since the Kconfig
language doesn't support inequalities, let preprocessor compare that with
NR_CPUS. But I don't think it's worth being user-configurable: for good
testing of both split and unsplit configs, split now at 4 cpus, and perhaps
change that to 8 later.
There is a benefit even for singly threaded processes: kswapd can be attacking
one part of the mm while another part is busy faulting.
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Prepare arm for the split page_table_lock: three issues.
Signal handling's preserve and restore of iwmmxt context currently involves
reading and writing that context to and from user space, while holding
page_table_lock to secure the user page(s) against kswapd. If we split the
lock, then the structure might span two pages, secured by to read into and
write from a kernel stack buffer, copying that out and in without locking (the
structure is 160 bytes in size, and here we're near the top of the kernel
stack). Or would the overhead be noticeable?
arm_syscall's cmpxchg emulation use pte_offset_map_lock, instead of
pte_offset_map and mm-wide page_table_lock; and strictly, it should now also
take mmap_sem before descending to pmd, to guard against another thread
munmapping, and the page table pulled out beneath this thread.
Updated two comments in fault-armv.c. adjust_pte is interesting, since its
modification of a pte in one part of the mm depends on the lock held when
calling update_mmu_cache for a pte in some other part of that mm. This can't
be done with a split page_table_lock (and we've already taken the lowest lock
in the hierarchy here): so we'll have to disable split on arm, unless
CONFIG_CPU_CACHE_VIPT to ensures adjust_pte never used.
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
check_user_page_readable is a problematic variant of follow_page. It's used
only by oprofile's i386 and arm backtrace code, at interrupt time, to
establish whether a userspace stackframe is currently readable.
This is problematic, because we want to push the page_table_lock down inside
follow_page, and later split it; whereas oprofile is doing a spin_trylock on
it (in the i386 case, forgotten in the arm case), and needs that to pin
perhaps two pages spanned by the stackframe (which might be covered by
different locks when we split).
I think oprofile is going about this in the wrong way: it doesn't need to know
the area is readable (neither i386 nor arm uses read protection of user
pages), it doesn't need to pin the memory, it should simply
__copy_from_user_inatomic, and see if that succeeds or not. Sorry, but I've
not got around to devising the sparse __user annotations for this.
Then we can eliminate check_user_page_readable, and return to a single
follow_page without the __follow_page variants.
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Convert those few architectures which are calling pud_alloc, pmd_alloc,
pte_alloc_map on a user mm, not to take the page_table_lock first, nor drop it
after. Each of these can continue to use pte_alloc_map, no need to change
over to pte_alloc_map_lock, they're neither racy nor swappable.
In the sparc64 io_remap_pfn_range, flush_tlb_range then falls outside of the
page_table_lock: that's okay, on sparc64 it's like flush_tlb_mm, and that has
always been called from outside of page_table_lock in dup_mmap.
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
First step in pushing down the page_table_lock. init_mm.page_table_lock has
been used throughout the architectures (usually for ioremap): not to serialize
kernel address space allocation (that's usually vmlist_lock), but because
pud_alloc,pmd_alloc,pte_alloc_kernel expect caller holds it.
Reverse that: don't lock or unlock init_mm.page_table_lock in any of the
architectures; instead rely on pud_alloc,pmd_alloc,pte_alloc_kernel to take
and drop it when allocating a new one, to check lest a racing task already
did. Similarly no page_table_lock in vmalloc's map_vm_area.
Some temporary ugliness in __pud_alloc and __pmd_alloc: since they also handle
user mms, which are converted only by a later patch, for now they have to lock
differently according to whether or not it's init_mm.
If sources get muddled, there's a danger that an arch source taking
init_mm.page_table_lock will be mixed with common source also taking it (or
neither take it). So break the rules and make another change, which should
break the build for such a mismatch: remove the redundant mm arg from
pte_alloc_kernel (ppc64 scrapped its distinct ioremap_mm in 2.6.13).
Exceptions: arm26 used pte_alloc_kernel on user mm, now pte_alloc_map; ia64
used pte_alloc_map on init_mm, now pte_alloc_kernel; parisc had bad args to
pmd_alloc and pte_alloc_kernel in unused USE_HPPA_IOREMAP code; ppc64
map_io_page forgot to unlock on failure; ppc mmu_mapin_ram and ppc64 im_free
took page_table_lock for no good reason.
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Patch from Nicolas Pitre
Since vmlinux.lds.S is preprocessed, we can use the defines already
present in asm/memory.h (allowed by patch #3060) for the XIP kernel link
address instead of relying on a duplicated Makefile hardcoded value, and
also get rid of its dependency on awk to handle it at the same time.
While at it let's clean XIP stuff even further and make things clearer
in head.S with a nice code reduction.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Nicolas Pitre
This patch allows for assorted type of cleanups by letting assembly code
use the same set of defines for constant values and avoid duplicated
definitions that might not always be in sync, or that might simply be
confusing due to the different names for the same thing.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Nicolas Pitre
Fix XIP support after recent bootmem code refactoring.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
This got dropped from the SA1100 flash driver a while back and
never added to the platform support file. Add it back.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
In PM v1, all devices were called at SUSPEND_DISABLE level. Then
all devices were called at SUSPEND_SAVE_STATE level, and finally
SUSPEND_POWER_DOWN level. However, with PM v2, to maintain
compatibility for platform devices, I arranged for the PM v2
suspend/resume callbacks to call the old PM v1 suspend/resume
callbacks three times with each level in order so that existing
drivers continued to work.
Since this is obsolete infrastructure which is no longer necessary,
we can remove it. Here's an (untested) patch to do exactly that.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Patch from Bellido Nicolas
The AAED-2000 is equiped with an 640x480 LCD.
This adds the parameters that will be passed to the AAEC-2000 platform code.
Signed-off-by: Nicolas Bellido <ml@acolin.be>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Bellido Nicolas
The AAEC-2000 has an ARM PrimeCell PL110 Color LCD Controller.
This patch contains the platform glue that will be used by specific boards.
Signed-off-by: Nicolas Bellido <ml@acolin.be>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Bellido Nicolas
Here is a preliminary clock interface support for the AAEC-2000.
Signed-off-by: Nicolas Bellido <ml@acolin.be>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Bellido Nicolas
This adds platform code for MTD devices on AAEC-2000.
Signed-off-by: Nicolas Bellido <ml@acolin.be>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Bellido Nicolas
The AAED-2000 board has GPIO pins on an external port.
This patch adds the defines, and do the necessary mapping.
Signed-off-by: Nicolas Bellido <ml@acolin.be>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Nicolas Pitre
This is the PXA2xx common IRDA driver, plus platform support
for Lubbock and Mainstone.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Acked-by: Jean Tourrilhes <jt@hpl.hp.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Nicolas Pitre
From: Daniel Jacobowitz <dan@debian.org>
> I also fixed a bug that confused me greatly while trying to debug: one
> SIGILL has long been a SIGSEGV because of some broken RISCOS
> compatibility code.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Todd Poynor
Add symbols for PXA2xx PWRMODE register M field that selects low-power
mode, replace unadorned constants. Honor power mode parameter of
pxa_cpu_suspend(mode), no longer force to 3 (sleep). Full Deep Sleep
low-power mode support for PXA27x is pending generic PM interfaces to
select more than 2 suspend-to-RAM-style power modes, but this is
expected soon. This can be hardcoded in the meantime by replacing the
pxa_cpu_suspend() parameter value. From David Burrage and Todd Poynor.
Try #2 removes one of the register copies and moves the code to save the
pxa_cpu_suspend parameter to immediately surround the call that requires
the parameter value be preserved.
Signed-off-by: Todd Poynor <tpoynor@mvista.com>
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Jon Ringle
Updated 2898/1 per comments:
- Removed fixup
- Moved code in mach-mp1000/ to mach-clps711x/
- Cleaned up code in mp1000-seprom.c. Eliminated code that displayed the contents of the eeprom
Please comment.
Signed-off-by: Jon Ringle
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Ian Campbell
As noted by Uli Luckas in the comments of 3025 there is a typo in the i2s platform device. The i2s platform device refers to the i2c resources.
Signed-off-by: Ian Campbell <icampbell@arcom.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Ben Dooks
Add platform data for framebuffer for the
onboard LCD module
Signed-off-by: Ben Dooks <ben-linux@fluff.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Ben Dooks
Add framebuffer platform data
Signed-off-by: Ben Dooks <ben-linux@fluff.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Ben Dooks
Change set_s3c2410fb_info to s3c2410_fb_set_platdata
and use kmalloc() for the copy of the information it
is passed.
Signed-off-by: Ben Dooks <ben-linux@fluff.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Ben Dooks
From: Guillaume GOURAT <guillaume.gourat@nexvision.fr>
A number of devices have an extra byte on the
end of their areas due to mis-calculating the
.end field of their resources
Signed-off-by: Guillaume GOURAT <guillaume.gourat@nexvision.fr>
Signed-off-by: Ben Dooks <ben-linux@fluff.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Ben Dooks
Add generic values for the parameters to the
s3c2410_gpio_cfgpin() function, so that a caller
does not need to know the exact constant for
the specified pin.
This is very useful for the case where a driver
is passed a gpio pin number and needs to reconfigure
the pin's function.
Signed-off-by: Ben Dooks <ben-linux@fluff.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Nicolas Pitre
Here's an ARM assembly SHA1 implementation to replace the default C
version. It is approximately 50% faster than the generic C version. On
an XScale processor running at 400MHz:
generic C version: 9.8 MB/s
my version: 14.5 MB/s
This code is useful to quite a few callers in the tree:
crypto/sha1.c: sha_transform(sctx->state, sctx->buffer, temp);
crypto/sha1.c: sha_transform(sctx->state, &data[i], temp);
drivers/char/random.c: sha_transform(buf, (__u8 *)r->pool+i, buf + 5);
drivers/char/random.c: sha_transform(buf, (__u8 *)data, buf + 5);
net/ipv4/syncookies.c: sha_transform(tmp + 16, (__u8 *)tmp, tmp + 16 + 5);
Signed-off-by: Nicolas Pitre <nico@cam.org>
Seems to work fine on big-endian as well.
Signed-off-by: Lennert Buytenhek <buytenh@wantstofly.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
This patch adds support for 36-bit static mapped I/O. While there
are no platforms in the tree ATM that use it, it has been tested
tested on the IXP2350 NPU and I would like to get the support for
that chipset upstream one piece at a time. There are also other
Intel chipset ports in development that are waiting on this to go
upstream.
The patch replaces the print formats for physical addresses with
%016llx which will create a bit extraneous output on 32-bit systems,
but I think that is cleaner than having #ifdefs, specially since
users will only see the output in error cases.
Depends on 3016/1.
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
Convert map_desc.physical to map_desc.pfn. This allows us to add
support for 36-bit addressed physical devices in the static maps
without having to resort to u64 variables.
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
Integrator map_desc.pfn conversion
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
iMX map_desc.pfn conversion
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
H720x map_desc.pfn conversion
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
Footbridge map_desc.pfn conversion
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
EBSA110 map_desc.pfn conversion
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
EPXA10DB mapd_desc.pfn conversion
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
Versatile map_desc.pfn conversion
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
CLSP7500 map_desc.pfn conversion
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
CLSP711x map_desc.pfn conversion
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
SA1100 map_desc.pfn conversion
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
Shark map_desc.pfn conversion
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
S3C2410 map_desc.pfn conversion
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
RiscPC map_desc.pfn conversion
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
PXA map_desc.pfn conversion
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Deepak Saxena
OMAP map_desc.pfn conversion
Signed-off-by: Deepak Saxena <dsaxena@plexity.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Hugh Dickins