Now that we're using "preferred local" policy for system default, we need to
make this as fast as possible. Because of the variable size of the mempolicy
structure [based on size of nodemasks], the preferred_node may be in a
different cacheline from the mode. This can result in accessing an extra
cacheline in the normal case of system default policy. Suspect this is the
cause of an observed 2-3% slowdown in page fault testing relative to kernel
without this patch series.
To alleviate this, use an internal mode flag, MPOL_F_LOCAL in the mempolicy
flags member which is guaranteed [?] to be in the same cacheline as the mode
itself.
Verified that reworked mempolicy now performs slightly better on 25-rc8-mm1
for both anon and shmem segments with system default and vma [preferred local]
policy.
Signed-off-by: Lee Schermerhorn <lee.schermerhorn@hp.com>
Cc: Christoph Lameter <clameter@sgi.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Mel Gorman <mel@csn.ul.ie>
Cc: Andi Kleen <ak@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Currently, when one specifies MPOL_DEFAULT via a NUMA memory policy API
[set_mempolicy(), mbind() and internal versions], the kernel simply installs a
NULL struct mempolicy pointer in the appropriate context: task policy, vma
policy, or shared policy. This causes any use of that policy to "fall back"
to the next most specific policy scope.
The only use of MPOL_DEFAULT to mean "local allocation" is in the system
default policy. This requires extra checks/cases for MPOL_DEFAULT in many
mempolicy.c functions.
There is another, "preferred" way to specify local allocation via the APIs.
That is using the MPOL_PREFERRED policy mode with an empty nodemask.
Internally, the empty nodemask gets converted to a preferred_node id of '-1'.
All internal usage of MPOL_PREFERRED will convert the '-1' to the id of the
node local to the cpu where the allocation occurs.
System default policy, except during boot, is hard-coded to "local
allocation". By using the MPOL_PREFERRED mode with a negative value of
preferred node for system default policy, MPOL_DEFAULT will never occur in the
'policy' member of a struct mempolicy. Thus, we can remove all checks for
MPOL_DEFAULT when converting policy to a node id/zonelist in the allocation
paths.
In slab_node() return local node id when policy pointer is NULL. No need to
set a pol value to take the switch default. Replace switch default with
BUG()--i.e., shouldn't happen.
With this patch MPOL_DEFAULT is only used in the APIs, including internal
calls to do_set_mempolicy() and in the display of policy in
/proc/<pid>/numa_maps. It always means "fall back" to the the next most
specific policy scope. This simplifies the description of memory policies
quite a bit, with no visible change in behavior.
get_mempolicy() continues to return MPOL_DEFAULT and an empty nodemask when
the requested policy [task or vma/shared] is NULL. These are the values one
would supply via set_mempolicy() or mbind() to achieve that condition--default
behavior.
This patch updates Documentation to reflect this change.
Signed-off-by: Lee Schermerhorn <lee.schermerhorn@hp.com>
Cc: Christoph Lameter <clameter@sgi.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Mel Gorman <mel@csn.ul.ie>
Cc: Andi Kleen <ak@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
After further discussion with Christoph Lameter, it has become clear that my
earlier attempts to clean up the mempolicy reference counting were a bit of
overkill in some areas, resulting in superflous ref/unref in what are usually
fast paths. In other areas, further inspection reveals that I botched the
unref for interleave policies.
A separate patch, suitable for upstream/stable trees, fixes up the known
errors in the previous attempt to fix reference counting.
This patch reworks the memory policy referencing counting and, one hopes,
simplifies the code. Maybe I'll get it right this time.
See the update to the numa_memory_policy.txt document for a discussion of
memory policy reference counting that motivates this patch.
Summary:
Lookup of mempolicy, based on (vma, address) need only add a reference for
shared policy, and we need only unref the policy when finished for shared
policies. So, this patch backs out all of the unneeded extra reference
counting added by my previous attempt. It then unrefs only shared policies
when we're finished with them, using the mpol_cond_put() [conditional put]
helper function introduced by this patch.
Note that shmem_swapin() calls read_swap_cache_async() with a dummy vma
containing just the policy. read_swap_cache_async() can call alloc_page_vma()
multiple times, so we can't let alloc_page_vma() unref the shared policy in
this case. To avoid this, we make a copy of any non-null shared policy and
remove the MPOL_F_SHARED flag from the copy. This copy occurs before reading
a page [or multiple pages] from swap, so the overhead should not be an issue
here.
I introduced a new static inline function "mpol_cond_copy()" to copy the
shared policy to an on-stack policy and remove the flags that would require a
conditional free. The current implementation of mpol_cond_copy() assumes that
the struct mempolicy contains no pointers to dynamically allocated structures
that must be duplicated or reference counted during copy.
Signed-off-by: Lee Schermerhorn <lee.schermerhorn@hp.com>
Cc: Christoph Lameter <clameter@sgi.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Mel Gorman <mel@csn.ul.ie>
Cc: Andi Kleen <ak@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The terms 'policy' and 'mode' are both used in various places to describe the
semantics of the value stored in the 'policy' member of struct mempolicy.
Furthermore, the term 'policy' is used to refer to that member, to the entire
struct mempolicy and to the more abstract concept of the tuple consisting of a
"mode" and an optional node or set of nodes. Recently, we have added "mode
flags" that are passed in the upper bits of the 'mode' [or sometimes,
'policy'] member of the numa APIs.
I'd like to resolve this confusion, which perhaps only exists in my mind, by
renaming the 'policy' member to 'mode' throughout, and fixing up the
Documentation. Man pages will be updated separately.
Signed-off-by: Lee Schermerhorn <lee.schermerhorn@hp.com>
Cc: Christoph Lameter <clameter@sgi.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Mel Gorman <mel@csn.ul.ie>
Cc: Andi Kleen <ak@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
MPOL_F_STATIC_NODES and MPOL_F_RELATIVE_NODES don't mean anything for
MPOL_PREFERRED policies that were created with an empty nodemask (for purely
local allocations). They'll never be invalidated because the allowed mems of
a task changes or need to be rebound relative to a cpuset's placement.
Also fixes a bug identified by Lee Schermerhorn that disallowed empty
nodemasks to be passed to MPOL_PREFERRED to specify local allocations. [A
different, somewhat incomplete, patch already existed in 25-rc5-mm1.]
Cc: Paul Jackson <pj@sgi.com>
Cc: Christoph Lameter <clameter@sgi.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
Cc: Andi Kleen <ak@suse.de>
Cc: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Lee Schermerhorn <lee.schermerhorn@hp.com>
Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Updates Documentation/vm/numa_memory_policy.txt and
Documentation/filesystems/tmpfs.txt to describe optional mempolicy mode flags.
Cc: Christoph Lameter <clameter@sgi.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
Cc: Andi Kleen <ak@suse.de>
Cc: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Paul Jackson <pj@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The MPOL_BIND policy creates a zonelist that is used for allocations
controlled by that mempolicy. As the per-node zonelist is already being
filtered based on a zone id, this patch adds a version of __alloc_pages() that
takes a nodemask for further filtering. This eliminates the need for
MPOL_BIND to create a custom zonelist.
A positive benefit of this is that allocations using MPOL_BIND now use the
local node's distance-ordered zonelist instead of a custom node-id-ordered
zonelist. I.e., pages will be allocated from the closest allowed node with
available memory.
[Lee.Schermerhorn@hp.com: Mempolicy: update stale documentation and comments]
[Lee.Schermerhorn@hp.com: Mempolicy: make dequeue_huge_page_vma() obey MPOL_BIND nodemask]
[Lee.Schermerhorn@hp.com: Mempolicy: make dequeue_huge_page_vma() obey MPOL_BIND nodemask rework]
Signed-off-by: Mel Gorman <mel@csn.ul.ie>
Acked-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Lee Schermerhorn <lee.schermerhorn@hp.com>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: Mel Gorman <mel@csn.ul.ie>
Cc: Hugh Dickins <hugh@veritas.com>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Nothing in the tree uses nopage any more. Remove support for it in the
core mm code and documentation (and a few stray references to it in
comments).
Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Add <linux/clk.h> to the generated kerneldoc, with some overview
to go along with those per-function descriptions.
Signed-off-by: David Brownell <dbrownell@users.sourceforge.net>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Alessandro Zummo <a.zummo@towertech.it>
Cc: "Randy.Dunlap" <rdunlap@xenotime.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* 'kvm-updates-2.6.26' of git://git.kernel.org/pub/scm/linux/kernel/git/avi/kvm: (147 commits)
KVM: kill file->f_count abuse in kvm
KVM: MMU: kvm_pv_mmu_op should not take mmap_sem
KVM: SVM: remove selective CR0 comment
KVM: SVM: remove now obsolete FIXME comment
KVM: SVM: disable CR8 intercept when tpr is not masking interrupts
KVM: SVM: sync V_TPR with LAPIC.TPR if CR8 write intercept is disabled
KVM: export kvm_lapic_set_tpr() to modules
KVM: SVM: sync TPR value to V_TPR field in the VMCB
KVM: ppc: PowerPC 440 KVM implementation
KVM: Add MAINTAINERS entry for PowerPC KVM
KVM: ppc: Add DCR access information to struct kvm_run
ppc: Export tlb_44x_hwater for KVM
KVM: Rename debugfs_dir to kvm_debugfs_dir
KVM: x86 emulator: fix lea to really get the effective address
KVM: x86 emulator: fix smsw and lmsw with a memory operand
KVM: x86 emulator: initialize src.val and dst.val for register operands
KVM: SVM: force a new asid when initializing the vmcb
KVM: fix kvm_vcpu_kick vs __vcpu_run race
KVM: add ioctls to save/store mpstate
KVM: Rename VCPU_MP_STATE_* to KVM_MP_STATE_*
...
This functionality is definitely experimental, but is capable of running
unmodified PowerPC 440 Linux kernels as guests on a PowerPC 440 host. (Only
tested with 440EP "Bamboo" guests so far, but with appropriate userspace
support other SoC/board combinations should work.)
See Documentation/powerpc/kvm_440.txt for technical details.
[stephen: build fix]
Signed-off-by: Hollis Blanchard <hollisb@us.ibm.com>
Acked-by: Paul Mackerras <paulus@samba.org>
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Avi Kivity <avi@qumranet.com>
We have "vlb|pci_clock=" parameters now.
Acked-by: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: Bartlomiej Zolnierkiewicz <bzolnier@gmail.com>
Remove obsoleted "idex=reset" kernel parameter
(it has been obsoleted since 1 Nov 2004).
Then remove corresponding code from ide_probe_port()
and no longer used ->reset field from ide_hwif_t.
Signed-off-by: Bartlomiej Zolnierkiewicz <bzolnier@gmail.com>
Remove obsoleted "idex=serialize" kernel parameter
(it has been obsoleted since 1 Nov 2004).
Signed-off-by: Bartlomiej Zolnierkiewicz <bzolnier@gmail.com>
Add "ignore_cable" parameter:
* "ide_core.ignore_cable=[interface_number]" boot option if IDE is built-in
(i.e. "ide_core.ignore_cable=1" to force ignoring cable for "ide1")
* "ignore_cable=[interface_number]" module parameter (for ide_core module)
if IDE is compiled as module
v2:
* Add ide_port_apply_params() helper
- use it in ide_device_add_all() and ide_scan_port().
* Make it possible to later disable ignoring cable detection by passing
"[interface_number]:0" to /sys/module/ide_core/parameters/ignore_cable
(however sysfs interface is not enabled yet since it needs some other
IDE changes to make it work reliable).
Signed-off-by: Bartlomiej Zolnierkiewicz <bzolnier@gmail.com>
This patch adds Documentation/s390/kvm.txt, which describes specifics of kvm's
user interface that are unique to s390 architecture.
Signed-off-by: Carsten Otte <cotte@de.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
Always use "fast" MWDMA support and remove dma_{black,white}_list
(they were based on completely bogus ->ide_dma_check implementation
which didn't set neither the host controller timings nor the device
for the desired transfer mode).
Signed-off-by: Bartlomiej Zolnierkiewicz <bzolnier@gmail.com>
Documentation for linked list of struct setup_data.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
As requested by Sam Ravnborg: Remove Documentation/smart-config.txt.
It is outdated and has been (functionally) replaced by
Documentation/kbuild/*.txt.
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc: (49 commits)
[POWERPC] Add zImage.iseries to arch/powerpc/boot/.gitignore
[POWERPC] bootwrapper: fix build error on virtex405-head.S
[POWERPC] 4xx: Fix 460GT support to not enable FPU
[POWERPC] 4xx: Add NOR FLASH entries to Canyonlands and Glacier dts
[POWERPC] Xilinx: of_serial support for Xilinx uart 16550.
[POWERPC] Xilinx: boot support for Xilinx uart 16550.
[POWERPC] celleb: Add support for PCI Express
[POWERPC] celleb: Move miscellaneous files for Beat
[POWERPC] celleb: Move a file for SPU on Beat
[POWERPC] celleb: Move files for Beat mmu and iommu
[POWERPC] celleb: Move files for Beat hvcall interfaces
[POWERPC] celleb: Move the SCC related code for celleb
[POWERPC] celleb: Move the files for celleb base support
[POWERPC] celleb: Consolidate io-workarounds code
[POWERPC] cell: Generalize io-workarounds code
[POWERPC] Add CONFIG_PPC_PSERIES_DEBUG to enable debugging for platforms/pseries
[POWERPC] Convert from DBG() to pr_debug() in platforms/pseries/
[POWERPC] Register udbg console early on pseries LPAR
[POWERPC] Mark udbg console as CON_ANYTIME, ie. callable early in boot
[POWERPC] Set udbg_console index to 0
...
* 'for-linus' of git://git.o-hand.com/linux-rpurdie-leds:
leds: Add default-on trigger
leds: Document the context brightness_set needs
leds: Add new driver for the LEDs on the Freecom FSG-3
leds: Add support to leds with readable status
leds: enable support for blink_set() platform hook in leds-gpio
leds: Cleanup various whitespace and code style issues
leds: disable triggers on brightness set
leds: Add mail LED support for "Clevo D400P"
* git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb-2.6: (120 commits)
usb: don't update devnum for wusb devices
wusb: make ep0_reinit available for modules
wusb: devices dont use a set address
wusb: teach choose_address() about wireless devices
wusb: add link wusb-usb device
wusb: add authenticathed bit to usb_dev
USB: remove unnecessary type casting of urb->context
usb serial: more fixes and groundwork for tty changes
USB: replace remaining __FUNCTION__ occurrences
USB: usbfs: export the URB_NO_INTERRUPT flag to userspace
USB: fix compile problems in ehci-hcd
USB: ehci: qh_completions cleanup and bugfix
USB: cdc-acm: signedness fix
USB: add documentation about callbacks
USB: don't explicitly reenable root-hub status interrupts
USB: OHCI: turn off RD when remote wakeup is disabled
USB: HCDs use the do_remote_wakeup flag
USB: g_file_storage: ignore bulk-out data after invalid CBW
USB: serial: remove endpoints setting checks from core and header
USB: serial: remove unneeded number endpoints settings
...
* git://git.kernel.org/pub/scm/linux/kernel/git/sam/kbuild:
kconfig: fix broken target update-po-config
kbuild: silence documentation GEN xml messages according to $(quiet)
* git://git.infradead.org/mtd-2.6: (82 commits)
[MTD] m25p80: Add Support for ATMEL AT25DF641 64-Megabit SPI Flash
[MTD] m25p80: add FAST_READ access support to M25Pxx
[MTD] [NAND] bf5xx_nand: Avoid crash if bfin_mac is installed.
[MTD] [NAND] at91_nand: control NCE signal
[MTD] [NAND] AT91 hardware ECC compile fix for at91sam9263 / at91sam9260
[MTD] [NAND] Hardware ECC controller on at91sam9263 / at91sam9260
[JFFS2] Introduce dbg_readinode2 log level, use it to shut read_dnode() up
[JFFS2] Fix jffs2_reserve_space() when all blocks are pending erasure.
[JFFS2] Add erase_checking_list to hold blocks being marked.
UBI: add a message
[JFFS2] Return values of jffs2_block_check_erase error paths
[MTD] Clean up AR7 partition map support
[MTD] [NOR] Fix Intel CFI driver for collie flash
[JFFS2] Finally remove redundant ref->__totlen field.
[JFFS2] Honour TEST_TOTLEN macro in debugging code. ref->__totlen is going!
[JFFS2] Add paranoia debugging for superblock counts
[JFFS2] Fix free space leak with in-band cleanmarkers
[JFFS2] Self-sufficient #includes in jffs2_fs_i.h: include <linux/mutex.h>
[MTD] [NAND] Verify probe by retrying to checking the results match
[MTD] [NAND] S3C2410 Allow ECC disable to be specified by the board
...
* git://git.kernel.org/pub/scm/linux/kernel/git/sam/kbuild:
kbuild: fix depmod comment
kbuild: Add new Kbuild variable KBUILD_EXTRA_SYMBOLS
kbuild: support loading extra symbols in modpost
Add option to enable -Wframe-larger-than= on gcc 4.4
kbuild: add kconfig symbols to tags output
kbuild: fix some minor typoes
kbuild: error out on missing MODULE_LICENSE
Add rules for gen_xml and its quiet & silent variants. This causes "make -s"
to be silent for gen_xml.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Acked-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
* 'for-linus' of git://linux-nfs.org/~bfields/linux:
nfsd: don't allow setting ctime over v4
Update to NFS/RDMA documentation
locks: don't call ->copy_lock methods on return of conflicting locks
lockd: unlock lockd locks held for a certain filesystem
lockd: unlock lockd locks associated with a given server ip
leases: remove unneeded variable from fcntl_setlease().
leases: move lock allocation earlier in generic_setlease()
leases: when unlocking, skip locking-related steps
leases: fix a return-value mixup
This patch adds a new (Kbuild) Makefile variable KBUILD_EXTRA_SYMBOLS.
The space separated list of file names assigned to KBUILD_EXTRA_SYMBOLS
is used when calling scripts/mod/modpost during stage 2 of the Kbuild
process for non-kernel-tree modules.
Signed-off-by: Richard Hacker <lerichi@gmx.net>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Update to the NFS/RDMA documentation to clarify how to configure the
exports file.
Signed-off-by: James Lentini <jlentini@netapp.com>
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Add description of dm-crypt to device-mapper documentation.
Signed-off-by: Milan Broz <mbroz@redhat.com>
Signed-off-by: Alasdair G Kergon <agk@redhat.com>
Sometimes the specific interaction between the platform and the PHY
requires special handling. For instance, to change where the PHY's
clock input is, or to add a delay to account for latency issues in the
data path. We add a mechanism for registering a callback with the PHY
Lib to be called on matching PHYs when they are brought up, or reset.
Signed-off-by: Andy Fleming <afleming@freescale.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>
This adds documentation about the new usb anchor infrastructure.
Signed-off-by: Oliver Neukum <oneukum@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This patch (as1048) extends the descriptor checking after a device is
reset. Now the SerialNumber string descriptor is compared to its old
value, in addition to the device and configuration descriptors.
As a consequence, the kmalloc() call in usb_string() is now on the
error-handling pathway for usb-storage. Hence its allocation type is
changed to GFO_NOIO.
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This patch (as1047) removes the USB_PERSIST Kconfig option, enabling
it permanently. It also prevents the power/persist attribute from
being created for hub devices; there's no point in having it since
USB-PERSIST is always turned on for hubs.
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Lee Schermerhorn
David Rientjes
Mel Gorman
Nick Piggin
David Brownell
Linus Torvalds
Hollis Blanchard
Bartlomiej Zolnierkiewicz
Borislav Petkov
Avi Kivity
Xiantao Zhang
Carsten Otte
Huang, Ying
Randy Dunlap
Mike Frysinger
Richard Hacker
Jonathan Corbet
James Lentini
Milan Broz
Andy Fleming
Oliver Neukum
Matti Linnanvuori
Alan Stern