linux/mm
Linus Torvalds 2b047252d0 Fix TLB gather virtual address range invalidation corner cases
Ben Tebulin reported:

 "Since v3.7.2 on two independent machines a very specific Git
  repository fails in 9/10 cases on git-fsck due to an SHA1/memory
  failures.  This only occurs on a very specific repository and can be
  reproduced stably on two independent laptops.  Git mailing list ran
  out of ideas and for me this looks like some very exotic kernel issue"

and bisected the failure to the backport of commit 53a59fc67f ("mm:
limit mmu_gather batching to fix soft lockups on !CONFIG_PREEMPT").

That commit itself is not actually buggy, but what it does is to make it
much more likely to hit the partial TLB invalidation case, since it
introduces a new case in tlb_next_batch() that previously only ever
happened when running out of memory.

The real bug is that the TLB gather virtual memory range setup is subtly
buggered.  It was introduced in commit 597e1c3580 ("mm/mmu_gather:
enable tlb flush range in generic mmu_gather"), and the range handling
was already fixed at least once in commit e6c495a96c ("mm: fix the TLB
range flushed when __tlb_remove_page() runs out of slots"), but that fix
was not complete.

The problem with the TLB gather virtual address range is that it isn't
set up by the initial tlb_gather_mmu() initialization (which didn't get
the TLB range information), but it is set up ad-hoc later by the
functions that actually flush the TLB.  And so any such case that forgot
to update the TLB range entries would potentially miss TLB invalidates.

Rather than try to figure out exactly which particular ad-hoc range
setup was missing (I personally suspect it's the hugetlb case in
zap_huge_pmd(), which didn't have the same logic as zap_pte_range()
did), this patch just gets rid of the problem at the source: make the
TLB range information available to tlb_gather_mmu(), and initialize it
when initializing all the other tlb gather fields.

This makes the patch larger, but conceptually much simpler.  And the end
result is much more understandable; even if you want to play games with
partial ranges when invalidating the TLB contents in chunks, now the
range information is always there, and anybody who doesn't want to
bother with it won't introduce subtle bugs.

Ben verified that this fixes his problem.

Reported-bisected-and-tested-by: Ben Tebulin <tebulin@googlemail.com>
Build-testing-by: Stephen Rothwell <sfr@canb.auug.org.au>
Build-testing-by: Richard Weinberger <richard.weinberger@gmail.com>
Reviewed-by: Michal Hocko <mhocko@suse.cz>
Acked-by: Peter Zijlstra <peterz@infradead.org>
Cc: stable@vger.kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-08-16 08:52:46 -07:00
..
backing-dev.c sysfs.h: add __ATTR_RW() macro 2013-07-16 10:57:36 -07:00
balloon_compaction.c
bootmem.c mm: kill free_all_bootmem_node() 2013-07-03 16:07:39 -07:00
bounce.c
cleancache.c
compaction.c
debug-pagealloc.c
dmapool.c dmapool: make DMAPOOL_DEBUG detect corruption of free marker 2012-12-11 17:22:24 -08:00
fadvise.c
failslab.c
filemap_xip.c
filemap.c mm: remove unused VM_<READfoo> macros and expand other in-place 2013-07-09 10:33:23 -07:00
fremap.c mm: save soft-dirty bits on file pages 2013-08-13 17:57:48 -07:00
frontswap.c
highmem.c
huge_memory.c thp, mm: avoid PageUnevictable on active/inactive lru lists 2013-07-31 14:41:03 -07:00
hugetlb_cgroup.c mm/hugetlb: create hugetlb cgroup file in hugetlb_init 2012-12-18 15:02:15 -08:00
hugetlb.c Fix TLB gather virtual address range invalidation corner cases 2013-08-16 08:52:46 -07:00
hwpoison-inject.c
init-mm.c
internal.h mm: remove unused __put_page() 2013-07-09 10:33:22 -07:00
interval_tree.c
Kconfig zswap: add to mm/ 2013-07-10 18:11:34 -07:00
Kconfig.debug
kmemcheck.c
kmemleak-test.c
kmemleak.c
ksm.c
maccess.c
madvise.c
Makefile zswap: add to mm/ 2013-07-10 18:11:34 -07:00
memblock.c mm/memblock.c: fix wrong comment in __next_free_mem_range() 2013-07-09 10:33:23 -07:00
memcontrol.c memcg: don't initialize kmem-cache destroying work for root caches 2013-08-13 17:57:47 -07:00
memory_hotplug.c mm/memory_hotplug.c: fix return value of online_pages() 2013-07-09 10:33:25 -07:00
memory-failure.c mm/memory-failure.c: fix memory leak in successful soft offlining 2013-07-03 16:07:31 -07:00
memory.c Fix TLB gather virtual address range invalidation corner cases 2013-08-16 08:52:46 -07:00
mempolicy.c mm: mempolicy: fix mbind_range() && vma_adjust() interaction 2013-07-31 14:41:02 -07:00
mempool.c
migrate.c
mincore.c
mlock.c
mm_init.c mm: tune vm_committed_as percpu_counter batching size 2013-07-03 16:07:32 -07:00
mmap.c Fix TLB gather virtual address range invalidation corner cases 2013-08-16 08:52:46 -07:00
mmu_context.c
mmu_notifier.c
mmzone.c
mprotect.c mm/mprotect.c: coding-style cleanups 2012-12-18 15:02:15 -08:00
mremap.c mm: mremap: validate input before taking lock 2013-07-09 10:33:20 -07:00
msync.c
nobootmem.c mm: concentrate modification of totalram_pages into the mm core 2013-07-03 16:07:33 -07:00
nommu.c mm: remove free_area_cache 2013-07-10 18:11:34 -07:00
oom_kill.c
page_alloc.c mm: honor min_free_kbytes set by user 2013-07-09 10:33:25 -07:00
page_cgroup.c
page_io.c
page_isolation.c
page-writeback.c kernel: delete __cpuinit usage from all core kernel files 2013-07-14 19:36:59 -04:00
pagewalk.c
percpu-km.c percpu: clear memory allocated with the km allocator 2010-09-10 10:56:24 +02:00
percpu-vm.c
percpu.c
pgtable-generic.c
process_vm_access.c Fix: compat_rw_copy_check_uvector() misuse in aio, readv, writev, and security keys 2013-03-12 11:05:45 -07:00
quicklist.c
readahead.c
rmap.c mm: save soft-dirty bits on file pages 2013-08-13 17:57:48 -07:00
shmem.c tmpfs: fix SEEK_DATA/SEEK_HOLE regression 2013-08-04 11:40:17 -07:00
slab_common.c Merge branch 'slab/for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/penberg/linux 2013-07-14 15:14:29 -07:00
slab.c kernel: delete __cpuinit usage from all core kernel files 2013-07-14 19:36:59 -04:00
slab.h mm/slab: Give s_next and s_stop slab-specific names 2013-07-08 11:02:17 +03:00
slob.c Merge branch 'slab/for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/penberg/linux 2013-07-14 15:14:29 -07:00
slub.c Revert "slub: do not put a slab to cpu partial list when cpu_partial is 0" 2013-08-08 09:06:37 -07:00
sparse-vmemmap.c
sparse.c mm/sparse.c: put clear_hwpoisoned_pages within CONFIG_MEMORY_HOTREMOVE 2013-07-09 10:33:22 -07:00
swap_state.c
swap.c thp, mm: avoid PageUnevictable on active/inactive lru lists 2013-07-31 14:41:03 -07:00
swapfile.c mm: save soft-dirty bits on swapped pages 2013-08-13 17:57:47 -07:00
truncate.c mm: teach truncate_inode_pages_range() to handle non page aligned ranges 2013-05-27 23:32:35 -04:00
util.c mm: remove free_area_cache 2013-07-10 18:11:34 -07:00
vmalloc.c mm/vmalloc.c: fix an overflow bug in alloc_vmap_area() 2013-07-09 10:33:23 -07:00
vmpressure.c vmpressure: make sure there are no events queued after memcg is offlined 2013-07-31 14:41:04 -07:00
vmscan.c mm: vmscan: do not scale writeback pages when deciding whether to set ZONE_WRITEBACK 2013-07-09 10:33:23 -07:00
vmstat.c kernel: delete __cpuinit usage from all core kernel files 2013-07-14 19:36:59 -04:00
zbud.c mm: zbud: fix condition check on allocation size 2013-07-31 14:41:03 -07:00
zswap.c zswap: add to mm/ 2013-07-10 18:11:34 -07:00