OpenE2K
/
linux
6
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
linux/arch/arm/include/asm
History
Minchan Kim 99baac21e4 mm: fix MADV_[FREE|DONTNEED] TLB flush miss problem 
Nadav reported parallel MADV_DONTNEED on same range has a stale TLB
problem and Mel fixed it[1] and found same problem on MADV_FREE[2].

Quote from Mel Gorman:
 "The race in question is CPU 0 running madv_free and updating some PTEs
  while CPU 1 is also running madv_free and looking at the same PTEs.
  CPU 1 may have writable TLB entries for a page but fail the pte_dirty
  check (because CPU 0 has updated it already) and potentially fail to
  flush.

  Hence, when madv_free on CPU 1 returns, there are still potentially
  writable TLB entries and the underlying PTE is still present so that a
  subsequent write does not necessarily propagate the dirty bit to the
  underlying PTE any more. Reclaim at some unknown time at the future
  may then see that the PTE is still clean and discard the page even
  though a write has happened in the meantime. I think this is possible
  but I could have missed some protection in madv_free that prevents it
  happening."

This patch aims for solving both problems all at once and is ready for
other problem with KSM, MADV_FREE and soft-dirty story[3].

TLB batch API(tlb_[gather|finish]_mmu] uses [inc|dec]_tlb_flush_pending
and mmu_tlb_flush_pending so that when tlb_finish_mmu is called, we can
catch there are parallel threads going on.  In that case, forcefully,
flush TLB to prevent for user to access memory via stale TLB entry
although it fail to gather page table entry.

I confirmed this patch works with [4] test program Nadav gave so this
patch supersedes "mm: Always flush VMA ranges affected by zap_page_range
v2" in current mmotm.

NOTE:

This patch modifies arch-specific TLB gathering interface(x86, ia64,
s390, sh, um).  It seems most of architecture are straightforward but
s390 need to be careful because tlb_flush_mmu works only if
mm->context.flush_mm is set to non-zero which happens only a pte entry
really is cleared by ptep_get_and_clear and friends.  However, this
problem never changes the pte entries but need to flush to prevent
memory access from stale tlb.

[1] http://lkml.kernel.org/r/20170725101230.5v7gvnjmcnkzzql3@techsingularity.net
[2] http://lkml.kernel.org/r/20170725100722.2dxnmgypmwnrfawp@suse.de
[3] http://lkml.kernel.org/r/BD3A0EBE-ECF4-41D4-87FA-C755EA9AB6BD@gmail.com
[4] https://patchwork.kernel.org/patch/9861621/

[minchan@kernel.org: decrease tlb flush pending count in tlb_finish_mmu]
  Link: http://lkml.kernel.org/r/20170808080821.GA31730@bbox
Link: http://lkml.kernel.org/r/20170802000818.4760-7-namit@vmware.com
Signed-off-by: Minchan Kim <minchan@kernel.org>
Signed-off-by: Nadav Amit <namit@vmware.com>
Reported-by: Nadav Amit <namit@vmware.com>
Reported-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Mel Gorman <mgorman@techsingularity.net>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Jeff Dike <jdike@addtoit.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Hugh Dickins <hughd@google.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Nadav Amit <nadav.amit@gmail.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
 		2017-08-10 15:54:07 -07:00
..
hardware ARM/clk: move the ICST library to drivers/clk 2017-04-07 11:58:12 +02:00
mach ARM/PCI: Convert PCI scan API to pci_scan_root_bus_bridge() 2017-06-28 15:13:55 -05:00
xen xen: avoid type warning in xchg_xen_ulong 2017-06-08 15:07:38 -07:00
Kbuild ARM: move generic-y of exported headers to uapi/asm/Kbuild 2017-07-10 03:43:20 +09:00
arch_gicv3.h ARM: gic-v3-its: Add 32bit support to GICv3 ITS 2016-11-29 09:14:48 +00:00
arch_timer.h clocksource: arch_timer: Fix code to use physical timers when requested 2014-12-04 23:30:26 -08:00
arm-cci.h arm-cci: Get rid of secure transactions for PMU driver 2015-03-27 13:44:35 +00:00
asm-offsets.h
…
assembler.h ARM: Prepare for randomized task_struct 2017-06-30 12:00:50 -07:00
atomic.h locking/atomic: Remove linux/atomic.h:atomic_fetch_or() 2016-06-16 10:48:32 +02:00
auxvec.h ARM: 8329/1: miscellaneous vdso infrastructure, preparation 2015-03-27 22:20:45 +00:00
bL_switcher.h ARM: bL_switcher: Add query interface to discover CPU affinities 2013-09-23 18:47:31 -04:00
barrier.h ARM: 8582/1: remove unused CONFIG_ARCH_HAS_BARRIERS 2016-07-02 11:01:08 +01:00
bitops.h ARM: 8679/1: bitops: Align prototypes to generic API 2017-06-18 22:36:33 +01:00
bitrev.h ARM: 8287/1: add bitrev.h file to support rbit instruction 2015-01-16 14:06:18 +00:00
bug.h debug: Fix WARN_ON_ONCE() for modules 2017-07-20 12:31:04 +02:00
bugs.h
…
cache.h
…
cacheflush.h randstruct: opt-out externally exposed function pointer structs 2017-06-30 12:00:52 -07:00
cachetype.h ARM: 8604/1: V7M: Add support for reading the CTR with read_cpuid_cachetype() 2016-09-06 15:51:07 +01:00
checksum.h ipv6: Pass proto to csum_ipv6_magic as __u8 instead of unsigned short 2016-03-13 23:55:13 -04:00
clocksource.h arm/arm64: arch_timer: Use archdata to indicate vdso suitability 2016-09-23 17:19:25 +01:00
cmpxchg.h ARM: 8443/1: Adding support for atomic half word exchange 2015-10-09 16:22:54 +01:00
compiler.h ARM: 8300/1: teach __asmeq that r11 == fp and r12 == ip 2015-02-10 10:23:11 +00:00
cp15.h ARM: Move system register accessors to asm/cp15.h 2016-09-22 13:22:10 +02:00
cpu.h ARM: kernel: add MIDR to per-CPU information data 2012-11-19 14:51:11 +00:00
cpufeature.h ARM: 8663/1: wire up HWCAP/HWCAP2 feature bits to the CPU modalias 2017-03-30 19:45:52 +01:00
cpuidle.h ARM: cpuidle: add const qualifier to cpuidle_ops member in structures 2016-04-20 06:59:10 +02:00
cputype.h ARM: 8634/1: hw_breakpoint: blacklist Scorpion CPUs 2017-01-10 23:32:26 +00:00
cti.h ARM: coresight: common definition for (OS) Lock Access Register key value 2013-01-10 21:13:05 +00:00
dcc.h ARM: tty: Move HVC DCC assembly to arch/arm 2014-05-28 13:33:18 -07:00
delay.h ARM: 8619/1: udelay: document the various constants 2016-10-19 10:52:36 +01:00
device.h ARM: dma-mapping: Don't tear down third-party mappings 2017-05-30 11:31:33 +02:00
div64.h ARM: 8504/1: __arch_xprod_64(): small optimization 2016-02-11 15:33:37 +00:00
dma-contiguous.h drivers: dma-contiguous: clean source code and prepare for device tree 2013-08-27 09:18:29 +02:00
dma-iommu.h arm: remove arch specific dma_supported implementation 2017-06-28 06:54:45 -07:00
dma-mapping.h ARM: NOMMU: Introduce dma operations for noMMU 2017-06-30 10:03:09 -07:00
dma.h remove <asm/scatterlist.h> 2015-05-05 13:35:39 -06:00
dmi.h efi/arm: Enable DMI/SMBIOS 2017-06-05 17:50:44 +02:00
domain.h ARM: 8573/1: domain: move {set,get}_domain under config guard 2016-05-05 19:03:02 +01:00
ecard.h
…
edac.h EDAC: Cleanup atomic_scrub mess 2015-05-28 15:31:53 +02:00
efi.h efi/arm-stub: Correct FDT and initrd allocation rules for arm64 2017-04-05 12:27:23 +02:00
elf.h arm: move ELF_ET_DYN_BASE to 4MB 2017-07-10 16:32:36 -07:00
entry-macro-multi.S ARM: replace BSYM() with badr assembly macro 2015-05-08 17:33:50 +01:00
exception.h arch, ftrace: for KASAN put hard/soft IRQ entries into separate sections 2016-03-25 16:37:42 -07:00
fb.h
…
fiq.h
…
firmware.h ARM: EXYNOS: add exynos_get_boot_addr() helper 2015-06-06 02:17:56 +09:00
fixmap.h ARM: 8667/3: Fix memory attribute inconsistencies when using fixmap 2017-04-20 11:19:42 +01:00
flat.h binfmt_flat: flat_{get,put}_addr_from_rp() should be able to fail 2017-07-03 18:44:02 -04:00
floppy.h ARM: 8584/1: floppy: avoid gcc-6 warning 2016-07-02 11:02:13 +01:00
fncpy.h
…
fpstate.h
…
ftrace.h ARM: 8678/1: ftrace: Adds support for CONFIG_DYNAMIC_FTRACE_WITH_REGS 2017-06-18 22:25:16 +01:00
futex.h ARM: uaccess: provide uaccess_save_and_enable() and uaccess_restore() 2015-08-25 16:14:43 +01:00
glue-cache.h ARM: 8607/1: V7M: Wire up caches for V7M processors with cache support. 2016-09-06 15:51:08 +01:00
glue-df.h ARM: 8065/1: remove last use of CONFIG_CPU_ARM710 2014-05-30 01:12:30 +01:00
glue-pf.h
…
glue-proc.h ARM: 8103/1: save/restore Cortex-A9 CP15 registers on suspend/resume 2014-07-18 12:29:37 +01:00
glue.h
…
gpio.h ARM: delete old reference to ARM_GPIOLIB_COMPLEX 2014-06-30 21:37:39 +02:00
hardirq.h ARM: 8487/1: Remove IPI_CALL_FUNC_SINGLE 2015-12-22 12:09:43 +00:00
highmem.h kmap_atomic_to_page() has no users, remove it 2015-11-09 15:11:24 -08:00
hugetlb-3level.h ARM: mm: HugeTLB support for LPAE systems. 2013-06-04 16:52:37 +01:00
hugetlb.h mm/hugetlb: remove arch_prepare/release_hugepage from arch headers 2015-06-25 17:00:35 -07:00
hw_breakpoint.h ARM: 8598/1: remove traces of perf_ops_bp 2016-08-23 10:09:23 +01:00
hw_irq.h ARM: Remove ununsed set_irq_flags 2015-09-16 16:53:39 +02:00
hwcap.h ARM: 7981/1: add support for AT_HWCAP2 ELF auxv entry 2014-02-25 11:40:48 +00:00
hypervisor.h arm: initial Xen support 2012-09-14 13:53:39 +00:00
ide.h
…
idmap.h ARM: KVM: move to a KVM provided HYP idmap 2013-04-28 22:23:08 -07:00
insn.h ARM: kprobes: enable OPTPROBES for ARM 32 2015-01-13 16:10:17 +00:00
io.h ARM: Implement pci_remap_cfgspace() interface 2017-04-24 13:53:13 -05:00
irq.h nmi_backtrace: add more trigger_*_cpu_backtrace() methods 2016-10-07 18:46:30 -07:00
irq_work.h arm: Tell irq work about self IPI support 2014-09-13 18:38:39 +02:00
irqflags.h ARM: 8422/1: enable imprecise aborts during early kernel startup 2015-09-22 08:13:56 +01:00
jump_label.h locking/static_keys: Add a new static_key interface 2015-08-03 11:34:15 +02:00
kexec.h ARM: kexec: fix failure to boot crash kernel 2017-07-20 11:37:42 +01:00
kgdb.h ARM: kdgb: use <asm/opcodes.h> for data to be assembled as intruction 2013-10-19 20:46:35 +01:00
kmap_types.h arm: remove km_type definitions 2012-07-24 15:27:28 +08:00
kprobes.h kprobes: move kprobe declarations to asm-generic/kprobes.h 2017-02-27 18:43:45 -08:00
kvm_arm.h arm/arm64: KVM: add guest SEA support 2017-06-22 18:22:05 +01:00
kvm_asm.h ARM: KVM: Gracefully handle hyp-stubs being restored from under our feet 2017-04-09 07:49:31 -07:00
kvm_coproc.h KVM: arm: plug potential guest hardware debug leakage 2017-05-15 14:29:19 +02:00
kvm_emulate.h arm: KVM: Add Virtual Abort injection helper 2016-09-08 12:53:00 +02:00
kvm_host.h KVM: arm: Handle VCPU device attributes in guest.c 2017-06-08 16:59:55 +02:00
kvm_hyp.h ARM: KVM: move asmlinkage before type 2017-07-12 16:26:04 -07:00
kvm_mmio.h KVM: arm/arm64: Export mmio_read/write_bus 2016-05-20 15:39:42 +02:00
kvm_mmu.h arm/arm64: KVM: Remove kvm_get_idmap_start 2017-04-09 07:49:32 -07:00
kvm_psci.h ARM/ARM64: KVM: Make kvm_psci_call() return convention more flexible 2014-04-30 04:18:57 -07:00
limits.h
…
linkage.h
…
mc146818rtc.h
…
mcpm.h ARM: MCPM: remove residency argument from mcpm_cpu_suspend() 2015-05-06 11:47:10 -04:00
mcs_spinlock.h ARM: 8098/1: mcs lock: implement wfe-based polling for MCS locking 2014-07-18 12:29:35 +01:00
memblock.h ARM: 8025/1: Get rid of meminfo 2014-06-01 01:17:12 +01:00
memory.h ARM: 8648/2: nommu: display vectors base 2017-02-28 11:06:14 +00:00
mmu.h ARM: 8329/1: miscellaneous vdso infrastructure, preparation 2015-03-27 22:20:45 +00:00
mmu_context.h sched/headers: Prepare to remove the <linux/mm_types.h> dependency from <linux/sched.h> 2017-03-02 08:42:37 +01:00
module.h ARM: 8662/1: module: split core and init PLT sections 2017-03-17 10:01:28 +00:00
mpu.h ARM: mpu: protect the vectors page with an MPU region 2013-06-17 15:13:18 +01:00
mtd-xip.h
…
neon.h ARM: add support for kernel mode NEON 2013-07-08 22:08:51 +01:00
nwflash.h
…
opcodes-sec.h ARM: opcodes: add opcodes definitions for ARM security extensions 2013-01-10 21:10:20 +00:00
opcodes-virt.h ARM: opcodes: add __ERET/__MSR_ELR_HYP instruction encoding 2012-09-19 08:32:49 +01:00
opcodes.h ARM: opcodes: add missing include of linux/linkage.h 2013-01-10 21:10:20 +00:00
outercache.h ARM: move heavy barrier support out of line 2015-07-25 15:28:05 +01:00
page-nommu.h ARM: 8684/1: NOMMU: Remove unused KTHREAD_SIZE definition 2017-06-27 14:58:10 +01:00
page.h ARM: make vectors page inaccessible from userspace 2013-08-01 14:31:58 +01:00
paravirt.h arm: introduce CONFIG_PARAVIRT, PARAVIRT_TIME_ACCOUNTING and pv_time_ops 2015-12-21 14:40:54 +00:00
patch.h ARM: probes: move all probe code to dedicate directory 2015-01-09 09:36:50 +00:00
pci.h ARM/PCI: Use generic pci_mmap_resource_range() 2017-04-20 08:47:47 -05:00
percpu.h ARM: 8174/1: Use global stack register variable for percpu 2014-11-13 23:58:06 +00:00
perf_event.h arm: perf: Fix callchain parse error with kernel tracepoint events 2015-05-27 16:12:05 +01:00
pgalloc.h arm: get rid of superfluous __GFP_REPEAT 2016-07-26 16:19:19 -07:00
pgtable-2level-hwdef.h ARM: 8612/1: LPAE: initialize cache policy correctly 2016-09-12 12:12:30 +01:00
pgtable-2level-types.h
…
pgtable-2level.h ARM: 8578/1: mm: ensure pmd_present only checks the valid bit 2016-06-09 17:51:47 +01:00
pgtable-3level-hwdef.h ARM: 8612/1: LPAE: initialize cache policy correctly 2016-09-12 12:12:30 +01:00
pgtable-3level-types.h
…
pgtable-3level.h ARM: 8579/1: mm: Fix definition of pmd_mknotpresent 2016-06-09 17:51:47 +01:00
pgtable-hwdef.h
…
pgtable-nommu.h ARM: 8676/1: NOMMU: provide pgprot_device() macro 2017-06-05 10:29:40 +01:00
pgtable.h arch, mm: convert all architectures to use 5level-fixup.h 2017-03-09 11:48:47 -08:00
probes.h ARM: optprobes: execute instruction during restoring if possible. 2015-01-14 12:24:52 +00:00
proc-fns.h ARM: soft-reboot into same mode that we entered the kernel 2017-04-09 07:49:24 -07:00
processor.h locking/core: Provide common cpu_relax_yield() definition 2016-11-17 08:17:36 +01:00
procinfo.h
…
prom.h of/fdt: move memreserve and dtb memory reservations into core 2014-04-30 00:59:17 -05:00
psci.h ARM: SoC cleanups for v4.5 2016-01-20 17:55:20 -08:00
ptrace.h ARM: save and reset the address limit when entering an exception 2016-07-07 16:01:01 +01:00
sections.h ARM: 8518/1: Use correct symbols for XIP_KERNEL 2016-02-11 15:43:14 +00:00
set_memory.h treewide: move set_memory_* functions away from cacheflush.h 2017-05-08 17:15:13 -07:00
setup.h ARM: 8495/1: ATAGS: move save_atags() to arch/arm/include/asm/setup.h 2016-01-04 11:26:00 +00:00
shmparam.h
…
signal.h arm: switch to generic old sigaction() 2013-02-03 18:15:47 -05:00
smp.h ARM: 8423/1: add const qualifier to smp_operations member in structures 2015-09-22 08:13:57 +01:00
smp_plat.h ARM: 8392/3: smp: Only expose /sys/.../cpuX/online if hotpluggable 2015-07-31 18:58:30 +01:00
smp_scu.h ARM: make it easier to check the CPU part number correctly 2014-07-18 12:29:02 +01:00
smp_twd.h ARM: smp_twd: convert to use CLKSRC_OF init 2013-03-11 08:42:08 -05:00
sparsemem.h ARM: 8411/1: Add default SPARSEMEM settings 2016-02-16 16:35:21 +00:00
spinlock.h locking/spinlock, arch: Update and fix spin_unlock_wait() implementations 2016-06-14 11:55:15 +02:00
spinlock_types.h ARM: locks: prefetch the destination word for write prior to strex 2013-09-30 16:42:55 +01:00
stackprotector.h
…
stacktrace.h ARM: 8070/1: Introduce arm_get_current_stack_frame() 2014-07-18 12:29:11 +01:00
stage2_pgtable.h kvm-arm: arm32: Introduce stage2 page table helpers 2016-04-21 14:57:26 +02:00
string.h
…
suspend.h ARM: 8389/1: Add cpu_resume_arm() for firmwares that resume in ARM state 2015-06-12 21:17:17 +01:00
swab.h UAPI: (Scripted) Disintegrate arch/arm/include/asm 2012-10-12 13:05:52 +01:00
switch_to.h sched, arm: Remove finish_arch_switch() 2015-08-04 09:37:52 +02:00
sync_bitops.h ARM: 7989/1: Delete asm/system.h 2014-02-25 11:33:37 +00:00
syscall.h ARM: 8152/1: Convert pr_warning to pr_warn 2014-09-26 14:39:53 +01:00
system_info.h ARM: 8355/1: arch: Show the serial number from devicetree in cpuinfo 2015-05-08 10:42:35 +01:00
system_misc.h arm/arm64: KVM: add guest SEA support 2017-06-22 18:22:05 +01:00
tcm.h
…
therm.h
…
thread_info.h Merge branch 'fixes' of git://ftp.arm.linux.org.uk/~rmk/linux-arm 2015-09-14 12:24:10 -07:00
thread_notify.h
…
timex.h ARM: drop <mach/timex.h> for !ARCH_MULTIPLATFORM, too 2013-12-20 17:17:16 +01:00
tlb.h mm: fix MADV_[FREE|DONTNEED] TLB flush miss problem 2017-08-10 15:54:07 -07:00
tlbflush.h sched/headers: Prepare to remove the <linux/mm_types.h> dependency from <linux/sched.h> 2017-03-02 08:42:37 +01:00
tls.h ARM: 8178/1: fix set_tls for !CONFIG_KUSER_HELPERS 2014-09-30 16:55:23 +01:00
topology.h sched/topology: Rename topology_thread_cpumask() to topology_sibling_cpumask() 2015-05-27 15:22:15 +02:00
traps.h
…
trusted_foundations.h treewide: remove redundant #include <linux/kconfig.h> 2016-10-11 15:06:33 -07:00
uaccess.h Merge branch 'work.uaccess-unaligned' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-07-15 11:17:52 -07:00
ucontext.h ARM: 8687/1: signal: Fix unparseable iwmmxt_sigframe in uc_regspace[] 2017-07-24 14:26:55 +01:00
unified.h ARM: replace BSYM() with badr assembly macro 2015-05-08 17:33:50 +01:00
unistd.h ARM: wire up new pkey syscalls 2016-10-19 10:18:47 +01:00
unwind.h
…
uprobes.h ARM: add uprobes support 2014-03-18 16:39:40 -04:00
user.h
…
v7m.h ARM: 8603/1: V7M: Add addresses for mem-mapped V7M cache operations 2016-09-06 15:51:06 +01:00
vdso.h ARM: 8329/1: miscellaneous vdso infrastructure, preparation 2015-03-27 22:20:45 +00:00
vdso_datapage.h ARM: 8329/1: miscellaneous vdso infrastructure, preparation 2015-03-27 22:20:45 +00:00
vfp.h ARM: vfp: Add vfp_disable for problematic platforms 2015-05-20 15:07:00 -07:00
vfpmacros.h ARM: 7566/1: vfp: fix save and restore when running on pre-VFPv3 and CONFIG_VFPv3 set 2012-10-29 10:04:05 +00:00
vga.h
…
virt.h ARM: hyp-stub/KVM: Kill __hyp_get_vectors 2017-04-09 07:49:34 -07:00
word-at-a-time.h ARM: 8322/1: keep .text and .fixup regions closer together 2015-03-29 23:11:56 +01:00
xor.h ARM: crypto: add NEON accelerated XOR implementation 2013-07-08 22:09:06 +01:00