linux/mm
Mike Galbraith d9859c5f3a scheduling while atomic in cgroup code
mm, memcg: make refill_stock() use get_cpu_light()

Nikita reported the following memcg scheduling while atomic bug:

Call Trace:
[e22d5a90] [c0007ea8] show_stack+0x4c/0x168 (unreliable)
[e22d5ad0] [c0618c04] __schedule_bug+0x94/0xb0
[e22d5ae0] [c060b9ec] __schedule+0x530/0x550
[e22d5bf0] [c060bacc] schedule+0x30/0xbc
[e22d5c00] [c060ca24] rt_spin_lock_slowlock+0x180/0x27c
[e22d5c70] [c00b39dc] res_counter_uncharge_until+0x40/0xc4
[e22d5ca0] [c013ca88] drain_stock.isra.20+0x54/0x98
[e22d5cc0] [c01402ac] __mem_cgroup_try_charge+0x2e8/0xbac
[e22d5d70] [c01410d4] mem_cgroup_charge_common+0x3c/0x70
[e22d5d90] [c0117284] __do_fault+0x38c/0x510
[e22d5df0] [c011a5f4] handle_pte_fault+0x98/0x858
[e22d5e50] [c060ed08] do_page_fault+0x42c/0x6fc
[e22d5f40] [c000f5b4] handle_page_fault+0xc/0x80

What happens:

   refill_stock()
      get_cpu_var()
      drain_stock()
         res_counter_uncharge()
            res_counter_uncharge_until()
               spin_lock() <== boom

Fix it by replacing get/put_cpu_var() with get/put_cpu_light().

Cc: stable-rt@vger.kernel.org
Reported-by: Nikita Yushchenko <nyushchenko@dev.rtsoft.ru>
Signed-off-by: Mike Galbraith <umgwanakikbuti@gmail.com>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2020-10-14 00:59:24 +03:00
..
Kconfig kconfig-disable-a-few-options-rt.patch 2020-10-14 00:59:10 +03:00
Kconfig.debug mm: more intensive memory corruption debugging 2012-01-10 16:30:42 -08:00
Makefile mm: per-thread vma caching 2014-10-09 12:21:29 -07:00
backing-dev.c bdi: avoid oops on device removal 2014-04-26 17:19:05 -07:00
balloon_compaction.c mm: print more details for bad_page() 2014-01-23 16:36:50 -08:00
bootmem.c mm/bootmem.c: remove unused local `map' 2013-11-13 12:09:09 +09:00
bounce.c mm: bounce: Use local_irq_save_nort 2020-10-14 00:59:12 +03:00
cleancache.c mm: dump page when hitting a VM_BUG_ON using VM_BUG_ON_PAGE 2014-01-23 16:36:50 -08:00
compaction.c mm/compaction: fix wrong order check in compact_finished() 2015-03-18 13:31:23 +01:00
debug-pagealloc.c mm, x86: Remove debug_pagealloc_enabled 2011-12-06 09:24:07 +01:00
dmapool.c dmapool: make DMAPOOL_DEBUG detect corruption of free marker 2012-12-11 17:22:24 -08:00
fadvise.c teach SYSCALL_DEFINE<n> how to deal with long long/unsigned long long 2013-03-03 22:46:22 -05:00
failslab.c switch debugfs to umode_t 2012-01-03 22:54:56 -05:00
filemap.c mm: get rid of radix tree gfp mask for pagecache_get_page 2015-01-29 17:40:53 -08:00
filemap_xip.c seqcount: Add lockdep functionality to seqcount/seqlock structures 2013-11-06 12:40:26 +01:00
fremap.c mm: fix bad rss-counter if remap_file_pages raced migration 2014-03-19 16:21:49 -07:00
frontswap.c mm: frontswap: invalidate expired data on a dup-store failure 2014-12-16 09:34:26 -08:00
highmem.c mm, rt: kmap_atomic scheduling 2020-10-14 00:59:20 +03:00
huge_memory.c mm, thp: only collapse hugepages to nodes with affinity for zone_reclaim_mode 2015-01-29 17:40:52 -08:00
hugetlb.c mm/hugetlb: add migration entry check in __unmap_hugepage_range 2015-03-18 13:31:23 +01:00
hugetlb_cgroup.c mm: dump page when hitting a VM_BUG_ON using VM_BUG_ON_PAGE 2014-01-23 16:36:50 -08:00
hwpoison-inject.c mm/hwpoison: add '#' to hwpoison_inject 2014-01-21 16:19:48 -08:00
init-mm.c atomic: use <linux/atomic.h> 2011-07-26 16:49:47 -07:00
internal.h mm, compaction: properly signal and act upon lock and need_sched() contention 2014-11-21 09:23:07 -08:00
interval_tree.c mm: add CONFIG_DEBUG_VM_RB build option 2012-10-09 16:22:42 +09:00
kmemcheck.c kmemcheck: Fix build errors due to missing slab.h 2010-03-30 22:02:32 +09:00
kmemleak-test.c kmemleak: remove memset by using kzalloc 2011-01-27 18:31:51 +00:00
kmemleak.c mm: kmemleak: avoid false negatives on vmalloc'ed objects 2013-11-13 12:09:07 +09:00
ksm.c vm: add VM_FAULT_SIGSEGV handling support 2015-04-29 10:31:55 +02:00
list_lru.c mm: list_lru: fix almost infinite loop causing effective livelock 2013-10-30 12:57:46 -07:00
maccess.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
madvise.c mm: madvise: fix MADV_WILLNEED on shmem swapouts 2014-11-21 09:23:06 -08:00
memblock.c memblock, memhotplug: fix wrong type in memblock_find_in_range_node(). 2014-10-05 14:52:17 -07:00
memcontrol.c scheduling while atomic in cgroup code 2020-10-14 00:59:24 +03:00
memory-failure.c mm: soft-offline: fix num_poisoned_pages counting on concurrent events 2015-05-17 09:53:49 -07:00
memory.c mm, rt: kmap_atomic scheduling 2020-10-14 00:59:20 +03:00
memory_hotplug.c mm/memory_hotplug.c: set zone->wait_table to null after freeing it 2015-06-22 17:01:23 -07:00
mempolicy.c mm, numa: really disable NUMA balancing by default on single node machines 2015-06-06 08:19:38 -07:00
mempool.c mm/mempool.c: convert kmalloc_node(...GFP_ZERO...) to kzalloc_node(...) 2013-09-11 15:58:14 -07:00
migrate.c mm: fix direct reclaim writeback regression 2014-11-21 09:23:07 -08:00
mincore.c mm + fs: prepare for non-page entries in page cache radix trees 2014-11-21 09:23:06 -08:00
mlock.c mm: try_to_unmap_cluster() should lock_page() before mlocking 2014-05-06 07:59:35 -07:00
mm_init.c mm: bring back /sys/kernel/mm 2014-01-27 21:02:39 -08:00
mmap.c mm/mmap.c: fix arithmetic overflow in __vm_enough_memory() 2015-03-18 13:31:23 +01:00
mmu_context.c mm: Protect activate_mm() by preempt_[disable&enable]_rt() 2020-10-14 00:59:18 +03:00
mmu_notifier.c mm: audit/fix non-modular users of module_init in core code 2014-01-23 16:36:52 -08:00
mmzone.c mm: numa: Change page last {nid,pid} into {cpu,pid} 2013-10-09 14:47:45 +02:00
mprotect.c mm: Use ptep/pmdp_set_numa() for updating _PAGE_NUMA bit 2014-02-17 11:19:36 +11:00
mremap.c mm, thp: close race between mremap() and split_huge_page() 2014-06-07 10:28:10 -07:00
msync.c sanitize vfs_fsync calling conventions 2010-05-21 18:31:21 -04:00
nobootmem.c mm/nobootmem: free_all_bootmem again 2014-01-23 16:36:52 -08:00
nommu.c mm/nommu.c: fix arithmetic overflow in __vm_enough_memory() 2015-03-18 13:31:23 +01:00
oom_kill.c OOM, PM: OOM killed task shouldn't escape PM suspend 2014-11-14 09:00:01 -08:00
page-writeback.c writeback: use |1 instead of +1 to protect against div by zero 2015-05-17 09:53:49 -07:00
page_alloc.c mm: page_alloc: Use local_lock_on() instead of plain spinlock 2020-10-14 00:59:12 +03:00
page_cgroup.c mm: Replace cgroup_page bit spinlock 2020-10-14 00:59:11 +03:00
page_io.c Merge branch 'for-3.14/core' of git://git.kernel.dk/linux-block 2014-01-30 11:19:05 -08:00
page_isolation.c mm: memory-hotplug: enable memory hotplug to handle hugepage 2013-09-11 15:57:48 -07:00
pagewalk.c mm: pagewalk: call pte_hole() for VM_PFNMAP during walk_page_range 2015-02-11 14:54:47 +08:00
percpu-km.c percpu: clear memory allocated with the km allocator 2010-10-02 10:28:42 +03:00
percpu-vm.c percpu: perform tlb flush after pcpu_map_pages() failure 2014-10-05 14:52:20 -07:00
percpu.c Revert "percpu: free percpu allocation info for uniprocessor system" 2014-11-14 08:59:45 -08:00
pgtable-generic.c mm: fix TLB flush race between migration, and change_protection_range 2013-12-18 19:04:51 -08:00
process_vm_access.c Fix: compat_rw_copy_check_uvector() misuse in aio, readv, writev, and security keys 2013-03-12 11:05:45 -07:00
quicklist.c mm: delete various needless include <linux/module.h> 2011-10-31 09:20:11 -04:00
readahead.c mm/readahead.c: inline ra_submit 2014-11-21 09:23:06 -08:00
rmap.c mm: fix anon_vma_clone() error treatment 2014-12-16 09:34:26 -08:00
shmem.c shmem: fix init_page_accessed use to stop !PageLRU bug 2015-01-29 17:40:52 -08:00
slab.c mm: optimize put_mems_allowed() usage 2014-10-09 12:21:28 -07:00
slab.h mm: Enable SLUB for RT 2020-10-14 00:59:12 +03:00
slab_common.c slab_common: fix the check for duplicate slab names 2014-07-31 12:52:55 -07:00
slob.c mm/sl[aou]b: Move kmallocXXX functions to common code 2013-09-04 20:51:33 +03:00
slub.c slub: delay ctor until the object is requested 2020-10-14 00:59:12 +03:00
sparse-vmemmap.c mm/sparse: use memblock apis for early memory allocations 2014-01-21 16:19:47 -08:00
sparse.c mm/sparse: use memblock apis for early memory allocations 2014-01-21 16:19:47 -08:00
swap.c mm: convert swap to percpu locked 2020-10-14 00:59:12 +03:00
swap_state.c mm: page_alloc: convert hot/cold parameter and immediate callers to bool 2015-01-29 17:40:51 -08:00
swapfile.c swap: change swap_list_head to plist, add swap_avail_head 2014-10-09 12:21:28 -07:00
truncate.c mm + fs: prepare for non-page entries in page cache radix trees 2014-11-21 09:23:06 -08:00
util.c vm_is_stack: use for_each_thread() rather then buggy while_each_thread() 2014-09-05 16:34:18 -07:00
vmacache.c mm: don't pointlessly use BUG_ON() for sanity check 2014-10-09 12:21:29 -07:00
vmalloc.c mm-vmalloc.patch 2020-10-14 00:59:18 +03:00
vmpressure.c mm/vmpressure.c: fix race in vmpressure_work_fn() 2014-12-16 09:34:26 -08:00
vmscan.c mm: move zone->pages_scanned into a vmstat counter 2015-01-29 17:40:53 -08:00
vmstat.c mm: make vmstat -rt aware 2020-10-14 00:59:12 +03:00
zbud.c mm/zbud: fix some trivial typos in comments 2013-09-11 15:57:35 -07:00
zsmalloc.c zsmalloc: add copyright 2014-01-30 16:56:55 -08:00
zswap.c mm/zswap.c: change params from hidden to ro 2014-01-23 16:36:50 -08:00