OpenE2K/linux
6
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
370aadd16d
linux/lib
History
David Jeffery dba0d9caa6 sbitmap: only queue kyber's wait callback if not already active 
[ Upstream commit df034c93f1 ]

Under heavy loads where the kyber I/O scheduler hits the token limits for
its scheduling domains, kyber can become stuck.  When active requests
complete, kyber may not be woken up leaving the I/O requests in kyber
stuck.

This stuck state is due to a race condition with kyber and the sbitmap
functions it uses to run a callback when enough requests have completed.
The running of a sbt_wait callback can race with the attempt to insert the
sbt_wait.  Since sbitmap_del_wait_queue removes the sbt_wait from the list
first then sets the sbq field to NULL, kyber can see the item as not on a
list but the call to sbitmap_add_wait_queue will see sbq as non-NULL. This
results in the sbt_wait being inserted onto the wait list but ws_active
doesn't get incremented.  So the sbitmap queue does not know there is a
waiter on a wait list.

Since sbitmap doesn't think there is a waiter, kyber may never be
informed that there are domain tokens available and the I/O never advances.
With the sbt_wait on a wait list, kyber believes it has an active waiter
so cannot insert a new waiter when reaching the domain's full state.

This race can be fixed by only adding the sbt_wait to the queue if the
sbq field is NULL.  If sbq is not NULL, there is already an action active
which will trigger the re-running of kyber.  Let it run and add the
sbt_wait to the wait list if still needing to wait.

Reviewed-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Jeffery <djeffery@redhat.com>
Reported-by: John Pittman <jpittman@redhat.com>
Tested-by: John Pittman <jpittman@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2020-01-12 12:21:44 +01:00
..
842 lib: 842: no need to check return value of debugfs_create functions 2019-07-03 16:57:17 +02:00
crypto crypto: sha256 - Remove sha256/224_init code duplication 2019-09-05 14:54:54 +10:00
dim lib/dim: Fix -Wunused-const-variable warnings 2019-07-25 11:34:39 -07:00
fonts fonts: Prefer a bigger font for high resolution screens 2019-06-20 10:11:24 +02:00
livepatch
lz4 lz4: do not export static symbol 2019-09-20 09:06:26 -07:00
lzo lib/lzo/lzo1x_compress.c: fix alignment bug in lzo-rle 2019-09-25 17:51:41 -07:00
math
mpi lib/mpi: Eliminate unused umul_ppmm definitions for MIPS 2019-08-22 14:39:36 +10:00
raid6 lib: raid6: fix awk build warnings 2019-12-17 19:56:09 +01:00
reed_solomon rslib: Make some functions static 2019-07-02 08:41:37 +02:00
vdso lib/vdso: Make clock_getres() POSIX compliant again 2019-10-23 14:48:23 +02:00
xz lib/xz: fix XZ_DYNALLOC to avoid useless memory reallocations 2019-11-15 18:34:00 -08:00
zlib_deflate
zlib_inflate
zstd lib/zstd/mem.h: replace __inline by inline 2019-09-15 19:42:16 +02:00
.gitignore
argv_split.c
ashldi3.c
ashrdi3.c
asn1_decoder.c
assoc_array.c
atomic64_test.c
atomic64.c locking/atomic: Use s64 for atomic64 2019-06-03 12:32:56 +02:00
audit.c
bcd.c
bch.c
bitmap.c cpumask: Implement cpumask_or_equal() 2019-07-25 15:47:37 +02:00
bitrev.c
bsearch.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 372 2019-06-05 17:37:10 +02:00
btree.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 431 2019-06-05 17:37:16 +02:00
bucket_locks.c
bug.c bug: move WARN_ON() "cut here" into exception handler 2019-09-25 17:51:41 -07:00
build_OID_registry
bust_spinlocks.c
chacha.c
check_signature.c
checksum.c
clz_ctz.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
clz_tab.c
cmdline.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230 2019-06-19 17:09:06 +02:00
cmpdi2.c
compat_audit.c
cpu_rmap.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
cpumask.c
crc4.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230 2019-06-19 17:09:06 +02:00
crc7.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230 2019-06-19 17:09:06 +02:00
crc8.c
crc16.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230 2019-06-19 17:09:06 +02:00
crc32.c
crc32defs.h
crc32test.c
crc64.c
crc-ccitt.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230 2019-06-19 17:09:06 +02:00
crc-itu-t.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230 2019-06-19 17:09:06 +02:00
crc-t10dif.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230 2019-06-19 17:09:06 +02:00
ctype.c
debug_info.c
debug_locks.c
debugobjects.c debugobjects: Move printk out of db->lock critical sections 2019-06-14 14:51:16 +02:00
dec_and_lock.c
decompress_bunzip2.c
decompress_inflate.c
decompress_unlz4.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
decompress_unlzma.c
decompress_unlzo.c
decompress_unxz.c
decompress.c
devres.c devres: allow const resource arguments 2019-07-05 11:12:07 +09:00
digsig.c Revert "Merge tag 'keys-acl-20190703' of git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs" 2019-07-10 18:43:43 -07:00
dump_stack.c dump_stack: avoid the livelock of the dump_lock 2019-11-06 08:47:50 -08:00
dynamic_debug.c lib: dynamic_debug: no need to check return value of debugfs_create functions 2019-06-18 13:47:24 +02:00
dynamic_queue_limits.c
earlycpio.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 283 2019-06-05 17:36:37 +02:00
error-inject.c
errseq.c
extable.c lib/extable.c: add missing prototypes 2019-09-25 17:51:39 -07:00
fault-inject.c fault-inject: clean up debugfs file creation logic 2019-06-18 13:47:24 +02:00
fdt_empty_tree.c
fdt_ro.c
fdt_rw.c
fdt_strerror.c
fdt_sw.c
fdt_wip.c
fdt.c
find_bit_benchmark.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 295 2019-06-05 17:36:38 +02:00
find_bit.c
flex_proportions.c
gen_crc32table.c
gen_crc64table.c
genalloc.c dma-mapping updates for Linux 5.3 2019-07-12 15:13:55 -07:00
generic-radix-tree.c lib/generic-radix-tree.c: add kmemleak annotations 2019-10-14 15:04:00 -07:00
glob.c
globtest.c
hexdump.c lib/hexdump: make print_hex_dump_bytes() a nop on !DEBUG builds 2019-09-25 17:51:39 -07:00
hweight.c
idr.c idr: Fix idr_get_next_ul race with idr_remove 2019-11-01 22:26:34 -04:00
inflate.c
interval_tree_test.c
interval_tree.c
iomap_copy.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 504 2019-06-19 17:09:56 +02:00
iomap.c
iommu-helper.c
ioremap.c mm/ioremap: probe platform for p4d huge map support 2019-07-16 19:23:22 -07:00
iov_iter.c mm: introduce page_size() 2019-09-24 15:54:08 -07:00
irq_poll.c
irq_regs.c
is_single_threaded.c
kasprintf.c
Kconfig lib: Remove select of inexistant GENERIC_IO 2019-11-10 10:38:43 -08:00
Kconfig.debug compiler: enable CONFIG_OPTIMIZE_INLINING forcibly 2019-09-25 17:51:40 -07:00
Kconfig.kasan kasan: add memory corruption identification for software tag-based mode 2019-09-24 15:54:07 -07:00
Kconfig.kgdb
Kconfig.ubsan
kfifo.c Partially revert "kfifo: fix kfifo_alloc() and kfifo_init()" 2019-08-30 18:47:15 -07:00
klist.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 248 2019-06-19 17:09:08 +02:00
kobject_uevent.c
kobject.c lib : kobject: fix refcount imblance on kobject_rename 2019-06-19 19:27:39 +02:00
kstrtox.c
kstrtox.h
libcrc32c.c
list_debug.c
list_sort.c lib: list_sort.c: add a blank line to avoid kernel-doc warnings 2019-06-20 14:07:34 -06:00
llist.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 333 2019-06-05 17:37:06 +02:00
locking-selftest-hardirq.h
locking-selftest-mutex.h
locking-selftest-rlock-hardirq.h
locking-selftest-rlock-softirq.h
locking-selftest-rlock.h
locking-selftest-rsem.h
locking-selftest-rtmutex.h
locking-selftest-softirq.h
locking-selftest-spin-hardirq.h
locking-selftest-spin-softirq.h
locking-selftest-spin.h
locking-selftest-wlock-hardirq.h
locking-selftest-wlock-softirq.h
locking-selftest-wlock.h
locking-selftest-wsem.h
locking-selftest.c
lockref.c lockref: Limit number of cmpxchg loop retries 2019-06-07 13:15:06 -07:00
logic_pio.c lib: logic_pio: Add logic_pio_unregister_range() 2019-08-13 14:54:24 +08:00
lru_cache.c
lshrdi3.c
Makefile Merge branch 'x86/cleanups' into x86/cpu, to pick up dependent changes 2019-09-06 07:30:23 +02:00
memcat_p.c
memory-notifier-error-inject.c
memweight.c
muldi3.c
net_utils.c
netdev-notifier-error-inject.c
nlattr.c
nmi_backtrace.c
nodemask.c
notifier-error-inject.c lib: notifier-error-inject: no need to check return value of debugfs_create functions 2019-07-03 16:57:18 +02:00
notifier-error-inject.h
objagg.c lib: objagg: Use struct_size() in kzalloc() 2019-06-05 19:03:39 -07:00
of-reconfig-notifier-error-inject.c
oid_registry.c
once.c
packing.c
parman.c
parser.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230 2019-06-19 17:09:06 +02:00
pci_iomap.c
percpu_counter.c
percpu_test.c
percpu-refcount.c Merge branch 'for-5.3' of git://git.kernel.org/pub/scm/linux/kernel/git/dennis/percpu 2019-07-14 16:17:18 -07:00
plist.c
pm-notifier-error-inject.c
radix-tree.c idr: Fix idr_alloc_u32 on 32-bit systems 2019-11-03 06:36:50 -05:00
random32.c
ratelimit.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 428 2019-06-05 17:37:16 +02:00
rbtree_test.c augmented rbtree: add new RB_DECLARE_CALLBACKS_MAX macro 2019-09-25 17:51:39 -07:00
rbtree.c lib/rbtree: avoid generating code twice for the cached versions 2019-07-16 19:23:22 -07:00
refcount.c
rhashtable.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
sbitmap.c sbitmap: only queue kyber's wait callback if not already active 2020-01-12 12:21:44 +01:00
scatterlist.c SCSI sg on 20190709 2019-07-11 15:17:41 -07:00
seq_buf.c
sg_pool.c SCSI sg on 20190709 2019-07-11 15:17:41 -07:00
sg_split.c lib: scatterlist: Fix to support no mapped sg 2019-08-08 07:45:01 -06:00
sha1.c
show_mem.c mm: remove quicklist page table caches 2019-09-24 15:54:09 -07:00
siphash.c
smp_processor_id.c sched/core: Provide a pointer to the valid CPU mask 2019-06-03 11:49:37 +02:00
sort.c media: lib/sort.c: implement sort() variant taking context argument 2019-08-19 13:14:53 -03:00
stackdepot.c lib/stackdepot: Fix outdated comments 2019-08-19 12:57:28 +02:00
stmp_device.c
string_helpers.c lib/string_helpers: fix some kerneldoc warnings 2019-07-16 19:23:22 -07:00
string.c lib/string: Make memzero_explicit() inline instead of external 2019-10-08 13:27:05 +02:00
strncpy_from_user.c lib: untag user pointers in strn*_user 2019-09-25 17:51:41 -07:00
strnlen_user.c lib: introduce copy_struct_from_user() helper 2019-10-01 15:45:03 +02:00
syscall.c
test_bitfield.c
test_bitmap.c
test_blackhole_dev.c blackhole_dev: add a selftest 2019-07-01 19:34:46 -07:00
test_bpf.c test_bpf: Fix a new clang warning about xor-ing two numbers 2019-08-20 17:07:29 +02:00
test_debug_virtual.c
test_firmware.c test_firmware: add support for request_firmware_into_buf 2019-09-04 13:31:28 +02:00
test_hash.c
test_hexdump.c
test_ida.c
test_kasan.c lib/test_kasan.c: add roundtrip tests 2019-09-24 15:54:08 -07:00
test_kmod.c
test_list_sort.c
test_memcat_p.c
test_meminit.c lib/test_meminit: add a kmem_cache_alloc_bulk() test 2019-10-14 15:04:01 -07:00
test_module.c
test_objagg.c
test_overflow.c lib/test_overflow.c: avoid tainting the kernel and fix wrap size 2019-07-16 19:23:22 -07:00
test_parman.c
test_printf.c lib/test_printf: Remove obvious comments from %pd and %pD tests 2019-08-15 17:03:16 +02:00
test_rhashtable.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
test_siphash.c
test_sort.c
test_stackinit.c lib/test_stackinit: Handle Clang auto-initialization pattern 2019-06-05 07:36:43 -07:00
test_static_key_base.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 282 2019-06-05 17:36:37 +02:00
test_static_keys.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 282 2019-06-05 17:36:37 +02:00
test_string.c lib/test_string.c: add some testcases for strchr and strnchr 2019-07-16 19:23:22 -07:00
test_strscpy.c
test_sysctl.c
test_ubsan.c
test_user_copy.c usercopy: Avoid soft lockups in test_check_nonzero_user() 2019-10-16 14:56:21 +02:00
test_uuid.c
test_vmalloc.c
test_xarray.c XArray: Fix xas_next() with a single entry at 0 2019-07-01 17:11:16 -04:00
test-kstrtox.c
test-string_helpers.c
textsearch.c lib: textsearch: fix escapes in example code 2019-10-03 12:12:23 -04:00
timerqueue.c lib/timerqueue: Rely on rbtree semantics for next timer 2019-07-24 17:38:01 +02:00
ts_bm.c
ts_fsm.c
ts_kmp.c
ubsan.c lib/ubsan: don't serialize UBSAN report 2020-01-09 10:20:07 +01:00
ubsan.h
ucmpdi2.c
ucs2_string.c
usercopy.c lib: introduce copy_struct_from_user() helper 2019-10-01 15:45:03 +02:00
uuid.c
vsprintf.c vsprintf: Prevent crash when dereferencing invalid pointers for %pD 2019-08-15 16:40:10 +02:00
win_minmax.c
xarray.c XArray: Fix xas_next() with a single entry at 0 2019-07-01 17:11:16 -04:00
xxhash.c