Commit Graph

44 Commits

Author SHA1 Message Date
Paul E. McKenney 1dfa55e019 Merge branches 'cond_resched.2017.12.04a', 'dyntick.2017.11.28a', 'fixes.2017.12.11a', 'srbd.2017.12.05a' and 'torture.2017.12.11a' into HEAD
cond_resched.2017.12.04a: Convert cond_resched_rcu_qs() to cond_resched()
dyntick.2017.11.28a: Make RCU dynticks handle interrupts from NMI
fixes.2017.12.11a: Miscellaneous fixes
srbd.2017.12.05a: Remove now-redundant smp_read_barrier_depends()
torture.2017.12.11a: Torture-testing update
2017-12-11 09:21:58 -08:00
Paul E. McKenney 9ad3c143d7 doc: De-emphasize smp_read_barrier_depends
This commit keeps only the historical and low-level discussion of
smp_read_barrier_depends().

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
[ paulmck: Adjusted to allow for David Howells feedback on prior commit. ]
2017-12-05 11:57:53 -08:00
Paul E. McKenney f2b1760aed doc: Eliminate cond_resched_rcu_qs() in favor of cond_resched()
Now that cond_resched() also provides RCU quiescent states when
needed, it can be used in place of cond_resched_rcu_qs().  This
commit therefore documents this change.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-12-04 10:28:59 -08:00
Paul E. McKenney 3af3999b9a doc: Update dyntick-idle design documentation for NMI/irq consolidation
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-11-28 15:51:22 -08:00
Linus Torvalds 6098850e7e Merge branch 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull RCU updates from Ingo Molnar:
 "The main changes in this cycle are:

   - Documentation updates

   - RCU CPU stall-warning updates

   - Torture-test updates

   - Miscellaneous fixes

  Size wise the biggest updates are to documentation. Excluding
  documentation most of the code increase comes from a single commit
  which expands debugging"

* 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (24 commits)
  srcu: Add parameters to SRCU docbook comments
  doc: Rewrite confusing statement about memory barriers
  memory-barriers.txt: Fix typo in pairing example
  rcu/segcblist: Include rcupdate.h
  rcu: Add extended-quiescent-state testing advice
  rcu: Suppress lockdep false-positive ->boost_mtx complaints
  rcu: Do not include rtmutex_common.h unconditionally
  torture: Provide TMPDIR environment variable to specify tmpdir
  rcutorture: Dump writer stack if stalled
  rcutorture: Add interrupt-disable capability to stall-warning tests
  rcu: Suppress RCU CPU stall warnings while dumping trace
  rcu: Turn off tracing before dumping trace
  rcu: Make RCU CPU stall warnings check for irq-disabled CPUs
  sched,rcu: Make cond_resched() provide RCU quiescent state
  sched: Make resched_cpu() unconditional
  irq_work: Map irq_work_on_queue() to irq_work_on() in !SMP
  rcu: Create call_rcu_tasks() kthread at boot time
  rcu: Fix up pending cbs check in rcu_prepare_for_idle
  memory-barriers: Rework multicopy-atomicity section
  memory-barriers: Replace uses of "transitive"
  ...
2017-11-13 12:18:10 -08:00
Tom Saeger d354a6f150 Documentation: fix ref to workqueue content
Signed-off-by: Tom Saeger <tom.saeger@oracle.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-10-19 12:57:26 -06:00
Paul E. McKenney bb7e5ce7dd documentation: RCU grace-period memory ordering guarantees
This commit provides text and diagrams showing how Tree RCU implements
its grace-period memory ordering guarantees.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
2017-10-09 14:23:36 -07:00
Paul E. McKenney 850bf6d592 doc: Set down RCU's scheduling-clock-interrupt needs
This commit documents the situations in which RCU needs the
scheduling-clock interrupt to be enabled, along with the consequences
of failing to meet RCU's needs in this area.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-08-17 07:31:14 -07:00
Paul E. McKenney ae91aa0adb rcu: Remove debugfs tracing
RCU's debugfs tracing used to be the only reasonable low-level debug
information available, but ftrace and event tracing has since surpassed
the RCU debugfs level of usefulness.  This commit therefore removes
RCU's debugfs tracing.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-06-08 18:52:43 -07:00
Paul E. McKenney 41a2901e7d rcu: Remove SPARSE_RCU_POINTER Kconfig option
The sparse-based checking for non-RCU accesses to RCU-protected pointers
has been around for a very long time, and it is now the only type of
sparse-based checking that is optional.  This commit therefore makes
it unconditional.

Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Fengguang Wu <fengguang.wu@intel.com>
2017-06-08 18:52:41 -07:00
Paul E. McKenney fe5ac724d8 rcu: Remove nohz_full full-system-idle state machine
The NO_HZ_FULL_SYSIDLE full-system-idle capability was added in 2013
by commit 0edd1b1784 ("nohz_full: Add full-system-idle state machine"),
but has not been used.  This commit therefore removes it.

If it turns out to be needed later, this commit can always be reverted.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Ingo Molnar <mingo@kernel.org>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-06-08 18:52:39 -07:00
Paul E. McKenney c75e9caaf8 doc: Take tail recursion into account in RCU requirements
This commit classifies tail recursion as an alternative way to write
a loop, with similar limitations.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-06-08 08:25:34 -07:00
Paul E. McKenney 09f501a0f0 srcu: Document auto-expediting requirement
This commit documents the auto-expediting requirement satisfied by
commits 2da4b2a7fd ("srcu: Expedite first synchronize_srcu() when idle")
and 22607d66bb ("srcu: Specify auto-expedite holdoff time").

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-06-08 08:25:33 -07:00
Paul E. McKenney f2094107ac Merge branches 'doc.2017.04.12a', 'fixes.2017.04.19a' and 'srcu.2017.04.21a' into HEAD
doc.2017.04.12a: Documentation updates
fixes.2017.04.19a: Miscellaneous fixes
srcu.2017.04.21a: Parallelize SRCU callback handling
2017-04-21 06:00:13 -07:00
Paul E. McKenney 9226b10d78 rcu: Place guard on rcu_all_qs() and rcu_note_context_switch() actions
The rcu_all_qs() and rcu_note_context_switch() do a series of checks,
taking various actions to supply RCU with quiescent states, depending
on the outcomes of the various checks.  This is a bit much for scheduling
fastpaths, so this commit creates a separate ->rcu_urgent_qs field in
the rcu_dynticks structure that acts as a global guard for these checks.
Thus, in the common case, rcu_all_qs() and rcu_note_context_switch()
check the ->rcu_urgent_qs field, find it false, and simply return.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
2017-04-18 11:38:18 -07:00
Paul E. McKenney 0f9be8cabb rcu: Eliminate flavor scan in rcu_momentary_dyntick_idle()
The rcu_momentary_dyntick_idle() function scans the RCU flavors, checking
that one of them still needs a quiescent state before doing an expensive
atomic operation on the ->dynticks counter.  However, this check reduces
overhead only after a rare race condition, and increases complexity.  This
commit therefore removes the scan and the mechanism enabling the scan.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-04-18 11:38:18 -07:00
Paul E. McKenney 9577df9a31 rcu: Pull rcu_qs_ctr into rcu_dynticks structure
The rcu_qs_ctr variable is yet another isolated per-CPU variable,
so this commit pulls it into the pre-existing rcu_dynticks per-CPU
structure.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-04-18 11:38:17 -07:00
Paul E. McKenney abb06b9948 rcu: Pull rcu_sched_qs_mask into rcu_dynticks structure
The rcu_sched_qs_mask variable is yet another isolated per-CPU variable,
so this commit pulls it into the pre-existing rcu_dynticks per-CPU
structure.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-04-18 11:38:17 -07:00
Paul E. McKenney 6771853bce doc: Update requirements based on recent changes
These changes include lighter-weight expedited grace periods, the fact
that expedited grace periods and rcu_barrier() no longer block CPU
hotplug, some HTML font fixups, noting that rcu_barrier() need not wait
for a grace period (even if callbacks are posted), the fact that SRCU
read-side critical sections can be used from offline CPUs, and the fact
that SRCU now maintains per-CPU callback lists.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-04-12 08:23:41 -07:00
Paul E. McKenney aa123a748e doc: Update RCU data-structure documentation for rcu_segcblist
The rcu_segcblist data structure, which contains segmented lists
of RCU callbacks, was recently added.  This commit updates the
documentation accordingly.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-04-12 08:23:41 -07:00
Paul E. McKenney b4553f0cfe doc: Add mid-boot operation to expedited grace periods
This commit adds a description of how expedited grace periods operate
during the mid-boot "dead zone", which starts when the scheduler spawns
the first kthread and ends when all of RCU's kthreads have been spawned.
In short, before mid-boot, synchronous grace periods can be a no-op.
After the end of mid-boot, workqueues may be used.  During mid-boot,
the requesting task drivees the expedited grace period.

For more detail, see https://lwn.net/Articles/716148/.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-04-12 08:23:40 -07:00
Paul E. McKenney f1387d7705 doc: Synchronous RCU grace periods are now legal throughout boot
This commit updates the "Early Boot" section of the RCU requirements
to describe how synchronous RCU grace periods are now legal throughout
the boot process.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2017-04-12 08:23:39 -07:00
Paul E. McKenney 4afac159c3 doc: Quick-Quiz answers are now inline
Now that quick-quiz answers are inline, there is no separate section
containing those answers.  This commit therefore removes the dangling
reference from the RCU data-structures design documentation.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
2017-01-14 21:30:33 -08:00
Tetsuo Handa 526914a0ae doc: Fix RCU requirements typos
Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
2017-01-14 21:29:10 -08:00
Paul E. McKenney e73c14ebc3 rcu: Design documentation for expedited grace periods
This commit adds design documentation for expedited grace periods.
This documentation is in HTML rather than the new documentation
format because (1) I have prototype documentation already in HTML,
and (2) Attempting to learn the new documentation format while
creating the design documentation seems likely to result in neither
happening in a timely fashion.

Once the design documentation is complete, we can start a conversion effort.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
2017-01-14 21:28:58 -08:00
Paul E. McKenney e2c85cb12c documentation: Present updated RCU guarantee
Recent memory-model work deduces the relationships of RCU read-side
critical sections and grace periods based on the relationships of
accesses within a critical section and accesses preceding and following
the grace period.  This commit therefore adds this viewpoint.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2016-11-14 10:39:36 -08:00
Paul E. McKenney ed2bec07fd documentation: Record reason for rcu_head two-byte alignment
There is an assertion in __call_rcu() that checks only the bottom
bit of the rcu_head pointer, rather than the bottom two (as might be
expected for 32-bit systems) or the bottom three (as might be expected
for 64-bit systems).  This choice might be a bit surprising in these days
of ubiquitous 32-bit and 64-bit systems.  This commit therefore records
the reason for this odd alignment check, namely that m68k guarantees
only two-byte alignment despite being a 32-bit architectures.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2016-08-22 09:25:33 -07:00
Paul E. McKenney c79dac758d documentation: Add RCU_NONIDLE() restrictions to requirements
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2016-06-14 16:00:59 -07:00
Paul E. McKenney 5c1458478c documentation: Add documentation for RCU's major data structures
This commit adds documentation for RCU's major data structures,
including rcu_state, rcu_node, rcu_data, rcu_dynticks, and rcu_head.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2016-04-19 14:48:12 -07:00
Paul E. McKenney 0c7d10e4b9 documentation: Emphasize the call_rcu() is illegal from idle
Although call_rcu()'s fastpath works just fine on an idle CPU,
some branches of the slowpath invoke the scheduler, which uses
RCU.  Therefore, this commit emphasizes the fact that call_rcu()
must not be invoked from an idle CPU.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2016-03-31 13:33:23 -07:00
Paul E. McKenney 5413e24c94 documentation: Sharpen up the no-readers quick quiz
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2016-03-31 13:33:23 -07:00
Paul E. McKenney 6146f8df48 documentation: Get rid of duplicate .htmlx file
This commit uses colors to obscure the quick-quiz answers, thus getting
rid of the .htmlx file.  Use your mouse to select the answer in order
to see the text.  Alternatively, use your favorite scripting language
to remove all occurences of "<font color="ffffff">" from the file.

Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2016-03-31 13:33:22 -07:00
Paul E. McKenney 11a65df573 documentation: Remove unnecessary images from requirements
This commit removes a cutesy cartoon and also a diagram that can
just as easily be represented by text.

Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2016-03-31 13:33:21 -07:00
Paul E. McKenney 514f1eb5f4 documentation: Document illegality of call_rcu() from offline CPUs
There is already a blanket statement about no member of RCU's API
being legal from an offline CPU, but add an explicit note where it
states that it is illegal to invoke call_rcu() from an NMI handler.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2016-03-31 13:33:21 -07:00
Paul E. McKenney d8936c0b7e documentation: Explain why rcu_read_lock() needs no barrier()
This commit adds a Quick Quiz whose answer explains why the compiler
code reordering enabled by CONFIG_PREEMPT=n's empty rcu_read_lock()
and rcu_read_unlock() functions does not hinder RCU's ability to figure
out which RCU read-side critical sections have completed and not.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2016-03-31 13:33:20 -07:00
Paul E. McKenney f43b62542e documentation: Add synchronize_rcu_mult() to the requirements
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2016-03-31 13:33:20 -07:00
Paul E. McKenney 41abcf321d documentation: Add real-time requirements from CPU-bound workloads
This commit records RCU's responsibility to avoid degrading latencies
of CPUs running tight loops within properly configured workloads,
both in kernel and in userspace.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2016-03-31 13:33:19 -07:00
Paul E. McKenney c64c4b0f9a documentation: Update RCU requirements based on expedited changes
Because RCU-sched expedited grace periods now use IPIs and interact
with rcu_read_unlock(), it is no longer sufficient to disable preemption
across RCU read-side critical sections that acquire and hold scheduler
locks.  It is now necessary to instead disable interrupts.  This commit
documents this change.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2015-12-05 12:34:32 -08:00
Paul E. McKenney 4b689330b1 documentation: Clarify RCU memory barriers and requirements
The RCU requirements do not make it absolutely clear that the
memory-barrier requirements are not intended to replace the fundamental
requirement that all pre-existing RCU readers complete before a grace
period completes.  This commit therefore pulls the memory-barrier
requirements into a separate section and explicitly calls out the
relationship between the memory-barrier requirements and the fundamental
requirement.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2015-12-05 12:33:37 -08:00
Paul E. McKenney a4b575627e documentation: Expand on scheduler/RCU deadlock requirements
This commit adds a second option for avoiding scheduler/RCU deadlocks,
namely that preemption be disabled across the entire RCU read-side
critical section in question.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2015-12-05 12:33:26 -08:00
Paul E. McKenney 0825458b1d documentation: Composability analogies
This commit expands on RCU's composability by comparing it to that of
transactional memory and of locking.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2015-12-05 12:33:11 -08:00
Paul E. McKenney 01d3ad3834 documentation: Cover requirements controlling stall warnings
This commit adds verbiage on boot and sysfs parameters that can be
used to control RCU CPU stall warnings, both to change the timeout
and to suppress these warnings entirely.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
2015-12-05 12:32:19 -08:00
Paul E. McKenney 701e80312f Documentation: Record bottom-bit-zero guarantee for ->next
This commit records RCU's guarantee that the bottom bit of the rcu_head
structure's ->next field will remain zero for callbacks posted via
call_rcu(), but not necessarily for <tt>kfree_rcu()</tt> or some
possible future call_rcu_lazy() variant that might one day be created
for energy-efficiency purposese.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
[ paulmck: Updates URLs as suggested by Josh Triplett. ]
2015-12-05 12:31:47 -08:00
Paul E. McKenney 649e4368ff documentation: Record RCU requirements
This commit adds RCU requirements as published in a 2015 LWN series.
Bringing these requirements in-tree allows them to be updated as changes
are discovered.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
[ paulmck: Updates to charset and URLs as suggested by Josh Triplett. ]
2015-12-05 12:19:07 -08:00