2005-04-17 00:20:36 +02:00
|
|
|
/*
|
|
|
|
* raid5.c : Multiple Devices driver for Linux
|
|
|
|
* Copyright (C) 1996, 1997 Ingo Molnar, Miguel de Icaza, Gadi Oxman
|
|
|
|
* Copyright (C) 1999, 2000 Ingo Molnar
|
2006-06-26 09:27:38 +02:00
|
|
|
* Copyright (C) 2002, 2003 H. Peter Anvin
|
2005-04-17 00:20:36 +02:00
|
|
|
*
|
2006-06-26 09:27:38 +02:00
|
|
|
* RAID-4/5/6 management functions.
|
|
|
|
* Thanks to Penguin Computing for making the RAID-6 development possible
|
|
|
|
* by donating a test server!
|
2005-04-17 00:20:36 +02:00
|
|
|
*
|
|
|
|
* This program is free software; you can redistribute it and/or modify
|
|
|
|
* it under the terms of the GNU General Public License as published by
|
|
|
|
* the Free Software Foundation; either version 2, or (at your option)
|
|
|
|
* any later version.
|
|
|
|
*
|
|
|
|
* You should have received a copy of the GNU General Public License
|
|
|
|
* (for example /usr/src/linux/COPYING); if not, write to the Free
|
|
|
|
* Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
|
|
|
|
*/
|
|
|
|
|
2006-07-10 13:44:17 +02:00
|
|
|
/*
|
|
|
|
* BITMAP UNPLUGGING:
|
|
|
|
*
|
|
|
|
* The sequencing for updating the bitmap reliably is a little
|
|
|
|
* subtle (and I got it wrong the first time) so it deserves some
|
|
|
|
* explanation.
|
|
|
|
*
|
|
|
|
* We group bitmap updates into batches. Each batch has a number.
|
|
|
|
* We may write out several batches at once, but that isn't very important.
|
2011-04-18 10:25:43 +02:00
|
|
|
* conf->seq_write is the number of the last batch successfully written.
|
|
|
|
* conf->seq_flush is the number of the last batch that was closed to
|
2006-07-10 13:44:17 +02:00
|
|
|
* new additions.
|
|
|
|
* When we discover that we will need to write to any block in a stripe
|
|
|
|
* (in add_stripe_bio) we update the in-memory bitmap and record in sh->bm_seq
|
2011-04-18 10:25:43 +02:00
|
|
|
* the number of the batch it will be in. This is seq_flush+1.
|
2006-07-10 13:44:17 +02:00
|
|
|
* When we are ready to do a write, if that batch hasn't been written yet,
|
|
|
|
* we plug the array and queue the stripe for later.
|
|
|
|
* When an unplug happens, we increment bm_flush, thus closing the current
|
|
|
|
* batch.
|
|
|
|
* When we notice that bm_flush > bm_write, we write out all pending updates
|
|
|
|
* to the bitmap, and advance bm_write to where bm_flush was.
|
|
|
|
* This may occasionally write a bit out twice, but is sure never to
|
|
|
|
* miss any bits.
|
|
|
|
*/
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2009-03-31 05:33:13 +02:00
|
|
|
#include <linux/blkdev.h>
|
2006-03-27 11:18:11 +02:00
|
|
|
#include <linux/kthread.h>
|
2009-03-31 06:09:39 +02:00
|
|
|
#include <linux/raid/pq.h>
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
#include <linux/async_tx.h>
|
2009-08-30 04:13:13 +02:00
|
|
|
#include <linux/async.h>
|
2009-03-31 05:33:13 +02:00
|
|
|
#include <linux/seq_file.h>
|
2009-07-14 20:48:22 +02:00
|
|
|
#include <linux/cpu.h>
|
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-24 09:04:11 +01:00
|
|
|
#include <linux/slab.h>
|
2009-03-31 05:33:13 +02:00
|
|
|
#include "md.h"
|
2009-03-31 05:33:13 +02:00
|
|
|
#include "raid5.h"
|
2010-03-08 06:02:42 +01:00
|
|
|
#include "raid0.h"
|
2009-03-31 05:27:03 +02:00
|
|
|
#include "bitmap.h"
|
2005-09-10 01:23:54 +02:00
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
/*
|
|
|
|
* Stripe cache
|
|
|
|
*/
|
|
|
|
|
|
|
|
#define NR_STRIPES 256
|
|
|
|
#define STRIPE_SIZE PAGE_SIZE
|
|
|
|
#define STRIPE_SHIFT (PAGE_SHIFT - 9)
|
|
|
|
#define STRIPE_SECTORS (STRIPE_SIZE>>9)
|
|
|
|
#define IO_THRESHOLD 1
|
2008-04-28 11:15:53 +02:00
|
|
|
#define BYPASS_THRESHOLD 1
|
2006-01-06 09:20:33 +01:00
|
|
|
#define NR_HASH (PAGE_SIZE / sizeof(struct hlist_head))
|
2005-04-17 00:20:36 +02:00
|
|
|
#define HASH_MASK (NR_HASH - 1)
|
|
|
|
|
2006-01-06 09:20:33 +01:00
|
|
|
#define stripe_hash(conf, sect) (&((conf)->stripe_hashtbl[((sect) >> STRIPE_SHIFT) & HASH_MASK]))
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
/* bio's attached to a stripe+device for I/O are linked together in bi_sector
|
|
|
|
* order without overlap. There may be several bio's per stripe+device, and
|
|
|
|
* a bio could span several devices.
|
|
|
|
* When walking this list for a particular stripe+device, we must never proceed
|
|
|
|
* beyond a bio that extends past this device, as the next bio might no longer
|
|
|
|
* be valid.
|
|
|
|
* This macro is used to determine the 'next' bio in the list, given the sector
|
|
|
|
* of the current stripe+device
|
|
|
|
*/
|
|
|
|
#define r5_next_bio(bio, sect) ( ( (bio)->bi_sector + ((bio)->bi_size>>9) < sect + STRIPE_SECTORS) ? (bio)->bi_next : NULL)
|
|
|
|
/*
|
|
|
|
* The following can be used to debug the driver
|
|
|
|
*/
|
|
|
|
#define RAID5_PARANOIA 1
|
|
|
|
#if RAID5_PARANOIA && defined(CONFIG_SMP)
|
|
|
|
# define CHECK_DEVLOCK() assert_spin_locked(&conf->device_lock)
|
|
|
|
#else
|
|
|
|
# define CHECK_DEVLOCK()
|
|
|
|
#endif
|
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
#ifdef DEBUG
|
2005-04-17 00:20:36 +02:00
|
|
|
#define inline
|
|
|
|
#define __inline__
|
|
|
|
#endif
|
|
|
|
|
2008-05-23 22:04:34 +02:00
|
|
|
#define printk_rl(args...) ((void) (printk_ratelimit() && printk(args)))
|
|
|
|
|
2008-08-15 10:41:18 +02:00
|
|
|
/*
|
2008-08-15 10:56:11 +02:00
|
|
|
* We maintain a biased count of active stripes in the bottom 16 bits of
|
|
|
|
* bi_phys_segments, and a count of processed stripes in the upper 16 bits
|
2008-08-15 10:41:18 +02:00
|
|
|
*/
|
|
|
|
static inline int raid5_bi_phys_segments(struct bio *bio)
|
|
|
|
{
|
2008-08-15 10:56:11 +02:00
|
|
|
return bio->bi_phys_segments & 0xffff;
|
2008-08-15 10:41:18 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
static inline int raid5_bi_hw_segments(struct bio *bio)
|
|
|
|
{
|
2008-08-15 10:56:11 +02:00
|
|
|
return (bio->bi_phys_segments >> 16) & 0xffff;
|
2008-08-15 10:41:18 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
static inline int raid5_dec_bi_phys_segments(struct bio *bio)
|
|
|
|
{
|
|
|
|
--bio->bi_phys_segments;
|
|
|
|
return raid5_bi_phys_segments(bio);
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline int raid5_dec_bi_hw_segments(struct bio *bio)
|
|
|
|
{
|
|
|
|
unsigned short val = raid5_bi_hw_segments(bio);
|
|
|
|
|
|
|
|
--val;
|
2008-08-15 10:56:11 +02:00
|
|
|
bio->bi_phys_segments = (val << 16) | raid5_bi_phys_segments(bio);
|
2008-08-15 10:41:18 +02:00
|
|
|
return val;
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline void raid5_set_bi_hw_segments(struct bio *bio, unsigned int cnt)
|
|
|
|
{
|
2011-06-13 07:48:22 +02:00
|
|
|
bio->bi_phys_segments = raid5_bi_phys_segments(bio) | (cnt << 16);
|
2008-08-15 10:41:18 +02:00
|
|
|
}
|
|
|
|
|
2009-03-31 05:39:38 +02:00
|
|
|
/* Find first data disk in a raid6 stripe */
|
|
|
|
static inline int raid6_d0(struct stripe_head *sh)
|
|
|
|
{
|
2009-03-31 05:39:38 +02:00
|
|
|
if (sh->ddf_layout)
|
|
|
|
/* ddf always start from first device */
|
|
|
|
return 0;
|
|
|
|
/* md starts just after Q block */
|
2009-03-31 05:39:38 +02:00
|
|
|
if (sh->qd_idx == sh->disks - 1)
|
|
|
|
return 0;
|
|
|
|
else
|
|
|
|
return sh->qd_idx + 1;
|
|
|
|
}
|
2006-06-26 09:27:38 +02:00
|
|
|
static inline int raid6_next_disk(int disk, int raid_disks)
|
|
|
|
{
|
|
|
|
disk++;
|
|
|
|
return (disk < raid_disks) ? disk : 0;
|
|
|
|
}
|
2007-07-09 20:56:43 +02:00
|
|
|
|
2009-03-31 05:39:38 +02:00
|
|
|
/* When walking through the disks in a raid5, starting at raid6_d0,
|
|
|
|
* We need to map each disk to a 'slot', where the data disks are slot
|
|
|
|
* 0 .. raid_disks-3, the parity disk is raid_disks-2 and the Q disk
|
|
|
|
* is raid_disks-1. This help does that mapping.
|
|
|
|
*/
|
2009-03-31 05:39:38 +02:00
|
|
|
static int raid6_idx_to_slot(int idx, struct stripe_head *sh,
|
|
|
|
int *count, int syndrome_disks)
|
2009-03-31 05:39:38 +02:00
|
|
|
{
|
2009-10-20 03:09:32 +02:00
|
|
|
int slot = *count;
|
2009-03-31 05:39:38 +02:00
|
|
|
|
2009-10-16 07:27:34 +02:00
|
|
|
if (sh->ddf_layout)
|
2009-10-20 03:09:32 +02:00
|
|
|
(*count)++;
|
2009-03-31 05:39:38 +02:00
|
|
|
if (idx == sh->pd_idx)
|
2009-03-31 05:39:38 +02:00
|
|
|
return syndrome_disks;
|
2009-03-31 05:39:38 +02:00
|
|
|
if (idx == sh->qd_idx)
|
2009-03-31 05:39:38 +02:00
|
|
|
return syndrome_disks + 1;
|
2009-10-16 07:27:34 +02:00
|
|
|
if (!sh->ddf_layout)
|
2009-10-20 03:09:32 +02:00
|
|
|
(*count)++;
|
2009-03-31 05:39:38 +02:00
|
|
|
return slot;
|
|
|
|
}
|
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
static void return_io(struct bio *return_bi)
|
|
|
|
{
|
|
|
|
struct bio *bi = return_bi;
|
|
|
|
while (bi) {
|
|
|
|
|
|
|
|
return_bi = bi->bi_next;
|
|
|
|
bi->bi_next = NULL;
|
|
|
|
bi->bi_size = 0;
|
2008-06-28 00:31:20 +02:00
|
|
|
bio_endio(bi, 0);
|
2007-07-09 20:56:43 +02:00
|
|
|
bi = return_bi;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
static void print_raid5_conf (raid5_conf_t *conf);
|
|
|
|
|
md: replace STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} with 'reconstruct_states'
From: Dan Williams <dan.j.williams@intel.com>
Track the state of reconstruct operations (recalculating the parity block
usually due to incoming writes, or as part of array expansion) Reduces the
scope of the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags to only tracking whether
a reconstruct operation has been requested via the ops_request field of struct
stripe_head_state.
This is the final step in the removal of ops.{pending,ack,complete,count}, i.e.
the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags only request an operation and do
not track the state of the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:32:05 +02:00
|
|
|
static int stripe_operations_active(struct stripe_head *sh)
|
|
|
|
{
|
|
|
|
return sh->check_state || sh->reconstruct_state ||
|
|
|
|
test_bit(STRIPE_BIOFILL_RUN, &sh->state) ||
|
|
|
|
test_bit(STRIPE_COMPUTE_RUN, &sh->state);
|
|
|
|
}
|
|
|
|
|
2006-01-14 22:20:43 +01:00
|
|
|
static void __release_stripe(raid5_conf_t *conf, struct stripe_head *sh)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
if (atomic_dec_and_test(&sh->count)) {
|
2006-04-02 13:31:42 +02:00
|
|
|
BUG_ON(!list_empty(&sh->lru));
|
|
|
|
BUG_ON(atomic_read(&conf->active_stripes)==0);
|
2005-04-17 00:20:36 +02:00
|
|
|
if (test_bit(STRIPE_HANDLE, &sh->state)) {
|
2011-04-18 10:25:42 +02:00
|
|
|
if (test_bit(STRIPE_DELAYED, &sh->state))
|
2005-04-17 00:20:36 +02:00
|
|
|
list_add_tail(&sh->lru, &conf->delayed_list);
|
2011-04-18 10:25:42 +02:00
|
|
|
else if (test_bit(STRIPE_BIT_DELAY, &sh->state) &&
|
|
|
|
sh->bm_seq - conf->seq_write > 0)
|
2005-09-10 01:23:54 +02:00
|
|
|
list_add_tail(&sh->lru, &conf->bitmap_list);
|
2011-04-18 10:25:42 +02:00
|
|
|
else {
|
2005-09-10 01:23:54 +02:00
|
|
|
clear_bit(STRIPE_BIT_DELAY, &sh->state);
|
2005-04-17 00:20:36 +02:00
|
|
|
list_add_tail(&sh->lru, &conf->handle_list);
|
2005-09-10 01:23:54 +02:00
|
|
|
}
|
2005-04-17 00:20:36 +02:00
|
|
|
md_wakeup_thread(conf->mddev->thread);
|
|
|
|
} else {
|
md: replace STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} with 'reconstruct_states'
From: Dan Williams <dan.j.williams@intel.com>
Track the state of reconstruct operations (recalculating the parity block
usually due to incoming writes, or as part of array expansion) Reduces the
scope of the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags to only tracking whether
a reconstruct operation has been requested via the ops_request field of struct
stripe_head_state.
This is the final step in the removal of ops.{pending,ack,complete,count}, i.e.
the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags only request an operation and do
not track the state of the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:32:05 +02:00
|
|
|
BUG_ON(stripe_operations_active(sh));
|
2005-04-17 00:20:36 +02:00
|
|
|
if (test_and_clear_bit(STRIPE_PREREAD_ACTIVE, &sh->state)) {
|
|
|
|
atomic_dec(&conf->preread_active_stripes);
|
|
|
|
if (atomic_read(&conf->preread_active_stripes) < IO_THRESHOLD)
|
|
|
|
md_wakeup_thread(conf->mddev->thread);
|
|
|
|
}
|
|
|
|
atomic_dec(&conf->active_stripes);
|
2006-03-27 11:18:09 +02:00
|
|
|
if (!test_bit(STRIPE_EXPANDING, &sh->state)) {
|
|
|
|
list_add_tail(&sh->lru, &conf->inactive_list);
|
2005-04-17 00:20:36 +02:00
|
|
|
wake_up(&conf->wait_for_stripe);
|
2006-12-10 11:20:47 +01:00
|
|
|
if (conf->retry_read_aligned)
|
|
|
|
md_wakeup_thread(conf->mddev->thread);
|
2006-03-27 11:18:09 +02:00
|
|
|
}
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
2009-03-31 05:39:38 +02:00
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
static void release_stripe(struct stripe_head *sh)
|
|
|
|
{
|
|
|
|
raid5_conf_t *conf = sh->raid_conf;
|
|
|
|
unsigned long flags;
|
2006-06-26 09:27:38 +02:00
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
spin_lock_irqsave(&conf->device_lock, flags);
|
|
|
|
__release_stripe(conf, sh);
|
|
|
|
spin_unlock_irqrestore(&conf->device_lock, flags);
|
|
|
|
}
|
|
|
|
|
2006-01-06 09:20:33 +01:00
|
|
|
static inline void remove_hash(struct stripe_head *sh)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("remove_hash(), stripe %llu\n",
|
|
|
|
(unsigned long long)sh->sector);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2006-01-06 09:20:33 +01:00
|
|
|
hlist_del_init(&sh->hash);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
2006-06-26 09:27:38 +02:00
|
|
|
static inline void insert_hash(raid5_conf_t *conf, struct stripe_head *sh)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
2006-01-06 09:20:33 +01:00
|
|
|
struct hlist_head *hp = stripe_hash(conf, sh->sector);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("insert_hash(), stripe %llu\n",
|
|
|
|
(unsigned long long)sh->sector);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
CHECK_DEVLOCK();
|
2006-01-06 09:20:33 +01:00
|
|
|
hlist_add_head(&sh->hash, hp);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/* find an idle stripe, make sure it is unhashed, and return it. */
|
|
|
|
static struct stripe_head *get_free_stripe(raid5_conf_t *conf)
|
|
|
|
{
|
|
|
|
struct stripe_head *sh = NULL;
|
|
|
|
struct list_head *first;
|
|
|
|
|
|
|
|
CHECK_DEVLOCK();
|
|
|
|
if (list_empty(&conf->inactive_list))
|
|
|
|
goto out;
|
|
|
|
first = conf->inactive_list.next;
|
|
|
|
sh = list_entry(first, struct stripe_head, lru);
|
|
|
|
list_del_init(first);
|
|
|
|
remove_hash(sh);
|
|
|
|
atomic_inc(&conf->active_stripes);
|
|
|
|
out:
|
|
|
|
return sh;
|
|
|
|
}
|
|
|
|
|
2010-06-16 08:45:16 +02:00
|
|
|
static void shrink_buffers(struct stripe_head *sh)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
struct page *p;
|
|
|
|
int i;
|
2010-06-16 08:45:16 +02:00
|
|
|
int num = sh->raid_conf->pool_size;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2010-06-16 08:45:16 +02:00
|
|
|
for (i = 0; i < num ; i++) {
|
2005-04-17 00:20:36 +02:00
|
|
|
p = sh->dev[i].page;
|
|
|
|
if (!p)
|
|
|
|
continue;
|
|
|
|
sh->dev[i].page = NULL;
|
2006-01-06 09:20:31 +01:00
|
|
|
put_page(p);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2010-06-16 08:45:16 +02:00
|
|
|
static int grow_buffers(struct stripe_head *sh)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
int i;
|
2010-06-16 08:45:16 +02:00
|
|
|
int num = sh->raid_conf->pool_size;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2010-06-16 08:45:16 +02:00
|
|
|
for (i = 0; i < num; i++) {
|
2005-04-17 00:20:36 +02:00
|
|
|
struct page *page;
|
|
|
|
|
|
|
|
if (!(page = alloc_page(GFP_KERNEL))) {
|
|
|
|
return 1;
|
|
|
|
}
|
|
|
|
sh->dev[i].page = page;
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2009-03-31 06:19:07 +02:00
|
|
|
static void raid5_build_block(struct stripe_head *sh, int i, int previous);
|
2009-03-31 05:39:38 +02:00
|
|
|
static void stripe_set_idx(sector_t stripe, raid5_conf_t *conf, int previous,
|
|
|
|
struct stripe_head *sh);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2009-03-31 05:39:38 +02:00
|
|
|
static void init_stripe(struct stripe_head *sh, sector_t sector, int previous)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
raid5_conf_t *conf = sh->raid_conf;
|
2006-03-27 11:18:08 +02:00
|
|
|
int i;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2006-04-02 13:31:42 +02:00
|
|
|
BUG_ON(atomic_read(&sh->count) != 0);
|
|
|
|
BUG_ON(test_bit(STRIPE_HANDLE, &sh->state));
|
md: replace STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} with 'reconstruct_states'
From: Dan Williams <dan.j.williams@intel.com>
Track the state of reconstruct operations (recalculating the parity block
usually due to incoming writes, or as part of array expansion) Reduces the
scope of the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags to only tracking whether
a reconstruct operation has been requested via the ops_request field of struct
stripe_head_state.
This is the final step in the removal of ops.{pending,ack,complete,count}, i.e.
the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags only request an operation and do
not track the state of the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:32:05 +02:00
|
|
|
BUG_ON(stripe_operations_active(sh));
|
2007-01-02 21:52:30 +01:00
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
CHECK_DEVLOCK();
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("init_stripe called, stripe %llu\n",
|
2005-04-17 00:20:36 +02:00
|
|
|
(unsigned long long)sh->sector);
|
|
|
|
|
|
|
|
remove_hash(sh);
|
2006-06-26 09:27:38 +02:00
|
|
|
|
2009-03-31 06:19:03 +02:00
|
|
|
sh->generation = conf->generation - previous;
|
2009-03-31 05:39:38 +02:00
|
|
|
sh->disks = previous ? conf->previous_raid_disks : conf->raid_disks;
|
2005-04-17 00:20:36 +02:00
|
|
|
sh->sector = sector;
|
2009-03-31 05:39:38 +02:00
|
|
|
stripe_set_idx(sector, conf, previous, sh);
|
2005-04-17 00:20:36 +02:00
|
|
|
sh->state = 0;
|
|
|
|
|
2006-03-27 11:18:08 +02:00
|
|
|
|
|
|
|
for (i = sh->disks; i--; ) {
|
2005-04-17 00:20:36 +02:00
|
|
|
struct r5dev *dev = &sh->dev[i];
|
|
|
|
|
2007-01-02 21:52:30 +01:00
|
|
|
if (dev->toread || dev->read || dev->towrite || dev->written ||
|
2005-04-17 00:20:36 +02:00
|
|
|
test_bit(R5_LOCKED, &dev->flags)) {
|
2007-01-02 21:52:30 +01:00
|
|
|
printk(KERN_ERR "sector=%llx i=%d %p %p %p %p %d\n",
|
2005-04-17 00:20:36 +02:00
|
|
|
(unsigned long long)sh->sector, i, dev->toread,
|
2007-01-02 21:52:30 +01:00
|
|
|
dev->read, dev->towrite, dev->written,
|
2005-04-17 00:20:36 +02:00
|
|
|
test_bit(R5_LOCKED, &dev->flags));
|
|
|
|
BUG();
|
|
|
|
}
|
|
|
|
dev->flags = 0;
|
2009-03-31 06:19:07 +02:00
|
|
|
raid5_build_block(sh, i, previous);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
insert_hash(conf, sh);
|
|
|
|
}
|
|
|
|
|
2009-03-31 06:19:03 +02:00
|
|
|
static struct stripe_head *__find_stripe(raid5_conf_t *conf, sector_t sector,
|
|
|
|
short generation)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
struct stripe_head *sh;
|
2006-01-06 09:20:33 +01:00
|
|
|
struct hlist_node *hn;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
CHECK_DEVLOCK();
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("__find_stripe, sector %llu\n", (unsigned long long)sector);
|
2006-01-06 09:20:33 +01:00
|
|
|
hlist_for_each_entry(sh, hn, stripe_hash(conf, sector), hash)
|
2009-03-31 06:19:03 +02:00
|
|
|
if (sh->sector == sector && sh->generation == generation)
|
2005-04-17 00:20:36 +02:00
|
|
|
return sh;
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("__stripe %llu not in cache\n", (unsigned long long)sector);
|
2005-04-17 00:20:36 +02:00
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
2010-06-16 09:17:53 +02:00
|
|
|
/*
|
|
|
|
* Need to check if array has failed when deciding whether to:
|
|
|
|
* - start an array
|
|
|
|
* - remove non-faulty devices
|
|
|
|
* - add a spare
|
|
|
|
* - allow a reshape
|
|
|
|
* This determination is simple when no reshape is happening.
|
|
|
|
* However if there is a reshape, we need to carefully check
|
|
|
|
* both the before and after sections.
|
|
|
|
* This is because some failed devices may only affect one
|
|
|
|
* of the two sections, and some non-in_sync devices may
|
|
|
|
* be insync in the section most affected by failed devices.
|
|
|
|
*/
|
|
|
|
static int has_failed(raid5_conf_t *conf)
|
|
|
|
{
|
|
|
|
int degraded;
|
|
|
|
int i;
|
|
|
|
if (conf->mddev->reshape_position == MaxSector)
|
|
|
|
return conf->mddev->degraded > conf->max_degraded;
|
|
|
|
|
|
|
|
rcu_read_lock();
|
|
|
|
degraded = 0;
|
|
|
|
for (i = 0; i < conf->previous_raid_disks; i++) {
|
|
|
|
mdk_rdev_t *rdev = rcu_dereference(conf->disks[i].rdev);
|
|
|
|
if (!rdev || test_bit(Faulty, &rdev->flags))
|
|
|
|
degraded++;
|
|
|
|
else if (test_bit(In_sync, &rdev->flags))
|
|
|
|
;
|
|
|
|
else
|
|
|
|
/* not in-sync or faulty.
|
|
|
|
* If the reshape increases the number of devices,
|
|
|
|
* this is being recovered by the reshape, so
|
|
|
|
* this 'previous' section is not in_sync.
|
|
|
|
* If the number of devices is being reduced however,
|
|
|
|
* the device can only be part of the array if
|
|
|
|
* we are reverting a reshape, so this section will
|
|
|
|
* be in-sync.
|
|
|
|
*/
|
|
|
|
if (conf->raid_disks >= conf->previous_raid_disks)
|
|
|
|
degraded++;
|
|
|
|
}
|
|
|
|
rcu_read_unlock();
|
|
|
|
if (degraded > conf->max_degraded)
|
|
|
|
return 1;
|
|
|
|
rcu_read_lock();
|
|
|
|
degraded = 0;
|
|
|
|
for (i = 0; i < conf->raid_disks; i++) {
|
|
|
|
mdk_rdev_t *rdev = rcu_dereference(conf->disks[i].rdev);
|
|
|
|
if (!rdev || test_bit(Faulty, &rdev->flags))
|
|
|
|
degraded++;
|
|
|
|
else if (test_bit(In_sync, &rdev->flags))
|
|
|
|
;
|
|
|
|
else
|
|
|
|
/* not in-sync or faulty.
|
|
|
|
* If reshape increases the number of devices, this
|
|
|
|
* section has already been recovered, else it
|
|
|
|
* almost certainly hasn't.
|
|
|
|
*/
|
|
|
|
if (conf->raid_disks <= conf->previous_raid_disks)
|
|
|
|
degraded++;
|
|
|
|
}
|
|
|
|
rcu_read_unlock();
|
|
|
|
if (degraded > conf->max_degraded)
|
|
|
|
return 1;
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2009-03-31 05:39:38 +02:00
|
|
|
static struct stripe_head *
|
|
|
|
get_active_stripe(raid5_conf_t *conf, sector_t sector,
|
2009-06-09 06:39:59 +02:00
|
|
|
int previous, int noblock, int noquiesce)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
struct stripe_head *sh;
|
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("get_stripe, sector %llu\n", (unsigned long long)sector);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
|
|
|
|
|
|
|
do {
|
2005-09-10 01:23:54 +02:00
|
|
|
wait_event_lock_irq(conf->wait_for_stripe,
|
2009-06-09 06:39:59 +02:00
|
|
|
conf->quiesce == 0 || noquiesce,
|
2005-09-10 01:23:54 +02:00
|
|
|
conf->device_lock, /* nothing */);
|
2009-03-31 06:19:03 +02:00
|
|
|
sh = __find_stripe(conf, sector, conf->generation - previous);
|
2005-04-17 00:20:36 +02:00
|
|
|
if (!sh) {
|
|
|
|
if (!conf->inactive_blocked)
|
|
|
|
sh = get_free_stripe(conf);
|
|
|
|
if (noblock && sh == NULL)
|
|
|
|
break;
|
|
|
|
if (!sh) {
|
|
|
|
conf->inactive_blocked = 1;
|
|
|
|
wait_event_lock_irq(conf->wait_for_stripe,
|
|
|
|
!list_empty(&conf->inactive_list) &&
|
2005-12-12 11:39:17 +01:00
|
|
|
(atomic_read(&conf->active_stripes)
|
|
|
|
< (conf->max_nr_stripes *3/4)
|
2005-04-17 00:20:36 +02:00
|
|
|
|| !conf->inactive_blocked),
|
|
|
|
conf->device_lock,
|
2011-04-18 10:25:43 +02:00
|
|
|
);
|
2005-04-17 00:20:36 +02:00
|
|
|
conf->inactive_blocked = 0;
|
|
|
|
} else
|
2009-03-31 05:39:38 +02:00
|
|
|
init_stripe(sh, sector, previous);
|
2005-04-17 00:20:36 +02:00
|
|
|
} else {
|
|
|
|
if (atomic_read(&sh->count)) {
|
2009-03-31 06:26:47 +02:00
|
|
|
BUG_ON(!list_empty(&sh->lru)
|
|
|
|
&& !test_bit(STRIPE_EXPANDING, &sh->state));
|
2005-04-17 00:20:36 +02:00
|
|
|
} else {
|
|
|
|
if (!test_bit(STRIPE_HANDLE, &sh->state))
|
|
|
|
atomic_inc(&conf->active_stripes);
|
2006-07-10 13:44:16 +02:00
|
|
|
if (list_empty(&sh->lru) &&
|
|
|
|
!test_bit(STRIPE_EXPANDING, &sh->state))
|
2006-06-26 09:27:38 +02:00
|
|
|
BUG();
|
|
|
|
list_del_init(&sh->lru);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
} while (sh == NULL);
|
|
|
|
|
|
|
|
if (sh)
|
|
|
|
atomic_inc(&sh->count);
|
|
|
|
|
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
return sh;
|
|
|
|
}
|
|
|
|
|
2007-09-27 12:47:43 +02:00
|
|
|
static void
|
|
|
|
raid5_end_read_request(struct bio *bi, int error);
|
|
|
|
static void
|
|
|
|
raid5_end_write_request(struct bio *bi, int error);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
2008-06-28 00:31:53 +02:00
|
|
|
static void ops_run_io(struct stripe_head *sh, struct stripe_head_state *s)
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
{
|
|
|
|
raid5_conf_t *conf = sh->raid_conf;
|
|
|
|
int i, disks = sh->disks;
|
|
|
|
|
|
|
|
might_sleep();
|
|
|
|
|
|
|
|
for (i = disks; i--; ) {
|
|
|
|
int rw;
|
|
|
|
struct bio *bi;
|
|
|
|
mdk_rdev_t *rdev;
|
2010-09-03 11:56:18 +02:00
|
|
|
if (test_and_clear_bit(R5_Wantwrite, &sh->dev[i].flags)) {
|
|
|
|
if (test_and_clear_bit(R5_WantFUA, &sh->dev[i].flags))
|
|
|
|
rw = WRITE_FUA;
|
|
|
|
else
|
|
|
|
rw = WRITE;
|
|
|
|
} else if (test_and_clear_bit(R5_Wantread, &sh->dev[i].flags))
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
rw = READ;
|
|
|
|
else
|
|
|
|
continue;
|
|
|
|
|
|
|
|
bi = &sh->dev[i].req;
|
|
|
|
|
|
|
|
bi->bi_rw = rw;
|
2011-06-14 06:20:19 +02:00
|
|
|
if (rw & WRITE)
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
bi->bi_end_io = raid5_end_write_request;
|
|
|
|
else
|
|
|
|
bi->bi_end_io = raid5_end_read_request;
|
|
|
|
|
|
|
|
rcu_read_lock();
|
|
|
|
rdev = rcu_dereference(conf->disks[i].rdev);
|
|
|
|
if (rdev && test_bit(Faulty, &rdev->flags))
|
|
|
|
rdev = NULL;
|
|
|
|
if (rdev)
|
|
|
|
atomic_inc(&rdev->nr_pending);
|
|
|
|
rcu_read_unlock();
|
|
|
|
|
|
|
|
if (rdev) {
|
2008-06-28 00:31:53 +02:00
|
|
|
if (s->syncing || s->expanding || s->expanded)
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
md_sync_acct(rdev->bdev, STRIPE_SECTORS);
|
|
|
|
|
2008-06-28 00:31:52 +02:00
|
|
|
set_bit(STRIPE_IO_STARTED, &sh->state);
|
|
|
|
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
bi->bi_bdev = rdev->bdev;
|
|
|
|
pr_debug("%s: for %llu schedule op %ld on disc %d\n",
|
2008-04-28 11:15:50 +02:00
|
|
|
__func__, (unsigned long long)sh->sector,
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
bi->bi_rw, i);
|
|
|
|
atomic_inc(&sh->count);
|
|
|
|
bi->bi_sector = sh->sector + rdev->data_offset;
|
|
|
|
bi->bi_flags = 1 << BIO_UPTODATE;
|
|
|
|
bi->bi_vcnt = 1;
|
|
|
|
bi->bi_max_vecs = 1;
|
|
|
|
bi->bi_idx = 0;
|
|
|
|
bi->bi_io_vec = &sh->dev[i].vec;
|
|
|
|
bi->bi_io_vec[0].bv_len = STRIPE_SIZE;
|
|
|
|
bi->bi_io_vec[0].bv_offset = 0;
|
|
|
|
bi->bi_size = STRIPE_SIZE;
|
|
|
|
bi->bi_next = NULL;
|
2011-06-14 06:20:19 +02:00
|
|
|
if ((rw & WRITE) &&
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
test_bit(R5_ReWrite, &sh->dev[i].flags))
|
|
|
|
atomic_add(STRIPE_SECTORS,
|
|
|
|
&rdev->corrected_errors);
|
|
|
|
generic_make_request(bi);
|
|
|
|
} else {
|
2011-06-14 06:20:19 +02:00
|
|
|
if (rw & WRITE)
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
set_bit(STRIPE_DEGRADED, &sh->state);
|
|
|
|
pr_debug("skip op %ld on disc %d for sector %llu\n",
|
|
|
|
bi->bi_rw, i, (unsigned long long)sh->sector);
|
|
|
|
clear_bit(R5_LOCKED, &sh->dev[i].flags);
|
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
static struct dma_async_tx_descriptor *
|
|
|
|
async_copy_data(int frombio, struct bio *bio, struct page *page,
|
|
|
|
sector_t sector, struct dma_async_tx_descriptor *tx)
|
|
|
|
{
|
|
|
|
struct bio_vec *bvl;
|
|
|
|
struct page *bio_page;
|
|
|
|
int i;
|
|
|
|
int page_offset;
|
2009-06-03 20:43:59 +02:00
|
|
|
struct async_submit_ctl submit;
|
2009-09-09 02:42:50 +02:00
|
|
|
enum async_tx_flags flags = 0;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
if (bio->bi_sector >= sector)
|
|
|
|
page_offset = (signed)(bio->bi_sector - sector) * 512;
|
|
|
|
else
|
|
|
|
page_offset = (signed)(sector - bio->bi_sector) * -512;
|
2009-06-03 20:43:59 +02:00
|
|
|
|
2009-09-09 02:42:50 +02:00
|
|
|
if (frombio)
|
|
|
|
flags |= ASYNC_TX_FENCE;
|
|
|
|
init_async_submit(&submit, flags, tx, NULL, NULL, NULL);
|
|
|
|
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
bio_for_each_segment(bvl, bio, i) {
|
2011-06-14 06:23:57 +02:00
|
|
|
int len = bvl->bv_len;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
int clen;
|
|
|
|
int b_offset = 0;
|
|
|
|
|
|
|
|
if (page_offset < 0) {
|
|
|
|
b_offset = -page_offset;
|
|
|
|
page_offset += b_offset;
|
|
|
|
len -= b_offset;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (len > 0 && page_offset + len > STRIPE_SIZE)
|
|
|
|
clen = STRIPE_SIZE - page_offset;
|
|
|
|
else
|
|
|
|
clen = len;
|
|
|
|
|
|
|
|
if (clen > 0) {
|
2011-06-14 06:23:57 +02:00
|
|
|
b_offset += bvl->bv_offset;
|
|
|
|
bio_page = bvl->bv_page;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
if (frombio)
|
|
|
|
tx = async_memcpy(page, bio_page, page_offset,
|
2009-06-03 20:43:59 +02:00
|
|
|
b_offset, clen, &submit);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
else
|
|
|
|
tx = async_memcpy(bio_page, page, b_offset,
|
2009-06-03 20:43:59 +02:00
|
|
|
page_offset, clen, &submit);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
}
|
2009-06-03 20:43:59 +02:00
|
|
|
/* chain the operations */
|
|
|
|
submit.depend_tx = tx;
|
|
|
|
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
if (clen < len) /* hit end of page */
|
|
|
|
break;
|
|
|
|
page_offset += len;
|
|
|
|
}
|
|
|
|
|
|
|
|
return tx;
|
|
|
|
}
|
|
|
|
|
|
|
|
static void ops_complete_biofill(void *stripe_head_ref)
|
|
|
|
{
|
|
|
|
struct stripe_head *sh = stripe_head_ref;
|
|
|
|
struct bio *return_bi = NULL;
|
|
|
|
raid5_conf_t *conf = sh->raid_conf;
|
2007-09-24 19:06:13 +02:00
|
|
|
int i;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
2008-04-28 11:15:50 +02:00
|
|
|
pr_debug("%s: stripe %llu\n", __func__,
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
(unsigned long long)sh->sector);
|
|
|
|
|
|
|
|
/* clear completed biofills */
|
2008-06-28 00:31:58 +02:00
|
|
|
spin_lock_irq(&conf->device_lock);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
for (i = sh->disks; i--; ) {
|
|
|
|
struct r5dev *dev = &sh->dev[i];
|
|
|
|
|
|
|
|
/* acknowledge completion of a biofill operation */
|
2007-09-24 19:06:13 +02:00
|
|
|
/* and check if we need to reply to a read request,
|
|
|
|
* new R5_Wantfill requests are held off until
|
2008-06-28 00:31:58 +02:00
|
|
|
* !STRIPE_BIOFILL_RUN
|
2007-09-24 19:06:13 +02:00
|
|
|
*/
|
|
|
|
if (test_and_clear_bit(R5_Wantfill, &dev->flags)) {
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
struct bio *rbi, *rbi2;
|
|
|
|
|
|
|
|
BUG_ON(!dev->read);
|
|
|
|
rbi = dev->read;
|
|
|
|
dev->read = NULL;
|
|
|
|
while (rbi && rbi->bi_sector <
|
|
|
|
dev->sector + STRIPE_SECTORS) {
|
|
|
|
rbi2 = r5_next_bio(rbi, dev->sector);
|
2008-08-15 10:41:18 +02:00
|
|
|
if (!raid5_dec_bi_phys_segments(rbi)) {
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
rbi->bi_next = return_bi;
|
|
|
|
return_bi = rbi;
|
|
|
|
}
|
|
|
|
rbi = rbi2;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
2008-06-28 00:31:58 +02:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
clear_bit(STRIPE_BIOFILL_RUN, &sh->state);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
return_io(return_bi);
|
|
|
|
|
2007-09-24 19:06:13 +02:00
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
release_stripe(sh);
|
|
|
|
}
|
|
|
|
|
|
|
|
static void ops_run_biofill(struct stripe_head *sh)
|
|
|
|
{
|
|
|
|
struct dma_async_tx_descriptor *tx = NULL;
|
|
|
|
raid5_conf_t *conf = sh->raid_conf;
|
2009-06-03 20:43:59 +02:00
|
|
|
struct async_submit_ctl submit;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
int i;
|
|
|
|
|
2008-04-28 11:15:50 +02:00
|
|
|
pr_debug("%s: stripe %llu\n", __func__,
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
(unsigned long long)sh->sector);
|
|
|
|
|
|
|
|
for (i = sh->disks; i--; ) {
|
|
|
|
struct r5dev *dev = &sh->dev[i];
|
|
|
|
if (test_bit(R5_Wantfill, &dev->flags)) {
|
|
|
|
struct bio *rbi;
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
|
|
|
dev->read = rbi = dev->toread;
|
|
|
|
dev->toread = NULL;
|
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
while (rbi && rbi->bi_sector <
|
|
|
|
dev->sector + STRIPE_SECTORS) {
|
|
|
|
tx = async_copy_data(0, rbi, dev->page,
|
|
|
|
dev->sector, tx);
|
|
|
|
rbi = r5_next_bio(rbi, dev->sector);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
atomic_inc(&sh->count);
|
2009-06-03 20:43:59 +02:00
|
|
|
init_async_submit(&submit, ASYNC_TX_ACK, tx, ops_complete_biofill, sh, NULL);
|
|
|
|
async_trigger_callback(&submit);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
}
|
|
|
|
|
2009-08-30 04:13:11 +02:00
|
|
|
static void mark_target_uptodate(struct stripe_head *sh, int target)
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
{
|
2009-08-30 04:13:11 +02:00
|
|
|
struct r5dev *tgt;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
2009-08-30 04:13:11 +02:00
|
|
|
if (target < 0)
|
|
|
|
return;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
2009-08-30 04:13:11 +02:00
|
|
|
tgt = &sh->dev[target];
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
set_bit(R5_UPTODATE, &tgt->flags);
|
|
|
|
BUG_ON(!test_bit(R5_Wantcompute, &tgt->flags));
|
|
|
|
clear_bit(R5_Wantcompute, &tgt->flags);
|
2009-08-30 04:13:11 +02:00
|
|
|
}
|
|
|
|
|
2009-07-14 22:40:19 +02:00
|
|
|
static void ops_complete_compute(void *stripe_head_ref)
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
{
|
|
|
|
struct stripe_head *sh = stripe_head_ref;
|
|
|
|
|
2008-04-28 11:15:50 +02:00
|
|
|
pr_debug("%s: stripe %llu\n", __func__,
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
(unsigned long long)sh->sector);
|
|
|
|
|
2009-07-14 22:40:19 +02:00
|
|
|
/* mark the computed target(s) as uptodate */
|
2009-08-30 04:13:11 +02:00
|
|
|
mark_target_uptodate(sh, sh->ops.target);
|
2009-07-14 22:40:19 +02:00
|
|
|
mark_target_uptodate(sh, sh->ops.target2);
|
2009-08-30 04:13:11 +02:00
|
|
|
|
2008-06-28 00:31:57 +02:00
|
|
|
clear_bit(STRIPE_COMPUTE_RUN, &sh->state);
|
|
|
|
if (sh->check_state == check_state_compute_run)
|
|
|
|
sh->check_state = check_state_compute_result;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
release_stripe(sh);
|
|
|
|
}
|
|
|
|
|
2009-07-14 20:50:52 +02:00
|
|
|
/* return a pointer to the address conversion region of the scribble buffer */
|
|
|
|
static addr_conv_t *to_addr_conv(struct stripe_head *sh,
|
|
|
|
struct raid5_percpu *percpu)
|
|
|
|
{
|
|
|
|
return percpu->scribble + sizeof(struct page *) * (sh->disks + 2);
|
|
|
|
}
|
|
|
|
|
|
|
|
static struct dma_async_tx_descriptor *
|
|
|
|
ops_run_compute5(struct stripe_head *sh, struct raid5_percpu *percpu)
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
{
|
|
|
|
int disks = sh->disks;
|
2009-07-14 20:50:52 +02:00
|
|
|
struct page **xor_srcs = percpu->scribble;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
int target = sh->ops.target;
|
|
|
|
struct r5dev *tgt = &sh->dev[target];
|
|
|
|
struct page *xor_dest = tgt->page;
|
|
|
|
int count = 0;
|
|
|
|
struct dma_async_tx_descriptor *tx;
|
2009-06-03 20:43:59 +02:00
|
|
|
struct async_submit_ctl submit;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
int i;
|
|
|
|
|
|
|
|
pr_debug("%s: stripe %llu block: %d\n",
|
2008-04-28 11:15:50 +02:00
|
|
|
__func__, (unsigned long long)sh->sector, target);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
BUG_ON(!test_bit(R5_Wantcompute, &tgt->flags));
|
|
|
|
|
|
|
|
for (i = disks; i--; )
|
|
|
|
if (i != target)
|
|
|
|
xor_srcs[count++] = sh->dev[i].page;
|
|
|
|
|
|
|
|
atomic_inc(&sh->count);
|
|
|
|
|
2009-09-09 02:42:50 +02:00
|
|
|
init_async_submit(&submit, ASYNC_TX_FENCE|ASYNC_TX_XOR_ZERO_DST, NULL,
|
2009-07-14 22:40:19 +02:00
|
|
|
ops_complete_compute, sh, to_addr_conv(sh, percpu));
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
if (unlikely(count == 1))
|
2009-06-03 20:43:59 +02:00
|
|
|
tx = async_memcpy(xor_dest, xor_srcs[0], 0, 0, STRIPE_SIZE, &submit);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
else
|
2009-06-03 20:43:59 +02:00
|
|
|
tx = async_xor(xor_dest, xor_srcs, 0, count, STRIPE_SIZE, &submit);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
return tx;
|
|
|
|
}
|
|
|
|
|
2009-07-14 22:40:19 +02:00
|
|
|
/* set_syndrome_sources - populate source buffers for gen_syndrome
|
|
|
|
* @srcs - (struct page *) array of size sh->disks
|
|
|
|
* @sh - stripe_head to parse
|
|
|
|
*
|
|
|
|
* Populates srcs in proper layout order for the stripe and returns the
|
|
|
|
* 'count' of sources to be used in a call to async_gen_syndrome. The P
|
|
|
|
* destination buffer is recorded in srcs[count] and the Q destination
|
|
|
|
* is recorded in srcs[count+1]].
|
|
|
|
*/
|
|
|
|
static int set_syndrome_sources(struct page **srcs, struct stripe_head *sh)
|
|
|
|
{
|
|
|
|
int disks = sh->disks;
|
|
|
|
int syndrome_disks = sh->ddf_layout ? disks : (disks - 2);
|
|
|
|
int d0_idx = raid6_d0(sh);
|
|
|
|
int count;
|
|
|
|
int i;
|
|
|
|
|
|
|
|
for (i = 0; i < disks; i++)
|
2009-10-16 07:40:25 +02:00
|
|
|
srcs[i] = NULL;
|
2009-07-14 22:40:19 +02:00
|
|
|
|
|
|
|
count = 0;
|
|
|
|
i = d0_idx;
|
|
|
|
do {
|
|
|
|
int slot = raid6_idx_to_slot(i, sh, &count, syndrome_disks);
|
|
|
|
|
|
|
|
srcs[slot] = sh->dev[i].page;
|
|
|
|
i = raid6_next_disk(i, disks);
|
|
|
|
} while (i != d0_idx);
|
|
|
|
|
2009-10-16 07:27:34 +02:00
|
|
|
return syndrome_disks;
|
2009-07-14 22:40:19 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
static struct dma_async_tx_descriptor *
|
|
|
|
ops_run_compute6_1(struct stripe_head *sh, struct raid5_percpu *percpu)
|
|
|
|
{
|
|
|
|
int disks = sh->disks;
|
|
|
|
struct page **blocks = percpu->scribble;
|
|
|
|
int target;
|
|
|
|
int qd_idx = sh->qd_idx;
|
|
|
|
struct dma_async_tx_descriptor *tx;
|
|
|
|
struct async_submit_ctl submit;
|
|
|
|
struct r5dev *tgt;
|
|
|
|
struct page *dest;
|
|
|
|
int i;
|
|
|
|
int count;
|
|
|
|
|
|
|
|
if (sh->ops.target < 0)
|
|
|
|
target = sh->ops.target2;
|
|
|
|
else if (sh->ops.target2 < 0)
|
|
|
|
target = sh->ops.target;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
else
|
2009-07-14 22:40:19 +02:00
|
|
|
/* we should only have one valid target */
|
|
|
|
BUG();
|
|
|
|
BUG_ON(target < 0);
|
|
|
|
pr_debug("%s: stripe %llu block: %d\n",
|
|
|
|
__func__, (unsigned long long)sh->sector, target);
|
|
|
|
|
|
|
|
tgt = &sh->dev[target];
|
|
|
|
BUG_ON(!test_bit(R5_Wantcompute, &tgt->flags));
|
|
|
|
dest = tgt->page;
|
|
|
|
|
|
|
|
atomic_inc(&sh->count);
|
|
|
|
|
|
|
|
if (target == qd_idx) {
|
|
|
|
count = set_syndrome_sources(blocks, sh);
|
|
|
|
blocks[count] = NULL; /* regenerating p is not necessary */
|
|
|
|
BUG_ON(blocks[count+1] != dest); /* q should already be set */
|
2009-09-09 02:42:50 +02:00
|
|
|
init_async_submit(&submit, ASYNC_TX_FENCE, NULL,
|
|
|
|
ops_complete_compute, sh,
|
2009-07-14 22:40:19 +02:00
|
|
|
to_addr_conv(sh, percpu));
|
|
|
|
tx = async_gen_syndrome(blocks, 0, count+2, STRIPE_SIZE, &submit);
|
|
|
|
} else {
|
|
|
|
/* Compute any data- or p-drive using XOR */
|
|
|
|
count = 0;
|
|
|
|
for (i = disks; i-- ; ) {
|
|
|
|
if (i == target || i == qd_idx)
|
|
|
|
continue;
|
|
|
|
blocks[count++] = sh->dev[i].page;
|
|
|
|
}
|
|
|
|
|
2009-09-09 02:42:50 +02:00
|
|
|
init_async_submit(&submit, ASYNC_TX_FENCE|ASYNC_TX_XOR_ZERO_DST,
|
|
|
|
NULL, ops_complete_compute, sh,
|
2009-07-14 22:40:19 +02:00
|
|
|
to_addr_conv(sh, percpu));
|
|
|
|
tx = async_xor(dest, blocks, 0, count, STRIPE_SIZE, &submit);
|
|
|
|
}
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
return tx;
|
|
|
|
}
|
|
|
|
|
2009-07-14 22:40:19 +02:00
|
|
|
static struct dma_async_tx_descriptor *
|
|
|
|
ops_run_compute6_2(struct stripe_head *sh, struct raid5_percpu *percpu)
|
|
|
|
{
|
|
|
|
int i, count, disks = sh->disks;
|
|
|
|
int syndrome_disks = sh->ddf_layout ? disks : disks-2;
|
|
|
|
int d0_idx = raid6_d0(sh);
|
|
|
|
int faila = -1, failb = -1;
|
|
|
|
int target = sh->ops.target;
|
|
|
|
int target2 = sh->ops.target2;
|
|
|
|
struct r5dev *tgt = &sh->dev[target];
|
|
|
|
struct r5dev *tgt2 = &sh->dev[target2];
|
|
|
|
struct dma_async_tx_descriptor *tx;
|
|
|
|
struct page **blocks = percpu->scribble;
|
|
|
|
struct async_submit_ctl submit;
|
|
|
|
|
|
|
|
pr_debug("%s: stripe %llu block1: %d block2: %d\n",
|
|
|
|
__func__, (unsigned long long)sh->sector, target, target2);
|
|
|
|
BUG_ON(target < 0 || target2 < 0);
|
|
|
|
BUG_ON(!test_bit(R5_Wantcompute, &tgt->flags));
|
|
|
|
BUG_ON(!test_bit(R5_Wantcompute, &tgt2->flags));
|
|
|
|
|
2009-09-16 21:24:54 +02:00
|
|
|
/* we need to open-code set_syndrome_sources to handle the
|
2009-07-14 22:40:19 +02:00
|
|
|
* slot number conversion for 'faila' and 'failb'
|
|
|
|
*/
|
|
|
|
for (i = 0; i < disks ; i++)
|
2009-10-16 07:40:25 +02:00
|
|
|
blocks[i] = NULL;
|
2009-07-14 22:40:19 +02:00
|
|
|
count = 0;
|
|
|
|
i = d0_idx;
|
|
|
|
do {
|
|
|
|
int slot = raid6_idx_to_slot(i, sh, &count, syndrome_disks);
|
|
|
|
|
|
|
|
blocks[slot] = sh->dev[i].page;
|
|
|
|
|
|
|
|
if (i == target)
|
|
|
|
faila = slot;
|
|
|
|
if (i == target2)
|
|
|
|
failb = slot;
|
|
|
|
i = raid6_next_disk(i, disks);
|
|
|
|
} while (i != d0_idx);
|
|
|
|
|
|
|
|
BUG_ON(faila == failb);
|
|
|
|
if (failb < faila)
|
|
|
|
swap(faila, failb);
|
|
|
|
pr_debug("%s: stripe: %llu faila: %d failb: %d\n",
|
|
|
|
__func__, (unsigned long long)sh->sector, faila, failb);
|
|
|
|
|
|
|
|
atomic_inc(&sh->count);
|
|
|
|
|
|
|
|
if (failb == syndrome_disks+1) {
|
|
|
|
/* Q disk is one of the missing disks */
|
|
|
|
if (faila == syndrome_disks) {
|
|
|
|
/* Missing P+Q, just recompute */
|
2009-09-09 02:42:50 +02:00
|
|
|
init_async_submit(&submit, ASYNC_TX_FENCE, NULL,
|
|
|
|
ops_complete_compute, sh,
|
|
|
|
to_addr_conv(sh, percpu));
|
2009-10-16 07:27:34 +02:00
|
|
|
return async_gen_syndrome(blocks, 0, syndrome_disks+2,
|
2009-07-14 22:40:19 +02:00
|
|
|
STRIPE_SIZE, &submit);
|
|
|
|
} else {
|
|
|
|
struct page *dest;
|
|
|
|
int data_target;
|
|
|
|
int qd_idx = sh->qd_idx;
|
|
|
|
|
|
|
|
/* Missing D+Q: recompute D from P, then recompute Q */
|
|
|
|
if (target == qd_idx)
|
|
|
|
data_target = target2;
|
|
|
|
else
|
|
|
|
data_target = target;
|
|
|
|
|
|
|
|
count = 0;
|
|
|
|
for (i = disks; i-- ; ) {
|
|
|
|
if (i == data_target || i == qd_idx)
|
|
|
|
continue;
|
|
|
|
blocks[count++] = sh->dev[i].page;
|
|
|
|
}
|
|
|
|
dest = sh->dev[data_target].page;
|
2009-09-09 02:42:50 +02:00
|
|
|
init_async_submit(&submit,
|
|
|
|
ASYNC_TX_FENCE|ASYNC_TX_XOR_ZERO_DST,
|
|
|
|
NULL, NULL, NULL,
|
|
|
|
to_addr_conv(sh, percpu));
|
2009-07-14 22:40:19 +02:00
|
|
|
tx = async_xor(dest, blocks, 0, count, STRIPE_SIZE,
|
|
|
|
&submit);
|
|
|
|
|
|
|
|
count = set_syndrome_sources(blocks, sh);
|
2009-09-09 02:42:50 +02:00
|
|
|
init_async_submit(&submit, ASYNC_TX_FENCE, tx,
|
|
|
|
ops_complete_compute, sh,
|
|
|
|
to_addr_conv(sh, percpu));
|
2009-07-14 22:40:19 +02:00
|
|
|
return async_gen_syndrome(blocks, 0, count+2,
|
|
|
|
STRIPE_SIZE, &submit);
|
|
|
|
}
|
|
|
|
} else {
|
2009-09-16 21:24:54 +02:00
|
|
|
init_async_submit(&submit, ASYNC_TX_FENCE, NULL,
|
|
|
|
ops_complete_compute, sh,
|
|
|
|
to_addr_conv(sh, percpu));
|
|
|
|
if (failb == syndrome_disks) {
|
|
|
|
/* We're missing D+P. */
|
|
|
|
return async_raid6_datap_recov(syndrome_disks+2,
|
|
|
|
STRIPE_SIZE, faila,
|
|
|
|
blocks, &submit);
|
|
|
|
} else {
|
|
|
|
/* We're missing D+D. */
|
|
|
|
return async_raid6_2data_recov(syndrome_disks+2,
|
|
|
|
STRIPE_SIZE, faila, failb,
|
|
|
|
blocks, &submit);
|
|
|
|
}
|
2009-07-14 22:40:19 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
static void ops_complete_prexor(void *stripe_head_ref)
|
|
|
|
{
|
|
|
|
struct stripe_head *sh = stripe_head_ref;
|
|
|
|
|
2008-04-28 11:15:50 +02:00
|
|
|
pr_debug("%s: stripe %llu\n", __func__,
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
(unsigned long long)sh->sector);
|
|
|
|
}
|
|
|
|
|
|
|
|
static struct dma_async_tx_descriptor *
|
2009-07-14 20:50:52 +02:00
|
|
|
ops_run_prexor(struct stripe_head *sh, struct raid5_percpu *percpu,
|
|
|
|
struct dma_async_tx_descriptor *tx)
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
{
|
|
|
|
int disks = sh->disks;
|
2009-07-14 20:50:52 +02:00
|
|
|
struct page **xor_srcs = percpu->scribble;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
int count = 0, pd_idx = sh->pd_idx, i;
|
2009-06-03 20:43:59 +02:00
|
|
|
struct async_submit_ctl submit;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
/* existing parity data subtracted */
|
|
|
|
struct page *xor_dest = xor_srcs[count++] = sh->dev[pd_idx].page;
|
|
|
|
|
2008-04-28 11:15:50 +02:00
|
|
|
pr_debug("%s: stripe %llu\n", __func__,
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
(unsigned long long)sh->sector);
|
|
|
|
|
|
|
|
for (i = disks; i--; ) {
|
|
|
|
struct r5dev *dev = &sh->dev[i];
|
|
|
|
/* Only process blocks that are known to be uptodate */
|
2008-06-28 00:32:06 +02:00
|
|
|
if (test_bit(R5_Wantdrain, &dev->flags))
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
xor_srcs[count++] = dev->page;
|
|
|
|
}
|
|
|
|
|
2009-09-09 02:42:50 +02:00
|
|
|
init_async_submit(&submit, ASYNC_TX_FENCE|ASYNC_TX_XOR_DROP_DST, tx,
|
2009-07-14 20:50:52 +02:00
|
|
|
ops_complete_prexor, sh, to_addr_conv(sh, percpu));
|
2009-06-03 20:43:59 +02:00
|
|
|
tx = async_xor(xor_dest, xor_srcs, 0, count, STRIPE_SIZE, &submit);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
return tx;
|
|
|
|
}
|
|
|
|
|
|
|
|
static struct dma_async_tx_descriptor *
|
2008-06-28 00:32:06 +02:00
|
|
|
ops_run_biodrain(struct stripe_head *sh, struct dma_async_tx_descriptor *tx)
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
{
|
|
|
|
int disks = sh->disks;
|
2008-06-28 00:32:06 +02:00
|
|
|
int i;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
2008-04-28 11:15:50 +02:00
|
|
|
pr_debug("%s: stripe %llu\n", __func__,
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
(unsigned long long)sh->sector);
|
|
|
|
|
|
|
|
for (i = disks; i--; ) {
|
|
|
|
struct r5dev *dev = &sh->dev[i];
|
|
|
|
struct bio *chosen;
|
|
|
|
|
2008-06-28 00:32:06 +02:00
|
|
|
if (test_and_clear_bit(R5_Wantdrain, &dev->flags)) {
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
struct bio *wbi;
|
|
|
|
|
2011-07-26 03:20:35 +02:00
|
|
|
spin_lock_irq(&sh->raid_conf->device_lock);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
chosen = dev->towrite;
|
|
|
|
dev->towrite = NULL;
|
|
|
|
BUG_ON(dev->written);
|
|
|
|
wbi = dev->written = chosen;
|
2011-07-26 03:20:35 +02:00
|
|
|
spin_unlock_irq(&sh->raid_conf->device_lock);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
while (wbi && wbi->bi_sector <
|
|
|
|
dev->sector + STRIPE_SECTORS) {
|
2010-09-03 11:56:18 +02:00
|
|
|
if (wbi->bi_rw & REQ_FUA)
|
|
|
|
set_bit(R5_WantFUA, &dev->flags);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
tx = async_copy_data(1, wbi, dev->page,
|
|
|
|
dev->sector, tx);
|
|
|
|
wbi = r5_next_bio(wbi, dev->sector);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
return tx;
|
|
|
|
}
|
|
|
|
|
2009-07-14 22:40:19 +02:00
|
|
|
static void ops_complete_reconstruct(void *stripe_head_ref)
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
{
|
|
|
|
struct stripe_head *sh = stripe_head_ref;
|
2009-07-14 22:40:19 +02:00
|
|
|
int disks = sh->disks;
|
|
|
|
int pd_idx = sh->pd_idx;
|
|
|
|
int qd_idx = sh->qd_idx;
|
|
|
|
int i;
|
2010-09-03 11:56:18 +02:00
|
|
|
bool fua = false;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
2008-04-28 11:15:50 +02:00
|
|
|
pr_debug("%s: stripe %llu\n", __func__,
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
(unsigned long long)sh->sector);
|
|
|
|
|
2010-09-03 11:56:18 +02:00
|
|
|
for (i = disks; i--; )
|
|
|
|
fua |= test_bit(R5_WantFUA, &sh->dev[i].flags);
|
|
|
|
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
for (i = disks; i--; ) {
|
|
|
|
struct r5dev *dev = &sh->dev[i];
|
2009-07-14 22:40:19 +02:00
|
|
|
|
2010-09-03 11:56:18 +02:00
|
|
|
if (dev->written || i == pd_idx || i == qd_idx) {
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
set_bit(R5_UPTODATE, &dev->flags);
|
2010-09-03 11:56:18 +02:00
|
|
|
if (fua)
|
|
|
|
set_bit(R5_WantFUA, &dev->flags);
|
|
|
|
}
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
}
|
|
|
|
|
2008-06-28 00:32:06 +02:00
|
|
|
if (sh->reconstruct_state == reconstruct_state_drain_run)
|
|
|
|
sh->reconstruct_state = reconstruct_state_drain_result;
|
|
|
|
else if (sh->reconstruct_state == reconstruct_state_prexor_drain_run)
|
|
|
|
sh->reconstruct_state = reconstruct_state_prexor_drain_result;
|
|
|
|
else {
|
|
|
|
BUG_ON(sh->reconstruct_state != reconstruct_state_run);
|
|
|
|
sh->reconstruct_state = reconstruct_state_result;
|
|
|
|
}
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
release_stripe(sh);
|
|
|
|
}
|
|
|
|
|
|
|
|
static void
|
2009-07-14 22:40:19 +02:00
|
|
|
ops_run_reconstruct5(struct stripe_head *sh, struct raid5_percpu *percpu,
|
|
|
|
struct dma_async_tx_descriptor *tx)
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
{
|
|
|
|
int disks = sh->disks;
|
2009-07-14 20:50:52 +02:00
|
|
|
struct page **xor_srcs = percpu->scribble;
|
2009-06-03 20:43:59 +02:00
|
|
|
struct async_submit_ctl submit;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
int count = 0, pd_idx = sh->pd_idx, i;
|
|
|
|
struct page *xor_dest;
|
2008-06-28 00:32:06 +02:00
|
|
|
int prexor = 0;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
unsigned long flags;
|
|
|
|
|
2008-04-28 11:15:50 +02:00
|
|
|
pr_debug("%s: stripe %llu\n", __func__,
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
(unsigned long long)sh->sector);
|
|
|
|
|
|
|
|
/* check if prexor is active which means only process blocks
|
|
|
|
* that are part of a read-modify-write (written)
|
|
|
|
*/
|
2008-06-28 00:32:06 +02:00
|
|
|
if (sh->reconstruct_state == reconstruct_state_prexor_drain_run) {
|
|
|
|
prexor = 1;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
xor_dest = xor_srcs[count++] = sh->dev[pd_idx].page;
|
|
|
|
for (i = disks; i--; ) {
|
|
|
|
struct r5dev *dev = &sh->dev[i];
|
|
|
|
if (dev->written)
|
|
|
|
xor_srcs[count++] = dev->page;
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
xor_dest = sh->dev[pd_idx].page;
|
|
|
|
for (i = disks; i--; ) {
|
|
|
|
struct r5dev *dev = &sh->dev[i];
|
|
|
|
if (i != pd_idx)
|
|
|
|
xor_srcs[count++] = dev->page;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/* 1/ if we prexor'd then the dest is reused as a source
|
|
|
|
* 2/ if we did not prexor then we are redoing the parity
|
|
|
|
* set ASYNC_TX_XOR_DROP_DST and ASYNC_TX_XOR_ZERO_DST
|
|
|
|
* for the synchronous xor case
|
|
|
|
*/
|
2009-04-10 01:16:18 +02:00
|
|
|
flags = ASYNC_TX_ACK |
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
(prexor ? ASYNC_TX_XOR_DROP_DST : ASYNC_TX_XOR_ZERO_DST);
|
|
|
|
|
|
|
|
atomic_inc(&sh->count);
|
|
|
|
|
2009-07-14 22:40:19 +02:00
|
|
|
init_async_submit(&submit, flags, tx, ops_complete_reconstruct, sh,
|
2009-07-14 20:50:52 +02:00
|
|
|
to_addr_conv(sh, percpu));
|
2009-06-03 20:43:59 +02:00
|
|
|
if (unlikely(count == 1))
|
|
|
|
tx = async_memcpy(xor_dest, xor_srcs[0], 0, 0, STRIPE_SIZE, &submit);
|
|
|
|
else
|
|
|
|
tx = async_xor(xor_dest, xor_srcs, 0, count, STRIPE_SIZE, &submit);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
}
|
|
|
|
|
2009-07-14 22:40:19 +02:00
|
|
|
static void
|
|
|
|
ops_run_reconstruct6(struct stripe_head *sh, struct raid5_percpu *percpu,
|
|
|
|
struct dma_async_tx_descriptor *tx)
|
|
|
|
{
|
|
|
|
struct async_submit_ctl submit;
|
|
|
|
struct page **blocks = percpu->scribble;
|
|
|
|
int count;
|
|
|
|
|
|
|
|
pr_debug("%s: stripe %llu\n", __func__, (unsigned long long)sh->sector);
|
|
|
|
|
|
|
|
count = set_syndrome_sources(blocks, sh);
|
|
|
|
|
|
|
|
atomic_inc(&sh->count);
|
|
|
|
|
|
|
|
init_async_submit(&submit, ASYNC_TX_ACK, tx, ops_complete_reconstruct,
|
|
|
|
sh, to_addr_conv(sh, percpu));
|
|
|
|
async_gen_syndrome(blocks, 0, count+2, STRIPE_SIZE, &submit);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
static void ops_complete_check(void *stripe_head_ref)
|
|
|
|
{
|
|
|
|
struct stripe_head *sh = stripe_head_ref;
|
|
|
|
|
2008-04-28 11:15:50 +02:00
|
|
|
pr_debug("%s: stripe %llu\n", __func__,
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
(unsigned long long)sh->sector);
|
|
|
|
|
2008-06-28 00:31:57 +02:00
|
|
|
sh->check_state = check_state_check_result;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
release_stripe(sh);
|
|
|
|
}
|
|
|
|
|
2009-07-14 22:40:19 +02:00
|
|
|
static void ops_run_check_p(struct stripe_head *sh, struct raid5_percpu *percpu)
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
{
|
|
|
|
int disks = sh->disks;
|
2009-07-14 22:40:19 +02:00
|
|
|
int pd_idx = sh->pd_idx;
|
|
|
|
int qd_idx = sh->qd_idx;
|
|
|
|
struct page *xor_dest;
|
2009-07-14 20:50:52 +02:00
|
|
|
struct page **xor_srcs = percpu->scribble;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
struct dma_async_tx_descriptor *tx;
|
2009-06-03 20:43:59 +02:00
|
|
|
struct async_submit_ctl submit;
|
2009-07-14 22:40:19 +02:00
|
|
|
int count;
|
|
|
|
int i;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
2008-04-28 11:15:50 +02:00
|
|
|
pr_debug("%s: stripe %llu\n", __func__,
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
(unsigned long long)sh->sector);
|
|
|
|
|
2009-07-14 22:40:19 +02:00
|
|
|
count = 0;
|
|
|
|
xor_dest = sh->dev[pd_idx].page;
|
|
|
|
xor_srcs[count++] = xor_dest;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
for (i = disks; i--; ) {
|
2009-07-14 22:40:19 +02:00
|
|
|
if (i == pd_idx || i == qd_idx)
|
|
|
|
continue;
|
|
|
|
xor_srcs[count++] = sh->dev[i].page;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
}
|
|
|
|
|
2009-07-14 20:50:52 +02:00
|
|
|
init_async_submit(&submit, 0, NULL, NULL, NULL,
|
|
|
|
to_addr_conv(sh, percpu));
|
2009-04-08 23:28:37 +02:00
|
|
|
tx = async_xor_val(xor_dest, xor_srcs, 0, count, STRIPE_SIZE,
|
2009-06-03 20:43:59 +02:00
|
|
|
&sh->ops.zero_sum_result, &submit);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
atomic_inc(&sh->count);
|
2009-06-03 20:43:59 +02:00
|
|
|
init_async_submit(&submit, ASYNC_TX_ACK, tx, ops_complete_check, sh, NULL);
|
|
|
|
tx = async_trigger_callback(&submit);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
}
|
|
|
|
|
2009-07-14 22:40:19 +02:00
|
|
|
static void ops_run_check_pq(struct stripe_head *sh, struct raid5_percpu *percpu, int checkp)
|
|
|
|
{
|
|
|
|
struct page **srcs = percpu->scribble;
|
|
|
|
struct async_submit_ctl submit;
|
|
|
|
int count;
|
|
|
|
|
|
|
|
pr_debug("%s: stripe %llu checkp: %d\n", __func__,
|
|
|
|
(unsigned long long)sh->sector, checkp);
|
|
|
|
|
|
|
|
count = set_syndrome_sources(srcs, sh);
|
|
|
|
if (!checkp)
|
|
|
|
srcs[count] = NULL;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
atomic_inc(&sh->count);
|
2009-07-14 22:40:19 +02:00
|
|
|
init_async_submit(&submit, ASYNC_TX_ACK, NULL, ops_complete_check,
|
|
|
|
sh, to_addr_conv(sh, percpu));
|
|
|
|
async_syndrome_val(srcs, 0, count+2, STRIPE_SIZE,
|
|
|
|
&sh->ops.zero_sum_result, percpu->spare_page, &submit);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
}
|
|
|
|
|
2009-10-16 07:25:22 +02:00
|
|
|
static void __raid_run_ops(struct stripe_head *sh, unsigned long ops_request)
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
{
|
|
|
|
int overlap_clear = 0, i, disks = sh->disks;
|
|
|
|
struct dma_async_tx_descriptor *tx = NULL;
|
2009-07-14 20:50:52 +02:00
|
|
|
raid5_conf_t *conf = sh->raid_conf;
|
2009-07-14 22:40:19 +02:00
|
|
|
int level = conf->level;
|
2009-07-14 20:50:52 +02:00
|
|
|
struct raid5_percpu *percpu;
|
|
|
|
unsigned long cpu;
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
2009-07-14 20:50:52 +02:00
|
|
|
cpu = get_cpu();
|
|
|
|
percpu = per_cpu_ptr(conf->percpu, cpu);
|
2008-06-28 00:31:58 +02:00
|
|
|
if (test_bit(STRIPE_OP_BIOFILL, &ops_request)) {
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
ops_run_biofill(sh);
|
|
|
|
overlap_clear++;
|
|
|
|
}
|
|
|
|
|
2008-06-28 00:32:09 +02:00
|
|
|
if (test_bit(STRIPE_OP_COMPUTE_BLK, &ops_request)) {
|
2009-07-14 22:40:19 +02:00
|
|
|
if (level < 6)
|
|
|
|
tx = ops_run_compute5(sh, percpu);
|
|
|
|
else {
|
|
|
|
if (sh->ops.target2 < 0 || sh->ops.target < 0)
|
|
|
|
tx = ops_run_compute6_1(sh, percpu);
|
|
|
|
else
|
|
|
|
tx = ops_run_compute6_2(sh, percpu);
|
|
|
|
}
|
|
|
|
/* terminate the chain if reconstruct is not set to be run */
|
|
|
|
if (tx && !test_bit(STRIPE_OP_RECONSTRUCT, &ops_request))
|
2008-06-28 00:32:09 +02:00
|
|
|
async_tx_ack(tx);
|
|
|
|
}
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
md: replace STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} with 'reconstruct_states'
From: Dan Williams <dan.j.williams@intel.com>
Track the state of reconstruct operations (recalculating the parity block
usually due to incoming writes, or as part of array expansion) Reduces the
scope of the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags to only tracking whether
a reconstruct operation has been requested via the ops_request field of struct
stripe_head_state.
This is the final step in the removal of ops.{pending,ack,complete,count}, i.e.
the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags only request an operation and do
not track the state of the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:32:05 +02:00
|
|
|
if (test_bit(STRIPE_OP_PREXOR, &ops_request))
|
2009-07-14 20:50:52 +02:00
|
|
|
tx = ops_run_prexor(sh, percpu, tx);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
md: replace STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} with 'reconstruct_states'
From: Dan Williams <dan.j.williams@intel.com>
Track the state of reconstruct operations (recalculating the parity block
usually due to incoming writes, or as part of array expansion) Reduces the
scope of the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags to only tracking whether
a reconstruct operation has been requested via the ops_request field of struct
stripe_head_state.
This is the final step in the removal of ops.{pending,ack,complete,count}, i.e.
the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags only request an operation and do
not track the state of the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:32:05 +02:00
|
|
|
if (test_bit(STRIPE_OP_BIODRAIN, &ops_request)) {
|
2008-06-28 00:32:06 +02:00
|
|
|
tx = ops_run_biodrain(sh, tx);
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
overlap_clear++;
|
|
|
|
}
|
|
|
|
|
2009-07-14 22:40:19 +02:00
|
|
|
if (test_bit(STRIPE_OP_RECONSTRUCT, &ops_request)) {
|
|
|
|
if (level < 6)
|
|
|
|
ops_run_reconstruct5(sh, percpu, tx);
|
|
|
|
else
|
|
|
|
ops_run_reconstruct6(sh, percpu, tx);
|
|
|
|
}
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
2009-07-14 22:40:19 +02:00
|
|
|
if (test_bit(STRIPE_OP_CHECK, &ops_request)) {
|
|
|
|
if (sh->check_state == check_state_run)
|
|
|
|
ops_run_check_p(sh, percpu);
|
|
|
|
else if (sh->check_state == check_state_run_q)
|
|
|
|
ops_run_check_pq(sh, percpu, 0);
|
|
|
|
else if (sh->check_state == check_state_run_pq)
|
|
|
|
ops_run_check_pq(sh, percpu, 1);
|
|
|
|
else
|
|
|
|
BUG();
|
|
|
|
}
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
if (overlap_clear)
|
|
|
|
for (i = disks; i--; ) {
|
|
|
|
struct r5dev *dev = &sh->dev[i];
|
|
|
|
if (test_and_clear_bit(R5_Overlap, &dev->flags))
|
|
|
|
wake_up(&sh->raid_conf->wait_for_overlap);
|
|
|
|
}
|
2009-07-14 20:50:52 +02:00
|
|
|
put_cpu();
|
md: raid5_run_ops - run stripe operations outside sh->lock
When the raid acceleration work was proposed, Neil laid out the following
attack plan:
1/ move the xor and copy operations outside spin_lock(&sh->lock)
2/ find/implement an asynchronous offload api
The raid5_run_ops routine uses the asynchronous offload api (async_tx) and
the stripe_operations member of a stripe_head to carry out xor+copy
operations asynchronously, outside the lock.
To perform operations outside the lock a new set of state flags is needed
to track new requests, in-flight requests, and completed requests. In this
new model handle_stripe is tasked with scanning the stripe_head for work,
updating the stripe_operations structure, and finally dropping the lock and
calling raid5_run_ops for processing. The following flags outline the
requests that handle_stripe can make of raid5_run_ops:
STRIPE_OP_BIOFILL
- copy data into request buffers to satisfy a read request
STRIPE_OP_COMPUTE_BLK
- generate a missing block in the cache from the other blocks
STRIPE_OP_PREXOR
- subtract existing data as part of the read-modify-write process
STRIPE_OP_BIODRAIN
- copy data out of request buffers to satisfy a write request
STRIPE_OP_POSTXOR
- recalculate parity for new data that has entered the cache
STRIPE_OP_CHECK
- verify that the parity is correct
STRIPE_OP_IO
- submit i/o to the member disks (note this was already performed outside
the stripe lock, but it made sense to add it as an operation type
The flow is:
1/ handle_stripe sets STRIPE_OP_* in sh->ops.pending
2/ raid5_run_ops reads sh->ops.pending, sets sh->ops.ack, and submits the
operation to the async_tx api
3/ async_tx triggers the completion callback routine to set
sh->ops.complete and release the stripe
4/ handle_stripe runs again to finish the operation and optionally submit
new operations that were previously blocked
Note this patch just defines raid5_run_ops, subsequent commits (one per
major operation type) modify handle_stripe to take advantage of this
routine.
Changelog:
* removed ops_complete_biodrain in favor of ops_complete_postxor and
ops_complete_write.
* removed the raid5_run_ops workqueue
* call bi_end_io for reads in ops_complete_biofill, saves a call to
handle_stripe
* explicitly handle the 2-disk raid5 case (xor becomes memcpy), Neil Brown
* fix race between async engines and bi_end_io call for reads, Neil Brown
* remove unnecessary spin_lock from ops_complete_biofill
* remove test_and_set/test_and_clear BUG_ONs, Neil Brown
* remove explicit interrupt handling for channel switching, this feature
was absorbed (i.e. it is now implicit) by the async_tx api
* use return_io in ops_complete_biofill
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
2007-01-02 21:52:30 +01:00
|
|
|
}
|
|
|
|
|
2009-10-16 07:25:22 +02:00
|
|
|
#ifdef CONFIG_MULTICORE_RAID456
|
|
|
|
static void async_run_ops(void *param, async_cookie_t cookie)
|
|
|
|
{
|
|
|
|
struct stripe_head *sh = param;
|
|
|
|
unsigned long ops_request = sh->ops.request;
|
|
|
|
|
|
|
|
clear_bit_unlock(STRIPE_OPS_REQ_PENDING, &sh->state);
|
|
|
|
wake_up(&sh->ops.wait_for_ops);
|
|
|
|
|
|
|
|
__raid_run_ops(sh, ops_request);
|
|
|
|
release_stripe(sh);
|
|
|
|
}
|
|
|
|
|
|
|
|
static void raid_run_ops(struct stripe_head *sh, unsigned long ops_request)
|
|
|
|
{
|
|
|
|
/* since handle_stripe can be called outside of raid5d context
|
|
|
|
* we need to ensure sh->ops.request is de-staged before another
|
|
|
|
* request arrives
|
|
|
|
*/
|
|
|
|
wait_event(sh->ops.wait_for_ops,
|
|
|
|
!test_and_set_bit_lock(STRIPE_OPS_REQ_PENDING, &sh->state));
|
|
|
|
sh->ops.request = ops_request;
|
|
|
|
|
|
|
|
atomic_inc(&sh->count);
|
|
|
|
async_schedule(async_run_ops, sh);
|
|
|
|
}
|
|
|
|
#else
|
|
|
|
#define raid_run_ops __raid_run_ops
|
|
|
|
#endif
|
|
|
|
|
2005-11-09 06:39:25 +01:00
|
|
|
static int grow_one_stripe(raid5_conf_t *conf)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
struct stripe_head *sh;
|
2011-07-18 09:38:50 +02:00
|
|
|
sh = kmem_cache_zalloc(conf->slab_cache, GFP_KERNEL);
|
2005-11-09 06:39:25 +01:00
|
|
|
if (!sh)
|
|
|
|
return 0;
|
2011-07-18 09:38:50 +02:00
|
|
|
|
2005-11-09 06:39:25 +01:00
|
|
|
sh->raid_conf = conf;
|
2009-10-16 07:25:22 +02:00
|
|
|
#ifdef CONFIG_MULTICORE_RAID456
|
|
|
|
init_waitqueue_head(&sh->ops.wait_for_ops);
|
|
|
|
#endif
|
2005-11-09 06:39:25 +01:00
|
|
|
|
2010-06-16 08:45:16 +02:00
|
|
|
if (grow_buffers(sh)) {
|
|
|
|
shrink_buffers(sh);
|
2005-11-09 06:39:25 +01:00
|
|
|
kmem_cache_free(conf->slab_cache, sh);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
/* we just created an active stripe so... */
|
|
|
|
atomic_set(&sh->count, 1);
|
|
|
|
atomic_inc(&conf->active_stripes);
|
|
|
|
INIT_LIST_HEAD(&sh->lru);
|
|
|
|
release_stripe(sh);
|
|
|
|
return 1;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int grow_stripes(raid5_conf_t *conf, int num)
|
|
|
|
{
|
2006-12-07 05:33:20 +01:00
|
|
|
struct kmem_cache *sc;
|
2009-10-16 07:35:30 +02:00
|
|
|
int devs = max(conf->raid_disks, conf->previous_raid_disks);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2010-06-01 11:37:25 +02:00
|
|
|
if (conf->mddev->gendisk)
|
|
|
|
sprintf(conf->cache_name[0],
|
|
|
|
"raid%d-%s", conf->level, mdname(conf->mddev));
|
|
|
|
else
|
|
|
|
sprintf(conf->cache_name[0],
|
|
|
|
"raid%d-%p", conf->level, conf->mddev);
|
|
|
|
sprintf(conf->cache_name[1], "%s-alt", conf->cache_name[0]);
|
|
|
|
|
2006-03-27 11:18:07 +02:00
|
|
|
conf->active_name = 0;
|
|
|
|
sc = kmem_cache_create(conf->cache_name[conf->active_name],
|
2005-04-17 00:20:36 +02:00
|
|
|
sizeof(struct stripe_head)+(devs-1)*sizeof(struct r5dev),
|
2007-07-20 03:11:58 +02:00
|
|
|
0, 0, NULL);
|
2005-04-17 00:20:36 +02:00
|
|
|
if (!sc)
|
|
|
|
return 1;
|
|
|
|
conf->slab_cache = sc;
|
2006-03-27 11:18:07 +02:00
|
|
|
conf->pool_size = devs;
|
2006-06-26 09:27:38 +02:00
|
|
|
while (num--)
|
2005-11-09 06:39:25 +01:00
|
|
|
if (!grow_one_stripe(conf))
|
2005-04-17 00:20:36 +02:00
|
|
|
return 1;
|
|
|
|
return 0;
|
|
|
|
}
|
2006-03-27 11:18:10 +02:00
|
|
|
|
2009-07-14 20:50:52 +02:00
|
|
|
/**
|
|
|
|
* scribble_len - return the required size of the scribble region
|
|
|
|
* @num - total number of disks in the array
|
|
|
|
*
|
|
|
|
* The size must be enough to contain:
|
|
|
|
* 1/ a struct page pointer for each device in the array +2
|
|
|
|
* 2/ room to convert each entry in (1) to its corresponding dma
|
|
|
|
* (dma_map_page()) or page (page_address()) address.
|
|
|
|
*
|
|
|
|
* Note: the +2 is for the destination buffers of the ddf/raid6 case where we
|
|
|
|
* calculate over all devices (not just the data blocks), using zeros in place
|
|
|
|
* of the P and Q blocks.
|
|
|
|
*/
|
|
|
|
static size_t scribble_len(int num)
|
|
|
|
{
|
|
|
|
size_t len;
|
|
|
|
|
|
|
|
len = sizeof(struct page *) * (num+2) + sizeof(addr_conv_t) * (num+2);
|
|
|
|
|
|
|
|
return len;
|
|
|
|
}
|
|
|
|
|
2006-03-27 11:18:07 +02:00
|
|
|
static int resize_stripes(raid5_conf_t *conf, int newsize)
|
|
|
|
{
|
|
|
|
/* Make all the stripes able to hold 'newsize' devices.
|
|
|
|
* New slots in each stripe get 'page' set to a new page.
|
|
|
|
*
|
|
|
|
* This happens in stages:
|
|
|
|
* 1/ create a new kmem_cache and allocate the required number of
|
|
|
|
* stripe_heads.
|
|
|
|
* 2/ gather all the old stripe_heads and tranfer the pages across
|
|
|
|
* to the new stripe_heads. This will have the side effect of
|
|
|
|
* freezing the array as once all stripe_heads have been collected,
|
|
|
|
* no IO will be possible. Old stripe heads are freed once their
|
|
|
|
* pages have been transferred over, and the old kmem_cache is
|
|
|
|
* freed when all stripes are done.
|
|
|
|
* 3/ reallocate conf->disks to be suitable bigger. If this fails,
|
|
|
|
* we simple return a failre status - no need to clean anything up.
|
|
|
|
* 4/ allocate new pages for the new slots in the new stripe_heads.
|
|
|
|
* If this fails, we don't bother trying the shrink the
|
|
|
|
* stripe_heads down again, we just leave them as they are.
|
|
|
|
* As each stripe_head is processed the new one is released into
|
|
|
|
* active service.
|
|
|
|
*
|
|
|
|
* Once step2 is started, we cannot afford to wait for a write,
|
|
|
|
* so we use GFP_NOIO allocations.
|
|
|
|
*/
|
|
|
|
struct stripe_head *osh, *nsh;
|
|
|
|
LIST_HEAD(newstripes);
|
|
|
|
struct disk_info *ndisks;
|
2009-07-14 20:50:52 +02:00
|
|
|
unsigned long cpu;
|
2008-06-28 06:44:04 +02:00
|
|
|
int err;
|
2006-12-07 05:33:20 +01:00
|
|
|
struct kmem_cache *sc;
|
2006-03-27 11:18:07 +02:00
|
|
|
int i;
|
|
|
|
|
|
|
|
if (newsize <= conf->pool_size)
|
|
|
|
return 0; /* never bother to shrink */
|
|
|
|
|
2008-06-28 06:44:04 +02:00
|
|
|
err = md_allow_write(conf->mddev);
|
|
|
|
if (err)
|
|
|
|
return err;
|
2007-01-26 09:57:11 +01:00
|
|
|
|
2006-03-27 11:18:07 +02:00
|
|
|
/* Step 1 */
|
|
|
|
sc = kmem_cache_create(conf->cache_name[1-conf->active_name],
|
|
|
|
sizeof(struct stripe_head)+(newsize-1)*sizeof(struct r5dev),
|
2007-07-20 03:11:58 +02:00
|
|
|
0, 0, NULL);
|
2006-03-27 11:18:07 +02:00
|
|
|
if (!sc)
|
|
|
|
return -ENOMEM;
|
|
|
|
|
|
|
|
for (i = conf->max_nr_stripes; i; i--) {
|
2011-07-18 09:38:50 +02:00
|
|
|
nsh = kmem_cache_zalloc(sc, GFP_KERNEL);
|
2006-03-27 11:18:07 +02:00
|
|
|
if (!nsh)
|
|
|
|
break;
|
|
|
|
|
|
|
|
nsh->raid_conf = conf;
|
2009-10-16 07:25:22 +02:00
|
|
|
#ifdef CONFIG_MULTICORE_RAID456
|
|
|
|
init_waitqueue_head(&nsh->ops.wait_for_ops);
|
|
|
|
#endif
|
2006-03-27 11:18:07 +02:00
|
|
|
|
|
|
|
list_add(&nsh->lru, &newstripes);
|
|
|
|
}
|
|
|
|
if (i) {
|
|
|
|
/* didn't get enough, give up */
|
|
|
|
while (!list_empty(&newstripes)) {
|
|
|
|
nsh = list_entry(newstripes.next, struct stripe_head, lru);
|
|
|
|
list_del(&nsh->lru);
|
|
|
|
kmem_cache_free(sc, nsh);
|
|
|
|
}
|
|
|
|
kmem_cache_destroy(sc);
|
|
|
|
return -ENOMEM;
|
|
|
|
}
|
|
|
|
/* Step 2 - Must use GFP_NOIO now.
|
|
|
|
* OK, we have enough stripes, start collecting inactive
|
|
|
|
* stripes and copying them over
|
|
|
|
*/
|
|
|
|
list_for_each_entry(nsh, &newstripes, lru) {
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
|
|
|
wait_event_lock_irq(conf->wait_for_stripe,
|
|
|
|
!list_empty(&conf->inactive_list),
|
|
|
|
conf->device_lock,
|
2011-04-18 10:25:42 +02:00
|
|
|
);
|
2006-03-27 11:18:07 +02:00
|
|
|
osh = get_free_stripe(conf);
|
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
atomic_set(&nsh->count, 1);
|
|
|
|
for(i=0; i<conf->pool_size; i++)
|
|
|
|
nsh->dev[i].page = osh->dev[i].page;
|
|
|
|
for( ; i<newsize; i++)
|
|
|
|
nsh->dev[i].page = NULL;
|
|
|
|
kmem_cache_free(conf->slab_cache, osh);
|
|
|
|
}
|
|
|
|
kmem_cache_destroy(conf->slab_cache);
|
|
|
|
|
|
|
|
/* Step 3.
|
|
|
|
* At this point, we are holding all the stripes so the array
|
|
|
|
* is completely stalled, so now is a good time to resize
|
2009-07-14 20:50:52 +02:00
|
|
|
* conf->disks and the scribble region
|
2006-03-27 11:18:07 +02:00
|
|
|
*/
|
|
|
|
ndisks = kzalloc(newsize * sizeof(struct disk_info), GFP_NOIO);
|
|
|
|
if (ndisks) {
|
|
|
|
for (i=0; i<conf->raid_disks; i++)
|
|
|
|
ndisks[i] = conf->disks[i];
|
|
|
|
kfree(conf->disks);
|
|
|
|
conf->disks = ndisks;
|
|
|
|
} else
|
|
|
|
err = -ENOMEM;
|
|
|
|
|
2009-07-14 20:50:52 +02:00
|
|
|
get_online_cpus();
|
|
|
|
conf->scribble_len = scribble_len(newsize);
|
|
|
|
for_each_present_cpu(cpu) {
|
|
|
|
struct raid5_percpu *percpu;
|
|
|
|
void *scribble;
|
|
|
|
|
|
|
|
percpu = per_cpu_ptr(conf->percpu, cpu);
|
|
|
|
scribble = kmalloc(conf->scribble_len, GFP_NOIO);
|
|
|
|
|
|
|
|
if (scribble) {
|
|
|
|
kfree(percpu->scribble);
|
|
|
|
percpu->scribble = scribble;
|
|
|
|
} else {
|
|
|
|
err = -ENOMEM;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
put_online_cpus();
|
|
|
|
|
2006-03-27 11:18:07 +02:00
|
|
|
/* Step 4, return new stripes to service */
|
|
|
|
while(!list_empty(&newstripes)) {
|
|
|
|
nsh = list_entry(newstripes.next, struct stripe_head, lru);
|
|
|
|
list_del_init(&nsh->lru);
|
2009-07-14 20:50:52 +02:00
|
|
|
|
2006-03-27 11:18:07 +02:00
|
|
|
for (i=conf->raid_disks; i < newsize; i++)
|
|
|
|
if (nsh->dev[i].page == NULL) {
|
|
|
|
struct page *p = alloc_page(GFP_NOIO);
|
|
|
|
nsh->dev[i].page = p;
|
|
|
|
if (!p)
|
|
|
|
err = -ENOMEM;
|
|
|
|
}
|
|
|
|
release_stripe(nsh);
|
|
|
|
}
|
|
|
|
/* critical section pass, GFP_NOIO no longer needed */
|
|
|
|
|
|
|
|
conf->slab_cache = sc;
|
|
|
|
conf->active_name = 1-conf->active_name;
|
|
|
|
conf->pool_size = newsize;
|
|
|
|
return err;
|
|
|
|
}
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2005-11-09 06:39:25 +01:00
|
|
|
static int drop_one_stripe(raid5_conf_t *conf)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
struct stripe_head *sh;
|
|
|
|
|
2005-11-09 06:39:25 +01:00
|
|
|
spin_lock_irq(&conf->device_lock);
|
|
|
|
sh = get_free_stripe(conf);
|
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
if (!sh)
|
|
|
|
return 0;
|
2006-04-02 13:31:42 +02:00
|
|
|
BUG_ON(atomic_read(&sh->count));
|
2010-06-16 08:45:16 +02:00
|
|
|
shrink_buffers(sh);
|
2005-11-09 06:39:25 +01:00
|
|
|
kmem_cache_free(conf->slab_cache, sh);
|
|
|
|
atomic_dec(&conf->active_stripes);
|
|
|
|
return 1;
|
|
|
|
}
|
|
|
|
|
|
|
|
static void shrink_stripes(raid5_conf_t *conf)
|
|
|
|
{
|
|
|
|
while (drop_one_stripe(conf))
|
|
|
|
;
|
|
|
|
|
2006-02-03 12:03:41 +01:00
|
|
|
if (conf->slab_cache)
|
|
|
|
kmem_cache_destroy(conf->slab_cache);
|
2005-04-17 00:20:36 +02:00
|
|
|
conf->slab_cache = NULL;
|
|
|
|
}
|
|
|
|
|
2007-09-27 12:47:43 +02:00
|
|
|
static void raid5_end_read_request(struct bio * bi, int error)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
2009-03-31 05:39:38 +02:00
|
|
|
struct stripe_head *sh = bi->bi_private;
|
2005-04-17 00:20:36 +02:00
|
|
|
raid5_conf_t *conf = sh->raid_conf;
|
2006-03-27 11:18:08 +02:00
|
|
|
int disks = sh->disks, i;
|
2005-04-17 00:20:36 +02:00
|
|
|
int uptodate = test_bit(BIO_UPTODATE, &bi->bi_flags);
|
2006-07-10 13:44:20 +02:00
|
|
|
char b[BDEVNAME_SIZE];
|
|
|
|
mdk_rdev_t *rdev;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
|
|
|
|
for (i=0 ; i<disks; i++)
|
|
|
|
if (bi == &sh->dev[i].req)
|
|
|
|
break;
|
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("end_read_request %llu/%d, count: %d, uptodate %d.\n",
|
|
|
|
(unsigned long long)sh->sector, i, atomic_read(&sh->count),
|
2005-04-17 00:20:36 +02:00
|
|
|
uptodate);
|
|
|
|
if (i == disks) {
|
|
|
|
BUG();
|
2007-09-27 12:47:43 +02:00
|
|
|
return;
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
if (uptodate) {
|
|
|
|
set_bit(R5_UPTODATE, &sh->dev[i].flags);
|
2005-11-09 06:39:22 +01:00
|
|
|
if (test_bit(R5_ReadError, &sh->dev[i].flags)) {
|
2006-07-10 13:44:20 +02:00
|
|
|
rdev = conf->disks[i].rdev;
|
2010-05-03 06:09:02 +02:00
|
|
|
printk_rl(KERN_INFO "md/raid:%s: read error corrected"
|
2008-05-23 22:04:34 +02:00
|
|
|
" (%lu sectors at %llu on %s)\n",
|
|
|
|
mdname(conf->mddev), STRIPE_SECTORS,
|
|
|
|
(unsigned long long)(sh->sector
|
|
|
|
+ rdev->data_offset),
|
|
|
|
bdevname(rdev->bdev, b));
|
2005-11-09 06:39:22 +01:00
|
|
|
clear_bit(R5_ReadError, &sh->dev[i].flags);
|
|
|
|
clear_bit(R5_ReWrite, &sh->dev[i].flags);
|
|
|
|
}
|
2005-11-09 06:39:31 +01:00
|
|
|
if (atomic_read(&conf->disks[i].rdev->read_errors))
|
|
|
|
atomic_set(&conf->disks[i].rdev->read_errors, 0);
|
2005-04-17 00:20:36 +02:00
|
|
|
} else {
|
2006-07-10 13:44:20 +02:00
|
|
|
const char *bdn = bdevname(conf->disks[i].rdev->bdev, b);
|
2005-11-09 06:39:31 +01:00
|
|
|
int retry = 0;
|
2006-07-10 13:44:20 +02:00
|
|
|
rdev = conf->disks[i].rdev;
|
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
clear_bit(R5_UPTODATE, &sh->dev[i].flags);
|
2006-07-10 13:44:20 +02:00
|
|
|
atomic_inc(&rdev->read_errors);
|
2010-04-28 03:51:17 +02:00
|
|
|
if (conf->mddev->degraded >= conf->max_degraded)
|
2008-05-23 22:04:34 +02:00
|
|
|
printk_rl(KERN_WARNING
|
2010-05-03 06:09:02 +02:00
|
|
|
"md/raid:%s: read error not correctable "
|
2008-05-23 22:04:34 +02:00
|
|
|
"(sector %llu on %s).\n",
|
|
|
|
mdname(conf->mddev),
|
|
|
|
(unsigned long long)(sh->sector
|
|
|
|
+ rdev->data_offset),
|
|
|
|
bdn);
|
2005-11-09 06:39:31 +01:00
|
|
|
else if (test_bit(R5_ReWrite, &sh->dev[i].flags))
|
2005-11-09 06:39:22 +01:00
|
|
|
/* Oh, no!!! */
|
2008-05-23 22:04:34 +02:00
|
|
|
printk_rl(KERN_WARNING
|
2010-05-03 06:09:02 +02:00
|
|
|
"md/raid:%s: read error NOT corrected!! "
|
2008-05-23 22:04:34 +02:00
|
|
|
"(sector %llu on %s).\n",
|
|
|
|
mdname(conf->mddev),
|
|
|
|
(unsigned long long)(sh->sector
|
|
|
|
+ rdev->data_offset),
|
|
|
|
bdn);
|
2006-07-10 13:44:20 +02:00
|
|
|
else if (atomic_read(&rdev->read_errors)
|
2005-11-09 06:39:31 +01:00
|
|
|
> conf->max_nr_stripes)
|
2006-01-06 09:20:14 +01:00
|
|
|
printk(KERN_WARNING
|
2010-05-03 06:09:02 +02:00
|
|
|
"md/raid:%s: Too many read errors, failing device %s.\n",
|
2006-07-10 13:44:20 +02:00
|
|
|
mdname(conf->mddev), bdn);
|
2005-11-09 06:39:31 +01:00
|
|
|
else
|
|
|
|
retry = 1;
|
|
|
|
if (retry)
|
|
|
|
set_bit(R5_ReadError, &sh->dev[i].flags);
|
|
|
|
else {
|
2005-11-09 06:39:22 +01:00
|
|
|
clear_bit(R5_ReadError, &sh->dev[i].flags);
|
|
|
|
clear_bit(R5_ReWrite, &sh->dev[i].flags);
|
2006-07-10 13:44:20 +02:00
|
|
|
md_error(conf->mddev, rdev);
|
2005-11-09 06:39:31 +01:00
|
|
|
}
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
rdev_dec_pending(conf->disks[i].rdev, conf->mddev);
|
|
|
|
clear_bit(R5_LOCKED, &sh->dev[i].flags);
|
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
release_stripe(sh);
|
|
|
|
}
|
|
|
|
|
2008-10-13 02:55:12 +02:00
|
|
|
static void raid5_end_write_request(struct bio *bi, int error)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
2009-03-31 05:39:38 +02:00
|
|
|
struct stripe_head *sh = bi->bi_private;
|
2005-04-17 00:20:36 +02:00
|
|
|
raid5_conf_t *conf = sh->raid_conf;
|
2006-03-27 11:18:08 +02:00
|
|
|
int disks = sh->disks, i;
|
2005-04-17 00:20:36 +02:00
|
|
|
int uptodate = test_bit(BIO_UPTODATE, &bi->bi_flags);
|
|
|
|
|
|
|
|
for (i=0 ; i<disks; i++)
|
|
|
|
if (bi == &sh->dev[i].req)
|
|
|
|
break;
|
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("end_write_request %llu/%d, count %d, uptodate: %d.\n",
|
2005-04-17 00:20:36 +02:00
|
|
|
(unsigned long long)sh->sector, i, atomic_read(&sh->count),
|
|
|
|
uptodate);
|
|
|
|
if (i == disks) {
|
|
|
|
BUG();
|
2007-09-27 12:47:43 +02:00
|
|
|
return;
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
if (!uptodate)
|
|
|
|
md_error(conf->mddev, conf->disks[i].rdev);
|
|
|
|
|
|
|
|
rdev_dec_pending(conf->disks[i].rdev, conf->mddev);
|
|
|
|
|
|
|
|
clear_bit(R5_LOCKED, &sh->dev[i].flags);
|
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
2006-10-03 10:15:53 +02:00
|
|
|
release_stripe(sh);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
2009-03-31 06:19:07 +02:00
|
|
|
static sector_t compute_blocknr(struct stripe_head *sh, int i, int previous);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2009-03-31 06:19:07 +02:00
|
|
|
static void raid5_build_block(struct stripe_head *sh, int i, int previous)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
struct r5dev *dev = &sh->dev[i];
|
|
|
|
|
|
|
|
bio_init(&dev->req);
|
|
|
|
dev->req.bi_io_vec = &dev->vec;
|
|
|
|
dev->req.bi_vcnt++;
|
|
|
|
dev->req.bi_max_vecs++;
|
|
|
|
dev->vec.bv_page = dev->page;
|
|
|
|
dev->vec.bv_len = STRIPE_SIZE;
|
|
|
|
dev->vec.bv_offset = 0;
|
|
|
|
|
|
|
|
dev->req.bi_sector = sh->sector;
|
|
|
|
dev->req.bi_private = sh;
|
|
|
|
|
|
|
|
dev->flags = 0;
|
2009-03-31 06:19:07 +02:00
|
|
|
dev->sector = compute_blocknr(sh, i, previous);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
static void error(mddev_t *mddev, mdk_rdev_t *rdev)
|
|
|
|
{
|
|
|
|
char b[BDEVNAME_SIZE];
|
2010-03-08 06:02:40 +01:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2010-05-03 06:09:02 +02:00
|
|
|
pr_debug("raid456: error called\n");
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2011-05-11 06:38:44 +02:00
|
|
|
if (test_and_clear_bit(In_sync, &rdev->flags)) {
|
|
|
|
unsigned long flags;
|
|
|
|
spin_lock_irqsave(&conf->device_lock, flags);
|
|
|
|
mddev->degraded++;
|
|
|
|
spin_unlock_irqrestore(&conf->device_lock, flags);
|
|
|
|
/*
|
|
|
|
* if recovery was running, make sure it aborts.
|
|
|
|
*/
|
|
|
|
set_bit(MD_RECOVERY_INTR, &mddev->recovery);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
2011-05-11 06:38:44 +02:00
|
|
|
set_bit(Faulty, &rdev->flags);
|
|
|
|
set_bit(MD_CHANGE_DEVS, &mddev->flags);
|
|
|
|
printk(KERN_ALERT
|
|
|
|
"md/raid:%s: Disk failure on %s, disabling device.\n"
|
|
|
|
"md/raid:%s: Operation continuing on %d devices.\n",
|
|
|
|
mdname(mddev),
|
|
|
|
bdevname(rdev->bdev, b),
|
|
|
|
mdname(mddev),
|
|
|
|
conf->raid_disks - mddev->degraded);
|
2006-06-26 09:27:38 +02:00
|
|
|
}
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Input: a 'big' sector number,
|
|
|
|
* Output: index of the data and parity disk, and the sector # in them.
|
|
|
|
*/
|
2009-03-31 05:39:38 +02:00
|
|
|
static sector_t raid5_compute_sector(raid5_conf_t *conf, sector_t r_sector,
|
2009-03-31 05:39:38 +02:00
|
|
|
int previous, int *dd_idx,
|
|
|
|
struct stripe_head *sh)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
2010-04-22 23:08:28 +02:00
|
|
|
sector_t stripe, stripe2;
|
2010-04-20 06:13:34 +02:00
|
|
|
sector_t chunk_number;
|
2005-04-17 00:20:36 +02:00
|
|
|
unsigned int chunk_offset;
|
2009-03-31 05:39:38 +02:00
|
|
|
int pd_idx, qd_idx;
|
2009-03-31 05:39:38 +02:00
|
|
|
int ddf_layout = 0;
|
2005-04-17 00:20:36 +02:00
|
|
|
sector_t new_sector;
|
2009-03-31 06:20:22 +02:00
|
|
|
int algorithm = previous ? conf->prev_algo
|
|
|
|
: conf->algorithm;
|
2009-06-18 00:45:55 +02:00
|
|
|
int sectors_per_chunk = previous ? conf->prev_chunk_sectors
|
|
|
|
: conf->chunk_sectors;
|
2009-03-31 05:39:38 +02:00
|
|
|
int raid_disks = previous ? conf->previous_raid_disks
|
|
|
|
: conf->raid_disks;
|
|
|
|
int data_disks = raid_disks - conf->max_degraded;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
/* First compute the information on this sector */
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Compute the chunk number and the sector offset inside the chunk
|
|
|
|
*/
|
|
|
|
chunk_offset = sector_div(r_sector, sectors_per_chunk);
|
|
|
|
chunk_number = r_sector;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Compute the stripe number
|
|
|
|
*/
|
2010-04-20 06:13:34 +02:00
|
|
|
stripe = chunk_number;
|
|
|
|
*dd_idx = sector_div(stripe, data_disks);
|
2010-04-22 23:08:28 +02:00
|
|
|
stripe2 = stripe;
|
2005-04-17 00:20:36 +02:00
|
|
|
/*
|
|
|
|
* Select the parity disk based on the user selected algorithm.
|
|
|
|
*/
|
2009-03-31 05:39:38 +02:00
|
|
|
pd_idx = qd_idx = ~0;
|
2006-06-26 09:27:38 +02:00
|
|
|
switch(conf->level) {
|
|
|
|
case 4:
|
2009-03-31 05:39:38 +02:00
|
|
|
pd_idx = data_disks;
|
2006-06-26 09:27:38 +02:00
|
|
|
break;
|
|
|
|
case 5:
|
2009-03-31 06:20:22 +02:00
|
|
|
switch (algorithm) {
|
2005-04-17 00:20:36 +02:00
|
|
|
case ALGORITHM_LEFT_ASYMMETRIC:
|
2010-04-22 23:08:28 +02:00
|
|
|
pd_idx = data_disks - sector_div(stripe2, raid_disks);
|
2009-03-31 05:39:38 +02:00
|
|
|
if (*dd_idx >= pd_idx)
|
2005-04-17 00:20:36 +02:00
|
|
|
(*dd_idx)++;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_RIGHT_ASYMMETRIC:
|
2010-04-22 23:08:28 +02:00
|
|
|
pd_idx = sector_div(stripe2, raid_disks);
|
2009-03-31 05:39:38 +02:00
|
|
|
if (*dd_idx >= pd_idx)
|
2005-04-17 00:20:36 +02:00
|
|
|
(*dd_idx)++;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_LEFT_SYMMETRIC:
|
2010-04-22 23:08:28 +02:00
|
|
|
pd_idx = data_disks - sector_div(stripe2, raid_disks);
|
2009-03-31 05:39:38 +02:00
|
|
|
*dd_idx = (pd_idx + 1 + *dd_idx) % raid_disks;
|
2005-04-17 00:20:36 +02:00
|
|
|
break;
|
|
|
|
case ALGORITHM_RIGHT_SYMMETRIC:
|
2010-04-22 23:08:28 +02:00
|
|
|
pd_idx = sector_div(stripe2, raid_disks);
|
2009-03-31 05:39:38 +02:00
|
|
|
*dd_idx = (pd_idx + 1 + *dd_idx) % raid_disks;
|
2005-04-17 00:20:36 +02:00
|
|
|
break;
|
2009-03-31 05:39:38 +02:00
|
|
|
case ALGORITHM_PARITY_0:
|
|
|
|
pd_idx = 0;
|
|
|
|
(*dd_idx)++;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_PARITY_N:
|
|
|
|
pd_idx = data_disks;
|
|
|
|
break;
|
2005-04-17 00:20:36 +02:00
|
|
|
default:
|
2009-03-31 05:39:38 +02:00
|
|
|
BUG();
|
2006-06-26 09:27:38 +02:00
|
|
|
}
|
|
|
|
break;
|
|
|
|
case 6:
|
|
|
|
|
2009-03-31 06:20:22 +02:00
|
|
|
switch (algorithm) {
|
2006-06-26 09:27:38 +02:00
|
|
|
case ALGORITHM_LEFT_ASYMMETRIC:
|
2010-04-22 23:08:28 +02:00
|
|
|
pd_idx = raid_disks - 1 - sector_div(stripe2, raid_disks);
|
2009-03-31 05:39:38 +02:00
|
|
|
qd_idx = pd_idx + 1;
|
|
|
|
if (pd_idx == raid_disks-1) {
|
2009-03-31 05:39:38 +02:00
|
|
|
(*dd_idx)++; /* Q D D D P */
|
2009-03-31 05:39:38 +02:00
|
|
|
qd_idx = 0;
|
|
|
|
} else if (*dd_idx >= pd_idx)
|
2006-06-26 09:27:38 +02:00
|
|
|
(*dd_idx) += 2; /* D D P Q D */
|
|
|
|
break;
|
|
|
|
case ALGORITHM_RIGHT_ASYMMETRIC:
|
2010-04-22 23:08:28 +02:00
|
|
|
pd_idx = sector_div(stripe2, raid_disks);
|
2009-03-31 05:39:38 +02:00
|
|
|
qd_idx = pd_idx + 1;
|
|
|
|
if (pd_idx == raid_disks-1) {
|
2009-03-31 05:39:38 +02:00
|
|
|
(*dd_idx)++; /* Q D D D P */
|
2009-03-31 05:39:38 +02:00
|
|
|
qd_idx = 0;
|
|
|
|
} else if (*dd_idx >= pd_idx)
|
2006-06-26 09:27:38 +02:00
|
|
|
(*dd_idx) += 2; /* D D P Q D */
|
|
|
|
break;
|
|
|
|
case ALGORITHM_LEFT_SYMMETRIC:
|
2010-04-22 23:08:28 +02:00
|
|
|
pd_idx = raid_disks - 1 - sector_div(stripe2, raid_disks);
|
2009-03-31 05:39:38 +02:00
|
|
|
qd_idx = (pd_idx + 1) % raid_disks;
|
|
|
|
*dd_idx = (pd_idx + 2 + *dd_idx) % raid_disks;
|
2006-06-26 09:27:38 +02:00
|
|
|
break;
|
|
|
|
case ALGORITHM_RIGHT_SYMMETRIC:
|
2010-04-22 23:08:28 +02:00
|
|
|
pd_idx = sector_div(stripe2, raid_disks);
|
2009-03-31 05:39:38 +02:00
|
|
|
qd_idx = (pd_idx + 1) % raid_disks;
|
|
|
|
*dd_idx = (pd_idx + 2 + *dd_idx) % raid_disks;
|
2006-06-26 09:27:38 +02:00
|
|
|
break;
|
2009-03-31 05:39:38 +02:00
|
|
|
|
|
|
|
case ALGORITHM_PARITY_0:
|
|
|
|
pd_idx = 0;
|
|
|
|
qd_idx = 1;
|
|
|
|
(*dd_idx) += 2;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_PARITY_N:
|
|
|
|
pd_idx = data_disks;
|
|
|
|
qd_idx = data_disks + 1;
|
|
|
|
break;
|
|
|
|
|
|
|
|
case ALGORITHM_ROTATING_ZERO_RESTART:
|
|
|
|
/* Exactly the same as RIGHT_ASYMMETRIC, but or
|
|
|
|
* of blocks for computing Q is different.
|
|
|
|
*/
|
2010-04-22 23:08:28 +02:00
|
|
|
pd_idx = sector_div(stripe2, raid_disks);
|
2009-03-31 05:39:38 +02:00
|
|
|
qd_idx = pd_idx + 1;
|
|
|
|
if (pd_idx == raid_disks-1) {
|
|
|
|
(*dd_idx)++; /* Q D D D P */
|
|
|
|
qd_idx = 0;
|
|
|
|
} else if (*dd_idx >= pd_idx)
|
|
|
|
(*dd_idx) += 2; /* D D P Q D */
|
2009-03-31 05:39:38 +02:00
|
|
|
ddf_layout = 1;
|
2009-03-31 05:39:38 +02:00
|
|
|
break;
|
|
|
|
|
|
|
|
case ALGORITHM_ROTATING_N_RESTART:
|
|
|
|
/* Same a left_asymmetric, by first stripe is
|
|
|
|
* D D D P Q rather than
|
|
|
|
* Q D D D P
|
|
|
|
*/
|
2010-04-22 23:08:28 +02:00
|
|
|
stripe2 += 1;
|
|
|
|
pd_idx = raid_disks - 1 - sector_div(stripe2, raid_disks);
|
2009-03-31 05:39:38 +02:00
|
|
|
qd_idx = pd_idx + 1;
|
|
|
|
if (pd_idx == raid_disks-1) {
|
|
|
|
(*dd_idx)++; /* Q D D D P */
|
|
|
|
qd_idx = 0;
|
|
|
|
} else if (*dd_idx >= pd_idx)
|
|
|
|
(*dd_idx) += 2; /* D D P Q D */
|
2009-03-31 05:39:38 +02:00
|
|
|
ddf_layout = 1;
|
2009-03-31 05:39:38 +02:00
|
|
|
break;
|
|
|
|
|
|
|
|
case ALGORITHM_ROTATING_N_CONTINUE:
|
|
|
|
/* Same as left_symmetric but Q is before P */
|
2010-04-22 23:08:28 +02:00
|
|
|
pd_idx = raid_disks - 1 - sector_div(stripe2, raid_disks);
|
2009-03-31 05:39:38 +02:00
|
|
|
qd_idx = (pd_idx + raid_disks - 1) % raid_disks;
|
|
|
|
*dd_idx = (pd_idx + 1 + *dd_idx) % raid_disks;
|
2009-03-31 05:39:38 +02:00
|
|
|
ddf_layout = 1;
|
2009-03-31 05:39:38 +02:00
|
|
|
break;
|
|
|
|
|
|
|
|
case ALGORITHM_LEFT_ASYMMETRIC_6:
|
|
|
|
/* RAID5 left_asymmetric, with Q on last device */
|
2010-04-22 23:08:28 +02:00
|
|
|
pd_idx = data_disks - sector_div(stripe2, raid_disks-1);
|
2009-03-31 05:39:38 +02:00
|
|
|
if (*dd_idx >= pd_idx)
|
|
|
|
(*dd_idx)++;
|
|
|
|
qd_idx = raid_disks - 1;
|
|
|
|
break;
|
|
|
|
|
|
|
|
case ALGORITHM_RIGHT_ASYMMETRIC_6:
|
2010-04-22 23:08:28 +02:00
|
|
|
pd_idx = sector_div(stripe2, raid_disks-1);
|
2009-03-31 05:39:38 +02:00
|
|
|
if (*dd_idx >= pd_idx)
|
|
|
|
(*dd_idx)++;
|
|
|
|
qd_idx = raid_disks - 1;
|
|
|
|
break;
|
|
|
|
|
|
|
|
case ALGORITHM_LEFT_SYMMETRIC_6:
|
2010-04-22 23:08:28 +02:00
|
|
|
pd_idx = data_disks - sector_div(stripe2, raid_disks-1);
|
2009-03-31 05:39:38 +02:00
|
|
|
*dd_idx = (pd_idx + 1 + *dd_idx) % (raid_disks-1);
|
|
|
|
qd_idx = raid_disks - 1;
|
|
|
|
break;
|
|
|
|
|
|
|
|
case ALGORITHM_RIGHT_SYMMETRIC_6:
|
2010-04-22 23:08:28 +02:00
|
|
|
pd_idx = sector_div(stripe2, raid_disks-1);
|
2009-03-31 05:39:38 +02:00
|
|
|
*dd_idx = (pd_idx + 1 + *dd_idx) % (raid_disks-1);
|
|
|
|
qd_idx = raid_disks - 1;
|
|
|
|
break;
|
|
|
|
|
|
|
|
case ALGORITHM_PARITY_0_6:
|
|
|
|
pd_idx = 0;
|
|
|
|
(*dd_idx)++;
|
|
|
|
qd_idx = raid_disks - 1;
|
|
|
|
break;
|
|
|
|
|
2006-06-26 09:27:38 +02:00
|
|
|
default:
|
2009-03-31 05:39:38 +02:00
|
|
|
BUG();
|
2006-06-26 09:27:38 +02:00
|
|
|
}
|
|
|
|
break;
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
2009-03-31 05:39:38 +02:00
|
|
|
if (sh) {
|
|
|
|
sh->pd_idx = pd_idx;
|
|
|
|
sh->qd_idx = qd_idx;
|
2009-03-31 05:39:38 +02:00
|
|
|
sh->ddf_layout = ddf_layout;
|
2009-03-31 05:39:38 +02:00
|
|
|
}
|
2005-04-17 00:20:36 +02:00
|
|
|
/*
|
|
|
|
* Finally, compute the new sector number
|
|
|
|
*/
|
|
|
|
new_sector = (sector_t)stripe * sectors_per_chunk + chunk_offset;
|
|
|
|
return new_sector;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2009-03-31 06:19:07 +02:00
|
|
|
static sector_t compute_blocknr(struct stripe_head *sh, int i, int previous)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
raid5_conf_t *conf = sh->raid_conf;
|
2006-12-10 11:20:49 +01:00
|
|
|
int raid_disks = sh->disks;
|
|
|
|
int data_disks = raid_disks - conf->max_degraded;
|
2005-04-17 00:20:36 +02:00
|
|
|
sector_t new_sector = sh->sector, check;
|
2009-06-18 00:45:55 +02:00
|
|
|
int sectors_per_chunk = previous ? conf->prev_chunk_sectors
|
|
|
|
: conf->chunk_sectors;
|
2009-03-31 06:20:22 +02:00
|
|
|
int algorithm = previous ? conf->prev_algo
|
|
|
|
: conf->algorithm;
|
2005-04-17 00:20:36 +02:00
|
|
|
sector_t stripe;
|
|
|
|
int chunk_offset;
|
2010-04-20 06:13:34 +02:00
|
|
|
sector_t chunk_number;
|
|
|
|
int dummy1, dd_idx = i;
|
2005-04-17 00:20:36 +02:00
|
|
|
sector_t r_sector;
|
2009-03-31 05:39:38 +02:00
|
|
|
struct stripe_head sh2;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2006-06-26 09:27:38 +02:00
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
chunk_offset = sector_div(new_sector, sectors_per_chunk);
|
|
|
|
stripe = new_sector;
|
|
|
|
|
2006-06-26 09:27:38 +02:00
|
|
|
if (i == sh->pd_idx)
|
|
|
|
return 0;
|
|
|
|
switch(conf->level) {
|
|
|
|
case 4: break;
|
|
|
|
case 5:
|
2009-03-31 06:20:22 +02:00
|
|
|
switch (algorithm) {
|
2005-04-17 00:20:36 +02:00
|
|
|
case ALGORITHM_LEFT_ASYMMETRIC:
|
|
|
|
case ALGORITHM_RIGHT_ASYMMETRIC:
|
|
|
|
if (i > sh->pd_idx)
|
|
|
|
i--;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_LEFT_SYMMETRIC:
|
|
|
|
case ALGORITHM_RIGHT_SYMMETRIC:
|
|
|
|
if (i < sh->pd_idx)
|
|
|
|
i += raid_disks;
|
|
|
|
i -= (sh->pd_idx + 1);
|
|
|
|
break;
|
2009-03-31 05:39:38 +02:00
|
|
|
case ALGORITHM_PARITY_0:
|
|
|
|
i -= 1;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_PARITY_N:
|
|
|
|
break;
|
2005-04-17 00:20:36 +02:00
|
|
|
default:
|
2009-03-31 05:39:38 +02:00
|
|
|
BUG();
|
2006-06-26 09:27:38 +02:00
|
|
|
}
|
|
|
|
break;
|
|
|
|
case 6:
|
2009-03-31 05:39:38 +02:00
|
|
|
if (i == sh->qd_idx)
|
2006-06-26 09:27:38 +02:00
|
|
|
return 0; /* It is the Q disk */
|
2009-03-31 06:20:22 +02:00
|
|
|
switch (algorithm) {
|
2006-06-26 09:27:38 +02:00
|
|
|
case ALGORITHM_LEFT_ASYMMETRIC:
|
|
|
|
case ALGORITHM_RIGHT_ASYMMETRIC:
|
2009-03-31 05:39:38 +02:00
|
|
|
case ALGORITHM_ROTATING_ZERO_RESTART:
|
|
|
|
case ALGORITHM_ROTATING_N_RESTART:
|
|
|
|
if (sh->pd_idx == raid_disks-1)
|
|
|
|
i--; /* Q D D D P */
|
2006-06-26 09:27:38 +02:00
|
|
|
else if (i > sh->pd_idx)
|
|
|
|
i -= 2; /* D D P Q D */
|
|
|
|
break;
|
|
|
|
case ALGORITHM_LEFT_SYMMETRIC:
|
|
|
|
case ALGORITHM_RIGHT_SYMMETRIC:
|
|
|
|
if (sh->pd_idx == raid_disks-1)
|
|
|
|
i--; /* Q D D D P */
|
|
|
|
else {
|
|
|
|
/* D D P Q D */
|
|
|
|
if (i < sh->pd_idx)
|
|
|
|
i += raid_disks;
|
|
|
|
i -= (sh->pd_idx + 2);
|
|
|
|
}
|
|
|
|
break;
|
2009-03-31 05:39:38 +02:00
|
|
|
case ALGORITHM_PARITY_0:
|
|
|
|
i -= 2;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_PARITY_N:
|
|
|
|
break;
|
|
|
|
case ALGORITHM_ROTATING_N_CONTINUE:
|
2009-10-16 07:27:34 +02:00
|
|
|
/* Like left_symmetric, but P is before Q */
|
2009-03-31 05:39:38 +02:00
|
|
|
if (sh->pd_idx == 0)
|
|
|
|
i--; /* P D D D Q */
|
2009-10-16 07:27:34 +02:00
|
|
|
else {
|
|
|
|
/* D D Q P D */
|
|
|
|
if (i < sh->pd_idx)
|
|
|
|
i += raid_disks;
|
|
|
|
i -= (sh->pd_idx + 1);
|
|
|
|
}
|
2009-03-31 05:39:38 +02:00
|
|
|
break;
|
|
|
|
case ALGORITHM_LEFT_ASYMMETRIC_6:
|
|
|
|
case ALGORITHM_RIGHT_ASYMMETRIC_6:
|
|
|
|
if (i > sh->pd_idx)
|
|
|
|
i--;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_LEFT_SYMMETRIC_6:
|
|
|
|
case ALGORITHM_RIGHT_SYMMETRIC_6:
|
|
|
|
if (i < sh->pd_idx)
|
|
|
|
i += data_disks + 1;
|
|
|
|
i -= (sh->pd_idx + 1);
|
|
|
|
break;
|
|
|
|
case ALGORITHM_PARITY_0_6:
|
|
|
|
i -= 1;
|
|
|
|
break;
|
2006-06-26 09:27:38 +02:00
|
|
|
default:
|
2009-03-31 05:39:38 +02:00
|
|
|
BUG();
|
2006-06-26 09:27:38 +02:00
|
|
|
}
|
|
|
|
break;
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
chunk_number = stripe * data_disks + i;
|
2010-04-20 06:13:34 +02:00
|
|
|
r_sector = chunk_number * sectors_per_chunk + chunk_offset;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2009-03-31 05:39:38 +02:00
|
|
|
check = raid5_compute_sector(conf, r_sector,
|
2009-03-31 06:19:07 +02:00
|
|
|
previous, &dummy1, &sh2);
|
2009-03-31 05:39:38 +02:00
|
|
|
if (check != sh->sector || dummy1 != dd_idx || sh2.pd_idx != sh->pd_idx
|
|
|
|
|| sh2.qd_idx != sh->qd_idx) {
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_ERR "md/raid:%s: compute_blocknr: map not correct\n",
|
|
|
|
mdname(conf->mddev));
|
2005-04-17 00:20:36 +02:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
return r_sector;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
md: replace STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} with 'reconstruct_states'
From: Dan Williams <dan.j.williams@intel.com>
Track the state of reconstruct operations (recalculating the parity block
usually due to incoming writes, or as part of array expansion) Reduces the
scope of the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags to only tracking whether
a reconstruct operation has been requested via the ops_request field of struct
stripe_head_state.
This is the final step in the removal of ops.{pending,ack,complete,count}, i.e.
the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags only request an operation and do
not track the state of the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:32:05 +02:00
|
|
|
static void
|
2009-08-30 04:13:12 +02:00
|
|
|
schedule_reconstruction(struct stripe_head *sh, struct stripe_head_state *s,
|
md: replace STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} with 'reconstruct_states'
From: Dan Williams <dan.j.williams@intel.com>
Track the state of reconstruct operations (recalculating the parity block
usually due to incoming writes, or as part of array expansion) Reduces the
scope of the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags to only tracking whether
a reconstruct operation has been requested via the ops_request field of struct
stripe_head_state.
This is the final step in the removal of ops.{pending,ack,complete,count}, i.e.
the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags only request an operation and do
not track the state of the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:32:05 +02:00
|
|
|
int rcw, int expand)
|
2007-01-02 21:52:30 +01:00
|
|
|
{
|
|
|
|
int i, pd_idx = sh->pd_idx, disks = sh->disks;
|
2009-08-30 04:13:12 +02:00
|
|
|
raid5_conf_t *conf = sh->raid_conf;
|
|
|
|
int level = conf->level;
|
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
if (rcw) {
|
|
|
|
/* if we are not expanding this is a proper write request, and
|
|
|
|
* there will be bios with new data to be drained into the
|
|
|
|
* stripe cache
|
|
|
|
*/
|
|
|
|
if (!expand) {
|
md: replace STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} with 'reconstruct_states'
From: Dan Williams <dan.j.williams@intel.com>
Track the state of reconstruct operations (recalculating the parity block
usually due to incoming writes, or as part of array expansion) Reduces the
scope of the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags to only tracking whether
a reconstruct operation has been requested via the ops_request field of struct
stripe_head_state.
This is the final step in the removal of ops.{pending,ack,complete,count}, i.e.
the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags only request an operation and do
not track the state of the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:32:05 +02:00
|
|
|
sh->reconstruct_state = reconstruct_state_drain_run;
|
|
|
|
set_bit(STRIPE_OP_BIODRAIN, &s->ops_request);
|
|
|
|
} else
|
|
|
|
sh->reconstruct_state = reconstruct_state_run;
|
2006-06-26 09:27:38 +02:00
|
|
|
|
2009-07-14 22:40:19 +02:00
|
|
|
set_bit(STRIPE_OP_RECONSTRUCT, &s->ops_request);
|
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
for (i = disks; i--; ) {
|
|
|
|
struct r5dev *dev = &sh->dev[i];
|
|
|
|
|
|
|
|
if (dev->towrite) {
|
|
|
|
set_bit(R5_LOCKED, &dev->flags);
|
2008-06-28 00:32:06 +02:00
|
|
|
set_bit(R5_Wantdrain, &dev->flags);
|
2007-01-02 21:52:30 +01:00
|
|
|
if (!expand)
|
|
|
|
clear_bit(R5_UPTODATE, &dev->flags);
|
md: replace STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} with 'reconstruct_states'
From: Dan Williams <dan.j.williams@intel.com>
Track the state of reconstruct operations (recalculating the parity block
usually due to incoming writes, or as part of array expansion) Reduces the
scope of the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags to only tracking whether
a reconstruct operation has been requested via the ops_request field of struct
stripe_head_state.
This is the final step in the removal of ops.{pending,ack,complete,count}, i.e.
the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags only request an operation and do
not track the state of the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:32:05 +02:00
|
|
|
s->locked++;
|
2007-01-02 21:52:30 +01:00
|
|
|
}
|
|
|
|
}
|
2009-08-30 04:13:12 +02:00
|
|
|
if (s->locked + conf->max_degraded == disks)
|
2008-04-28 11:15:53 +02:00
|
|
|
if (!test_and_set_bit(STRIPE_FULL_WRITE, &sh->state))
|
2009-08-30 04:13:12 +02:00
|
|
|
atomic_inc(&conf->pending_full_writes);
|
2007-01-02 21:52:30 +01:00
|
|
|
} else {
|
2009-08-30 04:13:12 +02:00
|
|
|
BUG_ON(level == 6);
|
2007-01-02 21:52:30 +01:00
|
|
|
BUG_ON(!(test_bit(R5_UPTODATE, &sh->dev[pd_idx].flags) ||
|
|
|
|
test_bit(R5_Wantcompute, &sh->dev[pd_idx].flags)));
|
|
|
|
|
2008-06-28 00:32:06 +02:00
|
|
|
sh->reconstruct_state = reconstruct_state_prexor_drain_run;
|
md: replace STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} with 'reconstruct_states'
From: Dan Williams <dan.j.williams@intel.com>
Track the state of reconstruct operations (recalculating the parity block
usually due to incoming writes, or as part of array expansion) Reduces the
scope of the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags to only tracking whether
a reconstruct operation has been requested via the ops_request field of struct
stripe_head_state.
This is the final step in the removal of ops.{pending,ack,complete,count}, i.e.
the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags only request an operation and do
not track the state of the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:32:05 +02:00
|
|
|
set_bit(STRIPE_OP_PREXOR, &s->ops_request);
|
|
|
|
set_bit(STRIPE_OP_BIODRAIN, &s->ops_request);
|
2009-07-14 22:40:19 +02:00
|
|
|
set_bit(STRIPE_OP_RECONSTRUCT, &s->ops_request);
|
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
for (i = disks; i--; ) {
|
|
|
|
struct r5dev *dev = &sh->dev[i];
|
|
|
|
if (i == pd_idx)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
if (dev->towrite &&
|
|
|
|
(test_bit(R5_UPTODATE, &dev->flags) ||
|
2008-06-28 00:32:06 +02:00
|
|
|
test_bit(R5_Wantcompute, &dev->flags))) {
|
|
|
|
set_bit(R5_Wantdrain, &dev->flags);
|
2007-01-02 21:52:30 +01:00
|
|
|
set_bit(R5_LOCKED, &dev->flags);
|
|
|
|
clear_bit(R5_UPTODATE, &dev->flags);
|
md: replace STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} with 'reconstruct_states'
From: Dan Williams <dan.j.williams@intel.com>
Track the state of reconstruct operations (recalculating the parity block
usually due to incoming writes, or as part of array expansion) Reduces the
scope of the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags to only tracking whether
a reconstruct operation has been requested via the ops_request field of struct
stripe_head_state.
This is the final step in the removal of ops.{pending,ack,complete,count}, i.e.
the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags only request an operation and do
not track the state of the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:32:05 +02:00
|
|
|
s->locked++;
|
2007-01-02 21:52:30 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2009-08-30 04:13:12 +02:00
|
|
|
/* keep the parity disk(s) locked while asynchronous operations
|
2007-01-02 21:52:30 +01:00
|
|
|
* are in flight
|
|
|
|
*/
|
|
|
|
set_bit(R5_LOCKED, &sh->dev[pd_idx].flags);
|
|
|
|
clear_bit(R5_UPTODATE, &sh->dev[pd_idx].flags);
|
md: replace STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} with 'reconstruct_states'
From: Dan Williams <dan.j.williams@intel.com>
Track the state of reconstruct operations (recalculating the parity block
usually due to incoming writes, or as part of array expansion) Reduces the
scope of the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags to only tracking whether
a reconstruct operation has been requested via the ops_request field of struct
stripe_head_state.
This is the final step in the removal of ops.{pending,ack,complete,count}, i.e.
the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags only request an operation and do
not track the state of the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:32:05 +02:00
|
|
|
s->locked++;
|
2007-01-02 21:52:30 +01:00
|
|
|
|
2009-08-30 04:13:12 +02:00
|
|
|
if (level == 6) {
|
|
|
|
int qd_idx = sh->qd_idx;
|
|
|
|
struct r5dev *dev = &sh->dev[qd_idx];
|
|
|
|
|
|
|
|
set_bit(R5_LOCKED, &dev->flags);
|
|
|
|
clear_bit(R5_UPTODATE, &dev->flags);
|
|
|
|
s->locked++;
|
|
|
|
}
|
|
|
|
|
md: replace STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} with 'reconstruct_states'
From: Dan Williams <dan.j.williams@intel.com>
Track the state of reconstruct operations (recalculating the parity block
usually due to incoming writes, or as part of array expansion) Reduces the
scope of the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags to only tracking whether
a reconstruct operation has been requested via the ops_request field of struct
stripe_head_state.
This is the final step in the removal of ops.{pending,ack,complete,count}, i.e.
the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags only request an operation and do
not track the state of the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:32:05 +02:00
|
|
|
pr_debug("%s: stripe %llu locked: %d ops_request: %lx\n",
|
2008-04-28 11:15:50 +02:00
|
|
|
__func__, (unsigned long long)sh->sector,
|
md: replace STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} with 'reconstruct_states'
From: Dan Williams <dan.j.williams@intel.com>
Track the state of reconstruct operations (recalculating the parity block
usually due to incoming writes, or as part of array expansion) Reduces the
scope of the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags to only tracking whether
a reconstruct operation has been requested via the ops_request field of struct
stripe_head_state.
This is the final step in the removal of ops.{pending,ack,complete,count}, i.e.
the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags only request an operation and do
not track the state of the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:32:05 +02:00
|
|
|
s->locked, s->ops_request);
|
2007-01-02 21:52:30 +01:00
|
|
|
}
|
2006-06-26 09:27:38 +02:00
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
/*
|
|
|
|
* Each stripe/dev can have one or more bion attached.
|
2006-06-26 09:27:38 +02:00
|
|
|
* toread/towrite point to the first in a chain.
|
2005-04-17 00:20:36 +02:00
|
|
|
* The bi_next chain must be in order.
|
|
|
|
*/
|
|
|
|
static int add_stripe_bio(struct stripe_head *sh, struct bio *bi, int dd_idx, int forwrite)
|
|
|
|
{
|
|
|
|
struct bio **bip;
|
|
|
|
raid5_conf_t *conf = sh->raid_conf;
|
2005-09-10 01:23:54 +02:00
|
|
|
int firstwrite=0;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2011-07-26 03:20:35 +02:00
|
|
|
pr_debug("adding bi b#%llu to stripe s#%llu\n",
|
2005-04-17 00:20:36 +02:00
|
|
|
(unsigned long long)bi->bi_sector,
|
|
|
|
(unsigned long long)sh->sector);
|
|
|
|
|
|
|
|
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
2005-09-10 01:23:54 +02:00
|
|
|
if (forwrite) {
|
2005-04-17 00:20:36 +02:00
|
|
|
bip = &sh->dev[dd_idx].towrite;
|
2005-09-10 01:23:54 +02:00
|
|
|
if (*bip == NULL && sh->dev[dd_idx].written == NULL)
|
|
|
|
firstwrite = 1;
|
|
|
|
} else
|
2005-04-17 00:20:36 +02:00
|
|
|
bip = &sh->dev[dd_idx].toread;
|
|
|
|
while (*bip && (*bip)->bi_sector < bi->bi_sector) {
|
|
|
|
if ((*bip)->bi_sector + ((*bip)->bi_size >> 9) > bi->bi_sector)
|
|
|
|
goto overlap;
|
|
|
|
bip = & (*bip)->bi_next;
|
|
|
|
}
|
|
|
|
if (*bip && (*bip)->bi_sector < bi->bi_sector + ((bi->bi_size)>>9))
|
|
|
|
goto overlap;
|
|
|
|
|
2006-04-02 13:31:42 +02:00
|
|
|
BUG_ON(*bip && bi->bi_next && (*bip) != bi->bi_next);
|
2005-04-17 00:20:36 +02:00
|
|
|
if (*bip)
|
|
|
|
bi->bi_next = *bip;
|
|
|
|
*bip = bi;
|
2008-08-15 10:41:18 +02:00
|
|
|
bi->bi_phys_segments++;
|
2005-09-10 01:23:54 +02:00
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
if (forwrite) {
|
|
|
|
/* check if page is covered */
|
|
|
|
sector_t sector = sh->dev[dd_idx].sector;
|
|
|
|
for (bi=sh->dev[dd_idx].towrite;
|
|
|
|
sector < sh->dev[dd_idx].sector + STRIPE_SECTORS &&
|
|
|
|
bi && bi->bi_sector <= sector;
|
|
|
|
bi = r5_next_bio(bi, sh->dev[dd_idx].sector)) {
|
|
|
|
if (bi->bi_sector + (bi->bi_size>>9) >= sector)
|
|
|
|
sector = bi->bi_sector + (bi->bi_size>>9);
|
|
|
|
}
|
|
|
|
if (sector >= sh->dev[dd_idx].sector + STRIPE_SECTORS)
|
|
|
|
set_bit(R5_OVERWRITE, &sh->dev[dd_idx].flags);
|
|
|
|
}
|
2011-07-26 03:20:35 +02:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
|
|
|
|
pr_debug("added bi b#%llu to stripe s#%llu, disk %d.\n",
|
|
|
|
(unsigned long long)(*bip)->bi_sector,
|
|
|
|
(unsigned long long)sh->sector, dd_idx);
|
|
|
|
|
|
|
|
if (conf->mddev->bitmap && firstwrite) {
|
|
|
|
bitmap_startwrite(conf->mddev->bitmap, sh->sector,
|
|
|
|
STRIPE_SECTORS, 0);
|
|
|
|
sh->bm_seq = conf->seq_flush+1;
|
|
|
|
set_bit(STRIPE_BIT_DELAY, &sh->state);
|
|
|
|
}
|
2005-04-17 00:20:36 +02:00
|
|
|
return 1;
|
|
|
|
|
|
|
|
overlap:
|
|
|
|
set_bit(R5_Overlap, &sh->dev[dd_idx].flags);
|
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2006-03-27 11:18:10 +02:00
|
|
|
static void end_reshape(raid5_conf_t *conf);
|
|
|
|
|
2009-03-31 05:39:38 +02:00
|
|
|
static void stripe_set_idx(sector_t stripe, raid5_conf_t *conf, int previous,
|
|
|
|
struct stripe_head *sh)
|
2006-03-27 11:18:09 +02:00
|
|
|
{
|
2009-03-31 06:19:07 +02:00
|
|
|
int sectors_per_chunk =
|
2009-06-18 00:45:55 +02:00
|
|
|
previous ? conf->prev_chunk_sectors : conf->chunk_sectors;
|
2009-03-31 05:39:38 +02:00
|
|
|
int dd_idx;
|
2006-10-03 10:15:50 +02:00
|
|
|
int chunk_offset = sector_div(stripe, sectors_per_chunk);
|
2009-03-31 05:39:38 +02:00
|
|
|
int disks = previous ? conf->previous_raid_disks : conf->raid_disks;
|
2006-10-03 10:15:50 +02:00
|
|
|
|
2009-03-31 05:39:38 +02:00
|
|
|
raid5_compute_sector(conf,
|
|
|
|
stripe * (disks - conf->max_degraded)
|
2006-12-10 11:20:49 +01:00
|
|
|
*sectors_per_chunk + chunk_offset,
|
2009-03-31 05:39:38 +02:00
|
|
|
previous,
|
2009-03-31 05:39:38 +02:00
|
|
|
&dd_idx, sh);
|
2006-03-27 11:18:09 +02:00
|
|
|
}
|
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
static void
|
2008-06-28 01:16:30 +02:00
|
|
|
handle_failed_stripe(raid5_conf_t *conf, struct stripe_head *sh,
|
2007-07-09 20:56:43 +02:00
|
|
|
struct stripe_head_state *s, int disks,
|
|
|
|
struct bio **return_bi)
|
|
|
|
{
|
|
|
|
int i;
|
|
|
|
for (i = disks; i--; ) {
|
|
|
|
struct bio *bi;
|
|
|
|
int bitmap_end = 0;
|
|
|
|
|
|
|
|
if (test_bit(R5_ReadError, &sh->dev[i].flags)) {
|
|
|
|
mdk_rdev_t *rdev;
|
|
|
|
rcu_read_lock();
|
|
|
|
rdev = rcu_dereference(conf->disks[i].rdev);
|
|
|
|
if (rdev && test_bit(In_sync, &rdev->flags))
|
|
|
|
/* multiple read failures in one stripe */
|
|
|
|
md_error(conf->mddev, rdev);
|
|
|
|
rcu_read_unlock();
|
|
|
|
}
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
|
|
|
/* fail all writes first */
|
|
|
|
bi = sh->dev[i].towrite;
|
|
|
|
sh->dev[i].towrite = NULL;
|
|
|
|
if (bi) {
|
|
|
|
s->to_write--;
|
|
|
|
bitmap_end = 1;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (test_and_clear_bit(R5_Overlap, &sh->dev[i].flags))
|
|
|
|
wake_up(&conf->wait_for_overlap);
|
|
|
|
|
|
|
|
while (bi && bi->bi_sector <
|
|
|
|
sh->dev[i].sector + STRIPE_SECTORS) {
|
|
|
|
struct bio *nextbi = r5_next_bio(bi, sh->dev[i].sector);
|
|
|
|
clear_bit(BIO_UPTODATE, &bi->bi_flags);
|
2008-08-15 10:41:18 +02:00
|
|
|
if (!raid5_dec_bi_phys_segments(bi)) {
|
2007-07-09 20:56:43 +02:00
|
|
|
md_write_end(conf->mddev);
|
|
|
|
bi->bi_next = *return_bi;
|
|
|
|
*return_bi = bi;
|
|
|
|
}
|
|
|
|
bi = nextbi;
|
|
|
|
}
|
|
|
|
/* and fail all 'written' */
|
|
|
|
bi = sh->dev[i].written;
|
|
|
|
sh->dev[i].written = NULL;
|
|
|
|
if (bi) bitmap_end = 1;
|
|
|
|
while (bi && bi->bi_sector <
|
|
|
|
sh->dev[i].sector + STRIPE_SECTORS) {
|
|
|
|
struct bio *bi2 = r5_next_bio(bi, sh->dev[i].sector);
|
|
|
|
clear_bit(BIO_UPTODATE, &bi->bi_flags);
|
2008-08-15 10:41:18 +02:00
|
|
|
if (!raid5_dec_bi_phys_segments(bi)) {
|
2007-07-09 20:56:43 +02:00
|
|
|
md_write_end(conf->mddev);
|
|
|
|
bi->bi_next = *return_bi;
|
|
|
|
*return_bi = bi;
|
|
|
|
}
|
|
|
|
bi = bi2;
|
|
|
|
}
|
|
|
|
|
2007-01-02 21:52:31 +01:00
|
|
|
/* fail any reads if this device is non-operational and
|
|
|
|
* the data has not reached the cache yet.
|
|
|
|
*/
|
|
|
|
if (!test_bit(R5_Wantfill, &sh->dev[i].flags) &&
|
|
|
|
(!test_bit(R5_Insync, &sh->dev[i].flags) ||
|
|
|
|
test_bit(R5_ReadError, &sh->dev[i].flags))) {
|
2007-07-09 20:56:43 +02:00
|
|
|
bi = sh->dev[i].toread;
|
|
|
|
sh->dev[i].toread = NULL;
|
|
|
|
if (test_and_clear_bit(R5_Overlap, &sh->dev[i].flags))
|
|
|
|
wake_up(&conf->wait_for_overlap);
|
|
|
|
if (bi) s->to_read--;
|
|
|
|
while (bi && bi->bi_sector <
|
|
|
|
sh->dev[i].sector + STRIPE_SECTORS) {
|
|
|
|
struct bio *nextbi =
|
|
|
|
r5_next_bio(bi, sh->dev[i].sector);
|
|
|
|
clear_bit(BIO_UPTODATE, &bi->bi_flags);
|
2008-08-15 10:41:18 +02:00
|
|
|
if (!raid5_dec_bi_phys_segments(bi)) {
|
2007-07-09 20:56:43 +02:00
|
|
|
bi->bi_next = *return_bi;
|
|
|
|
*return_bi = bi;
|
|
|
|
}
|
|
|
|
bi = nextbi;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
if (bitmap_end)
|
|
|
|
bitmap_endwrite(conf->mddev->bitmap, sh->sector,
|
|
|
|
STRIPE_SECTORS, 0, 0);
|
|
|
|
}
|
|
|
|
|
2008-04-28 11:15:53 +02:00
|
|
|
if (test_and_clear_bit(STRIPE_FULL_WRITE, &sh->state))
|
|
|
|
if (atomic_dec_and_test(&conf->pending_full_writes))
|
|
|
|
md_wakeup_thread(conf->mddev->thread);
|
2007-07-09 20:56:43 +02:00
|
|
|
}
|
|
|
|
|
2008-06-28 01:16:30 +02:00
|
|
|
/* fetch_block5 - checks the given member device to see if its data needs
|
|
|
|
* to be read or computed to satisfy a request.
|
|
|
|
*
|
|
|
|
* Returns 1 when no more member devices need to be checked, otherwise returns
|
|
|
|
* 0 to tell the loop in handle_stripe_fill5 to continue
|
2007-01-02 21:52:30 +01:00
|
|
|
*/
|
2008-06-28 01:16:30 +02:00
|
|
|
static int fetch_block5(struct stripe_head *sh, struct stripe_head_state *s,
|
|
|
|
int disk_idx, int disks)
|
2007-01-02 21:52:30 +01:00
|
|
|
{
|
|
|
|
struct r5dev *dev = &sh->dev[disk_idx];
|
2011-07-26 03:35:19 +02:00
|
|
|
struct r5dev *failed_dev = &sh->dev[s->failed_num[0]];
|
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
/* is the data in this block needed, and can we get it? */
|
|
|
|
if (!test_bit(R5_LOCKED, &dev->flags) &&
|
2008-06-28 01:16:30 +02:00
|
|
|
!test_bit(R5_UPTODATE, &dev->flags) &&
|
|
|
|
(dev->toread ||
|
|
|
|
(dev->towrite && !test_bit(R5_OVERWRITE, &dev->flags)) ||
|
|
|
|
s->syncing || s->expanding ||
|
2011-07-27 03:00:36 +02:00
|
|
|
(s->failed && failed_dev->toread) ||
|
|
|
|
(s->failed && failed_dev->towrite &&
|
2008-06-28 01:16:30 +02:00
|
|
|
!test_bit(R5_OVERWRITE, &failed_dev->flags)))))) {
|
2008-06-28 00:32:03 +02:00
|
|
|
/* We would like to get this block, possibly by computing it,
|
|
|
|
* otherwise read it if the backing disk is insync
|
2007-01-02 21:52:30 +01:00
|
|
|
*/
|
|
|
|
if ((s->uptodate == disks - 1) &&
|
2011-07-26 03:35:19 +02:00
|
|
|
(s->failed && disk_idx == s->failed_num[0])) {
|
2008-06-28 00:32:03 +02:00
|
|
|
set_bit(STRIPE_COMPUTE_RUN, &sh->state);
|
|
|
|
set_bit(STRIPE_OP_COMPUTE_BLK, &s->ops_request);
|
2007-01-02 21:52:30 +01:00
|
|
|
set_bit(R5_Wantcompute, &dev->flags);
|
|
|
|
sh->ops.target = disk_idx;
|
2009-07-14 22:40:19 +02:00
|
|
|
sh->ops.target2 = -1;
|
2007-01-02 21:52:30 +01:00
|
|
|
s->req_compute = 1;
|
|
|
|
/* Careful: from this point on 'uptodate' is in the eye
|
2009-07-14 22:40:19 +02:00
|
|
|
* of raid_run_ops which services 'compute' operations
|
2007-01-02 21:52:30 +01:00
|
|
|
* before writes. R5_Wantcompute flags a block that will
|
|
|
|
* be R5_UPTODATE by the time it is needed for a
|
|
|
|
* subsequent operation.
|
|
|
|
*/
|
|
|
|
s->uptodate++;
|
2008-06-28 01:16:30 +02:00
|
|
|
return 1; /* uptodate + compute == disks */
|
2008-07-10 13:54:57 +02:00
|
|
|
} else if (test_bit(R5_Insync, &dev->flags)) {
|
2007-01-02 21:52:30 +01:00
|
|
|
set_bit(R5_LOCKED, &dev->flags);
|
|
|
|
set_bit(R5_Wantread, &dev->flags);
|
|
|
|
s->locked++;
|
|
|
|
pr_debug("Reading block %d (sync=%d)\n", disk_idx,
|
|
|
|
s->syncing);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2008-06-28 01:16:30 +02:00
|
|
|
return 0;
|
2007-01-02 21:52:30 +01:00
|
|
|
}
|
|
|
|
|
2008-06-28 01:16:30 +02:00
|
|
|
/**
|
|
|
|
* handle_stripe_fill5 - read or compute data to satisfy pending requests.
|
|
|
|
*/
|
|
|
|
static void handle_stripe_fill5(struct stripe_head *sh,
|
2007-07-09 20:56:43 +02:00
|
|
|
struct stripe_head_state *s, int disks)
|
|
|
|
{
|
|
|
|
int i;
|
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
/* look for blocks to read/compute, skip this if a compute
|
|
|
|
* is already in flight, or if the stripe contents are in the
|
|
|
|
* midst of changing due to a write
|
|
|
|
*/
|
2008-06-28 00:32:03 +02:00
|
|
|
if (!test_bit(STRIPE_COMPUTE_RUN, &sh->state) && !sh->check_state &&
|
2008-06-28 01:16:30 +02:00
|
|
|
!sh->reconstruct_state)
|
2007-01-02 21:52:30 +01:00
|
|
|
for (i = disks; i--; )
|
2008-06-28 01:16:30 +02:00
|
|
|
if (fetch_block5(sh, s, i, disks))
|
2007-01-02 21:52:30 +01:00
|
|
|
break;
|
2007-07-09 20:56:43 +02:00
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
}
|
|
|
|
|
2009-08-30 04:13:12 +02:00
|
|
|
/* fetch_block6 - checks the given member device to see if its data needs
|
|
|
|
* to be read or computed to satisfy a request.
|
|
|
|
*
|
|
|
|
* Returns 1 when no more member devices need to be checked, otherwise returns
|
|
|
|
* 0 to tell the loop in handle_stripe_fill6 to continue
|
|
|
|
*/
|
|
|
|
static int fetch_block6(struct stripe_head *sh, struct stripe_head_state *s,
|
2011-07-26 03:35:19 +02:00
|
|
|
int disk_idx, int disks)
|
2007-07-09 20:56:43 +02:00
|
|
|
{
|
2009-08-30 04:13:12 +02:00
|
|
|
struct r5dev *dev = &sh->dev[disk_idx];
|
2011-07-26 03:35:19 +02:00
|
|
|
struct r5dev *fdev[2] = { &sh->dev[s->failed_num[0]],
|
|
|
|
&sh->dev[s->failed_num[1]] };
|
2009-08-30 04:13:12 +02:00
|
|
|
|
|
|
|
if (!test_bit(R5_LOCKED, &dev->flags) &&
|
|
|
|
!test_bit(R5_UPTODATE, &dev->flags) &&
|
|
|
|
(dev->toread ||
|
|
|
|
(dev->towrite && !test_bit(R5_OVERWRITE, &dev->flags)) ||
|
|
|
|
s->syncing || s->expanding ||
|
2011-07-27 03:00:36 +02:00
|
|
|
(s->failed >= 1 && fdev[0]->toread) ||
|
|
|
|
(s->failed >= 2 && fdev[1]->toread) ||
|
|
|
|
(s->failed && s->to_write)) {
|
2009-08-30 04:13:12 +02:00
|
|
|
/* we would like to get this block, possibly by computing it,
|
|
|
|
* otherwise read it if the backing disk is insync
|
|
|
|
*/
|
|
|
|
BUG_ON(test_bit(R5_Wantcompute, &dev->flags));
|
|
|
|
BUG_ON(test_bit(R5_Wantread, &dev->flags));
|
|
|
|
if ((s->uptodate == disks - 1) &&
|
2011-07-26 03:35:19 +02:00
|
|
|
(s->failed && (disk_idx == s->failed_num[0] ||
|
|
|
|
disk_idx == s->failed_num[1]))) {
|
2009-08-30 04:13:12 +02:00
|
|
|
/* have disk failed, and we're requested to fetch it;
|
|
|
|
* do compute it
|
2007-07-09 20:56:43 +02:00
|
|
|
*/
|
2009-08-30 04:13:12 +02:00
|
|
|
pr_debug("Computing stripe %llu block %d\n",
|
|
|
|
(unsigned long long)sh->sector, disk_idx);
|
|
|
|
set_bit(STRIPE_COMPUTE_RUN, &sh->state);
|
|
|
|
set_bit(STRIPE_OP_COMPUTE_BLK, &s->ops_request);
|
|
|
|
set_bit(R5_Wantcompute, &dev->flags);
|
|
|
|
sh->ops.target = disk_idx;
|
|
|
|
sh->ops.target2 = -1; /* no 2nd target */
|
|
|
|
s->req_compute = 1;
|
|
|
|
s->uptodate++;
|
|
|
|
return 1;
|
|
|
|
} else if (s->uptodate == disks-2 && s->failed >= 2) {
|
|
|
|
/* Computing 2-failure is *very* expensive; only
|
|
|
|
* do it if failed >= 2
|
|
|
|
*/
|
|
|
|
int other;
|
|
|
|
for (other = disks; other--; ) {
|
|
|
|
if (other == disk_idx)
|
|
|
|
continue;
|
|
|
|
if (!test_bit(R5_UPTODATE,
|
|
|
|
&sh->dev[other].flags))
|
|
|
|
break;
|
2007-07-09 20:56:43 +02:00
|
|
|
}
|
2009-08-30 04:13:12 +02:00
|
|
|
BUG_ON(other < 0);
|
|
|
|
pr_debug("Computing stripe %llu blocks %d,%d\n",
|
|
|
|
(unsigned long long)sh->sector,
|
|
|
|
disk_idx, other);
|
|
|
|
set_bit(STRIPE_COMPUTE_RUN, &sh->state);
|
|
|
|
set_bit(STRIPE_OP_COMPUTE_BLK, &s->ops_request);
|
|
|
|
set_bit(R5_Wantcompute, &sh->dev[disk_idx].flags);
|
|
|
|
set_bit(R5_Wantcompute, &sh->dev[other].flags);
|
|
|
|
sh->ops.target = disk_idx;
|
|
|
|
sh->ops.target2 = other;
|
|
|
|
s->uptodate += 2;
|
|
|
|
s->req_compute = 1;
|
|
|
|
return 1;
|
|
|
|
} else if (test_bit(R5_Insync, &dev->flags)) {
|
|
|
|
set_bit(R5_LOCKED, &dev->flags);
|
|
|
|
set_bit(R5_Wantread, &dev->flags);
|
|
|
|
s->locked++;
|
|
|
|
pr_debug("Reading block %d (sync=%d)\n",
|
|
|
|
disk_idx, s->syncing);
|
2007-07-09 20:56:43 +02:00
|
|
|
}
|
|
|
|
}
|
2009-08-30 04:13:12 +02:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* handle_stripe_fill6 - read or compute data to satisfy pending requests.
|
|
|
|
*/
|
|
|
|
static void handle_stripe_fill6(struct stripe_head *sh,
|
2011-07-26 03:35:19 +02:00
|
|
|
struct stripe_head_state *s,
|
2009-08-30 04:13:12 +02:00
|
|
|
int disks)
|
|
|
|
{
|
|
|
|
int i;
|
|
|
|
|
|
|
|
/* look for blocks to read/compute, skip this if a compute
|
|
|
|
* is already in flight, or if the stripe contents are in the
|
|
|
|
* midst of changing due to a write
|
|
|
|
*/
|
|
|
|
if (!test_bit(STRIPE_COMPUTE_RUN, &sh->state) && !sh->check_state &&
|
|
|
|
!sh->reconstruct_state)
|
|
|
|
for (i = disks; i--; )
|
2011-07-26 03:35:19 +02:00
|
|
|
if (fetch_block6(sh, s, i, disks))
|
2009-08-30 04:13:12 +02:00
|
|
|
break;
|
2007-07-09 20:56:43 +02:00
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2008-06-28 01:16:30 +02:00
|
|
|
/* handle_stripe_clean_event
|
2007-07-09 20:56:43 +02:00
|
|
|
* any written block on an uptodate or failed drive can be returned.
|
|
|
|
* Note that if we 'wrote' to a failed drive, it will be UPTODATE, but
|
|
|
|
* never LOCKED, so we don't need to test 'failed' directly.
|
|
|
|
*/
|
2008-06-28 01:16:30 +02:00
|
|
|
static void handle_stripe_clean_event(raid5_conf_t *conf,
|
2007-07-09 20:56:43 +02:00
|
|
|
struct stripe_head *sh, int disks, struct bio **return_bi)
|
|
|
|
{
|
|
|
|
int i;
|
|
|
|
struct r5dev *dev;
|
|
|
|
|
|
|
|
for (i = disks; i--; )
|
|
|
|
if (sh->dev[i].written) {
|
|
|
|
dev = &sh->dev[i];
|
|
|
|
if (!test_bit(R5_LOCKED, &dev->flags) &&
|
|
|
|
test_bit(R5_UPTODATE, &dev->flags)) {
|
|
|
|
/* We can return any write requests */
|
|
|
|
struct bio *wbi, *wbi2;
|
|
|
|
int bitmap_end = 0;
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("Return write for disc %d\n", i);
|
2007-07-09 20:56:43 +02:00
|
|
|
spin_lock_irq(&conf->device_lock);
|
|
|
|
wbi = dev->written;
|
|
|
|
dev->written = NULL;
|
|
|
|
while (wbi && wbi->bi_sector <
|
|
|
|
dev->sector + STRIPE_SECTORS) {
|
|
|
|
wbi2 = r5_next_bio(wbi, dev->sector);
|
2008-08-15 10:41:18 +02:00
|
|
|
if (!raid5_dec_bi_phys_segments(wbi)) {
|
2007-07-09 20:56:43 +02:00
|
|
|
md_write_end(conf->mddev);
|
|
|
|
wbi->bi_next = *return_bi;
|
|
|
|
*return_bi = wbi;
|
|
|
|
}
|
|
|
|
wbi = wbi2;
|
|
|
|
}
|
|
|
|
if (dev->towrite == NULL)
|
|
|
|
bitmap_end = 1;
|
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
if (bitmap_end)
|
|
|
|
bitmap_endwrite(conf->mddev->bitmap,
|
|
|
|
sh->sector,
|
|
|
|
STRIPE_SECTORS,
|
|
|
|
!test_bit(STRIPE_DEGRADED, &sh->state),
|
|
|
|
0);
|
|
|
|
}
|
|
|
|
}
|
2008-04-28 11:15:53 +02:00
|
|
|
|
|
|
|
if (test_and_clear_bit(STRIPE_FULL_WRITE, &sh->state))
|
|
|
|
if (atomic_dec_and_test(&conf->pending_full_writes))
|
|
|
|
md_wakeup_thread(conf->mddev->thread);
|
2007-07-09 20:56:43 +02:00
|
|
|
}
|
|
|
|
|
2008-06-28 01:16:30 +02:00
|
|
|
static void handle_stripe_dirtying5(raid5_conf_t *conf,
|
2007-07-09 20:56:43 +02:00
|
|
|
struct stripe_head *sh, struct stripe_head_state *s, int disks)
|
|
|
|
{
|
|
|
|
int rmw = 0, rcw = 0, i;
|
|
|
|
for (i = disks; i--; ) {
|
|
|
|
/* would I have to read this buffer for read_modify_write */
|
|
|
|
struct r5dev *dev = &sh->dev[i];
|
|
|
|
if ((dev->towrite || i == sh->pd_idx) &&
|
|
|
|
!test_bit(R5_LOCKED, &dev->flags) &&
|
2007-01-02 21:52:30 +01:00
|
|
|
!(test_bit(R5_UPTODATE, &dev->flags) ||
|
|
|
|
test_bit(R5_Wantcompute, &dev->flags))) {
|
2007-07-09 20:56:43 +02:00
|
|
|
if (test_bit(R5_Insync, &dev->flags))
|
|
|
|
rmw++;
|
|
|
|
else
|
|
|
|
rmw += 2*disks; /* cannot read it */
|
|
|
|
}
|
|
|
|
/* Would I have to read this buffer for reconstruct_write */
|
|
|
|
if (!test_bit(R5_OVERWRITE, &dev->flags) && i != sh->pd_idx &&
|
|
|
|
!test_bit(R5_LOCKED, &dev->flags) &&
|
2007-01-02 21:52:30 +01:00
|
|
|
!(test_bit(R5_UPTODATE, &dev->flags) ||
|
|
|
|
test_bit(R5_Wantcompute, &dev->flags))) {
|
|
|
|
if (test_bit(R5_Insync, &dev->flags)) rcw++;
|
2007-07-09 20:56:43 +02:00
|
|
|
else
|
|
|
|
rcw += 2*disks;
|
|
|
|
}
|
|
|
|
}
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("for sector %llu, rmw=%d rcw=%d\n",
|
2007-07-09 20:56:43 +02:00
|
|
|
(unsigned long long)sh->sector, rmw, rcw);
|
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
if (rmw < rcw && rmw > 0)
|
|
|
|
/* prefer read-modify-write, but need to get some data */
|
|
|
|
for (i = disks; i--; ) {
|
|
|
|
struct r5dev *dev = &sh->dev[i];
|
|
|
|
if ((dev->towrite || i == sh->pd_idx) &&
|
|
|
|
!test_bit(R5_LOCKED, &dev->flags) &&
|
2007-01-02 21:52:30 +01:00
|
|
|
!(test_bit(R5_UPTODATE, &dev->flags) ||
|
|
|
|
test_bit(R5_Wantcompute, &dev->flags)) &&
|
2007-07-09 20:56:43 +02:00
|
|
|
test_bit(R5_Insync, &dev->flags)) {
|
|
|
|
if (
|
|
|
|
test_bit(STRIPE_PREREAD_ACTIVE, &sh->state)) {
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("Read_old block "
|
2007-07-09 20:56:43 +02:00
|
|
|
"%d for r-m-w\n", i);
|
|
|
|
set_bit(R5_LOCKED, &dev->flags);
|
|
|
|
set_bit(R5_Wantread, &dev->flags);
|
|
|
|
s->locked++;
|
|
|
|
} else {
|
|
|
|
set_bit(STRIPE_DELAYED, &sh->state);
|
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
if (rcw <= rmw && rcw > 0)
|
|
|
|
/* want reconstruct write, but need to get some data */
|
|
|
|
for (i = disks; i--; ) {
|
|
|
|
struct r5dev *dev = &sh->dev[i];
|
|
|
|
if (!test_bit(R5_OVERWRITE, &dev->flags) &&
|
|
|
|
i != sh->pd_idx &&
|
|
|
|
!test_bit(R5_LOCKED, &dev->flags) &&
|
2007-01-02 21:52:30 +01:00
|
|
|
!(test_bit(R5_UPTODATE, &dev->flags) ||
|
|
|
|
test_bit(R5_Wantcompute, &dev->flags)) &&
|
2007-07-09 20:56:43 +02:00
|
|
|
test_bit(R5_Insync, &dev->flags)) {
|
|
|
|
if (
|
|
|
|
test_bit(STRIPE_PREREAD_ACTIVE, &sh->state)) {
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("Read_old block "
|
2007-07-09 20:56:43 +02:00
|
|
|
"%d for Reconstruct\n", i);
|
|
|
|
set_bit(R5_LOCKED, &dev->flags);
|
|
|
|
set_bit(R5_Wantread, &dev->flags);
|
|
|
|
s->locked++;
|
|
|
|
} else {
|
|
|
|
set_bit(STRIPE_DELAYED, &sh->state);
|
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
/* now if nothing is locked, and if we have enough data,
|
|
|
|
* we can start a write request
|
|
|
|
*/
|
2007-01-02 21:52:30 +01:00
|
|
|
/* since handle_stripe can be called at any time we need to handle the
|
|
|
|
* case where a compute block operation has been submitted and then a
|
2009-07-14 22:40:19 +02:00
|
|
|
* subsequent call wants to start a write request. raid_run_ops only
|
|
|
|
* handles the case where compute block and reconstruct are requested
|
2007-01-02 21:52:30 +01:00
|
|
|
* simultaneously. If this is not the case then new writes need to be
|
|
|
|
* held off until the compute completes.
|
|
|
|
*/
|
2008-06-28 00:32:03 +02:00
|
|
|
if ((s->req_compute || !test_bit(STRIPE_COMPUTE_RUN, &sh->state)) &&
|
|
|
|
(s->locked == 0 && (rcw == 0 || rmw == 0) &&
|
|
|
|
!test_bit(STRIPE_BIT_DELAY, &sh->state)))
|
2009-08-30 04:13:12 +02:00
|
|
|
schedule_reconstruction(sh, s, rcw == 0, 0);
|
2007-07-09 20:56:43 +02:00
|
|
|
}
|
|
|
|
|
2008-06-28 01:16:30 +02:00
|
|
|
static void handle_stripe_dirtying6(raid5_conf_t *conf,
|
2007-07-09 20:56:43 +02:00
|
|
|
struct stripe_head *sh, struct stripe_head_state *s,
|
2011-07-26 03:35:19 +02:00
|
|
|
int disks)
|
2007-07-09 20:56:43 +02:00
|
|
|
{
|
2009-08-30 04:13:12 +02:00
|
|
|
int rcw = 0, pd_idx = sh->pd_idx, i;
|
2009-03-31 06:10:16 +02:00
|
|
|
int qd_idx = sh->qd_idx;
|
2009-08-30 04:13:12 +02:00
|
|
|
|
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
2007-07-09 20:56:43 +02:00
|
|
|
for (i = disks; i--; ) {
|
|
|
|
struct r5dev *dev = &sh->dev[i];
|
2009-08-30 04:13:12 +02:00
|
|
|
/* check if we haven't enough data */
|
|
|
|
if (!test_bit(R5_OVERWRITE, &dev->flags) &&
|
|
|
|
i != pd_idx && i != qd_idx &&
|
|
|
|
!test_bit(R5_LOCKED, &dev->flags) &&
|
|
|
|
!(test_bit(R5_UPTODATE, &dev->flags) ||
|
|
|
|
test_bit(R5_Wantcompute, &dev->flags))) {
|
|
|
|
rcw++;
|
|
|
|
if (!test_bit(R5_Insync, &dev->flags))
|
|
|
|
continue; /* it's a failed drive */
|
|
|
|
|
|
|
|
if (
|
|
|
|
test_bit(STRIPE_PREREAD_ACTIVE, &sh->state)) {
|
|
|
|
pr_debug("Read_old stripe %llu "
|
|
|
|
"block %d for Reconstruct\n",
|
|
|
|
(unsigned long long)sh->sector, i);
|
|
|
|
set_bit(R5_LOCKED, &dev->flags);
|
|
|
|
set_bit(R5_Wantread, &dev->flags);
|
|
|
|
s->locked++;
|
|
|
|
} else {
|
|
|
|
pr_debug("Request delayed stripe %llu "
|
|
|
|
"block %d for Reconstruct\n",
|
|
|
|
(unsigned long long)sh->sector, i);
|
|
|
|
set_bit(STRIPE_DELAYED, &sh->state);
|
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
2007-07-09 20:56:43 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
/* now if nothing is locked, and if we have enough data, we can start a
|
|
|
|
* write request
|
|
|
|
*/
|
2009-08-30 04:13:12 +02:00
|
|
|
if ((s->req_compute || !test_bit(STRIPE_COMPUTE_RUN, &sh->state)) &&
|
|
|
|
s->locked == 0 && rcw == 0 &&
|
2007-07-09 20:56:43 +02:00
|
|
|
!test_bit(STRIPE_BIT_DELAY, &sh->state)) {
|
2009-08-30 04:13:12 +02:00
|
|
|
schedule_reconstruction(sh, s, 1, 0);
|
2007-07-09 20:56:43 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
static void handle_parity_checks5(raid5_conf_t *conf, struct stripe_head *sh,
|
|
|
|
struct stripe_head_state *s, int disks)
|
|
|
|
{
|
2008-06-28 00:31:57 +02:00
|
|
|
struct r5dev *dev = NULL;
|
2008-04-11 06:29:27 +02:00
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
2007-01-02 21:52:31 +01:00
|
|
|
|
2008-06-28 00:31:57 +02:00
|
|
|
switch (sh->check_state) {
|
|
|
|
case check_state_idle:
|
|
|
|
/* start a new check operation if there are no failures */
|
2008-04-11 06:29:27 +02:00
|
|
|
if (s->failed == 0) {
|
|
|
|
BUG_ON(s->uptodate != disks);
|
2008-06-28 00:31:57 +02:00
|
|
|
sh->check_state = check_state_run;
|
|
|
|
set_bit(STRIPE_OP_CHECK, &s->ops_request);
|
2008-04-11 06:29:27 +02:00
|
|
|
clear_bit(R5_UPTODATE, &sh->dev[sh->pd_idx].flags);
|
|
|
|
s->uptodate--;
|
2008-06-28 00:31:57 +02:00
|
|
|
break;
|
2008-04-11 06:29:27 +02:00
|
|
|
}
|
2011-07-26 03:35:19 +02:00
|
|
|
dev = &sh->dev[s->failed_num[0]];
|
2008-06-28 00:31:57 +02:00
|
|
|
/* fall through */
|
|
|
|
case check_state_compute_result:
|
|
|
|
sh->check_state = check_state_idle;
|
|
|
|
if (!dev)
|
|
|
|
dev = &sh->dev[sh->pd_idx];
|
|
|
|
|
|
|
|
/* check that a write has not made the stripe insync */
|
|
|
|
if (test_bit(STRIPE_INSYNC, &sh->state))
|
|
|
|
break;
|
2008-05-12 23:02:12 +02:00
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
/* either failed parity check, or recovery is happening */
|
|
|
|
BUG_ON(!test_bit(R5_UPTODATE, &dev->flags));
|
|
|
|
BUG_ON(s->uptodate != disks);
|
|
|
|
|
|
|
|
set_bit(R5_LOCKED, &dev->flags);
|
2008-06-28 00:31:57 +02:00
|
|
|
s->locked++;
|
2007-07-09 20:56:43 +02:00
|
|
|
set_bit(R5_Wantwrite, &dev->flags);
|
2007-01-02 21:52:31 +01:00
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
clear_bit(STRIPE_DEGRADED, &sh->state);
|
|
|
|
set_bit(STRIPE_INSYNC, &sh->state);
|
2008-06-28 00:31:57 +02:00
|
|
|
break;
|
|
|
|
case check_state_run:
|
|
|
|
break; /* we will be called again upon completion */
|
|
|
|
case check_state_check_result:
|
|
|
|
sh->check_state = check_state_idle;
|
|
|
|
|
|
|
|
/* if a failure occurred during the check operation, leave
|
|
|
|
* STRIPE_INSYNC not set and let the stripe be handled again
|
|
|
|
*/
|
|
|
|
if (s->failed)
|
|
|
|
break;
|
|
|
|
|
|
|
|
/* handle a successful check operation, if parity is correct
|
|
|
|
* we are done. Otherwise update the mismatch count and repair
|
|
|
|
* parity if !MD_RECOVERY_CHECK
|
|
|
|
*/
|
2009-08-30 04:09:26 +02:00
|
|
|
if ((sh->ops.zero_sum_result & SUM_CHECK_P_RESULT) == 0)
|
2008-06-28 00:31:57 +02:00
|
|
|
/* parity is correct (on disc,
|
|
|
|
* not in buffer any more)
|
|
|
|
*/
|
|
|
|
set_bit(STRIPE_INSYNC, &sh->state);
|
|
|
|
else {
|
|
|
|
conf->mddev->resync_mismatches += STRIPE_SECTORS;
|
|
|
|
if (test_bit(MD_RECOVERY_CHECK, &conf->mddev->recovery))
|
|
|
|
/* don't try to repair!! */
|
|
|
|
set_bit(STRIPE_INSYNC, &sh->state);
|
|
|
|
else {
|
|
|
|
sh->check_state = check_state_compute_run;
|
2008-06-28 00:32:03 +02:00
|
|
|
set_bit(STRIPE_COMPUTE_RUN, &sh->state);
|
2008-06-28 00:31:57 +02:00
|
|
|
set_bit(STRIPE_OP_COMPUTE_BLK, &s->ops_request);
|
|
|
|
set_bit(R5_Wantcompute,
|
|
|
|
&sh->dev[sh->pd_idx].flags);
|
|
|
|
sh->ops.target = sh->pd_idx;
|
2009-07-14 22:40:19 +02:00
|
|
|
sh->ops.target2 = -1;
|
2008-06-28 00:31:57 +02:00
|
|
|
s->uptodate++;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
break;
|
|
|
|
case check_state_compute_run:
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
printk(KERN_ERR "%s: unknown check_state: %d sector: %llu\n",
|
|
|
|
__func__, sh->check_state,
|
|
|
|
(unsigned long long) sh->sector);
|
|
|
|
BUG();
|
2007-07-09 20:56:43 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
static void handle_parity_checks6(raid5_conf_t *conf, struct stripe_head *sh,
|
2009-07-14 20:48:22 +02:00
|
|
|
struct stripe_head_state *s,
|
2011-07-26 03:35:19 +02:00
|
|
|
int disks)
|
2007-07-09 20:56:43 +02:00
|
|
|
{
|
|
|
|
int pd_idx = sh->pd_idx;
|
2009-03-31 06:10:16 +02:00
|
|
|
int qd_idx = sh->qd_idx;
|
2009-07-14 22:40:57 +02:00
|
|
|
struct r5dev *dev;
|
2007-07-09 20:56:43 +02:00
|
|
|
|
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
|
|
|
|
BUG_ON(s->failed > 2);
|
2009-07-14 22:40:57 +02:00
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
/* Want to check and possibly repair P and Q.
|
|
|
|
* However there could be one 'failed' device, in which
|
|
|
|
* case we can only check one of them, possibly using the
|
|
|
|
* other to generate missing data
|
|
|
|
*/
|
|
|
|
|
2009-07-14 22:40:57 +02:00
|
|
|
switch (sh->check_state) {
|
|
|
|
case check_state_idle:
|
|
|
|
/* start a new check operation if there are < 2 failures */
|
2011-07-26 03:35:19 +02:00
|
|
|
if (s->failed == s->q_failed) {
|
2009-07-14 22:40:57 +02:00
|
|
|
/* The only possible failed device holds Q, so it
|
2007-07-09 20:56:43 +02:00
|
|
|
* makes sense to check P (If anything else were failed,
|
|
|
|
* we would have used P to recreate it).
|
|
|
|
*/
|
2009-07-14 22:40:57 +02:00
|
|
|
sh->check_state = check_state_run;
|
2007-07-09 20:56:43 +02:00
|
|
|
}
|
2011-07-26 03:35:19 +02:00
|
|
|
if (!s->q_failed && s->failed < 2) {
|
2009-07-14 22:40:57 +02:00
|
|
|
/* Q is not failed, and we didn't use it to generate
|
2007-07-09 20:56:43 +02:00
|
|
|
* anything, so it makes sense to check it
|
|
|
|
*/
|
2009-07-14 22:40:57 +02:00
|
|
|
if (sh->check_state == check_state_run)
|
|
|
|
sh->check_state = check_state_run_pq;
|
|
|
|
else
|
|
|
|
sh->check_state = check_state_run_q;
|
2007-07-09 20:56:43 +02:00
|
|
|
}
|
|
|
|
|
2009-07-14 22:40:57 +02:00
|
|
|
/* discard potentially stale zero_sum_result */
|
|
|
|
sh->ops.zero_sum_result = 0;
|
2007-07-09 20:56:43 +02:00
|
|
|
|
2009-07-14 22:40:57 +02:00
|
|
|
if (sh->check_state == check_state_run) {
|
|
|
|
/* async_xor_zero_sum destroys the contents of P */
|
|
|
|
clear_bit(R5_UPTODATE, &sh->dev[pd_idx].flags);
|
|
|
|
s->uptodate--;
|
2007-07-09 20:56:43 +02:00
|
|
|
}
|
2009-07-14 22:40:57 +02:00
|
|
|
if (sh->check_state >= check_state_run &&
|
|
|
|
sh->check_state <= check_state_run_pq) {
|
|
|
|
/* async_syndrome_zero_sum preserves P and Q, so
|
|
|
|
* no need to mark them !uptodate here
|
|
|
|
*/
|
|
|
|
set_bit(STRIPE_OP_CHECK, &s->ops_request);
|
|
|
|
break;
|
2007-07-09 20:56:43 +02:00
|
|
|
}
|
|
|
|
|
2009-07-14 22:40:57 +02:00
|
|
|
/* we have 2-disk failure */
|
|
|
|
BUG_ON(s->failed != 2);
|
|
|
|
/* fall through */
|
|
|
|
case check_state_compute_result:
|
|
|
|
sh->check_state = check_state_idle;
|
2007-07-09 20:56:43 +02:00
|
|
|
|
2009-07-14 22:40:57 +02:00
|
|
|
/* check that a write has not made the stripe insync */
|
|
|
|
if (test_bit(STRIPE_INSYNC, &sh->state))
|
|
|
|
break;
|
2007-07-09 20:56:43 +02:00
|
|
|
|
|
|
|
/* now write out any block on a failed drive,
|
2009-07-14 22:40:57 +02:00
|
|
|
* or P or Q if they were recomputed
|
2007-07-09 20:56:43 +02:00
|
|
|
*/
|
2009-07-14 22:40:57 +02:00
|
|
|
BUG_ON(s->uptodate < disks - 1); /* We don't need Q to recover */
|
2007-07-09 20:56:43 +02:00
|
|
|
if (s->failed == 2) {
|
2011-07-26 03:35:19 +02:00
|
|
|
dev = &sh->dev[s->failed_num[1]];
|
2007-07-09 20:56:43 +02:00
|
|
|
s->locked++;
|
|
|
|
set_bit(R5_LOCKED, &dev->flags);
|
|
|
|
set_bit(R5_Wantwrite, &dev->flags);
|
|
|
|
}
|
|
|
|
if (s->failed >= 1) {
|
2011-07-26 03:35:19 +02:00
|
|
|
dev = &sh->dev[s->failed_num[0]];
|
2007-07-09 20:56:43 +02:00
|
|
|
s->locked++;
|
|
|
|
set_bit(R5_LOCKED, &dev->flags);
|
|
|
|
set_bit(R5_Wantwrite, &dev->flags);
|
|
|
|
}
|
2009-07-14 22:40:57 +02:00
|
|
|
if (sh->ops.zero_sum_result & SUM_CHECK_P_RESULT) {
|
2007-07-09 20:56:43 +02:00
|
|
|
dev = &sh->dev[pd_idx];
|
|
|
|
s->locked++;
|
|
|
|
set_bit(R5_LOCKED, &dev->flags);
|
|
|
|
set_bit(R5_Wantwrite, &dev->flags);
|
|
|
|
}
|
2009-07-14 22:40:57 +02:00
|
|
|
if (sh->ops.zero_sum_result & SUM_CHECK_Q_RESULT) {
|
2007-07-09 20:56:43 +02:00
|
|
|
dev = &sh->dev[qd_idx];
|
|
|
|
s->locked++;
|
|
|
|
set_bit(R5_LOCKED, &dev->flags);
|
|
|
|
set_bit(R5_Wantwrite, &dev->flags);
|
|
|
|
}
|
|
|
|
clear_bit(STRIPE_DEGRADED, &sh->state);
|
|
|
|
|
|
|
|
set_bit(STRIPE_INSYNC, &sh->state);
|
2009-07-14 22:40:57 +02:00
|
|
|
break;
|
|
|
|
case check_state_run:
|
|
|
|
case check_state_run_q:
|
|
|
|
case check_state_run_pq:
|
|
|
|
break; /* we will be called again upon completion */
|
|
|
|
case check_state_check_result:
|
|
|
|
sh->check_state = check_state_idle;
|
|
|
|
|
|
|
|
/* handle a successful check operation, if parity is correct
|
|
|
|
* we are done. Otherwise update the mismatch count and repair
|
|
|
|
* parity if !MD_RECOVERY_CHECK
|
|
|
|
*/
|
|
|
|
if (sh->ops.zero_sum_result == 0) {
|
|
|
|
/* both parities are correct */
|
|
|
|
if (!s->failed)
|
|
|
|
set_bit(STRIPE_INSYNC, &sh->state);
|
|
|
|
else {
|
|
|
|
/* in contrast to the raid5 case we can validate
|
|
|
|
* parity, but still have a failure to write
|
|
|
|
* back
|
|
|
|
*/
|
|
|
|
sh->check_state = check_state_compute_result;
|
|
|
|
/* Returning at this point means that we may go
|
|
|
|
* off and bring p and/or q uptodate again so
|
|
|
|
* we make sure to check zero_sum_result again
|
|
|
|
* to verify if p or q need writeback
|
|
|
|
*/
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
conf->mddev->resync_mismatches += STRIPE_SECTORS;
|
|
|
|
if (test_bit(MD_RECOVERY_CHECK, &conf->mddev->recovery))
|
|
|
|
/* don't try to repair!! */
|
|
|
|
set_bit(STRIPE_INSYNC, &sh->state);
|
|
|
|
else {
|
|
|
|
int *target = &sh->ops.target;
|
|
|
|
|
|
|
|
sh->ops.target = -1;
|
|
|
|
sh->ops.target2 = -1;
|
|
|
|
sh->check_state = check_state_compute_run;
|
|
|
|
set_bit(STRIPE_COMPUTE_RUN, &sh->state);
|
|
|
|
set_bit(STRIPE_OP_COMPUTE_BLK, &s->ops_request);
|
|
|
|
if (sh->ops.zero_sum_result & SUM_CHECK_P_RESULT) {
|
|
|
|
set_bit(R5_Wantcompute,
|
|
|
|
&sh->dev[pd_idx].flags);
|
|
|
|
*target = pd_idx;
|
|
|
|
target = &sh->ops.target2;
|
|
|
|
s->uptodate++;
|
|
|
|
}
|
|
|
|
if (sh->ops.zero_sum_result & SUM_CHECK_Q_RESULT) {
|
|
|
|
set_bit(R5_Wantcompute,
|
|
|
|
&sh->dev[qd_idx].flags);
|
|
|
|
*target = qd_idx;
|
|
|
|
s->uptodate++;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
break;
|
|
|
|
case check_state_compute_run:
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
printk(KERN_ERR "%s: unknown check_state: %d sector: %llu\n",
|
|
|
|
__func__, sh->check_state,
|
|
|
|
(unsigned long long) sh->sector);
|
|
|
|
BUG();
|
2007-07-09 20:56:43 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2011-07-27 03:00:36 +02:00
|
|
|
static void handle_stripe_expansion(raid5_conf_t *conf, struct stripe_head *sh)
|
2007-07-09 20:56:43 +02:00
|
|
|
{
|
|
|
|
int i;
|
|
|
|
|
|
|
|
/* We have read all the blocks in this stripe and now we need to
|
|
|
|
* copy some of them into a target stripe for expand.
|
|
|
|
*/
|
2007-01-02 21:52:31 +01:00
|
|
|
struct dma_async_tx_descriptor *tx = NULL;
|
2007-07-09 20:56:43 +02:00
|
|
|
clear_bit(STRIPE_EXPAND_SOURCE, &sh->state);
|
|
|
|
for (i = 0; i < sh->disks; i++)
|
2009-03-31 06:10:16 +02:00
|
|
|
if (i != sh->pd_idx && i != sh->qd_idx) {
|
2009-03-31 05:39:38 +02:00
|
|
|
int dd_idx, j;
|
2007-07-09 20:56:43 +02:00
|
|
|
struct stripe_head *sh2;
|
2009-06-03 20:43:59 +02:00
|
|
|
struct async_submit_ctl submit;
|
2007-07-09 20:56:43 +02:00
|
|
|
|
2009-03-31 06:19:07 +02:00
|
|
|
sector_t bn = compute_blocknr(sh, i, 1);
|
2009-03-31 05:39:38 +02:00
|
|
|
sector_t s = raid5_compute_sector(conf, bn, 0,
|
|
|
|
&dd_idx, NULL);
|
2009-06-09 06:39:59 +02:00
|
|
|
sh2 = get_active_stripe(conf, s, 0, 1, 1);
|
2007-07-09 20:56:43 +02:00
|
|
|
if (sh2 == NULL)
|
|
|
|
/* so far only the early blocks of this stripe
|
|
|
|
* have been requested. When later blocks
|
|
|
|
* get requested, we will try again
|
|
|
|
*/
|
|
|
|
continue;
|
|
|
|
if (!test_bit(STRIPE_EXPANDING, &sh2->state) ||
|
|
|
|
test_bit(R5_Expanded, &sh2->dev[dd_idx].flags)) {
|
|
|
|
/* must have already done this block */
|
|
|
|
release_stripe(sh2);
|
|
|
|
continue;
|
|
|
|
}
|
2007-01-02 21:52:31 +01:00
|
|
|
|
|
|
|
/* place all the copies on one channel */
|
2009-06-03 20:43:59 +02:00
|
|
|
init_async_submit(&submit, 0, tx, NULL, NULL, NULL);
|
2007-01-02 21:52:31 +01:00
|
|
|
tx = async_memcpy(sh2->dev[dd_idx].page,
|
2009-04-10 01:16:18 +02:00
|
|
|
sh->dev[i].page, 0, 0, STRIPE_SIZE,
|
2009-06-03 20:43:59 +02:00
|
|
|
&submit);
|
2007-01-02 21:52:31 +01:00
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
set_bit(R5_Expanded, &sh2->dev[dd_idx].flags);
|
|
|
|
set_bit(R5_UPTODATE, &sh2->dev[dd_idx].flags);
|
|
|
|
for (j = 0; j < conf->raid_disks; j++)
|
|
|
|
if (j != sh2->pd_idx &&
|
2011-07-27 03:00:36 +02:00
|
|
|
j != sh2->qd_idx &&
|
2007-07-09 20:56:43 +02:00
|
|
|
!test_bit(R5_Expanded, &sh2->dev[j].flags))
|
|
|
|
break;
|
|
|
|
if (j == conf->raid_disks) {
|
|
|
|
set_bit(STRIPE_EXPAND_READY, &sh2->state);
|
|
|
|
set_bit(STRIPE_HANDLE, &sh2->state);
|
|
|
|
}
|
|
|
|
release_stripe(sh2);
|
2007-01-02 21:52:31 +01:00
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
}
|
2007-09-12 00:23:36 +02:00
|
|
|
/* done submitting copies, wait for them to complete */
|
|
|
|
if (tx) {
|
|
|
|
async_tx_ack(tx);
|
|
|
|
dma_wait_for_async_tx(tx);
|
|
|
|
}
|
2007-07-09 20:56:43 +02:00
|
|
|
}
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2008-04-30 09:52:32 +02:00
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
/*
|
|
|
|
* handle_stripe - do things to a stripe.
|
|
|
|
*
|
|
|
|
* We lock the stripe and then examine the state of various bits
|
|
|
|
* to see what needs to be done.
|
|
|
|
* Possible results:
|
|
|
|
* return some read request which now have data
|
|
|
|
* return some write requests which are safely on disc
|
|
|
|
* schedule a read on some buffers
|
|
|
|
* schedule a write of some buffers
|
|
|
|
* return confirmation of parity correctness
|
|
|
|
*
|
|
|
|
* buffers are taken off read_list or write_list, and bh_cache buffers
|
|
|
|
* get BH_Lock set before the stripe lock is released.
|
|
|
|
*
|
|
|
|
*/
|
2007-07-09 20:56:43 +02:00
|
|
|
|
2011-07-27 03:00:36 +02:00
|
|
|
static int handle_stripe5(struct stripe_head *sh, struct stripe_head_state *s)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
raid5_conf_t *conf = sh->raid_conf;
|
2007-07-09 20:56:43 +02:00
|
|
|
int disks = sh->disks, i;
|
2005-04-17 00:20:36 +02:00
|
|
|
struct r5dev *dev;
|
2008-06-06 07:45:52 +02:00
|
|
|
int prexor;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2008-06-28 00:31:58 +02:00
|
|
|
/* Now to look around and see what can be done */
|
2006-01-06 09:20:24 +01:00
|
|
|
rcu_read_lock();
|
2011-07-26 03:34:20 +02:00
|
|
|
spin_lock_irq(&conf->device_lock);
|
2005-04-17 00:20:36 +02:00
|
|
|
for (i=disks; i--; ) {
|
|
|
|
mdk_rdev_t *rdev;
|
2009-09-23 10:06:41 +02:00
|
|
|
|
|
|
|
dev = &sh->dev[i];
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2007-01-02 21:52:31 +01:00
|
|
|
pr_debug("check %d: state 0x%lx toread %p read %p write %p "
|
|
|
|
"written %p\n", i, dev->flags, dev->toread, dev->read,
|
|
|
|
dev->towrite, dev->written);
|
|
|
|
|
|
|
|
/* maybe we can request a biofill operation
|
|
|
|
*
|
|
|
|
* new wantfill requests are only permitted while
|
2008-06-28 00:31:58 +02:00
|
|
|
* ops_complete_biofill is guaranteed to be inactive
|
2007-01-02 21:52:31 +01:00
|
|
|
*/
|
|
|
|
if (test_bit(R5_UPTODATE, &dev->flags) && dev->toread &&
|
2008-06-28 00:31:58 +02:00
|
|
|
!test_bit(STRIPE_BIOFILL_RUN, &sh->state))
|
2007-01-02 21:52:31 +01:00
|
|
|
set_bit(R5_Wantfill, &dev->flags);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
/* now count some things */
|
2011-07-26 03:35:35 +02:00
|
|
|
if (test_bit(R5_LOCKED, &dev->flags))
|
|
|
|
s->locked++;
|
|
|
|
if (test_bit(R5_UPTODATE, &dev->flags))
|
|
|
|
s->uptodate++;
|
|
|
|
if (test_bit(R5_Wantcompute, &dev->flags))
|
|
|
|
s->compute++;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2007-01-02 21:52:31 +01:00
|
|
|
if (test_bit(R5_Wantfill, &dev->flags))
|
2011-07-26 03:35:35 +02:00
|
|
|
s->to_fill++;
|
2007-01-02 21:52:31 +01:00
|
|
|
else if (dev->toread)
|
2011-07-26 03:35:35 +02:00
|
|
|
s->to_read++;
|
2005-04-17 00:20:36 +02:00
|
|
|
if (dev->towrite) {
|
2011-07-26 03:35:35 +02:00
|
|
|
s->to_write++;
|
2005-04-17 00:20:36 +02:00
|
|
|
if (!test_bit(R5_OVERWRITE, &dev->flags))
|
2011-07-26 03:35:35 +02:00
|
|
|
s->non_overwrite++;
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
2007-07-09 20:56:43 +02:00
|
|
|
if (dev->written)
|
2011-07-26 03:35:35 +02:00
|
|
|
s->written++;
|
2006-01-06 09:20:24 +01:00
|
|
|
rdev = rcu_dereference(conf->disks[i].rdev);
|
2011-07-26 03:35:35 +02:00
|
|
|
if (s->blocked_rdev == NULL &&
|
2008-08-05 07:54:13 +02:00
|
|
|
rdev && unlikely(test_bit(Blocked, &rdev->flags))) {
|
2011-07-26 03:35:35 +02:00
|
|
|
s->blocked_rdev = rdev;
|
2008-04-30 09:52:32 +02:00
|
|
|
atomic_inc(&rdev->nr_pending);
|
|
|
|
}
|
2010-06-17 09:25:21 +02:00
|
|
|
clear_bit(R5_Insync, &dev->flags);
|
|
|
|
if (!rdev)
|
|
|
|
/* Not in-sync */;
|
|
|
|
else if (test_bit(In_sync, &rdev->flags))
|
|
|
|
set_bit(R5_Insync, &dev->flags);
|
|
|
|
else {
|
|
|
|
/* could be in-sync depending on recovery/reshape status */
|
|
|
|
if (sh->sector + STRIPE_SECTORS <= rdev->recovery_offset)
|
|
|
|
set_bit(R5_Insync, &dev->flags);
|
|
|
|
}
|
|
|
|
if (!test_bit(R5_Insync, &dev->flags)) {
|
2006-01-06 09:20:14 +01:00
|
|
|
/* The ReadError flag will just be confusing now */
|
2005-11-09 06:39:22 +01:00
|
|
|
clear_bit(R5_ReadError, &dev->flags);
|
|
|
|
clear_bit(R5_ReWrite, &dev->flags);
|
|
|
|
}
|
2010-06-17 09:25:21 +02:00
|
|
|
if (test_bit(R5_ReadError, &dev->flags))
|
|
|
|
clear_bit(R5_Insync, &dev->flags);
|
|
|
|
if (!test_bit(R5_Insync, &dev->flags)) {
|
2011-07-26 03:35:35 +02:00
|
|
|
s->failed++;
|
|
|
|
s->failed_num[0] = i;
|
2010-06-17 09:25:21 +02:00
|
|
|
}
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
2011-07-26 03:34:20 +02:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
2006-01-06 09:20:24 +01:00
|
|
|
rcu_read_unlock();
|
2007-01-02 21:52:31 +01:00
|
|
|
|
2011-07-26 03:35:35 +02:00
|
|
|
if (unlikely(s->blocked_rdev)) {
|
|
|
|
if (s->syncing || s->expanding || s->expanded ||
|
|
|
|
s->to_write || s->written) {
|
2008-08-05 07:54:13 +02:00
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
2011-07-27 03:00:36 +02:00
|
|
|
return 1;
|
2008-08-05 07:54:13 +02:00
|
|
|
}
|
|
|
|
/* There is nothing for the blocked_rdev to block */
|
2011-07-26 03:35:35 +02:00
|
|
|
rdev_dec_pending(s->blocked_rdev, conf->mddev);
|
|
|
|
s->blocked_rdev = NULL;
|
2008-04-30 09:52:32 +02:00
|
|
|
}
|
|
|
|
|
2011-07-26 03:35:35 +02:00
|
|
|
if (s->to_fill && !test_bit(STRIPE_BIOFILL_RUN, &sh->state)) {
|
|
|
|
set_bit(STRIPE_OP_BIOFILL, &s->ops_request);
|
2008-06-28 00:31:58 +02:00
|
|
|
set_bit(STRIPE_BIOFILL_RUN, &sh->state);
|
|
|
|
}
|
2007-01-02 21:52:31 +01:00
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("locked=%d uptodate=%d to_read=%d"
|
2005-04-17 00:20:36 +02:00
|
|
|
" to_write=%d failed=%d failed_num=%d\n",
|
2011-07-26 03:35:35 +02:00
|
|
|
s->locked, s->uptodate, s->to_read, s->to_write,
|
|
|
|
s->failed, s->failed_num[0]);
|
2005-04-17 00:20:36 +02:00
|
|
|
/* check if the array has lost two devices and, if so, some requests might
|
|
|
|
* need to be failed
|
|
|
|
*/
|
2011-07-26 03:35:35 +02:00
|
|
|
if (s->failed > 1 && s->to_read+s->to_write+s->written)
|
|
|
|
handle_failed_stripe(conf, sh, s, disks, &s->return_bi);
|
|
|
|
if (s->failed > 1 && s->syncing) {
|
2005-04-17 00:20:36 +02:00
|
|
|
md_done_sync(conf->mddev, STRIPE_SECTORS,0);
|
|
|
|
clear_bit(STRIPE_SYNCING, &sh->state);
|
2011-07-26 03:35:35 +02:00
|
|
|
s->syncing = 0;
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
/* might be able to return some write requests if the parity block
|
|
|
|
* is safe, or on a failed drive
|
|
|
|
*/
|
|
|
|
dev = &sh->dev[sh->pd_idx];
|
2011-07-26 03:35:35 +02:00
|
|
|
if (s->written &&
|
|
|
|
((test_bit(R5_Insync, &dev->flags) &&
|
|
|
|
!test_bit(R5_LOCKED, &dev->flags) &&
|
|
|
|
test_bit(R5_UPTODATE, &dev->flags)) ||
|
|
|
|
(s->failed == 1 && s->failed_num[0] == sh->pd_idx)))
|
|
|
|
handle_stripe_clean_event(conf, sh, disks, &s->return_bi);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
/* Now we might consider reading some blocks, either to check/generate
|
|
|
|
* parity, or to satisfy requests
|
|
|
|
* or to load a block that is being partially written.
|
|
|
|
*/
|
2011-07-26 03:35:35 +02:00
|
|
|
if (s->to_read || s->non_overwrite ||
|
|
|
|
(s->syncing && (s->uptodate + s->compute < disks)) || s->expanding)
|
|
|
|
handle_stripe_fill5(sh, s, disks);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2007-01-02 21:52:30 +01:00
|
|
|
/* Now we check to see if any write operations have recently
|
|
|
|
* completed
|
|
|
|
*/
|
2008-06-06 07:45:52 +02:00
|
|
|
prexor = 0;
|
2008-06-28 00:32:06 +02:00
|
|
|
if (sh->reconstruct_state == reconstruct_state_prexor_drain_result)
|
2008-06-06 07:45:52 +02:00
|
|
|
prexor = 1;
|
2008-06-28 00:32:06 +02:00
|
|
|
if (sh->reconstruct_state == reconstruct_state_drain_result ||
|
|
|
|
sh->reconstruct_state == reconstruct_state_prexor_drain_result) {
|
md: replace STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} with 'reconstruct_states'
From: Dan Williams <dan.j.williams@intel.com>
Track the state of reconstruct operations (recalculating the parity block
usually due to incoming writes, or as part of array expansion) Reduces the
scope of the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags to only tracking whether
a reconstruct operation has been requested via the ops_request field of struct
stripe_head_state.
This is the final step in the removal of ops.{pending,ack,complete,count}, i.e.
the STRIPE_OP_{BIODRAIN,PREXOR,POSTXOR} flags only request an operation and do
not track the state of the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:32:05 +02:00
|
|
|
sh->reconstruct_state = reconstruct_state_idle;
|
2007-01-02 21:52:30 +01:00
|
|
|
|
|
|
|
/* All the 'written' buffers and the parity block are ready to
|
|
|
|
* be written back to disk
|
|
|
|
*/
|
|
|
|
BUG_ON(!test_bit(R5_UPTODATE, &sh->dev[sh->pd_idx].flags));
|
|
|
|
for (i = disks; i--; ) {
|
|
|
|
dev = &sh->dev[i];
|
|
|
|
if (test_bit(R5_LOCKED, &dev->flags) &&
|
|
|
|
(i == sh->pd_idx || dev->written)) {
|
|
|
|
pr_debug("Writing block %d\n", i);
|
|
|
|
set_bit(R5_Wantwrite, &dev->flags);
|
2008-06-06 07:45:52 +02:00
|
|
|
if (prexor)
|
|
|
|
continue;
|
2007-01-02 21:52:30 +01:00
|
|
|
if (!test_bit(R5_Insync, &dev->flags) ||
|
2011-07-26 03:35:35 +02:00
|
|
|
(i == sh->pd_idx && s->failed == 0))
|
2007-01-02 21:52:30 +01:00
|
|
|
set_bit(STRIPE_INSYNC, &sh->state);
|
|
|
|
}
|
|
|
|
}
|
2009-12-14 02:49:50 +01:00
|
|
|
if (test_and_clear_bit(STRIPE_PREREAD_ACTIVE, &sh->state))
|
2011-07-26 03:35:35 +02:00
|
|
|
s->dec_preread_active = 1;
|
2007-01-02 21:52:30 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/* Now to consider new write requests and what else, if anything
|
|
|
|
* should be read. We do not handle new writes when:
|
|
|
|
* 1/ A 'write' operation (copy+xor) is already in flight.
|
|
|
|
* 2/ A 'check' operation is in flight, as it may clobber the parity
|
|
|
|
* block.
|
|
|
|
*/
|
2011-07-26 03:35:35 +02:00
|
|
|
if (s->to_write && !sh->reconstruct_state && !sh->check_state)
|
|
|
|
handle_stripe_dirtying5(conf, sh, s, disks);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
/* maybe we need to check and possibly fix the parity for this stripe
|
2007-01-02 21:52:31 +01:00
|
|
|
* Any reads will already have been scheduled, so we just see if enough
|
|
|
|
* data is available. The parity check is held off while parity
|
|
|
|
* dependent operations are in flight.
|
2005-04-17 00:20:36 +02:00
|
|
|
*/
|
2008-06-28 00:31:57 +02:00
|
|
|
if (sh->check_state ||
|
2011-07-26 03:35:35 +02:00
|
|
|
(s->syncing && s->locked == 0 &&
|
2008-06-28 00:32:03 +02:00
|
|
|
!test_bit(STRIPE_COMPUTE_RUN, &sh->state) &&
|
2008-06-28 00:31:57 +02:00
|
|
|
!test_bit(STRIPE_INSYNC, &sh->state)))
|
2011-07-26 03:35:35 +02:00
|
|
|
handle_parity_checks5(conf, sh, s, disks);
|
2011-07-27 03:00:36 +02:00
|
|
|
return 0;
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
2011-07-27 03:00:36 +02:00
|
|
|
static int handle_stripe6(struct stripe_head *sh, struct stripe_head_state *s)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
2009-03-31 05:33:13 +02:00
|
|
|
raid5_conf_t *conf = sh->raid_conf;
|
2007-03-01 05:11:53 +01:00
|
|
|
int disks = sh->disks;
|
2009-03-31 06:10:16 +02:00
|
|
|
int i, pd_idx = sh->pd_idx, qd_idx = sh->qd_idx;
|
2006-06-26 09:27:38 +02:00
|
|
|
struct r5dev *dev, *pdev, *qdev;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2006-06-26 09:27:38 +02:00
|
|
|
/* Now to look around and see what can be done */
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
rcu_read_lock();
|
2011-07-26 03:34:20 +02:00
|
|
|
spin_lock_irq(&conf->device_lock);
|
2006-06-26 09:27:38 +02:00
|
|
|
for (i=disks; i--; ) {
|
|
|
|
mdk_rdev_t *rdev;
|
|
|
|
dev = &sh->dev[i];
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("check %d: state 0x%lx read %p write %p written %p\n",
|
2006-06-26 09:27:38 +02:00
|
|
|
i, dev->flags, dev->toread, dev->towrite, dev->written);
|
2009-08-30 04:13:13 +02:00
|
|
|
/* maybe we can reply to a read
|
|
|
|
*
|
|
|
|
* new wantfill requests are only permitted while
|
|
|
|
* ops_complete_biofill is guaranteed to be inactive
|
|
|
|
*/
|
|
|
|
if (test_bit(R5_UPTODATE, &dev->flags) && dev->toread &&
|
|
|
|
!test_bit(STRIPE_BIOFILL_RUN, &sh->state))
|
|
|
|
set_bit(R5_Wantfill, &dev->flags);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2006-06-26 09:27:38 +02:00
|
|
|
/* now count some things */
|
2011-07-26 03:35:35 +02:00
|
|
|
if (test_bit(R5_LOCKED, &dev->flags))
|
|
|
|
s->locked++;
|
|
|
|
if (test_bit(R5_UPTODATE, &dev->flags))
|
|
|
|
s->uptodate++;
|
2009-09-16 21:11:54 +02:00
|
|
|
if (test_bit(R5_Wantcompute, &dev->flags)) {
|
2011-07-26 03:35:35 +02:00
|
|
|
s->compute++;
|
|
|
|
BUG_ON(s->compute > 2);
|
2009-09-16 21:11:54 +02:00
|
|
|
}
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2009-08-30 04:13:13 +02:00
|
|
|
if (test_bit(R5_Wantfill, &dev->flags)) {
|
2011-07-26 03:35:35 +02:00
|
|
|
s->to_fill++;
|
2009-08-30 04:13:13 +02:00
|
|
|
} else if (dev->toread)
|
2011-07-26 03:35:35 +02:00
|
|
|
s->to_read++;
|
2006-06-26 09:27:38 +02:00
|
|
|
if (dev->towrite) {
|
2011-07-26 03:35:35 +02:00
|
|
|
s->to_write++;
|
2006-06-26 09:27:38 +02:00
|
|
|
if (!test_bit(R5_OVERWRITE, &dev->flags))
|
2011-07-26 03:35:35 +02:00
|
|
|
s->non_overwrite++;
|
2006-06-26 09:27:38 +02:00
|
|
|
}
|
2007-07-09 20:56:43 +02:00
|
|
|
if (dev->written)
|
2011-07-26 03:35:35 +02:00
|
|
|
s->written++;
|
2006-06-26 09:27:38 +02:00
|
|
|
rdev = rcu_dereference(conf->disks[i].rdev);
|
2011-07-26 03:35:35 +02:00
|
|
|
if (s->blocked_rdev == NULL &&
|
2008-08-05 07:54:13 +02:00
|
|
|
rdev && unlikely(test_bit(Blocked, &rdev->flags))) {
|
2011-07-26 03:35:35 +02:00
|
|
|
s->blocked_rdev = rdev;
|
2008-04-30 09:52:32 +02:00
|
|
|
atomic_inc(&rdev->nr_pending);
|
|
|
|
}
|
2010-06-17 09:25:21 +02:00
|
|
|
clear_bit(R5_Insync, &dev->flags);
|
|
|
|
if (!rdev)
|
|
|
|
/* Not in-sync */;
|
|
|
|
else if (test_bit(In_sync, &rdev->flags))
|
|
|
|
set_bit(R5_Insync, &dev->flags);
|
|
|
|
else {
|
|
|
|
/* in sync if before recovery_offset */
|
|
|
|
if (sh->sector + STRIPE_SECTORS <= rdev->recovery_offset)
|
|
|
|
set_bit(R5_Insync, &dev->flags);
|
|
|
|
}
|
|
|
|
if (!test_bit(R5_Insync, &dev->flags)) {
|
2006-06-26 09:27:38 +02:00
|
|
|
/* The ReadError flag will just be confusing now */
|
|
|
|
clear_bit(R5_ReadError, &dev->flags);
|
|
|
|
clear_bit(R5_ReWrite, &dev->flags);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
2010-06-17 09:25:21 +02:00
|
|
|
if (test_bit(R5_ReadError, &dev->flags))
|
|
|
|
clear_bit(R5_Insync, &dev->flags);
|
|
|
|
if (!test_bit(R5_Insync, &dev->flags)) {
|
2011-07-26 03:35:35 +02:00
|
|
|
if (s->failed < 2)
|
|
|
|
s->failed_num[s->failed] = i;
|
|
|
|
s->failed++;
|
2010-06-17 09:25:21 +02:00
|
|
|
}
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
2011-07-26 03:34:20 +02:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
2005-04-17 00:20:36 +02:00
|
|
|
rcu_read_unlock();
|
2008-04-30 09:52:32 +02:00
|
|
|
|
2011-07-26 03:35:35 +02:00
|
|
|
if (unlikely(s->blocked_rdev)) {
|
|
|
|
if (s->syncing || s->expanding || s->expanded ||
|
|
|
|
s->to_write || s->written) {
|
2008-08-05 07:54:13 +02:00
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
2011-07-27 03:00:36 +02:00
|
|
|
return 1;
|
2008-08-05 07:54:13 +02:00
|
|
|
}
|
|
|
|
/* There is nothing for the blocked_rdev to block */
|
2011-07-26 03:35:35 +02:00
|
|
|
rdev_dec_pending(s->blocked_rdev, conf->mddev);
|
|
|
|
s->blocked_rdev = NULL;
|
2008-04-30 09:52:32 +02:00
|
|
|
}
|
2008-08-05 07:54:13 +02:00
|
|
|
|
2011-07-26 03:35:35 +02:00
|
|
|
if (s->to_fill && !test_bit(STRIPE_BIOFILL_RUN, &sh->state)) {
|
|
|
|
set_bit(STRIPE_OP_BIOFILL, &s->ops_request);
|
2009-08-30 04:13:13 +02:00
|
|
|
set_bit(STRIPE_BIOFILL_RUN, &sh->state);
|
|
|
|
}
|
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("locked=%d uptodate=%d to_read=%d"
|
2006-06-26 09:27:38 +02:00
|
|
|
" to_write=%d failed=%d failed_num=%d,%d\n",
|
2011-07-26 03:35:35 +02:00
|
|
|
s->locked, s->uptodate, s->to_read, s->to_write, s->failed,
|
|
|
|
s->failed_num[0], s->failed_num[1]);
|
2007-07-09 20:56:43 +02:00
|
|
|
/* check if the array has lost >2 devices and, if so, some requests
|
|
|
|
* might need to be failed
|
2006-06-26 09:27:38 +02:00
|
|
|
*/
|
2011-07-26 03:35:35 +02:00
|
|
|
if (s->failed > 2 && s->to_read+s->to_write+s->written)
|
|
|
|
handle_failed_stripe(conf, sh, s, disks, &s->return_bi);
|
|
|
|
if (s->failed > 2 && s->syncing) {
|
2006-06-26 09:27:38 +02:00
|
|
|
md_done_sync(conf->mddev, STRIPE_SECTORS,0);
|
|
|
|
clear_bit(STRIPE_SYNCING, &sh->state);
|
2011-07-26 03:35:35 +02:00
|
|
|
s->syncing = 0;
|
2006-06-26 09:27:38 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* might be able to return some write requests if the parity blocks
|
|
|
|
* are safe, or on a failed drive
|
|
|
|
*/
|
|
|
|
pdev = &sh->dev[pd_idx];
|
2011-07-26 03:35:35 +02:00
|
|
|
s->p_failed = (s->failed >= 1 && s->failed_num[0] == pd_idx)
|
|
|
|
|| (s->failed >= 2 && s->failed_num[1] == pd_idx);
|
2009-03-31 06:10:16 +02:00
|
|
|
qdev = &sh->dev[qd_idx];
|
2011-07-26 03:35:35 +02:00
|
|
|
s->q_failed = (s->failed >= 1 && s->failed_num[0] == qd_idx)
|
|
|
|
|| (s->failed >= 2 && s->failed_num[1] == qd_idx);
|
2007-07-09 20:56:43 +02:00
|
|
|
|
2011-07-26 03:35:35 +02:00
|
|
|
if (s->written &&
|
|
|
|
(s->p_failed || ((test_bit(R5_Insync, &pdev->flags)
|
2006-06-26 09:27:38 +02:00
|
|
|
&& !test_bit(R5_LOCKED, &pdev->flags)
|
2007-07-09 20:56:43 +02:00
|
|
|
&& test_bit(R5_UPTODATE, &pdev->flags)))) &&
|
2011-07-26 03:35:35 +02:00
|
|
|
(s->q_failed || ((test_bit(R5_Insync, &qdev->flags)
|
2006-06-26 09:27:38 +02:00
|
|
|
&& !test_bit(R5_LOCKED, &qdev->flags)
|
2007-07-09 20:56:43 +02:00
|
|
|
&& test_bit(R5_UPTODATE, &qdev->flags)))))
|
2011-07-26 03:35:35 +02:00
|
|
|
handle_stripe_clean_event(conf, sh, disks, &s->return_bi);
|
2006-06-26 09:27:38 +02:00
|
|
|
|
|
|
|
/* Now we might consider reading some blocks, either to check/generate
|
|
|
|
* parity, or to satisfy requests
|
|
|
|
* or to load a block that is being partially written.
|
|
|
|
*/
|
2011-07-26 03:35:35 +02:00
|
|
|
if (s->to_read || s->non_overwrite || (s->to_write && s->failed) ||
|
|
|
|
(s->syncing && (s->uptodate + s->compute < disks)) || s->expanding)
|
|
|
|
handle_stripe_fill6(sh, s, disks);
|
2006-06-26 09:27:38 +02:00
|
|
|
|
2009-08-30 04:13:13 +02:00
|
|
|
/* Now we check to see if any write operations have recently
|
|
|
|
* completed
|
|
|
|
*/
|
|
|
|
if (sh->reconstruct_state == reconstruct_state_drain_result) {
|
|
|
|
|
|
|
|
sh->reconstruct_state = reconstruct_state_idle;
|
|
|
|
/* All the 'written' buffers and the parity blocks are ready to
|
|
|
|
* be written back to disk
|
|
|
|
*/
|
|
|
|
BUG_ON(!test_bit(R5_UPTODATE, &sh->dev[sh->pd_idx].flags));
|
|
|
|
BUG_ON(!test_bit(R5_UPTODATE, &sh->dev[qd_idx].flags));
|
|
|
|
for (i = disks; i--; ) {
|
|
|
|
dev = &sh->dev[i];
|
|
|
|
if (test_bit(R5_LOCKED, &dev->flags) &&
|
|
|
|
(i == sh->pd_idx || i == qd_idx ||
|
|
|
|
dev->written)) {
|
|
|
|
pr_debug("Writing block %d\n", i);
|
|
|
|
BUG_ON(!test_bit(R5_UPTODATE, &dev->flags));
|
|
|
|
set_bit(R5_Wantwrite, &dev->flags);
|
|
|
|
if (!test_bit(R5_Insync, &dev->flags) ||
|
|
|
|
((i == sh->pd_idx || i == qd_idx) &&
|
2011-07-26 03:35:35 +02:00
|
|
|
s->failed == 0))
|
2009-08-30 04:13:13 +02:00
|
|
|
set_bit(STRIPE_INSYNC, &sh->state);
|
|
|
|
}
|
|
|
|
}
|
2009-12-14 02:49:50 +01:00
|
|
|
if (test_and_clear_bit(STRIPE_PREREAD_ACTIVE, &sh->state))
|
2011-07-26 03:35:35 +02:00
|
|
|
s->dec_preread_active = 1;
|
2009-08-30 04:13:13 +02:00
|
|
|
}
|
|
|
|
|
2009-08-30 04:13:12 +02:00
|
|
|
/* Now to consider new write requests and what else, if anything
|
|
|
|
* should be read. We do not handle new writes when:
|
|
|
|
* 1/ A 'write' operation (copy+gen_syndrome) is already in flight.
|
|
|
|
* 2/ A 'check' operation is in flight, as it may clobber the parity
|
|
|
|
* block.
|
|
|
|
*/
|
2011-07-26 03:35:35 +02:00
|
|
|
if (s->to_write && !sh->reconstruct_state && !sh->check_state)
|
|
|
|
handle_stripe_dirtying6(conf, sh, s, disks);
|
2006-06-26 09:27:38 +02:00
|
|
|
|
|
|
|
/* maybe we need to check and possibly fix the parity for this stripe
|
2007-07-09 20:56:43 +02:00
|
|
|
* Any reads will already have been scheduled, so we just see if enough
|
2009-08-30 04:13:13 +02:00
|
|
|
* data is available. The parity check is held off while parity
|
|
|
|
* dependent operations are in flight.
|
2006-06-26 09:27:38 +02:00
|
|
|
*/
|
2009-08-30 04:13:13 +02:00
|
|
|
if (sh->check_state ||
|
2011-07-26 03:35:35 +02:00
|
|
|
(s->syncing && s->locked == 0 &&
|
2009-08-30 04:13:13 +02:00
|
|
|
!test_bit(STRIPE_COMPUTE_RUN, &sh->state) &&
|
|
|
|
!test_bit(STRIPE_INSYNC, &sh->state)))
|
2011-07-26 03:35:35 +02:00
|
|
|
handle_parity_checks6(conf, sh, s, disks);
|
2011-07-27 03:00:36 +02:00
|
|
|
return 0;
|
2011-07-26 03:35:35 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
static void handle_stripe(struct stripe_head *sh)
|
|
|
|
{
|
|
|
|
struct stripe_head_state s;
|
2011-07-27 03:00:36 +02:00
|
|
|
int done;
|
|
|
|
int i;
|
2011-07-26 03:35:35 +02:00
|
|
|
raid5_conf_t *conf = sh->raid_conf;
|
|
|
|
|
|
|
|
clear_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
if (test_and_set_bit(STRIPE_ACTIVE, &sh->state)) {
|
|
|
|
/* already being handled, ensure it gets handled
|
|
|
|
* again when current action finishes */
|
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (test_and_clear_bit(STRIPE_SYNC_REQUESTED, &sh->state)) {
|
|
|
|
set_bit(STRIPE_SYNCING, &sh->state);
|
|
|
|
clear_bit(STRIPE_INSYNC, &sh->state);
|
|
|
|
}
|
|
|
|
clear_bit(STRIPE_DELAYED, &sh->state);
|
|
|
|
|
|
|
|
pr_debug("handling stripe %llu, state=%#lx cnt=%d, "
|
|
|
|
"pd_idx=%d, qd_idx=%d\n, check:%d, reconstruct:%d\n",
|
|
|
|
(unsigned long long)sh->sector, sh->state,
|
|
|
|
atomic_read(&sh->count), sh->pd_idx, sh->qd_idx,
|
|
|
|
sh->check_state, sh->reconstruct_state);
|
|
|
|
memset(&s, 0, sizeof(s));
|
|
|
|
|
|
|
|
s.syncing = test_bit(STRIPE_SYNCING, &sh->state);
|
|
|
|
s.expanding = test_bit(STRIPE_EXPAND_SOURCE, &sh->state);
|
|
|
|
s.expanded = test_bit(STRIPE_EXPAND_READY, &sh->state);
|
|
|
|
|
|
|
|
if (conf->level == 6)
|
2011-07-27 03:00:36 +02:00
|
|
|
done = handle_stripe6(sh, &s);
|
2011-07-26 03:35:35 +02:00
|
|
|
else
|
2011-07-27 03:00:36 +02:00
|
|
|
done = handle_stripe5(sh, &s);
|
|
|
|
|
|
|
|
if (done)
|
|
|
|
goto finish;
|
2011-07-27 03:00:36 +02:00
|
|
|
|
|
|
|
|
|
|
|
if (s.syncing && s.locked == 0 && test_bit(STRIPE_INSYNC, &sh->state)) {
|
|
|
|
md_done_sync(conf->mddev, STRIPE_SECTORS, 1);
|
|
|
|
clear_bit(STRIPE_SYNCING, &sh->state);
|
|
|
|
}
|
|
|
|
|
|
|
|
/* If the failed drives are just a ReadError, then we might need
|
|
|
|
* to progress the repair/check process
|
|
|
|
*/
|
|
|
|
if (s.failed <= conf->max_degraded && !conf->mddev->ro)
|
|
|
|
for (i = 0; i < s.failed; i++) {
|
|
|
|
struct r5dev *dev = &sh->dev[s.failed_num[i]];
|
|
|
|
if (test_bit(R5_ReadError, &dev->flags)
|
|
|
|
&& !test_bit(R5_LOCKED, &dev->flags)
|
|
|
|
&& test_bit(R5_UPTODATE, &dev->flags)
|
|
|
|
) {
|
|
|
|
if (!test_bit(R5_ReWrite, &dev->flags)) {
|
|
|
|
set_bit(R5_Wantwrite, &dev->flags);
|
|
|
|
set_bit(R5_ReWrite, &dev->flags);
|
|
|
|
set_bit(R5_LOCKED, &dev->flags);
|
|
|
|
s.locked++;
|
|
|
|
} else {
|
|
|
|
/* let's read it back */
|
|
|
|
set_bit(R5_Wantread, &dev->flags);
|
|
|
|
set_bit(R5_LOCKED, &dev->flags);
|
|
|
|
s.locked++;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2011-07-27 03:00:36 +02:00
|
|
|
/* Finish reconstruct operations initiated by the expansion process */
|
|
|
|
if (sh->reconstruct_state == reconstruct_state_result) {
|
|
|
|
struct stripe_head *sh_src
|
|
|
|
= get_active_stripe(conf, sh->sector, 1, 1, 1);
|
|
|
|
if (sh_src && test_bit(STRIPE_EXPAND_SOURCE, &sh_src->state)) {
|
|
|
|
/* sh cannot be written until sh_src has been read.
|
|
|
|
* so arrange for sh to be delayed a little
|
|
|
|
*/
|
|
|
|
set_bit(STRIPE_DELAYED, &sh->state);
|
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
if (!test_and_set_bit(STRIPE_PREREAD_ACTIVE,
|
|
|
|
&sh_src->state))
|
|
|
|
atomic_inc(&conf->preread_active_stripes);
|
|
|
|
release_stripe(sh_src);
|
|
|
|
goto finish;
|
|
|
|
}
|
|
|
|
if (sh_src)
|
|
|
|
release_stripe(sh_src);
|
|
|
|
|
|
|
|
sh->reconstruct_state = reconstruct_state_idle;
|
|
|
|
clear_bit(STRIPE_EXPANDING, &sh->state);
|
|
|
|
for (i = conf->raid_disks; i--; ) {
|
|
|
|
set_bit(R5_Wantwrite, &sh->dev[i].flags);
|
|
|
|
set_bit(R5_LOCKED, &sh->dev[i].flags);
|
|
|
|
s.locked++;
|
|
|
|
}
|
|
|
|
}
|
2007-03-01 05:11:53 +01:00
|
|
|
|
2011-07-27 03:00:36 +02:00
|
|
|
if (s.expanded && test_bit(STRIPE_EXPANDING, &sh->state) &&
|
|
|
|
!sh->reconstruct_state) {
|
|
|
|
/* Need to write out all blocks after computing parity */
|
|
|
|
sh->disks = conf->raid_disks;
|
|
|
|
stripe_set_idx(sh->sector, conf, 0, sh);
|
|
|
|
schedule_reconstruction(sh, &s, 1, 1);
|
|
|
|
} else if (s.expanded && !sh->reconstruct_state && s.locked == 0) {
|
|
|
|
clear_bit(STRIPE_EXPAND_READY, &sh->state);
|
|
|
|
atomic_dec(&conf->reshape_stripes);
|
|
|
|
wake_up(&conf->wait_for_overlap);
|
|
|
|
md_done_sync(conf->mddev, STRIPE_SECTORS, 1);
|
|
|
|
}
|
|
|
|
|
|
|
|
if (s.expanding && s.locked == 0 &&
|
|
|
|
!test_bit(STRIPE_COMPUTE_RUN, &sh->state))
|
|
|
|
handle_stripe_expansion(conf, sh);
|
2006-06-26 09:27:38 +02:00
|
|
|
|
2011-07-27 03:00:36 +02:00
|
|
|
finish:
|
2008-04-30 09:52:32 +02:00
|
|
|
/* wait for this device to become unblocked */
|
2011-07-26 03:35:20 +02:00
|
|
|
if (unlikely(s.blocked_rdev))
|
|
|
|
md_wait_for_blocked_rdev(s.blocked_rdev, conf->mddev);
|
2008-04-30 09:52:32 +02:00
|
|
|
|
2009-08-30 04:13:13 +02:00
|
|
|
if (s.ops_request)
|
|
|
|
raid_run_ops(sh, s.ops_request);
|
|
|
|
|
2008-06-28 00:31:55 +02:00
|
|
|
ops_run_io(sh, &s);
|
2006-06-26 09:27:38 +02:00
|
|
|
|
2009-12-14 02:49:50 +01:00
|
|
|
|
2011-07-26 03:35:20 +02:00
|
|
|
if (s.dec_preread_active) {
|
2009-12-14 02:49:50 +01:00
|
|
|
/* We delay this until after ops_run_io so that if make_request
|
2010-09-03 11:56:18 +02:00
|
|
|
* is waiting on a flush, it won't continue until the writes
|
2009-12-14 02:49:50 +01:00
|
|
|
* have actually been submitted.
|
|
|
|
*/
|
|
|
|
atomic_dec(&conf->preread_active_stripes);
|
|
|
|
if (atomic_read(&conf->preread_active_stripes) <
|
|
|
|
IO_THRESHOLD)
|
|
|
|
md_wakeup_thread(conf->mddev->thread);
|
|
|
|
}
|
|
|
|
|
2011-07-26 03:35:20 +02:00
|
|
|
return_io(s.return_bi);
|
2006-06-26 09:27:38 +02:00
|
|
|
|
2011-07-26 03:34:20 +02:00
|
|
|
clear_bit(STRIPE_ACTIVE, &sh->state);
|
2006-06-26 09:27:38 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
static void raid5_activate_delayed(raid5_conf_t *conf)
|
|
|
|
{
|
|
|
|
if (atomic_read(&conf->preread_active_stripes) < IO_THRESHOLD) {
|
|
|
|
while (!list_empty(&conf->delayed_list)) {
|
|
|
|
struct list_head *l = conf->delayed_list.next;
|
|
|
|
struct stripe_head *sh;
|
|
|
|
sh = list_entry(l, struct stripe_head, lru);
|
|
|
|
list_del_init(l);
|
|
|
|
clear_bit(STRIPE_DELAYED, &sh->state);
|
|
|
|
if (!test_and_set_bit(STRIPE_PREREAD_ACTIVE, &sh->state))
|
|
|
|
atomic_inc(&conf->preread_active_stripes);
|
2008-04-28 11:15:53 +02:00
|
|
|
list_add_tail(&sh->lru, &conf->hold_list);
|
2006-06-26 09:27:38 +02:00
|
|
|
}
|
2011-04-18 10:25:42 +02:00
|
|
|
}
|
2006-06-26 09:27:38 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
static void activate_bit_delay(raid5_conf_t *conf)
|
|
|
|
{
|
|
|
|
/* device_lock is held */
|
|
|
|
struct list_head head;
|
|
|
|
list_add(&head, &conf->bitmap_list);
|
|
|
|
list_del_init(&conf->bitmap_list);
|
|
|
|
while (!list_empty(&head)) {
|
|
|
|
struct stripe_head *sh = list_entry(head.next, struct stripe_head, lru);
|
|
|
|
list_del_init(&sh->lru);
|
|
|
|
atomic_inc(&sh->count);
|
|
|
|
__release_stripe(conf, sh);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2010-07-26 03:57:07 +02:00
|
|
|
int md_raid5_congested(mddev_t *mddev, int bits)
|
2006-10-03 10:15:56 +02:00
|
|
|
{
|
2009-06-16 08:54:21 +02:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2006-10-03 10:15:56 +02:00
|
|
|
|
|
|
|
/* No difference between reads and writes. Just check
|
|
|
|
* how busy the stripe_cache is
|
|
|
|
*/
|
2009-09-23 10:10:29 +02:00
|
|
|
|
2006-10-03 10:15:56 +02:00
|
|
|
if (conf->inactive_blocked)
|
|
|
|
return 1;
|
|
|
|
if (conf->quiesce)
|
|
|
|
return 1;
|
|
|
|
if (list_empty_careful(&conf->inactive_list))
|
|
|
|
return 1;
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
2010-07-26 03:57:07 +02:00
|
|
|
EXPORT_SYMBOL_GPL(md_raid5_congested);
|
|
|
|
|
|
|
|
static int raid5_congested(void *data, int bits)
|
|
|
|
{
|
|
|
|
mddev_t *mddev = data;
|
|
|
|
|
|
|
|
return mddev_congested(mddev, bits) ||
|
|
|
|
md_raid5_congested(mddev, bits);
|
|
|
|
}
|
2006-10-03 10:15:56 +02:00
|
|
|
|
2006-12-10 11:20:45 +01:00
|
|
|
/* We want read requests to align with chunks where possible,
|
|
|
|
* but write requests don't need to.
|
|
|
|
*/
|
2008-07-03 09:53:43 +02:00
|
|
|
static int raid5_mergeable_bvec(struct request_queue *q,
|
|
|
|
struct bvec_merge_data *bvm,
|
|
|
|
struct bio_vec *biovec)
|
2006-12-10 11:20:45 +01:00
|
|
|
{
|
|
|
|
mddev_t *mddev = q->queuedata;
|
2008-07-03 09:53:43 +02:00
|
|
|
sector_t sector = bvm->bi_sector + get_start_sect(bvm->bi_bdev);
|
2006-12-10 11:20:45 +01:00
|
|
|
int max;
|
2009-06-18 00:45:01 +02:00
|
|
|
unsigned int chunk_sectors = mddev->chunk_sectors;
|
2008-07-03 09:53:43 +02:00
|
|
|
unsigned int bio_sectors = bvm->bi_size >> 9;
|
2006-12-10 11:20:45 +01:00
|
|
|
|
2008-07-03 09:53:43 +02:00
|
|
|
if ((bvm->bi_rw & 1) == WRITE)
|
2006-12-10 11:20:45 +01:00
|
|
|
return biovec->bv_len; /* always allow writes to be mergeable */
|
|
|
|
|
2009-06-18 00:45:27 +02:00
|
|
|
if (mddev->new_chunk_sectors < mddev->chunk_sectors)
|
|
|
|
chunk_sectors = mddev->new_chunk_sectors;
|
2006-12-10 11:20:45 +01:00
|
|
|
max = (chunk_sectors - ((sector & (chunk_sectors - 1)) + bio_sectors)) << 9;
|
|
|
|
if (max < 0) max = 0;
|
|
|
|
if (max <= biovec->bv_len && bio_sectors == 0)
|
|
|
|
return biovec->bv_len;
|
|
|
|
else
|
|
|
|
return max;
|
|
|
|
}
|
|
|
|
|
2006-12-10 11:20:46 +01:00
|
|
|
|
|
|
|
static int in_chunk_boundary(mddev_t *mddev, struct bio *bio)
|
|
|
|
{
|
|
|
|
sector_t sector = bio->bi_sector + get_start_sect(bio->bi_bdev);
|
2009-06-18 00:45:01 +02:00
|
|
|
unsigned int chunk_sectors = mddev->chunk_sectors;
|
2006-12-10 11:20:46 +01:00
|
|
|
unsigned int bio_sectors = bio->bi_size >> 9;
|
|
|
|
|
2009-06-18 00:45:27 +02:00
|
|
|
if (mddev->new_chunk_sectors < mddev->chunk_sectors)
|
|
|
|
chunk_sectors = mddev->new_chunk_sectors;
|
2006-12-10 11:20:46 +01:00
|
|
|
return chunk_sectors >=
|
|
|
|
((sector & (chunk_sectors - 1)) + bio_sectors);
|
|
|
|
}
|
|
|
|
|
2006-12-10 11:20:47 +01:00
|
|
|
/*
|
|
|
|
* add bio to the retry LIFO ( in O(1) ... we are in interrupt )
|
|
|
|
* later sampled by raid5d.
|
|
|
|
*/
|
|
|
|
static void add_bio_to_retry(struct bio *bi,raid5_conf_t *conf)
|
|
|
|
{
|
|
|
|
unsigned long flags;
|
|
|
|
|
|
|
|
spin_lock_irqsave(&conf->device_lock, flags);
|
|
|
|
|
|
|
|
bi->bi_next = conf->retry_read_aligned_list;
|
|
|
|
conf->retry_read_aligned_list = bi;
|
|
|
|
|
|
|
|
spin_unlock_irqrestore(&conf->device_lock, flags);
|
|
|
|
md_wakeup_thread(conf->mddev->thread);
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
static struct bio *remove_bio_from_retry(raid5_conf_t *conf)
|
|
|
|
{
|
|
|
|
struct bio *bi;
|
|
|
|
|
|
|
|
bi = conf->retry_read_aligned;
|
|
|
|
if (bi) {
|
|
|
|
conf->retry_read_aligned = NULL;
|
|
|
|
return bi;
|
|
|
|
}
|
|
|
|
bi = conf->retry_read_aligned_list;
|
|
|
|
if(bi) {
|
2007-02-08 23:20:29 +01:00
|
|
|
conf->retry_read_aligned_list = bi->bi_next;
|
2006-12-10 11:20:47 +01:00
|
|
|
bi->bi_next = NULL;
|
2008-08-15 10:41:18 +02:00
|
|
|
/*
|
|
|
|
* this sets the active strip count to 1 and the processed
|
|
|
|
* strip count to zero (upper 8 bits)
|
|
|
|
*/
|
2006-12-10 11:20:47 +01:00
|
|
|
bi->bi_phys_segments = 1; /* biased count of active stripes */
|
|
|
|
}
|
|
|
|
|
|
|
|
return bi;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2006-12-10 11:20:46 +01:00
|
|
|
/*
|
|
|
|
* The "raid5_align_endio" should check if the read succeeded and if it
|
|
|
|
* did, call bio_endio on the original bio (having bio_put the new bio
|
|
|
|
* first).
|
|
|
|
* If the read failed..
|
|
|
|
*/
|
2007-09-27 12:47:43 +02:00
|
|
|
static void raid5_align_endio(struct bio *bi, int error)
|
2006-12-10 11:20:46 +01:00
|
|
|
{
|
|
|
|
struct bio* raid_bi = bi->bi_private;
|
2006-12-10 11:20:47 +01:00
|
|
|
mddev_t *mddev;
|
|
|
|
raid5_conf_t *conf;
|
|
|
|
int uptodate = test_bit(BIO_UPTODATE, &bi->bi_flags);
|
|
|
|
mdk_rdev_t *rdev;
|
|
|
|
|
2006-12-10 11:20:46 +01:00
|
|
|
bio_put(bi);
|
2006-12-10 11:20:47 +01:00
|
|
|
|
|
|
|
rdev = (void*)raid_bi->bi_next;
|
|
|
|
raid_bi->bi_next = NULL;
|
2010-03-25 06:06:03 +01:00
|
|
|
mddev = rdev->mddev;
|
|
|
|
conf = mddev->private;
|
2006-12-10 11:20:47 +01:00
|
|
|
|
|
|
|
rdev_dec_pending(rdev, conf->mddev);
|
|
|
|
|
|
|
|
if (!error && uptodate) {
|
2007-09-27 12:47:43 +02:00
|
|
|
bio_endio(raid_bi, 0);
|
2006-12-10 11:20:47 +01:00
|
|
|
if (atomic_dec_and_test(&conf->active_aligned_reads))
|
|
|
|
wake_up(&conf->wait_for_stripe);
|
2007-09-27 12:47:43 +02:00
|
|
|
return;
|
2006-12-10 11:20:47 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("raid5_align_endio : io error...handing IO for a retry\n");
|
2006-12-10 11:20:47 +01:00
|
|
|
|
|
|
|
add_bio_to_retry(raid_bi, conf);
|
2006-12-10 11:20:46 +01:00
|
|
|
}
|
|
|
|
|
2007-02-08 23:20:29 +01:00
|
|
|
static int bio_fits_rdev(struct bio *bi)
|
|
|
|
{
|
2007-07-24 09:28:11 +02:00
|
|
|
struct request_queue *q = bdev_get_queue(bi->bi_bdev);
|
2007-02-08 23:20:29 +01:00
|
|
|
|
2009-05-22 23:17:50 +02:00
|
|
|
if ((bi->bi_size>>9) > queue_max_sectors(q))
|
2007-02-08 23:20:29 +01:00
|
|
|
return 0;
|
|
|
|
blk_recount_segments(q, bi);
|
2010-02-26 06:20:39 +01:00
|
|
|
if (bi->bi_phys_segments > queue_max_segments(q))
|
2007-02-08 23:20:29 +01:00
|
|
|
return 0;
|
|
|
|
|
|
|
|
if (q->merge_bvec_fn)
|
|
|
|
/* it's too hard to apply the merge_bvec_fn at this stage,
|
|
|
|
* just just give up
|
|
|
|
*/
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
return 1;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2010-04-01 06:02:13 +02:00
|
|
|
static int chunk_aligned_read(mddev_t *mddev, struct bio * raid_bio)
|
2006-12-10 11:20:46 +01:00
|
|
|
{
|
2009-06-16 08:54:21 +02:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2009-12-14 02:49:47 +01:00
|
|
|
int dd_idx;
|
2006-12-10 11:20:46 +01:00
|
|
|
struct bio* align_bi;
|
|
|
|
mdk_rdev_t *rdev;
|
|
|
|
|
|
|
|
if (!in_chunk_boundary(mddev, raid_bio)) {
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("chunk_aligned_read : non aligned\n");
|
2006-12-10 11:20:46 +01:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
/*
|
2010-10-26 09:31:13 +02:00
|
|
|
* use bio_clone_mddev to make a copy of the bio
|
2006-12-10 11:20:46 +01:00
|
|
|
*/
|
2010-10-26 09:31:13 +02:00
|
|
|
align_bi = bio_clone_mddev(raid_bio, GFP_NOIO, mddev);
|
2006-12-10 11:20:46 +01:00
|
|
|
if (!align_bi)
|
|
|
|
return 0;
|
|
|
|
/*
|
|
|
|
* set bi_end_io to a new function, and set bi_private to the
|
|
|
|
* original bio.
|
|
|
|
*/
|
|
|
|
align_bi->bi_end_io = raid5_align_endio;
|
|
|
|
align_bi->bi_private = raid_bio;
|
|
|
|
/*
|
|
|
|
* compute position
|
|
|
|
*/
|
2009-03-31 05:39:38 +02:00
|
|
|
align_bi->bi_sector = raid5_compute_sector(conf, raid_bio->bi_sector,
|
|
|
|
0,
|
2009-03-31 05:39:38 +02:00
|
|
|
&dd_idx, NULL);
|
2006-12-10 11:20:46 +01:00
|
|
|
|
|
|
|
rcu_read_lock();
|
|
|
|
rdev = rcu_dereference(conf->disks[dd_idx].rdev);
|
|
|
|
if (rdev && test_bit(In_sync, &rdev->flags)) {
|
|
|
|
atomic_inc(&rdev->nr_pending);
|
|
|
|
rcu_read_unlock();
|
2006-12-10 11:20:47 +01:00
|
|
|
raid_bio->bi_next = (void*)rdev;
|
|
|
|
align_bi->bi_bdev = rdev->bdev;
|
|
|
|
align_bi->bi_flags &= ~(1 << BIO_SEG_VALID);
|
|
|
|
align_bi->bi_sector += rdev->data_offset;
|
|
|
|
|
2007-02-08 23:20:29 +01:00
|
|
|
if (!bio_fits_rdev(align_bi)) {
|
|
|
|
/* too big in some way */
|
|
|
|
bio_put(align_bi);
|
|
|
|
rdev_dec_pending(rdev, mddev);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2006-12-10 11:20:47 +01:00
|
|
|
spin_lock_irq(&conf->device_lock);
|
|
|
|
wait_event_lock_irq(conf->wait_for_stripe,
|
|
|
|
conf->quiesce == 0,
|
|
|
|
conf->device_lock, /* nothing */);
|
|
|
|
atomic_inc(&conf->active_aligned_reads);
|
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
|
2006-12-10 11:20:46 +01:00
|
|
|
generic_make_request(align_bi);
|
|
|
|
return 1;
|
|
|
|
} else {
|
|
|
|
rcu_read_unlock();
|
2006-12-10 11:20:47 +01:00
|
|
|
bio_put(align_bi);
|
2006-12-10 11:20:46 +01:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2008-04-28 11:15:53 +02:00
|
|
|
/* __get_priority_stripe - get the next stripe to process
|
|
|
|
*
|
|
|
|
* Full stripe writes are allowed to pass preread active stripes up until
|
|
|
|
* the bypass_threshold is exceeded. In general the bypass_count
|
|
|
|
* increments when the handle_list is handled before the hold_list; however, it
|
|
|
|
* will not be incremented when STRIPE_IO_STARTED is sampled set signifying a
|
|
|
|
* stripe with in flight i/o. The bypass_count will be reset when the
|
|
|
|
* head of the hold_list has changed, i.e. the head was promoted to the
|
|
|
|
* handle_list.
|
|
|
|
*/
|
|
|
|
static struct stripe_head *__get_priority_stripe(raid5_conf_t *conf)
|
|
|
|
{
|
|
|
|
struct stripe_head *sh;
|
|
|
|
|
|
|
|
pr_debug("%s: handle: %s hold: %s full_writes: %d bypass_count: %d\n",
|
|
|
|
__func__,
|
|
|
|
list_empty(&conf->handle_list) ? "empty" : "busy",
|
|
|
|
list_empty(&conf->hold_list) ? "empty" : "busy",
|
|
|
|
atomic_read(&conf->pending_full_writes), conf->bypass_count);
|
|
|
|
|
|
|
|
if (!list_empty(&conf->handle_list)) {
|
|
|
|
sh = list_entry(conf->handle_list.next, typeof(*sh), lru);
|
|
|
|
|
|
|
|
if (list_empty(&conf->hold_list))
|
|
|
|
conf->bypass_count = 0;
|
|
|
|
else if (!test_bit(STRIPE_IO_STARTED, &sh->state)) {
|
|
|
|
if (conf->hold_list.next == conf->last_hold)
|
|
|
|
conf->bypass_count++;
|
|
|
|
else {
|
|
|
|
conf->last_hold = conf->hold_list.next;
|
|
|
|
conf->bypass_count -= conf->bypass_threshold;
|
|
|
|
if (conf->bypass_count < 0)
|
|
|
|
conf->bypass_count = 0;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
} else if (!list_empty(&conf->hold_list) &&
|
|
|
|
((conf->bypass_threshold &&
|
|
|
|
conf->bypass_count > conf->bypass_threshold) ||
|
|
|
|
atomic_read(&conf->pending_full_writes) == 0)) {
|
|
|
|
sh = list_entry(conf->hold_list.next,
|
|
|
|
typeof(*sh), lru);
|
|
|
|
conf->bypass_count -= conf->bypass_threshold;
|
|
|
|
if (conf->bypass_count < 0)
|
|
|
|
conf->bypass_count = 0;
|
|
|
|
} else
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
list_del_init(&sh->lru);
|
|
|
|
atomic_inc(&sh->count);
|
|
|
|
BUG_ON(atomic_read(&sh->count) != 1);
|
|
|
|
return sh;
|
|
|
|
}
|
2006-12-10 11:20:46 +01:00
|
|
|
|
2010-04-01 06:02:13 +02:00
|
|
|
static int make_request(mddev_t *mddev, struct bio * bi)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
2009-06-16 08:54:21 +02:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2009-03-31 05:39:38 +02:00
|
|
|
int dd_idx;
|
2005-04-17 00:20:36 +02:00
|
|
|
sector_t new_sector;
|
|
|
|
sector_t logical_sector, last_sector;
|
|
|
|
struct stripe_head *sh;
|
2005-11-01 09:26:16 +01:00
|
|
|
const int rw = bio_data_dir(bi);
|
2010-03-25 06:20:56 +01:00
|
|
|
int remaining;
|
2011-04-18 10:25:43 +02:00
|
|
|
int plugged;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2010-09-03 11:56:18 +02:00
|
|
|
if (unlikely(bi->bi_rw & REQ_FLUSH)) {
|
|
|
|
md_flush_request(mddev, bi);
|
2005-09-10 01:23:41 +02:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2005-06-22 02:17:26 +02:00
|
|
|
md_write_start(mddev, bi);
|
2005-06-22 02:17:12 +02:00
|
|
|
|
2006-12-13 09:34:13 +01:00
|
|
|
if (rw == READ &&
|
2006-12-10 11:20:48 +01:00
|
|
|
mddev->reshape_position == MaxSector &&
|
2010-04-01 06:02:13 +02:00
|
|
|
chunk_aligned_read(mddev,bi))
|
2009-03-31 05:39:38 +02:00
|
|
|
return 0;
|
2006-12-10 11:20:48 +01:00
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
logical_sector = bi->bi_sector & ~((sector_t)STRIPE_SECTORS-1);
|
|
|
|
last_sector = bi->bi_sector + (bi->bi_size>>9);
|
|
|
|
bi->bi_next = NULL;
|
|
|
|
bi->bi_phys_segments = 1; /* over-loaded to count active stripes */
|
2005-06-22 02:17:12 +02:00
|
|
|
|
2011-04-18 10:25:43 +02:00
|
|
|
plugged = mddev_check_plugged(mddev);
|
2005-04-17 00:20:36 +02:00
|
|
|
for (;logical_sector < last_sector; logical_sector += STRIPE_SECTORS) {
|
|
|
|
DEFINE_WAIT(w);
|
2006-06-26 09:27:38 +02:00
|
|
|
int disks, data_disks;
|
2009-03-31 05:39:38 +02:00
|
|
|
int previous;
|
2006-03-27 11:18:12 +02:00
|
|
|
|
2006-03-27 11:18:08 +02:00
|
|
|
retry:
|
2009-03-31 05:39:38 +02:00
|
|
|
previous = 0;
|
2009-03-31 06:27:18 +02:00
|
|
|
disks = conf->raid_disks;
|
2006-03-27 11:18:12 +02:00
|
|
|
prepare_to_wait(&conf->wait_for_overlap, &w, TASK_UNINTERRUPTIBLE);
|
2009-03-31 06:27:18 +02:00
|
|
|
if (unlikely(conf->reshape_progress != MaxSector)) {
|
2009-03-31 06:16:46 +02:00
|
|
|
/* spinlock is needed as reshape_progress may be
|
2006-03-27 11:18:15 +02:00
|
|
|
* 64bit on a 32bit platform, and so it might be
|
|
|
|
* possible to see a half-updated value
|
2011-04-10 18:06:17 +02:00
|
|
|
* Of course reshape_progress could change after
|
2006-03-27 11:18:15 +02:00
|
|
|
* the lock is dropped, so once we get a reference
|
|
|
|
* to the stripe that we think it is, we will have
|
|
|
|
* to check again.
|
|
|
|
*/
|
2006-03-27 11:18:08 +02:00
|
|
|
spin_lock_irq(&conf->device_lock);
|
2009-03-31 06:16:46 +02:00
|
|
|
if (mddev->delta_disks < 0
|
|
|
|
? logical_sector < conf->reshape_progress
|
|
|
|
: logical_sector >= conf->reshape_progress) {
|
2006-03-27 11:18:08 +02:00
|
|
|
disks = conf->previous_raid_disks;
|
2009-03-31 05:39:38 +02:00
|
|
|
previous = 1;
|
|
|
|
} else {
|
2009-03-31 06:16:46 +02:00
|
|
|
if (mddev->delta_disks < 0
|
|
|
|
? logical_sector < conf->reshape_safe
|
|
|
|
: logical_sector >= conf->reshape_safe) {
|
2006-03-27 11:18:12 +02:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
schedule();
|
|
|
|
goto retry;
|
|
|
|
}
|
|
|
|
}
|
2006-03-27 11:18:08 +02:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
}
|
2006-06-26 09:27:38 +02:00
|
|
|
data_disks = disks - conf->max_degraded;
|
|
|
|
|
2009-03-31 05:39:38 +02:00
|
|
|
new_sector = raid5_compute_sector(conf, logical_sector,
|
|
|
|
previous,
|
2009-03-31 05:39:38 +02:00
|
|
|
&dd_idx, NULL);
|
2010-05-03 06:09:02 +02:00
|
|
|
pr_debug("raid456: make_request, sector %llu logical %llu\n",
|
2005-04-17 00:20:36 +02:00
|
|
|
(unsigned long long)new_sector,
|
|
|
|
(unsigned long long)logical_sector);
|
|
|
|
|
2009-03-31 05:39:38 +02:00
|
|
|
sh = get_active_stripe(conf, new_sector, previous,
|
2009-06-09 06:39:59 +02:00
|
|
|
(bi->bi_rw&RWA_MASK), 0);
|
2005-04-17 00:20:36 +02:00
|
|
|
if (sh) {
|
2009-03-31 06:27:18 +02:00
|
|
|
if (unlikely(previous)) {
|
2006-03-27 11:18:08 +02:00
|
|
|
/* expansion might have moved on while waiting for a
|
2006-03-27 11:18:15 +02:00
|
|
|
* stripe, so we must do the range check again.
|
|
|
|
* Expansion could still move past after this
|
|
|
|
* test, but as we are holding a reference to
|
|
|
|
* 'sh', we know that if that happens,
|
|
|
|
* STRIPE_EXPANDING will get set and the expansion
|
|
|
|
* won't proceed until we finish with the stripe.
|
2006-03-27 11:18:08 +02:00
|
|
|
*/
|
|
|
|
int must_retry = 0;
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
2009-03-31 06:27:18 +02:00
|
|
|
if (mddev->delta_disks < 0
|
|
|
|
? logical_sector >= conf->reshape_progress
|
|
|
|
: logical_sector < conf->reshape_progress)
|
2006-03-27 11:18:08 +02:00
|
|
|
/* mismatch, need to try again */
|
|
|
|
must_retry = 1;
|
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
if (must_retry) {
|
|
|
|
release_stripe(sh);
|
2009-06-17 01:00:33 +02:00
|
|
|
schedule();
|
2006-03-27 11:18:08 +02:00
|
|
|
goto retry;
|
|
|
|
}
|
|
|
|
}
|
2009-07-01 05:15:35 +02:00
|
|
|
|
2011-07-18 09:38:51 +02:00
|
|
|
if (rw == WRITE &&
|
2009-07-01 05:15:35 +02:00
|
|
|
logical_sector >= mddev->suspend_lo &&
|
2006-03-27 11:18:14 +02:00
|
|
|
logical_sector < mddev->suspend_hi) {
|
|
|
|
release_stripe(sh);
|
2009-07-01 05:15:35 +02:00
|
|
|
/* As the suspend_* range is controlled by
|
|
|
|
* userspace, we want an interruptible
|
|
|
|
* wait.
|
|
|
|
*/
|
|
|
|
flush_signals(current);
|
|
|
|
prepare_to_wait(&conf->wait_for_overlap,
|
|
|
|
&w, TASK_INTERRUPTIBLE);
|
|
|
|
if (logical_sector >= mddev->suspend_lo &&
|
|
|
|
logical_sector < mddev->suspend_hi)
|
|
|
|
schedule();
|
2006-03-27 11:18:14 +02:00
|
|
|
goto retry;
|
|
|
|
}
|
2006-03-27 11:18:08 +02:00
|
|
|
|
|
|
|
if (test_bit(STRIPE_EXPANDING, &sh->state) ||
|
2011-07-18 09:38:51 +02:00
|
|
|
!add_stripe_bio(sh, bi, dd_idx, rw)) {
|
2006-03-27 11:18:08 +02:00
|
|
|
/* Stripe is busy expanding or
|
|
|
|
* add failed due to overlap. Flush everything
|
2005-04-17 00:20:36 +02:00
|
|
|
* and wait a while
|
|
|
|
*/
|
2011-04-18 10:25:42 +02:00
|
|
|
md_wakeup_thread(mddev->thread);
|
2005-04-17 00:20:36 +02:00
|
|
|
release_stripe(sh);
|
|
|
|
schedule();
|
|
|
|
goto retry;
|
|
|
|
}
|
|
|
|
finish_wait(&conf->wait_for_overlap, &w);
|
2008-02-06 10:40:00 +01:00
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
clear_bit(STRIPE_DELAYED, &sh->state);
|
2010-09-03 11:56:18 +02:00
|
|
|
if ((bi->bi_rw & REQ_SYNC) &&
|
2009-12-14 02:49:50 +01:00
|
|
|
!test_and_set_bit(STRIPE_PREREAD_ACTIVE, &sh->state))
|
|
|
|
atomic_inc(&conf->preread_active_stripes);
|
2005-04-17 00:20:36 +02:00
|
|
|
release_stripe(sh);
|
|
|
|
} else {
|
|
|
|
/* cannot get stripe for read-ahead, just give-up */
|
|
|
|
clear_bit(BIO_UPTODATE, &bi->bi_flags);
|
|
|
|
finish_wait(&conf->wait_for_overlap, &w);
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
|
|
|
|
}
|
2011-04-18 10:25:43 +02:00
|
|
|
if (!plugged)
|
|
|
|
md_wakeup_thread(mddev->thread);
|
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
spin_lock_irq(&conf->device_lock);
|
2008-08-15 10:41:18 +02:00
|
|
|
remaining = raid5_dec_bi_phys_segments(bi);
|
2006-03-27 11:18:17 +02:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
if (remaining == 0) {
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2006-06-26 09:27:38 +02:00
|
|
|
if ( rw == WRITE )
|
2005-04-17 00:20:36 +02:00
|
|
|
md_write_end(mddev);
|
2007-09-27 12:47:43 +02:00
|
|
|
|
2008-06-28 00:31:20 +02:00
|
|
|
bio_endio(bi, 0);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
2009-12-14 02:49:50 +01:00
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2009-03-31 06:00:31 +02:00
|
|
|
static sector_t raid5_size(mddev_t *mddev, sector_t sectors, int raid_disks);
|
|
|
|
|
2006-06-26 09:27:43 +02:00
|
|
|
static sector_t reshape_request(mddev_t *mddev, sector_t sector_nr, int *skipped)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
2006-06-26 09:27:43 +02:00
|
|
|
/* reshaping is quite different to recovery/resync so it is
|
|
|
|
* handled quite separately ... here.
|
|
|
|
*
|
|
|
|
* On each call to sync_request, we gather one chunk worth of
|
|
|
|
* destination stripes and flag them as expanding.
|
|
|
|
* Then we find all the source stripes and request reads.
|
|
|
|
* As the reads complete, handle_stripe will copy the data
|
|
|
|
* into the destination stripe and release that stripe.
|
|
|
|
*/
|
2010-03-08 06:02:40 +01:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2005-04-17 00:20:36 +02:00
|
|
|
struct stripe_head *sh;
|
2006-03-27 11:18:09 +02:00
|
|
|
sector_t first_sector, last_sector;
|
2007-03-01 05:11:53 +01:00
|
|
|
int raid_disks = conf->previous_raid_disks;
|
|
|
|
int data_disks = raid_disks - conf->max_degraded;
|
|
|
|
int new_data_disks = conf->raid_disks - conf->max_degraded;
|
2006-06-26 09:27:43 +02:00
|
|
|
int i;
|
|
|
|
int dd_idx;
|
2009-03-31 06:28:40 +02:00
|
|
|
sector_t writepos, readpos, safepos;
|
2009-03-31 06:17:38 +02:00
|
|
|
sector_t stripe_addr;
|
2009-03-31 06:21:40 +02:00
|
|
|
int reshape_sectors;
|
2009-03-31 06:26:47 +02:00
|
|
|
struct list_head stripes;
|
2006-06-26 09:27:43 +02:00
|
|
|
|
2009-03-31 06:16:46 +02:00
|
|
|
if (sector_nr == 0) {
|
|
|
|
/* If restarting in the middle, skip the initial sectors */
|
|
|
|
if (mddev->delta_disks < 0 &&
|
|
|
|
conf->reshape_progress < raid5_size(mddev, 0, 0)) {
|
|
|
|
sector_nr = raid5_size(mddev, 0, 0)
|
|
|
|
- conf->reshape_progress;
|
2009-08-13 02:13:00 +02:00
|
|
|
} else if (mddev->delta_disks >= 0 &&
|
2009-03-31 06:16:46 +02:00
|
|
|
conf->reshape_progress > 0)
|
|
|
|
sector_nr = conf->reshape_progress;
|
2007-03-01 05:11:53 +01:00
|
|
|
sector_div(sector_nr, new_data_disks);
|
2009-03-31 06:16:46 +02:00
|
|
|
if (sector_nr) {
|
2009-11-06 04:59:29 +01:00
|
|
|
mddev->curr_resync_completed = sector_nr;
|
|
|
|
sysfs_notify(&mddev->kobj, NULL, "sync_completed");
|
2009-03-31 06:16:46 +02:00
|
|
|
*skipped = 1;
|
|
|
|
return sector_nr;
|
|
|
|
}
|
2006-06-26 09:27:43 +02:00
|
|
|
}
|
|
|
|
|
2009-03-31 06:21:40 +02:00
|
|
|
/* We need to process a full chunk at a time.
|
|
|
|
* If old and new chunk sizes differ, we need to process the
|
|
|
|
* largest of these
|
|
|
|
*/
|
2009-06-18 00:45:27 +02:00
|
|
|
if (mddev->new_chunk_sectors > mddev->chunk_sectors)
|
|
|
|
reshape_sectors = mddev->new_chunk_sectors;
|
2009-03-31 06:21:40 +02:00
|
|
|
else
|
2009-06-18 00:45:01 +02:00
|
|
|
reshape_sectors = mddev->chunk_sectors;
|
2009-03-31 06:21:40 +02:00
|
|
|
|
2006-06-26 09:27:43 +02:00
|
|
|
/* we update the metadata when there is more than 3Meg
|
|
|
|
* in the block range (that is rather arbitrary, should
|
|
|
|
* probably be time based) or when the data about to be
|
|
|
|
* copied would over-write the source of the data at
|
|
|
|
* the front of the range.
|
2009-03-31 06:16:46 +02:00
|
|
|
* i.e. one new_stripe along from reshape_progress new_maps
|
|
|
|
* to after where reshape_safe old_maps to
|
2006-06-26 09:27:43 +02:00
|
|
|
*/
|
2009-03-31 06:16:46 +02:00
|
|
|
writepos = conf->reshape_progress;
|
2007-03-01 05:11:53 +01:00
|
|
|
sector_div(writepos, new_data_disks);
|
2009-03-31 06:28:40 +02:00
|
|
|
readpos = conf->reshape_progress;
|
|
|
|
sector_div(readpos, data_disks);
|
2009-03-31 06:16:46 +02:00
|
|
|
safepos = conf->reshape_safe;
|
2007-03-01 05:11:53 +01:00
|
|
|
sector_div(safepos, data_disks);
|
2009-03-31 06:16:46 +02:00
|
|
|
if (mddev->delta_disks < 0) {
|
2009-05-27 13:39:05 +02:00
|
|
|
writepos -= min_t(sector_t, reshape_sectors, writepos);
|
2009-03-31 06:28:40 +02:00
|
|
|
readpos += reshape_sectors;
|
2009-03-31 06:21:40 +02:00
|
|
|
safepos += reshape_sectors;
|
2009-03-31 06:16:46 +02:00
|
|
|
} else {
|
2009-03-31 06:21:40 +02:00
|
|
|
writepos += reshape_sectors;
|
2009-05-27 13:39:05 +02:00
|
|
|
readpos -= min_t(sector_t, reshape_sectors, readpos);
|
|
|
|
safepos -= min_t(sector_t, reshape_sectors, safepos);
|
2009-03-31 06:16:46 +02:00
|
|
|
}
|
2006-06-26 09:27:43 +02:00
|
|
|
|
2009-03-31 06:28:40 +02:00
|
|
|
/* 'writepos' is the most advanced device address we might write.
|
|
|
|
* 'readpos' is the least advanced device address we might read.
|
|
|
|
* 'safepos' is the least address recorded in the metadata as having
|
|
|
|
* been reshaped.
|
|
|
|
* If 'readpos' is behind 'writepos', then there is no way that we can
|
|
|
|
* ensure safety in the face of a crash - that must be done by userspace
|
|
|
|
* making a backup of the data. So in that case there is no particular
|
|
|
|
* rush to update metadata.
|
|
|
|
* Otherwise if 'safepos' is behind 'writepos', then we really need to
|
|
|
|
* update the metadata to advance 'safepos' to match 'readpos' so that
|
|
|
|
* we can be safe in the event of a crash.
|
|
|
|
* So we insist on updating metadata if safepos is behind writepos and
|
|
|
|
* readpos is beyond writepos.
|
|
|
|
* In any case, update the metadata every 10 seconds.
|
|
|
|
* Maybe that number should be configurable, but I'm not sure it is
|
|
|
|
* worth it.... maybe it could be a multiple of safemode_delay???
|
|
|
|
*/
|
2009-03-31 06:16:46 +02:00
|
|
|
if ((mddev->delta_disks < 0
|
2009-03-31 06:28:40 +02:00
|
|
|
? (safepos > writepos && readpos < writepos)
|
|
|
|
: (safepos < writepos && readpos > writepos)) ||
|
|
|
|
time_after(jiffies, conf->reshape_checkpoint + 10*HZ)) {
|
2006-06-26 09:27:43 +02:00
|
|
|
/* Cannot proceed until we've updated the superblock... */
|
|
|
|
wait_event(conf->wait_for_overlap,
|
|
|
|
atomic_read(&conf->reshape_stripes)==0);
|
2009-03-31 06:16:46 +02:00
|
|
|
mddev->reshape_position = conf->reshape_progress;
|
2011-01-13 23:14:34 +01:00
|
|
|
mddev->curr_resync_completed = sector_nr;
|
2009-03-31 06:28:40 +02:00
|
|
|
conf->reshape_checkpoint = jiffies;
|
2006-10-03 10:15:46 +02:00
|
|
|
set_bit(MD_CHANGE_DEVS, &mddev->flags);
|
2006-06-26 09:27:43 +02:00
|
|
|
md_wakeup_thread(mddev->thread);
|
2006-10-03 10:15:46 +02:00
|
|
|
wait_event(mddev->sb_wait, mddev->flags == 0 ||
|
2006-06-26 09:27:43 +02:00
|
|
|
kthread_should_stop());
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
2009-03-31 06:16:46 +02:00
|
|
|
conf->reshape_safe = mddev->reshape_position;
|
2006-06-26 09:27:43 +02:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
wake_up(&conf->wait_for_overlap);
|
2009-04-14 08:28:34 +02:00
|
|
|
sysfs_notify(&mddev->kobj, NULL, "sync_completed");
|
2006-06-26 09:27:43 +02:00
|
|
|
}
|
|
|
|
|
2009-03-31 06:17:38 +02:00
|
|
|
if (mddev->delta_disks < 0) {
|
|
|
|
BUG_ON(conf->reshape_progress == 0);
|
|
|
|
stripe_addr = writepos;
|
|
|
|
BUG_ON((mddev->dev_sectors &
|
2009-03-31 06:21:40 +02:00
|
|
|
~((sector_t)reshape_sectors - 1))
|
|
|
|
- reshape_sectors - stripe_addr
|
2009-03-31 06:17:38 +02:00
|
|
|
!= sector_nr);
|
|
|
|
} else {
|
2009-03-31 06:21:40 +02:00
|
|
|
BUG_ON(writepos != sector_nr + reshape_sectors);
|
2009-03-31 06:17:38 +02:00
|
|
|
stripe_addr = sector_nr;
|
|
|
|
}
|
2009-03-31 06:26:47 +02:00
|
|
|
INIT_LIST_HEAD(&stripes);
|
2009-03-31 06:21:40 +02:00
|
|
|
for (i = 0; i < reshape_sectors; i += STRIPE_SECTORS) {
|
2006-06-26 09:27:43 +02:00
|
|
|
int j;
|
2009-09-23 10:06:41 +02:00
|
|
|
int skipped_disk = 0;
|
2009-06-09 06:39:59 +02:00
|
|
|
sh = get_active_stripe(conf, stripe_addr+i, 0, 0, 1);
|
2006-06-26 09:27:43 +02:00
|
|
|
set_bit(STRIPE_EXPANDING, &sh->state);
|
|
|
|
atomic_inc(&conf->reshape_stripes);
|
|
|
|
/* If any of this stripe is beyond the end of the old
|
|
|
|
* array, then we need to zero those blocks
|
|
|
|
*/
|
|
|
|
for (j=sh->disks; j--;) {
|
|
|
|
sector_t s;
|
|
|
|
if (j == sh->pd_idx)
|
|
|
|
continue;
|
2007-03-01 05:11:53 +01:00
|
|
|
if (conf->level == 6 &&
|
2009-03-31 05:39:38 +02:00
|
|
|
j == sh->qd_idx)
|
2007-03-01 05:11:53 +01:00
|
|
|
continue;
|
2009-03-31 06:19:07 +02:00
|
|
|
s = compute_blocknr(sh, j, 0);
|
2009-03-31 06:00:31 +02:00
|
|
|
if (s < raid5_size(mddev, 0, 0)) {
|
2009-09-23 10:06:41 +02:00
|
|
|
skipped_disk = 1;
|
2006-06-26 09:27:43 +02:00
|
|
|
continue;
|
|
|
|
}
|
|
|
|
memset(page_address(sh->dev[j].page), 0, STRIPE_SIZE);
|
|
|
|
set_bit(R5_Expanded, &sh->dev[j].flags);
|
|
|
|
set_bit(R5_UPTODATE, &sh->dev[j].flags);
|
|
|
|
}
|
2009-09-23 10:06:41 +02:00
|
|
|
if (!skipped_disk) {
|
2006-06-26 09:27:43 +02:00
|
|
|
set_bit(STRIPE_EXPAND_READY, &sh->state);
|
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
}
|
2009-03-31 06:26:47 +02:00
|
|
|
list_add(&sh->lru, &stripes);
|
2006-06-26 09:27:43 +02:00
|
|
|
}
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
2009-03-31 06:16:46 +02:00
|
|
|
if (mddev->delta_disks < 0)
|
2009-03-31 06:21:40 +02:00
|
|
|
conf->reshape_progress -= reshape_sectors * new_data_disks;
|
2009-03-31 06:16:46 +02:00
|
|
|
else
|
2009-03-31 06:21:40 +02:00
|
|
|
conf->reshape_progress += reshape_sectors * new_data_disks;
|
2006-06-26 09:27:43 +02:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
/* Ok, those stripe are ready. We can start scheduling
|
|
|
|
* reads on the source stripes.
|
|
|
|
* The source stripes are determined by mapping the first and last
|
|
|
|
* block on the destination stripes.
|
|
|
|
*/
|
|
|
|
first_sector =
|
2009-03-31 06:17:38 +02:00
|
|
|
raid5_compute_sector(conf, stripe_addr*(new_data_disks),
|
2009-03-31 05:39:38 +02:00
|
|
|
1, &dd_idx, NULL);
|
2006-06-26 09:27:43 +02:00
|
|
|
last_sector =
|
2009-06-09 08:32:22 +02:00
|
|
|
raid5_compute_sector(conf, ((stripe_addr+reshape_sectors)
|
2009-06-18 00:45:55 +02:00
|
|
|
* new_data_disks - 1),
|
2009-03-31 05:39:38 +02:00
|
|
|
1, &dd_idx, NULL);
|
2009-03-31 05:33:13 +02:00
|
|
|
if (last_sector >= mddev->dev_sectors)
|
|
|
|
last_sector = mddev->dev_sectors - 1;
|
2006-06-26 09:27:43 +02:00
|
|
|
while (first_sector <= last_sector) {
|
2009-06-09 06:39:59 +02:00
|
|
|
sh = get_active_stripe(conf, first_sector, 1, 0, 1);
|
2006-06-26 09:27:43 +02:00
|
|
|
set_bit(STRIPE_EXPAND_SOURCE, &sh->state);
|
|
|
|
set_bit(STRIPE_HANDLE, &sh->state);
|
|
|
|
release_stripe(sh);
|
|
|
|
first_sector += STRIPE_SECTORS;
|
|
|
|
}
|
2009-03-31 06:26:47 +02:00
|
|
|
/* Now that the sources are clearly marked, we can release
|
|
|
|
* the destination stripes
|
|
|
|
*/
|
|
|
|
while (!list_empty(&stripes)) {
|
|
|
|
sh = list_entry(stripes.next, struct stripe_head, lru);
|
|
|
|
list_del_init(&sh->lru);
|
|
|
|
release_stripe(sh);
|
|
|
|
}
|
2008-02-06 10:39:52 +01:00
|
|
|
/* If this takes us to the resync_max point where we have to pause,
|
|
|
|
* then we need to write out the superblock.
|
|
|
|
*/
|
2009-03-31 06:21:40 +02:00
|
|
|
sector_nr += reshape_sectors;
|
2009-04-17 03:06:30 +02:00
|
|
|
if ((sector_nr - mddev->curr_resync_completed) * 2
|
|
|
|
>= mddev->resync_max - mddev->curr_resync_completed) {
|
2008-02-06 10:39:52 +01:00
|
|
|
/* Cannot proceed until we've updated the superblock... */
|
|
|
|
wait_event(conf->wait_for_overlap,
|
|
|
|
atomic_read(&conf->reshape_stripes) == 0);
|
2009-03-31 06:16:46 +02:00
|
|
|
mddev->reshape_position = conf->reshape_progress;
|
2011-01-13 23:14:34 +01:00
|
|
|
mddev->curr_resync_completed = sector_nr;
|
2009-03-31 06:28:40 +02:00
|
|
|
conf->reshape_checkpoint = jiffies;
|
2008-02-06 10:39:52 +01:00
|
|
|
set_bit(MD_CHANGE_DEVS, &mddev->flags);
|
|
|
|
md_wakeup_thread(mddev->thread);
|
|
|
|
wait_event(mddev->sb_wait,
|
|
|
|
!test_bit(MD_CHANGE_DEVS, &mddev->flags)
|
|
|
|
|| kthread_should_stop());
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
2009-03-31 06:16:46 +02:00
|
|
|
conf->reshape_safe = mddev->reshape_position;
|
2008-02-06 10:39:52 +01:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
wake_up(&conf->wait_for_overlap);
|
2009-04-14 08:28:34 +02:00
|
|
|
sysfs_notify(&mddev->kobj, NULL, "sync_completed");
|
2008-02-06 10:39:52 +01:00
|
|
|
}
|
2009-03-31 06:21:40 +02:00
|
|
|
return reshape_sectors;
|
2006-06-26 09:27:43 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
/* FIXME go_faster isn't used */
|
|
|
|
static inline sector_t sync_request(mddev_t *mddev, sector_t sector_nr, int *skipped, int go_faster)
|
|
|
|
{
|
2010-03-08 06:02:40 +01:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2006-06-26 09:27:43 +02:00
|
|
|
struct stripe_head *sh;
|
2009-03-31 05:33:13 +02:00
|
|
|
sector_t max_sector = mddev->dev_sectors;
|
2010-10-19 01:03:39 +02:00
|
|
|
sector_t sync_blocks;
|
2006-06-26 09:27:38 +02:00
|
|
|
int still_degraded = 0;
|
|
|
|
int i;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2005-09-10 01:23:54 +02:00
|
|
|
if (sector_nr >= max_sector) {
|
2005-04-17 00:20:36 +02:00
|
|
|
/* just being told to finish up .. nothing much to do */
|
2009-03-31 06:15:05 +02:00
|
|
|
|
2006-03-27 11:18:10 +02:00
|
|
|
if (test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery)) {
|
|
|
|
end_reshape(conf);
|
|
|
|
return 0;
|
|
|
|
}
|
2005-09-10 01:23:54 +02:00
|
|
|
|
|
|
|
if (mddev->curr_resync < max_sector) /* aborted */
|
|
|
|
bitmap_end_sync(mddev->bitmap, mddev->curr_resync,
|
|
|
|
&sync_blocks, 1);
|
2006-06-26 09:27:38 +02:00
|
|
|
else /* completed sync */
|
2005-09-10 01:23:54 +02:00
|
|
|
conf->fullsync = 0;
|
|
|
|
bitmap_close_sync(mddev->bitmap);
|
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
return 0;
|
|
|
|
}
|
2006-03-27 11:18:09 +02:00
|
|
|
|
2009-08-03 02:59:58 +02:00
|
|
|
/* Allow raid5_quiesce to complete */
|
|
|
|
wait_event(conf->wait_for_overlap, conf->quiesce != 2);
|
|
|
|
|
2006-06-26 09:27:43 +02:00
|
|
|
if (test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery))
|
|
|
|
return reshape_request(mddev, sector_nr, skipped);
|
2006-03-27 11:18:11 +02:00
|
|
|
|
2008-02-06 10:39:52 +01:00
|
|
|
/* No need to check resync_max as we never do more than one
|
|
|
|
* stripe, and as resync_max will always be on a chunk boundary,
|
|
|
|
* if the check in md_do_sync didn't fire, there is no chance
|
|
|
|
* of overstepping resync_max here
|
|
|
|
*/
|
|
|
|
|
2006-06-26 09:27:38 +02:00
|
|
|
/* if there is too many failed drives and we are trying
|
2005-04-17 00:20:36 +02:00
|
|
|
* to resync, then assert that we are finished, because there is
|
|
|
|
* nothing we can do.
|
|
|
|
*/
|
2006-06-26 09:27:55 +02:00
|
|
|
if (mddev->degraded >= conf->max_degraded &&
|
2006-06-26 09:27:38 +02:00
|
|
|
test_bit(MD_RECOVERY_SYNC, &mddev->recovery)) {
|
2009-03-31 05:33:13 +02:00
|
|
|
sector_t rv = mddev->dev_sectors - sector_nr;
|
2005-06-22 02:17:13 +02:00
|
|
|
*skipped = 1;
|
2005-04-17 00:20:36 +02:00
|
|
|
return rv;
|
|
|
|
}
|
2005-09-10 01:23:54 +02:00
|
|
|
if (!bitmap_start_sync(mddev->bitmap, sector_nr, &sync_blocks, 1) &&
|
2005-11-09 06:39:38 +01:00
|
|
|
!test_bit(MD_RECOVERY_REQUESTED, &mddev->recovery) &&
|
2005-09-10 01:23:54 +02:00
|
|
|
!conf->fullsync && sync_blocks >= STRIPE_SECTORS) {
|
|
|
|
/* we can skip this block, and probably more */
|
|
|
|
sync_blocks /= STRIPE_SECTORS;
|
|
|
|
*skipped = 1;
|
|
|
|
return sync_blocks * STRIPE_SECTORS; /* keep things rounded to whole stripes */
|
|
|
|
}
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2008-02-06 10:39:50 +01:00
|
|
|
|
|
|
|
bitmap_cond_end_sync(mddev->bitmap, sector_nr);
|
|
|
|
|
2009-06-09 06:39:59 +02:00
|
|
|
sh = get_active_stripe(conf, sector_nr, 0, 1, 0);
|
2005-04-17 00:20:36 +02:00
|
|
|
if (sh == NULL) {
|
2009-06-09 06:39:59 +02:00
|
|
|
sh = get_active_stripe(conf, sector_nr, 0, 0, 0);
|
2005-04-17 00:20:36 +02:00
|
|
|
/* make sure we don't swamp the stripe cache if someone else
|
2006-06-26 09:27:38 +02:00
|
|
|
* is trying to get access
|
2005-04-17 00:20:36 +02:00
|
|
|
*/
|
2005-11-07 10:01:17 +01:00
|
|
|
schedule_timeout_uninterruptible(1);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
2006-06-26 09:27:38 +02:00
|
|
|
/* Need to check if array will still be degraded after recovery/resync
|
|
|
|
* We don't need to check the 'failed' flag as when that gets set,
|
|
|
|
* recovery aborts.
|
|
|
|
*/
|
2009-06-09 06:30:31 +02:00
|
|
|
for (i = 0; i < conf->raid_disks; i++)
|
2006-06-26 09:27:38 +02:00
|
|
|
if (conf->disks[i].rdev == NULL)
|
|
|
|
still_degraded = 1;
|
|
|
|
|
|
|
|
bitmap_start_sync(mddev->bitmap, sector_nr, &sync_blocks, still_degraded);
|
|
|
|
|
2011-07-26 03:19:49 +02:00
|
|
|
set_bit(STRIPE_SYNC_REQUESTED, &sh->state);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2009-10-16 06:55:25 +02:00
|
|
|
handle_stripe(sh);
|
2005-04-17 00:20:36 +02:00
|
|
|
release_stripe(sh);
|
|
|
|
|
|
|
|
return STRIPE_SECTORS;
|
|
|
|
}
|
|
|
|
|
2006-12-10 11:20:47 +01:00
|
|
|
static int retry_aligned_read(raid5_conf_t *conf, struct bio *raid_bio)
|
|
|
|
{
|
|
|
|
/* We may not be able to submit a whole bio at once as there
|
|
|
|
* may not be enough stripe_heads available.
|
|
|
|
* We cannot pre-allocate enough stripe_heads as we may need
|
|
|
|
* more than exist in the cache (if we allow ever large chunks).
|
|
|
|
* So we do one stripe head at a time and record in
|
|
|
|
* ->bi_hw_segments how many have been done.
|
|
|
|
*
|
|
|
|
* We *know* that this entire raid_bio is in one chunk, so
|
|
|
|
* it will be only one 'dd_idx' and only need one call to raid5_compute_sector.
|
|
|
|
*/
|
|
|
|
struct stripe_head *sh;
|
2009-03-31 05:39:38 +02:00
|
|
|
int dd_idx;
|
2006-12-10 11:20:47 +01:00
|
|
|
sector_t sector, logical_sector, last_sector;
|
|
|
|
int scnt = 0;
|
|
|
|
int remaining;
|
|
|
|
int handled = 0;
|
|
|
|
|
|
|
|
logical_sector = raid_bio->bi_sector & ~((sector_t)STRIPE_SECTORS-1);
|
2009-03-31 05:39:38 +02:00
|
|
|
sector = raid5_compute_sector(conf, logical_sector,
|
2009-03-31 05:39:38 +02:00
|
|
|
0, &dd_idx, NULL);
|
2006-12-10 11:20:47 +01:00
|
|
|
last_sector = raid_bio->bi_sector + (raid_bio->bi_size>>9);
|
|
|
|
|
|
|
|
for (; logical_sector < last_sector;
|
2007-02-08 23:20:29 +01:00
|
|
|
logical_sector += STRIPE_SECTORS,
|
|
|
|
sector += STRIPE_SECTORS,
|
|
|
|
scnt++) {
|
2006-12-10 11:20:47 +01:00
|
|
|
|
2008-08-15 10:41:18 +02:00
|
|
|
if (scnt < raid5_bi_hw_segments(raid_bio))
|
2006-12-10 11:20:47 +01:00
|
|
|
/* already done this stripe */
|
|
|
|
continue;
|
|
|
|
|
2009-06-09 06:39:59 +02:00
|
|
|
sh = get_active_stripe(conf, sector, 0, 1, 0);
|
2006-12-10 11:20:47 +01:00
|
|
|
|
|
|
|
if (!sh) {
|
|
|
|
/* failed to get a stripe - must wait */
|
2008-08-15 10:41:18 +02:00
|
|
|
raid5_set_bi_hw_segments(raid_bio, scnt);
|
2006-12-10 11:20:47 +01:00
|
|
|
conf->retry_read_aligned = raid_bio;
|
|
|
|
return handled;
|
|
|
|
}
|
|
|
|
|
|
|
|
set_bit(R5_ReadError, &sh->dev[dd_idx].flags);
|
2007-02-08 23:20:29 +01:00
|
|
|
if (!add_stripe_bio(sh, raid_bio, dd_idx, 0)) {
|
|
|
|
release_stripe(sh);
|
2008-08-15 10:41:18 +02:00
|
|
|
raid5_set_bi_hw_segments(raid_bio, scnt);
|
2007-02-08 23:20:29 +01:00
|
|
|
conf->retry_read_aligned = raid_bio;
|
|
|
|
return handled;
|
|
|
|
}
|
|
|
|
|
2009-07-14 20:48:22 +02:00
|
|
|
handle_stripe(sh);
|
2006-12-10 11:20:47 +01:00
|
|
|
release_stripe(sh);
|
|
|
|
handled++;
|
|
|
|
}
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
2008-08-15 10:41:18 +02:00
|
|
|
remaining = raid5_dec_bi_phys_segments(raid_bio);
|
2006-12-10 11:20:47 +01:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
2008-06-28 00:31:20 +02:00
|
|
|
if (remaining == 0)
|
|
|
|
bio_endio(raid_bio, 0);
|
2006-12-10 11:20:47 +01:00
|
|
|
if (atomic_dec_and_test(&conf->active_aligned_reads))
|
|
|
|
wake_up(&conf->wait_for_stripe);
|
|
|
|
return handled;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
/*
|
|
|
|
* This is our raid5 kernel thread.
|
|
|
|
*
|
|
|
|
* We scan the hash table for stripes which can be handled now.
|
|
|
|
* During the scan, completed stripes are saved for us by the interrupt
|
|
|
|
* handler, so that they will not have to wait for our next wakeup.
|
|
|
|
*/
|
2008-02-06 10:40:00 +01:00
|
|
|
static void raid5d(mddev_t *mddev)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
struct stripe_head *sh;
|
2009-06-16 08:54:21 +02:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2005-04-17 00:20:36 +02:00
|
|
|
int handled;
|
2011-04-18 10:25:41 +02:00
|
|
|
struct blk_plug plug;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("+++ raid5d active\n");
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
md_check_recovery(mddev);
|
|
|
|
|
2011-04-18 10:25:41 +02:00
|
|
|
blk_start_plug(&plug);
|
2005-04-17 00:20:36 +02:00
|
|
|
handled = 0;
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
|
|
|
while (1) {
|
2006-12-10 11:20:47 +01:00
|
|
|
struct bio *bio;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2011-04-18 10:25:43 +02:00
|
|
|
if (atomic_read(&mddev->plug_cnt) == 0 &&
|
|
|
|
!list_empty(&conf->bitmap_list)) {
|
|
|
|
/* Now is a good time to flush some bitmap updates */
|
|
|
|
conf->seq_flush++;
|
2005-11-28 22:44:10 +01:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
2005-09-10 01:23:54 +02:00
|
|
|
bitmap_unplug(mddev->bitmap);
|
2005-11-28 22:44:10 +01:00
|
|
|
spin_lock_irq(&conf->device_lock);
|
2011-04-18 10:25:43 +02:00
|
|
|
conf->seq_write = conf->seq_flush;
|
2005-09-10 01:23:54 +02:00
|
|
|
activate_bit_delay(conf);
|
|
|
|
}
|
2011-04-18 10:25:43 +02:00
|
|
|
if (atomic_read(&mddev->plug_cnt) == 0)
|
|
|
|
raid5_activate_delayed(conf);
|
2005-09-10 01:23:54 +02:00
|
|
|
|
2006-12-10 11:20:47 +01:00
|
|
|
while ((bio = remove_bio_from_retry(conf))) {
|
|
|
|
int ok;
|
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
ok = retry_aligned_read(conf, bio);
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
|
|
|
if (!ok)
|
|
|
|
break;
|
|
|
|
handled++;
|
|
|
|
}
|
|
|
|
|
2008-04-28 11:15:53 +02:00
|
|
|
sh = __get_priority_stripe(conf);
|
|
|
|
|
2008-07-23 21:05:51 +02:00
|
|
|
if (!sh)
|
2005-04-17 00:20:36 +02:00
|
|
|
break;
|
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
|
|
|
|
handled++;
|
2009-10-16 07:25:22 +02:00
|
|
|
handle_stripe(sh);
|
|
|
|
release_stripe(sh);
|
|
|
|
cond_resched();
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
|
|
|
}
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("%d stripes handled\n", handled);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
|
2008-07-23 21:05:51 +02:00
|
|
|
async_tx_issue_pending_all();
|
2011-04-18 10:25:41 +02:00
|
|
|
blk_finish_plug(&plug);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
pr_debug("--- raid5d inactive\n");
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
2005-11-09 06:39:25 +01:00
|
|
|
static ssize_t
|
2005-11-09 06:39:30 +01:00
|
|
|
raid5_show_stripe_cache_size(mddev_t *mddev, char *page)
|
2005-11-09 06:39:25 +01:00
|
|
|
{
|
2009-06-16 08:54:21 +02:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2005-11-09 06:39:39 +01:00
|
|
|
if (conf)
|
|
|
|
return sprintf(page, "%d\n", conf->max_nr_stripes);
|
|
|
|
else
|
|
|
|
return 0;
|
2005-11-09 06:39:25 +01:00
|
|
|
}
|
|
|
|
|
2010-06-01 11:37:24 +02:00
|
|
|
int
|
|
|
|
raid5_set_cache_size(mddev_t *mddev, int size)
|
2005-11-09 06:39:25 +01:00
|
|
|
{
|
2009-06-16 08:54:21 +02:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2008-06-28 06:44:04 +02:00
|
|
|
int err;
|
|
|
|
|
2010-06-01 11:37:24 +02:00
|
|
|
if (size <= 16 || size > 32768)
|
2005-11-09 06:39:25 +01:00
|
|
|
return -EINVAL;
|
2010-06-01 11:37:24 +02:00
|
|
|
while (size < conf->max_nr_stripes) {
|
2005-11-09 06:39:25 +01:00
|
|
|
if (drop_one_stripe(conf))
|
|
|
|
conf->max_nr_stripes--;
|
|
|
|
else
|
|
|
|
break;
|
|
|
|
}
|
2008-06-28 06:44:04 +02:00
|
|
|
err = md_allow_write(mddev);
|
|
|
|
if (err)
|
|
|
|
return err;
|
2010-06-01 11:37:24 +02:00
|
|
|
while (size > conf->max_nr_stripes) {
|
2005-11-09 06:39:25 +01:00
|
|
|
if (grow_one_stripe(conf))
|
|
|
|
conf->max_nr_stripes++;
|
|
|
|
else break;
|
|
|
|
}
|
2010-06-01 11:37:24 +02:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL(raid5_set_cache_size);
|
|
|
|
|
|
|
|
static ssize_t
|
|
|
|
raid5_store_stripe_cache_size(mddev_t *mddev, const char *page, size_t len)
|
|
|
|
{
|
|
|
|
raid5_conf_t *conf = mddev->private;
|
|
|
|
unsigned long new;
|
|
|
|
int err;
|
|
|
|
|
|
|
|
if (len >= PAGE_SIZE)
|
|
|
|
return -EINVAL;
|
|
|
|
if (!conf)
|
|
|
|
return -ENODEV;
|
|
|
|
|
|
|
|
if (strict_strtoul(page, 10, &new))
|
|
|
|
return -EINVAL;
|
|
|
|
err = raid5_set_cache_size(mddev, new);
|
|
|
|
if (err)
|
|
|
|
return err;
|
2005-11-09 06:39:25 +01:00
|
|
|
return len;
|
|
|
|
}
|
2005-11-09 06:39:30 +01:00
|
|
|
|
2005-11-09 06:39:39 +01:00
|
|
|
static struct md_sysfs_entry
|
|
|
|
raid5_stripecache_size = __ATTR(stripe_cache_size, S_IRUGO | S_IWUSR,
|
|
|
|
raid5_show_stripe_cache_size,
|
|
|
|
raid5_store_stripe_cache_size);
|
2005-11-09 06:39:25 +01:00
|
|
|
|
2008-04-28 11:15:53 +02:00
|
|
|
static ssize_t
|
|
|
|
raid5_show_preread_threshold(mddev_t *mddev, char *page)
|
|
|
|
{
|
2009-06-16 08:54:21 +02:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2008-04-28 11:15:53 +02:00
|
|
|
if (conf)
|
|
|
|
return sprintf(page, "%d\n", conf->bypass_threshold);
|
|
|
|
else
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static ssize_t
|
|
|
|
raid5_store_preread_threshold(mddev_t *mddev, const char *page, size_t len)
|
|
|
|
{
|
2009-06-16 08:54:21 +02:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2008-04-28 11:15:54 +02:00
|
|
|
unsigned long new;
|
2008-04-28 11:15:53 +02:00
|
|
|
if (len >= PAGE_SIZE)
|
|
|
|
return -EINVAL;
|
|
|
|
if (!conf)
|
|
|
|
return -ENODEV;
|
|
|
|
|
2008-04-28 11:15:54 +02:00
|
|
|
if (strict_strtoul(page, 10, &new))
|
2008-04-28 11:15:53 +02:00
|
|
|
return -EINVAL;
|
2008-04-28 11:15:54 +02:00
|
|
|
if (new > conf->max_nr_stripes)
|
2008-04-28 11:15:53 +02:00
|
|
|
return -EINVAL;
|
|
|
|
conf->bypass_threshold = new;
|
|
|
|
return len;
|
|
|
|
}
|
|
|
|
|
|
|
|
static struct md_sysfs_entry
|
|
|
|
raid5_preread_bypass_threshold = __ATTR(preread_bypass_threshold,
|
|
|
|
S_IRUGO | S_IWUSR,
|
|
|
|
raid5_show_preread_threshold,
|
|
|
|
raid5_store_preread_threshold);
|
|
|
|
|
2005-11-09 06:39:25 +01:00
|
|
|
static ssize_t
|
2005-11-09 06:39:39 +01:00
|
|
|
stripe_cache_active_show(mddev_t *mddev, char *page)
|
2005-11-09 06:39:25 +01:00
|
|
|
{
|
2009-06-16 08:54:21 +02:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2005-11-09 06:39:39 +01:00
|
|
|
if (conf)
|
|
|
|
return sprintf(page, "%d\n", atomic_read(&conf->active_stripes));
|
|
|
|
else
|
|
|
|
return 0;
|
2005-11-09 06:39:25 +01:00
|
|
|
}
|
|
|
|
|
2005-11-09 06:39:39 +01:00
|
|
|
static struct md_sysfs_entry
|
|
|
|
raid5_stripecache_active = __ATTR_RO(stripe_cache_active);
|
2005-11-09 06:39:25 +01:00
|
|
|
|
2005-11-09 06:39:30 +01:00
|
|
|
static struct attribute *raid5_attrs[] = {
|
2005-11-09 06:39:25 +01:00
|
|
|
&raid5_stripecache_size.attr,
|
|
|
|
&raid5_stripecache_active.attr,
|
2008-04-28 11:15:53 +02:00
|
|
|
&raid5_preread_bypass_threshold.attr,
|
2005-11-09 06:39:25 +01:00
|
|
|
NULL,
|
|
|
|
};
|
2005-11-09 06:39:30 +01:00
|
|
|
static struct attribute_group raid5_attrs_group = {
|
|
|
|
.name = NULL,
|
|
|
|
.attrs = raid5_attrs,
|
2005-11-09 06:39:25 +01:00
|
|
|
};
|
|
|
|
|
2009-03-18 02:10:40 +01:00
|
|
|
static sector_t
|
|
|
|
raid5_size(mddev_t *mddev, sector_t sectors, int raid_disks)
|
|
|
|
{
|
2009-06-16 08:54:21 +02:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2009-03-18 02:10:40 +01:00
|
|
|
|
|
|
|
if (!sectors)
|
|
|
|
sectors = mddev->dev_sectors;
|
2009-10-16 07:35:30 +02:00
|
|
|
if (!raid_disks)
|
2009-03-31 06:10:36 +02:00
|
|
|
/* size is defined by the smallest of previous and new size */
|
2009-10-16 07:35:30 +02:00
|
|
|
raid_disks = min(conf->raid_disks, conf->previous_raid_disks);
|
2009-03-18 02:10:40 +01:00
|
|
|
|
2009-06-18 00:45:01 +02:00
|
|
|
sectors &= ~((sector_t)mddev->chunk_sectors - 1);
|
2009-06-18 00:45:27 +02:00
|
|
|
sectors &= ~((sector_t)mddev->new_chunk_sectors - 1);
|
2009-03-18 02:10:40 +01:00
|
|
|
return sectors * (raid_disks - conf->max_degraded);
|
|
|
|
}
|
|
|
|
|
2009-07-14 20:48:22 +02:00
|
|
|
static void raid5_free_percpu(raid5_conf_t *conf)
|
|
|
|
{
|
|
|
|
struct raid5_percpu *percpu;
|
|
|
|
unsigned long cpu;
|
|
|
|
|
|
|
|
if (!conf->percpu)
|
|
|
|
return;
|
|
|
|
|
|
|
|
get_online_cpus();
|
|
|
|
for_each_possible_cpu(cpu) {
|
|
|
|
percpu = per_cpu_ptr(conf->percpu, cpu);
|
|
|
|
safe_put_page(percpu->spare_page);
|
2009-07-14 20:50:52 +02:00
|
|
|
kfree(percpu->scribble);
|
2009-07-14 20:48:22 +02:00
|
|
|
}
|
|
|
|
#ifdef CONFIG_HOTPLUG_CPU
|
|
|
|
unregister_cpu_notifier(&conf->cpu_notify);
|
|
|
|
#endif
|
|
|
|
put_online_cpus();
|
|
|
|
|
|
|
|
free_percpu(conf->percpu);
|
|
|
|
}
|
|
|
|
|
2009-07-31 04:39:15 +02:00
|
|
|
static void free_conf(raid5_conf_t *conf)
|
|
|
|
{
|
|
|
|
shrink_stripes(conf);
|
2009-07-14 20:48:22 +02:00
|
|
|
raid5_free_percpu(conf);
|
2009-07-31 04:39:15 +02:00
|
|
|
kfree(conf->disks);
|
|
|
|
kfree(conf->stripe_hashtbl);
|
|
|
|
kfree(conf);
|
|
|
|
}
|
|
|
|
|
2009-07-14 20:48:22 +02:00
|
|
|
#ifdef CONFIG_HOTPLUG_CPU
|
|
|
|
static int raid456_cpu_notify(struct notifier_block *nfb, unsigned long action,
|
|
|
|
void *hcpu)
|
|
|
|
{
|
|
|
|
raid5_conf_t *conf = container_of(nfb, raid5_conf_t, cpu_notify);
|
|
|
|
long cpu = (long)hcpu;
|
|
|
|
struct raid5_percpu *percpu = per_cpu_ptr(conf->percpu, cpu);
|
|
|
|
|
|
|
|
switch (action) {
|
|
|
|
case CPU_UP_PREPARE:
|
|
|
|
case CPU_UP_PREPARE_FROZEN:
|
2009-07-14 20:50:52 +02:00
|
|
|
if (conf->level == 6 && !percpu->spare_page)
|
2009-07-14 20:48:22 +02:00
|
|
|
percpu->spare_page = alloc_page(GFP_KERNEL);
|
2009-07-14 20:50:52 +02:00
|
|
|
if (!percpu->scribble)
|
|
|
|
percpu->scribble = kmalloc(conf->scribble_len, GFP_KERNEL);
|
|
|
|
|
|
|
|
if (!percpu->scribble ||
|
|
|
|
(conf->level == 6 && !percpu->spare_page)) {
|
|
|
|
safe_put_page(percpu->spare_page);
|
|
|
|
kfree(percpu->scribble);
|
2009-07-14 20:48:22 +02:00
|
|
|
pr_err("%s: failed memory allocation for cpu%ld\n",
|
|
|
|
__func__, cpu);
|
2010-05-26 23:43:35 +02:00
|
|
|
return notifier_from_errno(-ENOMEM);
|
2009-07-14 20:48:22 +02:00
|
|
|
}
|
|
|
|
break;
|
|
|
|
case CPU_DEAD:
|
|
|
|
case CPU_DEAD_FROZEN:
|
|
|
|
safe_put_page(percpu->spare_page);
|
2009-07-14 20:50:52 +02:00
|
|
|
kfree(percpu->scribble);
|
2009-07-14 20:48:22 +02:00
|
|
|
percpu->spare_page = NULL;
|
2009-07-14 20:50:52 +02:00
|
|
|
percpu->scribble = NULL;
|
2009-07-14 20:48:22 +02:00
|
|
|
break;
|
|
|
|
default:
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
return NOTIFY_OK;
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
|
|
|
static int raid5_alloc_percpu(raid5_conf_t *conf)
|
|
|
|
{
|
|
|
|
unsigned long cpu;
|
|
|
|
struct page *spare_page;
|
2010-02-02 06:39:15 +01:00
|
|
|
struct raid5_percpu __percpu *allcpus;
|
2009-07-14 20:50:52 +02:00
|
|
|
void *scribble;
|
2009-07-14 20:48:22 +02:00
|
|
|
int err;
|
|
|
|
|
|
|
|
allcpus = alloc_percpu(struct raid5_percpu);
|
|
|
|
if (!allcpus)
|
|
|
|
return -ENOMEM;
|
|
|
|
conf->percpu = allcpus;
|
|
|
|
|
|
|
|
get_online_cpus();
|
|
|
|
err = 0;
|
|
|
|
for_each_present_cpu(cpu) {
|
2009-07-14 20:50:52 +02:00
|
|
|
if (conf->level == 6) {
|
|
|
|
spare_page = alloc_page(GFP_KERNEL);
|
|
|
|
if (!spare_page) {
|
|
|
|
err = -ENOMEM;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
per_cpu_ptr(conf->percpu, cpu)->spare_page = spare_page;
|
|
|
|
}
|
2009-10-16 07:35:30 +02:00
|
|
|
scribble = kmalloc(conf->scribble_len, GFP_KERNEL);
|
2009-07-14 20:50:52 +02:00
|
|
|
if (!scribble) {
|
2009-07-14 20:48:22 +02:00
|
|
|
err = -ENOMEM;
|
|
|
|
break;
|
|
|
|
}
|
2009-07-14 20:50:52 +02:00
|
|
|
per_cpu_ptr(conf->percpu, cpu)->scribble = scribble;
|
2009-07-14 20:48:22 +02:00
|
|
|
}
|
|
|
|
#ifdef CONFIG_HOTPLUG_CPU
|
|
|
|
conf->cpu_notify.notifier_call = raid456_cpu_notify;
|
|
|
|
conf->cpu_notify.priority = 0;
|
|
|
|
if (err == 0)
|
|
|
|
err = register_cpu_notifier(&conf->cpu_notify);
|
|
|
|
#endif
|
|
|
|
put_online_cpus();
|
|
|
|
|
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2009-03-31 05:39:39 +02:00
|
|
|
static raid5_conf_t *setup_conf(mddev_t *mddev)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
raid5_conf_t *conf;
|
2009-10-16 07:35:30 +02:00
|
|
|
int raid_disk, memory, max_disks;
|
2005-04-17 00:20:36 +02:00
|
|
|
mdk_rdev_t *rdev;
|
|
|
|
struct disk_info *disk;
|
|
|
|
|
2009-03-31 05:39:39 +02:00
|
|
|
if (mddev->new_level != 5
|
|
|
|
&& mddev->new_level != 4
|
|
|
|
&& mddev->new_level != 6) {
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_ERR "md/raid:%s: raid level not set to 4/5/6 (%d)\n",
|
2009-03-31 05:39:39 +02:00
|
|
|
mdname(mddev), mddev->new_level);
|
|
|
|
return ERR_PTR(-EIO);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
2009-03-31 05:39:39 +02:00
|
|
|
if ((mddev->new_level == 5
|
|
|
|
&& !algorithm_valid_raid5(mddev->new_layout)) ||
|
|
|
|
(mddev->new_level == 6
|
|
|
|
&& !algorithm_valid_raid6(mddev->new_layout))) {
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_ERR "md/raid:%s: layout %d not supported\n",
|
2009-03-31 05:39:39 +02:00
|
|
|
mdname(mddev), mddev->new_layout);
|
|
|
|
return ERR_PTR(-EIO);
|
2009-03-31 05:39:38 +02:00
|
|
|
}
|
2009-03-31 05:39:39 +02:00
|
|
|
if (mddev->new_level == 6 && mddev->raid_disks < 4) {
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_ERR "md/raid:%s: not enough configured devices (%d, minimum 4)\n",
|
2009-03-31 05:39:39 +02:00
|
|
|
mdname(mddev), mddev->raid_disks);
|
|
|
|
return ERR_PTR(-EINVAL);
|
2008-10-13 02:55:12 +02:00
|
|
|
}
|
|
|
|
|
2009-06-18 00:45:27 +02:00
|
|
|
if (!mddev->new_chunk_sectors ||
|
|
|
|
(mddev->new_chunk_sectors << 9) % PAGE_SIZE ||
|
|
|
|
!is_power_of_2(mddev->new_chunk_sectors)) {
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_ERR "md/raid:%s: invalid chunk size %d\n",
|
|
|
|
mdname(mddev), mddev->new_chunk_sectors << 9);
|
2009-03-31 05:39:39 +02:00
|
|
|
return ERR_PTR(-EINVAL);
|
2006-03-27 11:18:11 +02:00
|
|
|
}
|
|
|
|
|
2009-03-31 05:39:39 +02:00
|
|
|
conf = kzalloc(sizeof(raid5_conf_t), GFP_KERNEL);
|
|
|
|
if (conf == NULL)
|
2005-04-17 00:20:36 +02:00
|
|
|
goto abort;
|
2009-10-16 06:55:38 +02:00
|
|
|
spin_lock_init(&conf->device_lock);
|
|
|
|
init_waitqueue_head(&conf->wait_for_stripe);
|
|
|
|
init_waitqueue_head(&conf->wait_for_overlap);
|
|
|
|
INIT_LIST_HEAD(&conf->handle_list);
|
|
|
|
INIT_LIST_HEAD(&conf->hold_list);
|
|
|
|
INIT_LIST_HEAD(&conf->delayed_list);
|
|
|
|
INIT_LIST_HEAD(&conf->bitmap_list);
|
|
|
|
INIT_LIST_HEAD(&conf->inactive_list);
|
|
|
|
atomic_set(&conf->active_stripes, 0);
|
|
|
|
atomic_set(&conf->preread_active_stripes, 0);
|
|
|
|
atomic_set(&conf->active_aligned_reads, 0);
|
|
|
|
conf->bypass_threshold = BYPASS_THRESHOLD;
|
2009-03-31 05:39:39 +02:00
|
|
|
|
|
|
|
conf->raid_disks = mddev->raid_disks;
|
|
|
|
if (mddev->reshape_position == MaxSector)
|
|
|
|
conf->previous_raid_disks = mddev->raid_disks;
|
|
|
|
else
|
2006-03-27 11:18:11 +02:00
|
|
|
conf->previous_raid_disks = mddev->raid_disks - mddev->delta_disks;
|
2009-10-16 07:35:30 +02:00
|
|
|
max_disks = max(conf->raid_disks, conf->previous_raid_disks);
|
|
|
|
conf->scribble_len = scribble_len(max_disks);
|
2006-03-27 11:18:11 +02:00
|
|
|
|
2009-10-16 07:35:30 +02:00
|
|
|
conf->disks = kzalloc(max_disks * sizeof(struct disk_info),
|
2006-03-27 11:18:06 +02:00
|
|
|
GFP_KERNEL);
|
|
|
|
if (!conf->disks)
|
|
|
|
goto abort;
|
2006-01-06 09:20:32 +01:00
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
conf->mddev = mddev;
|
|
|
|
|
2006-01-06 09:20:33 +01:00
|
|
|
if ((conf->stripe_hashtbl = kzalloc(PAGE_SIZE, GFP_KERNEL)) == NULL)
|
2005-04-17 00:20:36 +02:00
|
|
|
goto abort;
|
|
|
|
|
2009-07-14 20:48:22 +02:00
|
|
|
conf->level = mddev->new_level;
|
|
|
|
if (raid5_alloc_percpu(conf) != 0)
|
|
|
|
goto abort;
|
|
|
|
|
2010-05-03 06:09:02 +02:00
|
|
|
pr_debug("raid456: run(%s) called.\n", mdname(mddev));
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2009-01-08 22:31:08 +01:00
|
|
|
list_for_each_entry(rdev, &mddev->disks, same_set) {
|
2005-04-17 00:20:36 +02:00
|
|
|
raid_disk = rdev->raid_disk;
|
2009-10-16 07:35:30 +02:00
|
|
|
if (raid_disk >= max_disks
|
2005-04-17 00:20:36 +02:00
|
|
|
|| raid_disk < 0)
|
|
|
|
continue;
|
|
|
|
disk = conf->disks + raid_disk;
|
|
|
|
|
|
|
|
disk->rdev = rdev;
|
|
|
|
|
2005-11-09 06:39:31 +01:00
|
|
|
if (test_bit(In_sync, &rdev->flags)) {
|
2005-04-17 00:20:36 +02:00
|
|
|
char b[BDEVNAME_SIZE];
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_INFO "md/raid:%s: device %s operational as raid"
|
|
|
|
" disk %d\n",
|
|
|
|
mdname(mddev), bdevname(rdev->bdev, b), raid_disk);
|
2011-06-09 01:00:28 +02:00
|
|
|
} else if (rdev->saved_raid_disk != raid_disk)
|
Ensure interrupted recovery completed properly (v1 metadata plus bitmap)
If, while assembling an array, we find a device which is not fully
in-sync with the array, it is important to set the "fullsync" flags.
This is an exact analog to the setting of this flag in hot_add_disk
methods.
Currently, only v1.x metadata supports having devices in an array
which are not fully in-sync (it keep track of how in sync they are).
The 'fullsync' flag only makes a difference when a write-intent bitmap
is being used. In this case it tells recovery to ignore the bitmap
and recovery all blocks.
This fix is already in place for raid1, but not raid5/6 or raid10.
So without this fix, a raid1 ir raid4/5/6 array with version 1.x
metadata and a write intent bitmaps, that is stopped in the middle
of a recovery, will appear to complete the recovery instantly
after it is reassembled, but the recovery will not be correct.
If you might have an array like that, issueing
echo repair > /sys/block/mdXX/md/sync_action
will make sure recovery completes properly.
Cc: <stable@kernel.org>
Signed-off-by: Neil Brown <neilb@suse.de>
2008-06-28 00:30:52 +02:00
|
|
|
/* Cannot rely on bitmap to complete recovery */
|
|
|
|
conf->fullsync = 1;
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
2009-06-18 00:45:55 +02:00
|
|
|
conf->chunk_sectors = mddev->new_chunk_sectors;
|
2009-03-31 05:39:39 +02:00
|
|
|
conf->level = mddev->new_level;
|
2006-06-26 09:27:38 +02:00
|
|
|
if (conf->level == 6)
|
|
|
|
conf->max_degraded = 2;
|
|
|
|
else
|
|
|
|
conf->max_degraded = 1;
|
2009-03-31 05:39:39 +02:00
|
|
|
conf->algorithm = mddev->new_layout;
|
2005-04-17 00:20:36 +02:00
|
|
|
conf->max_nr_stripes = NR_STRIPES;
|
2009-03-31 06:16:46 +02:00
|
|
|
conf->reshape_progress = mddev->reshape_position;
|
2009-03-31 06:20:22 +02:00
|
|
|
if (conf->reshape_progress != MaxSector) {
|
2009-06-18 00:45:55 +02:00
|
|
|
conf->prev_chunk_sectors = mddev->chunk_sectors;
|
2009-03-31 06:20:22 +02:00
|
|
|
conf->prev_algo = mddev->layout;
|
|
|
|
}
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2009-03-31 05:39:39 +02:00
|
|
|
memory = conf->max_nr_stripes * (sizeof(struct stripe_head) +
|
2009-10-16 07:35:30 +02:00
|
|
|
max_disks * ((sizeof(struct bio) + PAGE_SIZE))) / 1024;
|
2009-03-31 05:39:39 +02:00
|
|
|
if (grow_stripes(conf, conf->max_nr_stripes)) {
|
|
|
|
printk(KERN_ERR
|
2010-05-03 06:09:02 +02:00
|
|
|
"md/raid:%s: couldn't allocate %dkB for buffers\n",
|
|
|
|
mdname(mddev), memory);
|
2009-03-31 05:39:39 +02:00
|
|
|
goto abort;
|
|
|
|
} else
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_INFO "md/raid:%s: allocated %dkB\n",
|
|
|
|
mdname(mddev), memory);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2009-09-23 10:09:45 +02:00
|
|
|
conf->thread = md_register_thread(raid5d, mddev, NULL);
|
2009-03-31 05:39:39 +02:00
|
|
|
if (!conf->thread) {
|
|
|
|
printk(KERN_ERR
|
2010-05-03 06:09:02 +02:00
|
|
|
"md/raid:%s: couldn't allocate thread.\n",
|
2009-03-31 05:39:39 +02:00
|
|
|
mdname(mddev));
|
2006-06-26 09:27:38 +02:00
|
|
|
goto abort;
|
|
|
|
}
|
2009-03-31 05:39:39 +02:00
|
|
|
|
|
|
|
return conf;
|
|
|
|
|
|
|
|
abort:
|
|
|
|
if (conf) {
|
2009-07-31 04:39:15 +02:00
|
|
|
free_conf(conf);
|
2009-03-31 05:39:39 +02:00
|
|
|
return ERR_PTR(-EIO);
|
|
|
|
} else
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
}
|
|
|
|
|
2009-11-13 07:47:00 +01:00
|
|
|
|
|
|
|
static int only_parity(int raid_disk, int algo, int raid_disks, int max_degraded)
|
|
|
|
{
|
|
|
|
switch (algo) {
|
|
|
|
case ALGORITHM_PARITY_0:
|
|
|
|
if (raid_disk < max_degraded)
|
|
|
|
return 1;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_PARITY_N:
|
|
|
|
if (raid_disk >= raid_disks - max_degraded)
|
|
|
|
return 1;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_PARITY_0_6:
|
|
|
|
if (raid_disk == 0 ||
|
|
|
|
raid_disk == raid_disks - 1)
|
|
|
|
return 1;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_LEFT_ASYMMETRIC_6:
|
|
|
|
case ALGORITHM_RIGHT_ASYMMETRIC_6:
|
|
|
|
case ALGORITHM_LEFT_SYMMETRIC_6:
|
|
|
|
case ALGORITHM_RIGHT_SYMMETRIC_6:
|
|
|
|
if (raid_disk == raid_disks - 1)
|
|
|
|
return 1;
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2009-03-31 05:39:39 +02:00
|
|
|
static int run(mddev_t *mddev)
|
|
|
|
{
|
|
|
|
raid5_conf_t *conf;
|
2010-07-26 04:04:13 +02:00
|
|
|
int working_disks = 0;
|
2009-11-13 07:47:00 +01:00
|
|
|
int dirty_parity_disks = 0;
|
2009-03-31 05:39:39 +02:00
|
|
|
mdk_rdev_t *rdev;
|
2009-11-13 07:47:00 +01:00
|
|
|
sector_t reshape_offset = 0;
|
2009-03-31 05:39:39 +02:00
|
|
|
|
2009-06-18 00:48:06 +02:00
|
|
|
if (mddev->recovery_cp != MaxSector)
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_NOTICE "md/raid:%s: not clean"
|
2009-06-18 00:48:06 +02:00
|
|
|
" -- starting background reconstruction\n",
|
|
|
|
mdname(mddev));
|
2009-03-31 05:39:39 +02:00
|
|
|
if (mddev->reshape_position != MaxSector) {
|
|
|
|
/* Check that we can continue the reshape.
|
|
|
|
* Currently only disks can change, it must
|
|
|
|
* increase, and we must be past the point where
|
|
|
|
* a stripe over-writes itself
|
|
|
|
*/
|
|
|
|
sector_t here_new, here_old;
|
|
|
|
int old_disks;
|
2009-03-31 06:00:56 +02:00
|
|
|
int max_degraded = (mddev->level == 6 ? 2 : 1);
|
2009-03-31 05:39:39 +02:00
|
|
|
|
2009-03-31 06:24:23 +02:00
|
|
|
if (mddev->new_level != mddev->level) {
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_ERR "md/raid:%s: unsupported reshape "
|
2009-03-31 05:39:39 +02:00
|
|
|
"required - aborting.\n",
|
|
|
|
mdname(mddev));
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
old_disks = mddev->raid_disks - mddev->delta_disks;
|
|
|
|
/* reshape_position must be on a new-stripe boundary, and one
|
|
|
|
* further up in new geometry must map after here in old
|
|
|
|
* geometry.
|
|
|
|
*/
|
|
|
|
here_new = mddev->reshape_position;
|
2009-06-18 00:45:27 +02:00
|
|
|
if (sector_div(here_new, mddev->new_chunk_sectors *
|
2009-03-31 05:39:39 +02:00
|
|
|
(mddev->raid_disks - max_degraded))) {
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_ERR "md/raid:%s: reshape_position not "
|
|
|
|
"on a stripe boundary\n", mdname(mddev));
|
2009-03-31 05:39:39 +02:00
|
|
|
return -EINVAL;
|
|
|
|
}
|
2009-11-13 07:47:00 +01:00
|
|
|
reshape_offset = here_new * mddev->new_chunk_sectors;
|
2009-03-31 05:39:39 +02:00
|
|
|
/* here_new is the stripe we will write to */
|
|
|
|
here_old = mddev->reshape_position;
|
2009-06-18 00:45:01 +02:00
|
|
|
sector_div(here_old, mddev->chunk_sectors *
|
2009-03-31 05:39:39 +02:00
|
|
|
(old_disks-max_degraded));
|
|
|
|
/* here_old is the first stripe that we might need to read
|
|
|
|
* from */
|
2009-08-13 02:06:24 +02:00
|
|
|
if (mddev->delta_disks == 0) {
|
|
|
|
/* We cannot be sure it is safe to start an in-place
|
|
|
|
* reshape. It is only safe if user-space if monitoring
|
|
|
|
* and taking constant backups.
|
|
|
|
* mdadm always starts a situation like this in
|
|
|
|
* readonly mode so it can take control before
|
|
|
|
* allowing any writes. So just check for that.
|
|
|
|
*/
|
|
|
|
if ((here_new * mddev->new_chunk_sectors !=
|
|
|
|
here_old * mddev->chunk_sectors) ||
|
|
|
|
mddev->ro == 0) {
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_ERR "md/raid:%s: in-place reshape must be started"
|
|
|
|
" in read-only mode - aborting\n",
|
|
|
|
mdname(mddev));
|
2009-08-13 02:06:24 +02:00
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
} else if (mddev->delta_disks < 0
|
|
|
|
? (here_new * mddev->new_chunk_sectors <=
|
|
|
|
here_old * mddev->chunk_sectors)
|
|
|
|
: (here_new * mddev->new_chunk_sectors >=
|
|
|
|
here_old * mddev->chunk_sectors)) {
|
2009-03-31 05:39:39 +02:00
|
|
|
/* Reading from the same stripe as writing to - bad */
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_ERR "md/raid:%s: reshape_position too early for "
|
|
|
|
"auto-recovery - aborting.\n",
|
|
|
|
mdname(mddev));
|
2009-03-31 05:39:39 +02:00
|
|
|
return -EINVAL;
|
|
|
|
}
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_INFO "md/raid:%s: reshape will continue\n",
|
|
|
|
mdname(mddev));
|
2009-03-31 05:39:39 +02:00
|
|
|
/* OK, we should be able to continue; */
|
|
|
|
} else {
|
|
|
|
BUG_ON(mddev->level != mddev->new_level);
|
|
|
|
BUG_ON(mddev->layout != mddev->new_layout);
|
2009-06-18 00:45:27 +02:00
|
|
|
BUG_ON(mddev->chunk_sectors != mddev->new_chunk_sectors);
|
2009-03-31 05:39:39 +02:00
|
|
|
BUG_ON(mddev->delta_disks != 0);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
2009-03-31 05:39:39 +02:00
|
|
|
|
2009-03-31 05:39:39 +02:00
|
|
|
if (mddev->private == NULL)
|
|
|
|
conf = setup_conf(mddev);
|
|
|
|
else
|
|
|
|
conf = mddev->private;
|
|
|
|
|
2009-03-31 05:39:39 +02:00
|
|
|
if (IS_ERR(conf))
|
|
|
|
return PTR_ERR(conf);
|
|
|
|
|
|
|
|
mddev->thread = conf->thread;
|
|
|
|
conf->thread = NULL;
|
|
|
|
mddev->private = conf;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* 0 for a fully functional array, 1 or 2 for a degraded array.
|
|
|
|
*/
|
2009-11-13 07:47:00 +01:00
|
|
|
list_for_each_entry(rdev, &mddev->disks, same_set) {
|
|
|
|
if (rdev->raid_disk < 0)
|
|
|
|
continue;
|
2010-06-17 09:41:03 +02:00
|
|
|
if (test_bit(In_sync, &rdev->flags)) {
|
2009-03-31 05:39:39 +02:00
|
|
|
working_disks++;
|
2010-06-17 09:41:03 +02:00
|
|
|
continue;
|
|
|
|
}
|
2009-11-13 07:47:00 +01:00
|
|
|
/* This disc is not fully in-sync. However if it
|
|
|
|
* just stored parity (beyond the recovery_offset),
|
|
|
|
* when we don't need to be concerned about the
|
|
|
|
* array being dirty.
|
|
|
|
* When reshape goes 'backwards', we never have
|
|
|
|
* partially completed devices, so we only need
|
|
|
|
* to worry about reshape going forwards.
|
|
|
|
*/
|
|
|
|
/* Hack because v0.91 doesn't store recovery_offset properly. */
|
|
|
|
if (mddev->major_version == 0 &&
|
|
|
|
mddev->minor_version > 90)
|
|
|
|
rdev->recovery_offset = reshape_offset;
|
|
|
|
|
|
|
|
if (rdev->recovery_offset < reshape_offset) {
|
|
|
|
/* We need to check old and new layout */
|
|
|
|
if (!only_parity(rdev->raid_disk,
|
|
|
|
conf->algorithm,
|
|
|
|
conf->raid_disks,
|
|
|
|
conf->max_degraded))
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
if (!only_parity(rdev->raid_disk,
|
|
|
|
conf->prev_algo,
|
|
|
|
conf->previous_raid_disks,
|
|
|
|
conf->max_degraded))
|
|
|
|
continue;
|
|
|
|
dirty_parity_disks++;
|
|
|
|
}
|
2009-03-31 05:39:39 +02:00
|
|
|
|
2009-10-16 07:35:30 +02:00
|
|
|
mddev->degraded = (max(conf->raid_disks, conf->previous_raid_disks)
|
|
|
|
- working_disks);
|
2009-03-31 05:39:39 +02:00
|
|
|
|
2010-06-16 09:17:53 +02:00
|
|
|
if (has_failed(conf)) {
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_ERR "md/raid:%s: not enough operational devices"
|
2005-04-17 00:20:36 +02:00
|
|
|
" (%d/%d failed)\n",
|
2006-10-03 10:15:47 +02:00
|
|
|
mdname(mddev), mddev->degraded, conf->raid_disks);
|
2005-04-17 00:20:36 +02:00
|
|
|
goto abort;
|
|
|
|
}
|
|
|
|
|
2009-03-31 05:39:39 +02:00
|
|
|
/* device size must be a multiple of chunk size */
|
2009-06-18 00:45:01 +02:00
|
|
|
mddev->dev_sectors &= ~(mddev->chunk_sectors - 1);
|
2009-03-31 05:39:39 +02:00
|
|
|
mddev->resync_max_sectors = mddev->dev_sectors;
|
|
|
|
|
2009-11-13 07:47:00 +01:00
|
|
|
if (mddev->degraded > dirty_parity_disks &&
|
2005-04-17 00:20:36 +02:00
|
|
|
mddev->recovery_cp != MaxSector) {
|
2006-01-06 09:20:15 +01:00
|
|
|
if (mddev->ok_start_degraded)
|
|
|
|
printk(KERN_WARNING
|
2010-05-03 06:09:02 +02:00
|
|
|
"md/raid:%s: starting dirty degraded array"
|
|
|
|
" - data corruption possible.\n",
|
2006-01-06 09:20:15 +01:00
|
|
|
mdname(mddev));
|
|
|
|
else {
|
|
|
|
printk(KERN_ERR
|
2010-05-03 06:09:02 +02:00
|
|
|
"md/raid:%s: cannot start dirty degraded array.\n",
|
2006-01-06 09:20:15 +01:00
|
|
|
mdname(mddev));
|
|
|
|
goto abort;
|
|
|
|
}
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
if (mddev->degraded == 0)
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_INFO "md/raid:%s: raid level %d active with %d out of %d"
|
|
|
|
" devices, algorithm %d\n", mdname(mddev), conf->level,
|
2009-03-31 06:20:22 +02:00
|
|
|
mddev->raid_disks-mddev->degraded, mddev->raid_disks,
|
|
|
|
mddev->new_layout);
|
2005-04-17 00:20:36 +02:00
|
|
|
else
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_ALERT "md/raid:%s: raid level %d active with %d"
|
|
|
|
" out of %d devices, algorithm %d\n",
|
|
|
|
mdname(mddev), conf->level,
|
|
|
|
mddev->raid_disks - mddev->degraded,
|
|
|
|
mddev->raid_disks, mddev->new_layout);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
print_raid5_conf(conf);
|
|
|
|
|
2009-03-31 06:16:46 +02:00
|
|
|
if (conf->reshape_progress != MaxSector) {
|
|
|
|
conf->reshape_safe = conf->reshape_progress;
|
2006-03-27 11:18:11 +02:00
|
|
|
atomic_set(&conf->reshape_stripes, 0);
|
|
|
|
clear_bit(MD_RECOVERY_SYNC, &mddev->recovery);
|
|
|
|
clear_bit(MD_RECOVERY_CHECK, &mddev->recovery);
|
|
|
|
set_bit(MD_RECOVERY_RESHAPE, &mddev->recovery);
|
|
|
|
set_bit(MD_RECOVERY_RUNNING, &mddev->recovery);
|
|
|
|
mddev->sync_thread = md_register_thread(md_do_sync, mddev,
|
2009-09-23 10:09:45 +02:00
|
|
|
"reshape");
|
2006-03-27 11:18:11 +02:00
|
|
|
}
|
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
/* Ok, everything is just fine now */
|
2010-04-14 09:15:37 +02:00
|
|
|
if (mddev->to_remove == &raid5_attrs_group)
|
|
|
|
mddev->to_remove = NULL;
|
2010-06-01 11:37:23 +02:00
|
|
|
else if (mddev->kobj.sd &&
|
|
|
|
sysfs_create_group(&mddev->kobj, &raid5_attrs_group))
|
2007-03-27 07:32:14 +02:00
|
|
|
printk(KERN_WARNING
|
2010-06-01 11:37:28 +02:00
|
|
|
"raid5: failed to create sysfs attributes for %s\n",
|
2007-03-27 07:32:14 +02:00
|
|
|
mdname(mddev));
|
2010-06-01 11:37:28 +02:00
|
|
|
md_set_array_sectors(mddev, raid5_size(mddev, 0, 0));
|
2005-05-17 06:53:16 +02:00
|
|
|
|
2010-06-01 11:37:28 +02:00
|
|
|
if (mddev->queue) {
|
2010-07-26 04:04:13 +02:00
|
|
|
int chunk_size;
|
2010-06-01 11:37:28 +02:00
|
|
|
/* read-ahead size must cover two whole stripes, which
|
|
|
|
* is 2 * (datadisks) * chunksize where 'n' is the
|
|
|
|
* number of raid devices
|
|
|
|
*/
|
|
|
|
int data_disks = conf->previous_raid_disks - conf->max_degraded;
|
|
|
|
int stripe = data_disks *
|
|
|
|
((mddev->chunk_sectors << 9) / PAGE_SIZE);
|
|
|
|
if (mddev->queue->backing_dev_info.ra_pages < 2 * stripe)
|
|
|
|
mddev->queue->backing_dev_info.ra_pages = 2 * stripe;
|
2009-03-31 05:39:39 +02:00
|
|
|
|
2010-06-01 11:37:28 +02:00
|
|
|
blk_queue_merge_bvec(mddev->queue, raid5_mergeable_bvec);
|
2006-10-03 10:15:56 +02:00
|
|
|
|
2010-07-26 03:57:07 +02:00
|
|
|
mddev->queue->backing_dev_info.congested_data = mddev;
|
|
|
|
mddev->queue->backing_dev_info.congested_fn = raid5_congested;
|
2005-05-17 06:53:16 +02:00
|
|
|
|
2010-07-26 04:04:13 +02:00
|
|
|
chunk_size = mddev->chunk_sectors << 9;
|
|
|
|
blk_queue_io_min(mddev->queue, chunk_size);
|
|
|
|
blk_queue_io_opt(mddev->queue, chunk_size *
|
|
|
|
(conf->raid_disks - conf->max_degraded));
|
2009-07-01 03:13:45 +02:00
|
|
|
|
2010-07-26 04:04:13 +02:00
|
|
|
list_for_each_entry(rdev, &mddev->disks, same_set)
|
|
|
|
disk_stack_limits(mddev->gendisk, rdev->bdev,
|
|
|
|
rdev->data_offset << 9);
|
|
|
|
}
|
2006-12-10 11:20:45 +01:00
|
|
|
|
2005-04-17 00:20:36 +02:00
|
|
|
return 0;
|
|
|
|
abort:
|
2009-03-31 05:39:39 +02:00
|
|
|
md_unregister_thread(mddev->thread);
|
2009-03-31 05:39:39 +02:00
|
|
|
mddev->thread = NULL;
|
2005-04-17 00:20:36 +02:00
|
|
|
if (conf) {
|
|
|
|
print_raid5_conf(conf);
|
2009-07-31 04:39:15 +02:00
|
|
|
free_conf(conf);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
mddev->private = NULL;
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_ALERT "md/raid:%s: failed to run raid set.\n", mdname(mddev));
|
2005-04-17 00:20:36 +02:00
|
|
|
return -EIO;
|
|
|
|
}
|
|
|
|
|
2005-11-09 06:39:25 +01:00
|
|
|
static int stop(mddev_t *mddev)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
2010-03-08 06:02:40 +01:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
md_unregister_thread(mddev->thread);
|
|
|
|
mddev->thread = NULL;
|
2010-07-26 03:57:07 +02:00
|
|
|
if (mddev->queue)
|
|
|
|
mddev->queue->backing_dev_info.congested_fn = NULL;
|
2009-07-31 04:39:15 +02:00
|
|
|
free_conf(conf);
|
2010-04-14 09:15:37 +02:00
|
|
|
mddev->private = NULL;
|
|
|
|
mddev->to_remove = &raid5_attrs_group;
|
2005-04-17 00:20:36 +02:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2007-07-09 20:56:43 +02:00
|
|
|
#ifdef DEBUG
|
2008-10-13 02:55:12 +02:00
|
|
|
static void print_sh(struct seq_file *seq, struct stripe_head *sh)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
int i;
|
|
|
|
|
2006-06-26 09:27:38 +02:00
|
|
|
seq_printf(seq, "sh %llu, pd_idx %d, state %ld.\n",
|
|
|
|
(unsigned long long)sh->sector, sh->pd_idx, sh->state);
|
|
|
|
seq_printf(seq, "sh %llu, count %d.\n",
|
|
|
|
(unsigned long long)sh->sector, atomic_read(&sh->count));
|
|
|
|
seq_printf(seq, "sh %llu, ", (unsigned long long)sh->sector);
|
2006-03-27 11:18:08 +02:00
|
|
|
for (i = 0; i < sh->disks; i++) {
|
2006-06-26 09:27:38 +02:00
|
|
|
seq_printf(seq, "(cache%d: %p %ld) ",
|
|
|
|
i, sh->dev[i].page, sh->dev[i].flags);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
2006-06-26 09:27:38 +02:00
|
|
|
seq_printf(seq, "\n");
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
2008-10-13 02:55:12 +02:00
|
|
|
static void printall(struct seq_file *seq, raid5_conf_t *conf)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
struct stripe_head *sh;
|
2006-01-06 09:20:33 +01:00
|
|
|
struct hlist_node *hn;
|
2005-04-17 00:20:36 +02:00
|
|
|
int i;
|
|
|
|
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
|
|
|
for (i = 0; i < NR_HASH; i++) {
|
2006-01-06 09:20:33 +01:00
|
|
|
hlist_for_each_entry(sh, hn, &conf->stripe_hashtbl[i], hash) {
|
2005-04-17 00:20:36 +02:00
|
|
|
if (sh->raid_conf != conf)
|
|
|
|
continue;
|
2006-06-26 09:27:38 +02:00
|
|
|
print_sh(seq, sh);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
2008-10-13 02:55:12 +02:00
|
|
|
static void status(struct seq_file *seq, mddev_t *mddev)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
2010-03-08 06:02:40 +01:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2005-04-17 00:20:36 +02:00
|
|
|
int i;
|
|
|
|
|
2009-06-18 00:45:01 +02:00
|
|
|
seq_printf(seq, " level %d, %dk chunk, algorithm %d", mddev->level,
|
|
|
|
mddev->chunk_sectors / 2, mddev->layout);
|
2006-10-03 10:15:47 +02:00
|
|
|
seq_printf (seq, " [%d/%d] [", conf->raid_disks, conf->raid_disks - mddev->degraded);
|
2005-04-17 00:20:36 +02:00
|
|
|
for (i = 0; i < conf->raid_disks; i++)
|
|
|
|
seq_printf (seq, "%s",
|
|
|
|
conf->disks[i].rdev &&
|
2005-11-09 06:39:31 +01:00
|
|
|
test_bit(In_sync, &conf->disks[i].rdev->flags) ? "U" : "_");
|
2005-04-17 00:20:36 +02:00
|
|
|
seq_printf (seq, "]");
|
2007-07-09 20:56:43 +02:00
|
|
|
#ifdef DEBUG
|
2006-06-26 09:27:38 +02:00
|
|
|
seq_printf (seq, "\n");
|
|
|
|
printall(seq, conf);
|
2005-04-17 00:20:36 +02:00
|
|
|
#endif
|
|
|
|
}
|
|
|
|
|
|
|
|
static void print_raid5_conf (raid5_conf_t *conf)
|
|
|
|
{
|
|
|
|
int i;
|
|
|
|
struct disk_info *tmp;
|
|
|
|
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_DEBUG "RAID conf printout:\n");
|
2005-04-17 00:20:36 +02:00
|
|
|
if (!conf) {
|
|
|
|
printk("(conf==NULL)\n");
|
|
|
|
return;
|
|
|
|
}
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_DEBUG " --- level:%d rd:%d wd:%d\n", conf->level,
|
|
|
|
conf->raid_disks,
|
|
|
|
conf->raid_disks - conf->mddev->degraded);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
for (i = 0; i < conf->raid_disks; i++) {
|
|
|
|
char b[BDEVNAME_SIZE];
|
|
|
|
tmp = conf->disks + i;
|
|
|
|
if (tmp->rdev)
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_DEBUG " disk %d, o:%d, dev:%s\n",
|
|
|
|
i, !test_bit(Faulty, &tmp->rdev->flags),
|
|
|
|
bdevname(tmp->rdev->bdev, b));
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
static int raid5_spare_active(mddev_t *mddev)
|
|
|
|
{
|
|
|
|
int i;
|
|
|
|
raid5_conf_t *conf = mddev->private;
|
|
|
|
struct disk_info *tmp;
|
2010-08-18 03:56:59 +02:00
|
|
|
int count = 0;
|
|
|
|
unsigned long flags;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
for (i = 0; i < conf->raid_disks; i++) {
|
|
|
|
tmp = conf->disks + i;
|
|
|
|
if (tmp->rdev
|
2010-06-16 09:01:25 +02:00
|
|
|
&& tmp->rdev->recovery_offset == MaxSector
|
2005-11-09 06:39:31 +01:00
|
|
|
&& !test_bit(Faulty, &tmp->rdev->flags)
|
2006-10-03 10:15:53 +02:00
|
|
|
&& !test_and_set_bit(In_sync, &tmp->rdev->flags)) {
|
2010-08-18 03:56:59 +02:00
|
|
|
count++;
|
2011-01-13 23:14:33 +01:00
|
|
|
sysfs_notify_dirent_safe(tmp->rdev->sysfs_state);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
}
|
2010-08-18 03:56:59 +02:00
|
|
|
spin_lock_irqsave(&conf->device_lock, flags);
|
|
|
|
mddev->degraded -= count;
|
|
|
|
spin_unlock_irqrestore(&conf->device_lock, flags);
|
2005-04-17 00:20:36 +02:00
|
|
|
print_raid5_conf(conf);
|
2010-08-18 03:56:59 +02:00
|
|
|
return count;
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
static int raid5_remove_disk(mddev_t *mddev, int number)
|
|
|
|
{
|
|
|
|
raid5_conf_t *conf = mddev->private;
|
|
|
|
int err = 0;
|
|
|
|
mdk_rdev_t *rdev;
|
|
|
|
struct disk_info *p = conf->disks + number;
|
|
|
|
|
|
|
|
print_raid5_conf(conf);
|
|
|
|
rdev = p->rdev;
|
|
|
|
if (rdev) {
|
2009-03-31 06:17:38 +02:00
|
|
|
if (number >= conf->raid_disks &&
|
|
|
|
conf->reshape_progress == MaxSector)
|
|
|
|
clear_bit(In_sync, &rdev->flags);
|
|
|
|
|
2005-11-09 06:39:31 +01:00
|
|
|
if (test_bit(In_sync, &rdev->flags) ||
|
2005-04-17 00:20:36 +02:00
|
|
|
atomic_read(&rdev->nr_pending)) {
|
|
|
|
err = -EBUSY;
|
|
|
|
goto abort;
|
|
|
|
}
|
md: restart recovery cleanly after device failure.
When we get any IO error during a recovery (rebuilding a spare), we abort
the recovery and restart it.
For RAID6 (and multi-drive RAID1) it may not be best to restart at the
beginning: when multiple failures can be tolerated, the recovery may be
able to continue and re-doing all that has already been done doesn't make
sense.
We already have the infrastructure to record where a recovery is up to
and restart from there, but it is not being used properly.
This is because:
- We sometimes abort with MD_RECOVERY_ERR rather than just MD_RECOVERY_INTR,
which causes the recovery not be be checkpointed.
- We remove spares and then re-added them which loses important state
information.
The distinction between MD_RECOVERY_ERR and MD_RECOVERY_INTR really isn't
needed. If there is an error, the relevant drive will be marked as
Faulty, and that is enough to ensure correct handling of the error. So we
first remove MD_RECOVERY_ERR, changing some of the uses of it to
MD_RECOVERY_INTR.
Then we cause the attempt to remove a non-faulty device from an array to
fail (unless recovery is impossible as the array is too degraded). Then
when remove_and_add_spares attempts to remove the devices on which
recovery can continue, it will fail, they will remain in place, and
recovery will continue on them as desired.
Issue: If we are halfway through rebuilding a spare and another drive
fails, and a new spare is immediately available, do we want to:
1/ complete the current rebuild, then go back and rebuild the new spare or
2/ restart the rebuild from the start and rebuild both devices in
parallel.
Both options can be argued for. The code currently takes option 2 as
a/ this requires least code change
b/ this results in a minimally-degraded array in minimal time.
Cc: "Eivind Sarto" <ivan@kasenna.com>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-05-23 22:04:39 +02:00
|
|
|
/* Only remove non-faulty devices if recovery
|
|
|
|
* isn't possible.
|
|
|
|
*/
|
|
|
|
if (!test_bit(Faulty, &rdev->flags) &&
|
2010-06-16 09:17:53 +02:00
|
|
|
!has_failed(conf) &&
|
2009-03-31 06:17:38 +02:00
|
|
|
number < conf->raid_disks) {
|
md: restart recovery cleanly after device failure.
When we get any IO error during a recovery (rebuilding a spare), we abort
the recovery and restart it.
For RAID6 (and multi-drive RAID1) it may not be best to restart at the
beginning: when multiple failures can be tolerated, the recovery may be
able to continue and re-doing all that has already been done doesn't make
sense.
We already have the infrastructure to record where a recovery is up to
and restart from there, but it is not being used properly.
This is because:
- We sometimes abort with MD_RECOVERY_ERR rather than just MD_RECOVERY_INTR,
which causes the recovery not be be checkpointed.
- We remove spares and then re-added them which loses important state
information.
The distinction between MD_RECOVERY_ERR and MD_RECOVERY_INTR really isn't
needed. If there is an error, the relevant drive will be marked as
Faulty, and that is enough to ensure correct handling of the error. So we
first remove MD_RECOVERY_ERR, changing some of the uses of it to
MD_RECOVERY_INTR.
Then we cause the attempt to remove a non-faulty device from an array to
fail (unless recovery is impossible as the array is too degraded). Then
when remove_and_add_spares attempts to remove the devices on which
recovery can continue, it will fail, they will remain in place, and
recovery will continue on them as desired.
Issue: If we are halfway through rebuilding a spare and another drive
fails, and a new spare is immediately available, do we want to:
1/ complete the current rebuild, then go back and rebuild the new spare or
2/ restart the rebuild from the start and rebuild both devices in
parallel.
Both options can be argued for. The code currently takes option 2 as
a/ this requires least code change
b/ this results in a minimally-degraded array in minimal time.
Cc: "Eivind Sarto" <ivan@kasenna.com>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-05-23 22:04:39 +02:00
|
|
|
err = -EBUSY;
|
|
|
|
goto abort;
|
|
|
|
}
|
2005-04-17 00:20:36 +02:00
|
|
|
p->rdev = NULL;
|
2005-05-01 17:59:04 +02:00
|
|
|
synchronize_rcu();
|
2005-04-17 00:20:36 +02:00
|
|
|
if (atomic_read(&rdev->nr_pending)) {
|
|
|
|
/* lost the race, try later */
|
|
|
|
err = -EBUSY;
|
|
|
|
p->rdev = rdev;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
abort:
|
|
|
|
|
|
|
|
print_raid5_conf(conf);
|
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int raid5_add_disk(mddev_t *mddev, mdk_rdev_t *rdev)
|
|
|
|
{
|
|
|
|
raid5_conf_t *conf = mddev->private;
|
2008-06-28 00:31:33 +02:00
|
|
|
int err = -EEXIST;
|
2005-04-17 00:20:36 +02:00
|
|
|
int disk;
|
|
|
|
struct disk_info *p;
|
2008-06-28 00:31:31 +02:00
|
|
|
int first = 0;
|
|
|
|
int last = conf->raid_disks - 1;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2010-06-16 09:17:53 +02:00
|
|
|
if (has_failed(conf))
|
2005-04-17 00:20:36 +02:00
|
|
|
/* no point adding a device */
|
2008-06-28 00:31:33 +02:00
|
|
|
return -EINVAL;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2008-06-28 00:31:31 +02:00
|
|
|
if (rdev->raid_disk >= 0)
|
|
|
|
first = last = rdev->raid_disk;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
|
|
|
/*
|
2006-06-26 09:27:38 +02:00
|
|
|
* find the disk ... but prefer rdev->saved_raid_disk
|
|
|
|
* if possible.
|
2005-04-17 00:20:36 +02:00
|
|
|
*/
|
2006-06-26 09:27:38 +02:00
|
|
|
if (rdev->saved_raid_disk >= 0 &&
|
2008-06-28 00:31:31 +02:00
|
|
|
rdev->saved_raid_disk >= first &&
|
2006-06-26 09:27:38 +02:00
|
|
|
conf->disks[rdev->saved_raid_disk].rdev == NULL)
|
|
|
|
disk = rdev->saved_raid_disk;
|
|
|
|
else
|
2008-06-28 00:31:31 +02:00
|
|
|
disk = first;
|
|
|
|
for ( ; disk <= last ; disk++)
|
2005-04-17 00:20:36 +02:00
|
|
|
if ((p=conf->disks + disk)->rdev == NULL) {
|
2005-11-09 06:39:31 +01:00
|
|
|
clear_bit(In_sync, &rdev->flags);
|
2005-04-17 00:20:36 +02:00
|
|
|
rdev->raid_disk = disk;
|
2008-06-28 00:31:33 +02:00
|
|
|
err = 0;
|
2005-09-10 01:23:54 +02:00
|
|
|
if (rdev->saved_raid_disk != disk)
|
|
|
|
conf->fullsync = 1;
|
2005-11-09 06:39:27 +01:00
|
|
|
rcu_assign_pointer(p->rdev, rdev);
|
2005-04-17 00:20:36 +02:00
|
|
|
break;
|
|
|
|
}
|
|
|
|
print_raid5_conf(conf);
|
2008-06-28 00:31:33 +02:00
|
|
|
return err;
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
static int raid5_resize(mddev_t *mddev, sector_t sectors)
|
|
|
|
{
|
|
|
|
/* no resync is happening, and there is enough space
|
|
|
|
* on all devices, so we can resize.
|
|
|
|
* We need to make sure resync covers any new space.
|
|
|
|
* If the array is shrinking we should possibly wait until
|
|
|
|
* any io in the removed space completes, but it hardly seems
|
|
|
|
* worth it.
|
|
|
|
*/
|
2009-06-18 00:45:01 +02:00
|
|
|
sectors &= ~((sector_t)mddev->chunk_sectors - 1);
|
2009-03-31 05:59:03 +02:00
|
|
|
md_set_array_sectors(mddev, raid5_size(mddev, sectors,
|
|
|
|
mddev->raid_disks));
|
2009-03-31 06:00:31 +02:00
|
|
|
if (mddev->array_sectors >
|
|
|
|
raid5_size(mddev, sectors, mddev->raid_disks))
|
|
|
|
return -EINVAL;
|
2008-07-21 09:05:22 +02:00
|
|
|
set_capacity(mddev->gendisk, mddev->array_sectors);
|
2009-08-03 02:59:58 +02:00
|
|
|
revalidate_disk(mddev->gendisk);
|
2011-05-11 07:52:21 +02:00
|
|
|
if (sectors > mddev->dev_sectors &&
|
|
|
|
mddev->recovery_cp > mddev->dev_sectors) {
|
2009-03-31 05:33:13 +02:00
|
|
|
mddev->recovery_cp = mddev->dev_sectors;
|
2005-04-17 00:20:36 +02:00
|
|
|
set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
|
|
|
|
}
|
2009-03-31 05:33:13 +02:00
|
|
|
mddev->dev_sectors = sectors;
|
2005-07-27 20:43:28 +02:00
|
|
|
mddev->resync_max_sectors = sectors;
|
2005-04-17 00:20:36 +02:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2009-06-18 00:47:20 +02:00
|
|
|
static int check_stripe_cache(mddev_t *mddev)
|
|
|
|
{
|
|
|
|
/* Can only proceed if there are plenty of stripe_heads.
|
|
|
|
* We need a minimum of one full stripe,, and for sensible progress
|
|
|
|
* it is best to have about 4 times that.
|
|
|
|
* If we require 4 times, then the default 256 4K stripe_heads will
|
|
|
|
* allow for chunk sizes up to 256K, which is probably OK.
|
|
|
|
* If the chunk size is greater, user-space should request more
|
|
|
|
* stripe_heads first.
|
|
|
|
*/
|
|
|
|
raid5_conf_t *conf = mddev->private;
|
|
|
|
if (((mddev->chunk_sectors << 9) / STRIPE_SIZE) * 4
|
|
|
|
> conf->max_nr_stripes ||
|
|
|
|
((mddev->new_chunk_sectors << 9) / STRIPE_SIZE) * 4
|
|
|
|
> conf->max_nr_stripes) {
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_WARNING "md/raid:%s: reshape: not enough stripes. Needed %lu\n",
|
|
|
|
mdname(mddev),
|
2009-06-18 00:47:20 +02:00
|
|
|
((max(mddev->chunk_sectors, mddev->new_chunk_sectors) << 9)
|
|
|
|
/ STRIPE_SIZE)*4);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
return 1;
|
|
|
|
}
|
|
|
|
|
2009-06-18 00:47:55 +02:00
|
|
|
static int check_reshape(mddev_t *mddev)
|
2006-03-27 11:18:10 +02:00
|
|
|
{
|
2009-06-16 08:54:21 +02:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2006-03-27 11:18:10 +02:00
|
|
|
|
2009-03-31 06:24:23 +02:00
|
|
|
if (mddev->delta_disks == 0 &&
|
|
|
|
mddev->new_layout == mddev->layout &&
|
2009-06-18 00:45:27 +02:00
|
|
|
mddev->new_chunk_sectors == mddev->chunk_sectors)
|
2009-06-18 00:47:55 +02:00
|
|
|
return 0; /* nothing to do */
|
2008-08-05 07:54:13 +02:00
|
|
|
if (mddev->bitmap)
|
|
|
|
/* Cannot grow a bitmap yet */
|
|
|
|
return -EBUSY;
|
2010-06-16 09:17:53 +02:00
|
|
|
if (has_failed(conf))
|
2009-03-31 06:17:38 +02:00
|
|
|
return -EINVAL;
|
|
|
|
if (mddev->delta_disks < 0) {
|
|
|
|
/* We might be able to shrink, but the devices must
|
|
|
|
* be made bigger first.
|
|
|
|
* For raid6, 4 is the minimum size.
|
|
|
|
* Otherwise 2 is the minimum
|
|
|
|
*/
|
|
|
|
int min = 2;
|
|
|
|
if (mddev->level == 6)
|
|
|
|
min = 4;
|
|
|
|
if (mddev->raid_disks + mddev->delta_disks < min)
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
2006-03-27 11:18:10 +02:00
|
|
|
|
2009-06-18 00:47:20 +02:00
|
|
|
if (!check_stripe_cache(mddev))
|
2006-03-27 11:18:10 +02:00
|
|
|
return -ENOSPC;
|
|
|
|
|
2009-03-31 06:17:38 +02:00
|
|
|
return resize_stripes(conf, conf->raid_disks + mddev->delta_disks);
|
2006-03-27 11:18:13 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
static int raid5_start_reshape(mddev_t *mddev)
|
|
|
|
{
|
2009-06-16 08:54:21 +02:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2006-03-27 11:18:13 +02:00
|
|
|
mdk_rdev_t *rdev;
|
|
|
|
int spares = 0;
|
2006-10-03 10:15:53 +02:00
|
|
|
unsigned long flags;
|
2006-03-27 11:18:13 +02:00
|
|
|
|
2007-03-01 05:11:53 +01:00
|
|
|
if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery))
|
2006-03-27 11:18:13 +02:00
|
|
|
return -EBUSY;
|
|
|
|
|
2009-06-18 00:47:20 +02:00
|
|
|
if (!check_stripe_cache(mddev))
|
|
|
|
return -ENOSPC;
|
|
|
|
|
2009-01-08 22:31:08 +01:00
|
|
|
list_for_each_entry(rdev, &mddev->disks, same_set)
|
2011-01-31 01:57:43 +01:00
|
|
|
if (!test_bit(In_sync, &rdev->flags)
|
|
|
|
&& !test_bit(Faulty, &rdev->flags))
|
2006-03-27 11:18:10 +02:00
|
|
|
spares++;
|
2006-03-27 11:18:13 +02:00
|
|
|
|
2007-03-01 05:11:53 +01:00
|
|
|
if (spares - mddev->degraded < mddev->delta_disks - conf->max_degraded)
|
2006-03-27 11:18:10 +02:00
|
|
|
/* Not enough devices even to make a degraded array
|
|
|
|
* of that size
|
|
|
|
*/
|
|
|
|
return -EINVAL;
|
|
|
|
|
2009-03-31 06:17:38 +02:00
|
|
|
/* Refuse to reduce size of the array. Any reductions in
|
|
|
|
* array size must be through explicit setting of array_size
|
|
|
|
* attribute.
|
|
|
|
*/
|
|
|
|
if (raid5_size(mddev, 0, conf->raid_disks + mddev->delta_disks)
|
|
|
|
< mddev->array_sectors) {
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_ERR "md/raid:%s: array size must be reduced "
|
2009-03-31 06:17:38 +02:00
|
|
|
"before number of disks\n", mdname(mddev));
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
|
2006-03-27 11:18:11 +02:00
|
|
|
atomic_set(&conf->reshape_stripes, 0);
|
2006-03-27 11:18:10 +02:00
|
|
|
spin_lock_irq(&conf->device_lock);
|
|
|
|
conf->previous_raid_disks = conf->raid_disks;
|
2006-03-27 11:18:13 +02:00
|
|
|
conf->raid_disks += mddev->delta_disks;
|
2009-06-18 00:45:55 +02:00
|
|
|
conf->prev_chunk_sectors = conf->chunk_sectors;
|
|
|
|
conf->chunk_sectors = mddev->new_chunk_sectors;
|
2009-03-31 06:24:23 +02:00
|
|
|
conf->prev_algo = conf->algorithm;
|
|
|
|
conf->algorithm = mddev->new_layout;
|
2009-03-31 06:16:46 +02:00
|
|
|
if (mddev->delta_disks < 0)
|
|
|
|
conf->reshape_progress = raid5_size(mddev, 0, 0);
|
|
|
|
else
|
|
|
|
conf->reshape_progress = 0;
|
|
|
|
conf->reshape_safe = conf->reshape_progress;
|
2009-03-31 06:19:03 +02:00
|
|
|
conf->generation++;
|
2006-03-27 11:18:10 +02:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
|
|
|
|
/* Add some new drives, as many as will fit.
|
|
|
|
* We know there are enough to make the newly sized array work.
|
2010-06-17 09:48:26 +02:00
|
|
|
* Don't add devices if we are reducing the number of
|
|
|
|
* devices in the array. This is because it is not possible
|
|
|
|
* to correctly record the "partially reconstructed" state of
|
|
|
|
* such devices during the reshape and confusion could result.
|
2006-03-27 11:18:10 +02:00
|
|
|
*/
|
2011-01-31 01:57:43 +01:00
|
|
|
if (mddev->delta_disks >= 0) {
|
|
|
|
int added_devices = 0;
|
|
|
|
list_for_each_entry(rdev, &mddev->disks, same_set)
|
|
|
|
if (rdev->raid_disk < 0 &&
|
|
|
|
!test_bit(Faulty, &rdev->flags)) {
|
|
|
|
if (raid5_add_disk(mddev, rdev) == 0) {
|
|
|
|
char nm[20];
|
|
|
|
if (rdev->raid_disk
|
|
|
|
>= conf->previous_raid_disks) {
|
|
|
|
set_bit(In_sync, &rdev->flags);
|
|
|
|
added_devices++;
|
|
|
|
} else
|
|
|
|
rdev->recovery_offset = 0;
|
|
|
|
sprintf(nm, "rd%d", rdev->raid_disk);
|
|
|
|
if (sysfs_create_link(&mddev->kobj,
|
|
|
|
&rdev->kobj, nm))
|
|
|
|
/* Failure here is OK */;
|
2011-01-31 01:57:43 +01:00
|
|
|
}
|
2011-01-31 01:57:43 +01:00
|
|
|
} else if (rdev->raid_disk >= conf->previous_raid_disks
|
|
|
|
&& !test_bit(Faulty, &rdev->flags)) {
|
|
|
|
/* This is a spare that was manually added */
|
|
|
|
set_bit(In_sync, &rdev->flags);
|
|
|
|
added_devices++;
|
|
|
|
}
|
2006-03-27 11:18:10 +02:00
|
|
|
|
2011-01-31 01:57:43 +01:00
|
|
|
/* When a reshape changes the number of devices,
|
|
|
|
* ->degraded is measured against the larger of the
|
|
|
|
* pre and post number of devices.
|
|
|
|
*/
|
2009-03-31 06:17:38 +02:00
|
|
|
spin_lock_irqsave(&conf->device_lock, flags);
|
2010-02-09 02:31:47 +01:00
|
|
|
mddev->degraded += (conf->raid_disks - conf->previous_raid_disks)
|
2009-03-31 06:17:38 +02:00
|
|
|
- added_devices;
|
|
|
|
spin_unlock_irqrestore(&conf->device_lock, flags);
|
|
|
|
}
|
2006-03-27 11:18:13 +02:00
|
|
|
mddev->raid_disks = conf->raid_disks;
|
2009-08-03 02:59:57 +02:00
|
|
|
mddev->reshape_position = conf->reshape_progress;
|
2006-10-03 10:15:46 +02:00
|
|
|
set_bit(MD_CHANGE_DEVS, &mddev->flags);
|
2006-03-27 11:18:11 +02:00
|
|
|
|
2006-03-27 11:18:10 +02:00
|
|
|
clear_bit(MD_RECOVERY_SYNC, &mddev->recovery);
|
|
|
|
clear_bit(MD_RECOVERY_CHECK, &mddev->recovery);
|
|
|
|
set_bit(MD_RECOVERY_RESHAPE, &mddev->recovery);
|
|
|
|
set_bit(MD_RECOVERY_RUNNING, &mddev->recovery);
|
|
|
|
mddev->sync_thread = md_register_thread(md_do_sync, mddev,
|
2009-09-23 10:09:45 +02:00
|
|
|
"reshape");
|
2006-03-27 11:18:10 +02:00
|
|
|
if (!mddev->sync_thread) {
|
|
|
|
mddev->recovery = 0;
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
|
|
|
mddev->raid_disks = conf->raid_disks = conf->previous_raid_disks;
|
2009-03-31 06:16:46 +02:00
|
|
|
conf->reshape_progress = MaxSector;
|
2006-03-27 11:18:10 +02:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
return -EAGAIN;
|
|
|
|
}
|
2009-03-31 06:28:40 +02:00
|
|
|
conf->reshape_checkpoint = jiffies;
|
2006-03-27 11:18:10 +02:00
|
|
|
md_wakeup_thread(mddev->sync_thread);
|
|
|
|
md_new_event(mddev);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2009-03-31 06:17:38 +02:00
|
|
|
/* This is called from the reshape thread and should make any
|
|
|
|
* changes needed in 'conf'
|
|
|
|
*/
|
2006-03-27 11:18:10 +02:00
|
|
|
static void end_reshape(raid5_conf_t *conf)
|
|
|
|
{
|
|
|
|
|
2006-03-27 11:18:11 +02:00
|
|
|
if (!test_bit(MD_RECOVERY_INTR, &conf->mddev->recovery)) {
|
|
|
|
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
2009-03-31 06:15:05 +02:00
|
|
|
conf->previous_raid_disks = conf->raid_disks;
|
2009-03-31 06:16:46 +02:00
|
|
|
conf->reshape_progress = MaxSector;
|
2006-03-27 11:18:11 +02:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
2009-03-31 06:27:18 +02:00
|
|
|
wake_up(&conf->wait_for_overlap);
|
2006-06-26 09:27:38 +02:00
|
|
|
|
|
|
|
/* read-ahead size must cover two whole stripes, which is
|
|
|
|
* 2 * (datadisks) * chunksize where 'n' is the number of raid devices
|
|
|
|
*/
|
2010-06-01 11:37:28 +02:00
|
|
|
if (conf->mddev->queue) {
|
2009-03-31 06:15:05 +02:00
|
|
|
int data_disks = conf->raid_disks - conf->max_degraded;
|
2009-06-18 00:45:55 +02:00
|
|
|
int stripe = data_disks * ((conf->chunk_sectors << 9)
|
2009-03-31 06:15:05 +02:00
|
|
|
/ PAGE_SIZE);
|
2006-06-26 09:27:38 +02:00
|
|
|
if (conf->mddev->queue->backing_dev_info.ra_pages < 2 * stripe)
|
|
|
|
conf->mddev->queue->backing_dev_info.ra_pages = 2 * stripe;
|
|
|
|
}
|
2006-03-27 11:18:10 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2009-03-31 06:17:38 +02:00
|
|
|
/* This is called from the raid5d thread with mddev_lock held.
|
|
|
|
* It makes config changes to the device.
|
|
|
|
*/
|
2009-03-31 06:15:05 +02:00
|
|
|
static void raid5_finish_reshape(mddev_t *mddev)
|
|
|
|
{
|
2009-06-16 08:54:21 +02:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2009-03-31 06:15:05 +02:00
|
|
|
|
|
|
|
if (!test_bit(MD_RECOVERY_INTR, &mddev->recovery)) {
|
|
|
|
|
2009-03-31 06:17:38 +02:00
|
|
|
if (mddev->delta_disks > 0) {
|
|
|
|
md_set_array_sectors(mddev, raid5_size(mddev, 0, 0));
|
|
|
|
set_capacity(mddev->gendisk, mddev->array_sectors);
|
2009-08-03 02:59:58 +02:00
|
|
|
revalidate_disk(mddev->gendisk);
|
2009-03-31 06:17:38 +02:00
|
|
|
} else {
|
|
|
|
int d;
|
|
|
|
mddev->degraded = conf->raid_disks;
|
|
|
|
for (d = 0; d < conf->raid_disks ; d++)
|
|
|
|
if (conf->disks[d].rdev &&
|
|
|
|
test_bit(In_sync,
|
|
|
|
&conf->disks[d].rdev->flags))
|
|
|
|
mddev->degraded--;
|
|
|
|
for (d = conf->raid_disks ;
|
|
|
|
d < conf->raid_disks - mddev->delta_disks;
|
2009-08-13 02:41:49 +02:00
|
|
|
d++) {
|
|
|
|
mdk_rdev_t *rdev = conf->disks[d].rdev;
|
|
|
|
if (rdev && raid5_remove_disk(mddev, d) == 0) {
|
|
|
|
char nm[20];
|
|
|
|
sprintf(nm, "rd%d", rdev->raid_disk);
|
|
|
|
sysfs_remove_link(&mddev->kobj, nm);
|
|
|
|
rdev->raid_disk = -1;
|
|
|
|
}
|
|
|
|
}
|
2009-03-31 06:15:05 +02:00
|
|
|
}
|
2009-03-31 06:24:23 +02:00
|
|
|
mddev->layout = conf->algorithm;
|
2009-06-18 00:45:55 +02:00
|
|
|
mddev->chunk_sectors = conf->chunk_sectors;
|
2009-03-31 06:17:38 +02:00
|
|
|
mddev->reshape_position = MaxSector;
|
|
|
|
mddev->delta_disks = 0;
|
2009-03-31 06:15:05 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2005-09-10 01:23:54 +02:00
|
|
|
static void raid5_quiesce(mddev_t *mddev, int state)
|
|
|
|
{
|
2009-06-16 08:54:21 +02:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2005-09-10 01:23:54 +02:00
|
|
|
|
|
|
|
switch(state) {
|
2006-03-27 11:18:14 +02:00
|
|
|
case 2: /* resume for a suspend */
|
|
|
|
wake_up(&conf->wait_for_overlap);
|
|
|
|
break;
|
|
|
|
|
2005-09-10 01:23:54 +02:00
|
|
|
case 1: /* stop all writes */
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
2009-08-03 02:59:58 +02:00
|
|
|
/* '2' tells resync/reshape to pause so that all
|
|
|
|
* active stripes can drain
|
|
|
|
*/
|
|
|
|
conf->quiesce = 2;
|
2005-09-10 01:23:54 +02:00
|
|
|
wait_event_lock_irq(conf->wait_for_stripe,
|
2006-12-10 11:20:47 +01:00
|
|
|
atomic_read(&conf->active_stripes) == 0 &&
|
|
|
|
atomic_read(&conf->active_aligned_reads) == 0,
|
2005-09-10 01:23:54 +02:00
|
|
|
conf->device_lock, /* nothing */);
|
2009-08-03 02:59:58 +02:00
|
|
|
conf->quiesce = 1;
|
2005-09-10 01:23:54 +02:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
2009-08-03 02:59:58 +02:00
|
|
|
/* allow reshape to continue */
|
|
|
|
wake_up(&conf->wait_for_overlap);
|
2005-09-10 01:23:54 +02:00
|
|
|
break;
|
|
|
|
|
|
|
|
case 0: /* re-enable writes */
|
|
|
|
spin_lock_irq(&conf->device_lock);
|
|
|
|
conf->quiesce = 0;
|
|
|
|
wake_up(&conf->wait_for_stripe);
|
2006-03-27 11:18:14 +02:00
|
|
|
wake_up(&conf->wait_for_overlap);
|
2005-09-10 01:23:54 +02:00
|
|
|
spin_unlock_irq(&conf->device_lock);
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
2006-01-06 09:20:16 +01:00
|
|
|
|
2009-03-31 05:39:39 +02:00
|
|
|
|
2010-05-02 03:09:05 +02:00
|
|
|
static void *raid45_takeover_raid0(mddev_t *mddev, int level)
|
2010-03-08 06:02:42 +01:00
|
|
|
{
|
2010-05-02 03:09:05 +02:00
|
|
|
struct raid0_private_data *raid0_priv = mddev->private;
|
2011-04-21 18:07:26 +02:00
|
|
|
sector_t sectors;
|
2010-03-08 06:02:42 +01:00
|
|
|
|
2010-05-02 03:09:05 +02:00
|
|
|
/* for raid0 takeover only one zone is supported */
|
|
|
|
if (raid0_priv->nr_strip_zones > 1) {
|
2010-05-03 06:09:02 +02:00
|
|
|
printk(KERN_ERR "md/raid:%s: cannot takeover raid0 with more than one zone.\n",
|
|
|
|
mdname(mddev));
|
2010-05-02 03:09:05 +02:00
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
}
|
|
|
|
|
2011-04-20 07:38:18 +02:00
|
|
|
sectors = raid0_priv->strip_zone[0].zone_end;
|
|
|
|
sector_div(sectors, raid0_priv->strip_zone[0].nb_dev);
|
|
|
|
mddev->dev_sectors = sectors;
|
2010-05-02 03:09:05 +02:00
|
|
|
mddev->new_level = level;
|
2010-03-08 06:02:42 +01:00
|
|
|
mddev->new_layout = ALGORITHM_PARITY_N;
|
|
|
|
mddev->new_chunk_sectors = mddev->chunk_sectors;
|
|
|
|
mddev->raid_disks += 1;
|
|
|
|
mddev->delta_disks = 1;
|
|
|
|
/* make sure it will be not marked as dirty */
|
|
|
|
mddev->recovery_cp = MaxSector;
|
|
|
|
|
|
|
|
return setup_conf(mddev);
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2009-03-31 05:39:39 +02:00
|
|
|
static void *raid5_takeover_raid1(mddev_t *mddev)
|
|
|
|
{
|
|
|
|
int chunksect;
|
|
|
|
|
|
|
|
if (mddev->raid_disks != 2 ||
|
|
|
|
mddev->degraded > 1)
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
|
|
|
|
/* Should check if there are write-behind devices? */
|
|
|
|
|
|
|
|
chunksect = 64*2; /* 64K by default */
|
|
|
|
|
|
|
|
/* The array must be an exact multiple of chunksize */
|
|
|
|
while (chunksect && (mddev->array_sectors & (chunksect-1)))
|
|
|
|
chunksect >>= 1;
|
|
|
|
|
|
|
|
if ((chunksect<<9) < STRIPE_SIZE)
|
|
|
|
/* array size does not allow a suitable chunk size */
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
|
|
|
|
mddev->new_level = 5;
|
|
|
|
mddev->new_layout = ALGORITHM_LEFT_SYMMETRIC;
|
2009-06-18 00:45:27 +02:00
|
|
|
mddev->new_chunk_sectors = chunksect;
|
2009-03-31 05:39:39 +02:00
|
|
|
|
|
|
|
return setup_conf(mddev);
|
|
|
|
}
|
|
|
|
|
2009-03-31 05:57:20 +02:00
|
|
|
static void *raid5_takeover_raid6(mddev_t *mddev)
|
|
|
|
{
|
|
|
|
int new_layout;
|
|
|
|
|
|
|
|
switch (mddev->layout) {
|
|
|
|
case ALGORITHM_LEFT_ASYMMETRIC_6:
|
|
|
|
new_layout = ALGORITHM_LEFT_ASYMMETRIC;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_RIGHT_ASYMMETRIC_6:
|
|
|
|
new_layout = ALGORITHM_RIGHT_ASYMMETRIC;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_LEFT_SYMMETRIC_6:
|
|
|
|
new_layout = ALGORITHM_LEFT_SYMMETRIC;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_RIGHT_SYMMETRIC_6:
|
|
|
|
new_layout = ALGORITHM_RIGHT_SYMMETRIC;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_PARITY_0_6:
|
|
|
|
new_layout = ALGORITHM_PARITY_0;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_PARITY_N:
|
|
|
|
new_layout = ALGORITHM_PARITY_N;
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
}
|
|
|
|
mddev->new_level = 5;
|
|
|
|
mddev->new_layout = new_layout;
|
|
|
|
mddev->delta_disks = -1;
|
|
|
|
mddev->raid_disks -= 1;
|
|
|
|
return setup_conf(mddev);
|
|
|
|
}
|
|
|
|
|
2009-03-31 05:39:39 +02:00
|
|
|
|
2009-06-18 00:47:55 +02:00
|
|
|
static int raid5_check_reshape(mddev_t *mddev)
|
2009-03-31 05:56:41 +02:00
|
|
|
{
|
2009-03-31 06:24:23 +02:00
|
|
|
/* For a 2-drive array, the layout and chunk size can be changed
|
|
|
|
* immediately as not restriping is needed.
|
|
|
|
* For larger arrays we record the new value - after validation
|
|
|
|
* to be used by a reshape pass.
|
2009-03-31 05:56:41 +02:00
|
|
|
*/
|
2009-06-16 08:54:21 +02:00
|
|
|
raid5_conf_t *conf = mddev->private;
|
2009-06-18 00:47:42 +02:00
|
|
|
int new_chunk = mddev->new_chunk_sectors;
|
2009-03-31 05:56:41 +02:00
|
|
|
|
2009-06-18 00:47:42 +02:00
|
|
|
if (mddev->new_layout >= 0 && !algorithm_valid_raid5(mddev->new_layout))
|
2009-03-31 05:56:41 +02:00
|
|
|
return -EINVAL;
|
|
|
|
if (new_chunk > 0) {
|
2009-06-18 00:46:10 +02:00
|
|
|
if (!is_power_of_2(new_chunk))
|
2009-03-31 05:56:41 +02:00
|
|
|
return -EINVAL;
|
2009-06-18 00:47:42 +02:00
|
|
|
if (new_chunk < (PAGE_SIZE>>9))
|
2009-03-31 05:56:41 +02:00
|
|
|
return -EINVAL;
|
2009-06-18 00:47:42 +02:00
|
|
|
if (mddev->array_sectors & (new_chunk-1))
|
2009-03-31 05:56:41 +02:00
|
|
|
/* not factor of array size */
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* They look valid */
|
|
|
|
|
2009-03-31 06:24:23 +02:00
|
|
|
if (mddev->raid_disks == 2) {
|
2009-06-18 00:47:42 +02:00
|
|
|
/* can make the change immediately */
|
|
|
|
if (mddev->new_layout >= 0) {
|
|
|
|
conf->algorithm = mddev->new_layout;
|
|
|
|
mddev->layout = mddev->new_layout;
|
2009-03-31 06:24:23 +02:00
|
|
|
}
|
|
|
|
if (new_chunk > 0) {
|
2009-06-18 00:47:42 +02:00
|
|
|
conf->chunk_sectors = new_chunk ;
|
|
|
|
mddev->chunk_sectors = new_chunk;
|
2009-03-31 06:24:23 +02:00
|
|
|
}
|
|
|
|
set_bit(MD_CHANGE_DEVS, &mddev->flags);
|
|
|
|
md_wakeup_thread(mddev->thread);
|
2009-03-31 05:56:41 +02:00
|
|
|
}
|
2009-06-18 00:47:55 +02:00
|
|
|
return check_reshape(mddev);
|
2009-03-31 06:24:23 +02:00
|
|
|
}
|
|
|
|
|
2009-06-18 00:47:55 +02:00
|
|
|
static int raid6_check_reshape(mddev_t *mddev)
|
2009-03-31 06:24:23 +02:00
|
|
|
{
|
2009-06-18 00:47:42 +02:00
|
|
|
int new_chunk = mddev->new_chunk_sectors;
|
2009-06-18 00:47:55 +02:00
|
|
|
|
2009-06-18 00:47:42 +02:00
|
|
|
if (mddev->new_layout >= 0 && !algorithm_valid_raid6(mddev->new_layout))
|
2009-03-31 06:24:23 +02:00
|
|
|
return -EINVAL;
|
2009-03-31 05:56:41 +02:00
|
|
|
if (new_chunk > 0) {
|
2009-06-18 00:46:10 +02:00
|
|
|
if (!is_power_of_2(new_chunk))
|
2009-03-31 06:24:23 +02:00
|
|
|
return -EINVAL;
|
2009-06-18 00:47:42 +02:00
|
|
|
if (new_chunk < (PAGE_SIZE >> 9))
|
2009-03-31 06:24:23 +02:00
|
|
|
return -EINVAL;
|
2009-06-18 00:47:42 +02:00
|
|
|
if (mddev->array_sectors & (new_chunk-1))
|
2009-03-31 06:24:23 +02:00
|
|
|
/* not factor of array size */
|
|
|
|
return -EINVAL;
|
2009-03-31 05:56:41 +02:00
|
|
|
}
|
2009-03-31 06:24:23 +02:00
|
|
|
|
|
|
|
/* They look valid */
|
2009-06-18 00:47:55 +02:00
|
|
|
return check_reshape(mddev);
|
2009-03-31 05:56:41 +02:00
|
|
|
}
|
|
|
|
|
2009-03-31 05:39:39 +02:00
|
|
|
static void *raid5_takeover(mddev_t *mddev)
|
|
|
|
{
|
|
|
|
/* raid5 can take over:
|
2010-05-02 03:09:05 +02:00
|
|
|
* raid0 - if there is only one strip zone - make it a raid4 layout
|
2009-03-31 05:39:39 +02:00
|
|
|
* raid1 - if there are two drives. We need to know the chunk size
|
|
|
|
* raid4 - trivial - just use a raid4 layout.
|
|
|
|
* raid6 - Providing it is a *_6 layout
|
|
|
|
*/
|
2010-05-02 03:09:05 +02:00
|
|
|
if (mddev->level == 0)
|
|
|
|
return raid45_takeover_raid0(mddev, 5);
|
2009-03-31 05:39:39 +02:00
|
|
|
if (mddev->level == 1)
|
|
|
|
return raid5_takeover_raid1(mddev);
|
2009-03-31 05:57:09 +02:00
|
|
|
if (mddev->level == 4) {
|
|
|
|
mddev->new_layout = ALGORITHM_PARITY_N;
|
|
|
|
mddev->new_level = 5;
|
|
|
|
return setup_conf(mddev);
|
|
|
|
}
|
2009-03-31 05:57:20 +02:00
|
|
|
if (mddev->level == 6)
|
|
|
|
return raid5_takeover_raid6(mddev);
|
2009-03-31 05:39:39 +02:00
|
|
|
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
}
|
|
|
|
|
2010-03-22 06:53:49 +01:00
|
|
|
static void *raid4_takeover(mddev_t *mddev)
|
|
|
|
{
|
2010-05-02 03:09:05 +02:00
|
|
|
/* raid4 can take over:
|
|
|
|
* raid0 - if there is only one strip zone
|
|
|
|
* raid5 - if layout is right
|
2010-03-22 06:53:49 +01:00
|
|
|
*/
|
2010-05-02 03:09:05 +02:00
|
|
|
if (mddev->level == 0)
|
|
|
|
return raid45_takeover_raid0(mddev, 4);
|
2010-03-22 06:53:49 +01:00
|
|
|
if (mddev->level == 5 &&
|
|
|
|
mddev->layout == ALGORITHM_PARITY_N) {
|
|
|
|
mddev->new_layout = 0;
|
|
|
|
mddev->new_level = 4;
|
|
|
|
return setup_conf(mddev);
|
|
|
|
}
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
}
|
2009-03-31 05:39:39 +02:00
|
|
|
|
2009-03-31 05:39:39 +02:00
|
|
|
static struct mdk_personality raid5_personality;
|
|
|
|
|
|
|
|
static void *raid6_takeover(mddev_t *mddev)
|
|
|
|
{
|
|
|
|
/* Currently can only take over a raid5. We map the
|
|
|
|
* personality to an equivalent raid6 personality
|
|
|
|
* with the Q block at the end.
|
|
|
|
*/
|
|
|
|
int new_layout;
|
|
|
|
|
|
|
|
if (mddev->pers != &raid5_personality)
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
if (mddev->degraded > 1)
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
if (mddev->raid_disks > 253)
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
if (mddev->raid_disks < 3)
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
|
|
|
|
switch (mddev->layout) {
|
|
|
|
case ALGORITHM_LEFT_ASYMMETRIC:
|
|
|
|
new_layout = ALGORITHM_LEFT_ASYMMETRIC_6;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_RIGHT_ASYMMETRIC:
|
|
|
|
new_layout = ALGORITHM_RIGHT_ASYMMETRIC_6;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_LEFT_SYMMETRIC:
|
|
|
|
new_layout = ALGORITHM_LEFT_SYMMETRIC_6;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_RIGHT_SYMMETRIC:
|
|
|
|
new_layout = ALGORITHM_RIGHT_SYMMETRIC_6;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_PARITY_0:
|
|
|
|
new_layout = ALGORITHM_PARITY_0_6;
|
|
|
|
break;
|
|
|
|
case ALGORITHM_PARITY_N:
|
|
|
|
new_layout = ALGORITHM_PARITY_N;
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
}
|
|
|
|
mddev->new_level = 6;
|
|
|
|
mddev->new_layout = new_layout;
|
|
|
|
mddev->delta_disks = 1;
|
|
|
|
mddev->raid_disks += 1;
|
|
|
|
return setup_conf(mddev);
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2006-06-26 09:27:38 +02:00
|
|
|
static struct mdk_personality raid6_personality =
|
|
|
|
{
|
|
|
|
.name = "raid6",
|
|
|
|
.level = 6,
|
|
|
|
.owner = THIS_MODULE,
|
|
|
|
.make_request = make_request,
|
|
|
|
.run = run,
|
|
|
|
.stop = stop,
|
|
|
|
.status = status,
|
|
|
|
.error_handler = error,
|
|
|
|
.hot_add_disk = raid5_add_disk,
|
|
|
|
.hot_remove_disk= raid5_remove_disk,
|
|
|
|
.spare_active = raid5_spare_active,
|
|
|
|
.sync_request = sync_request,
|
|
|
|
.resize = raid5_resize,
|
2009-03-18 02:10:40 +01:00
|
|
|
.size = raid5_size,
|
2009-06-18 00:47:55 +02:00
|
|
|
.check_reshape = raid6_check_reshape,
|
2007-03-01 05:11:53 +01:00
|
|
|
.start_reshape = raid5_start_reshape,
|
2009-03-31 06:15:05 +02:00
|
|
|
.finish_reshape = raid5_finish_reshape,
|
2006-06-26 09:27:38 +02:00
|
|
|
.quiesce = raid5_quiesce,
|
2009-03-31 05:39:39 +02:00
|
|
|
.takeover = raid6_takeover,
|
2006-06-26 09:27:38 +02:00
|
|
|
};
|
2006-01-06 09:20:36 +01:00
|
|
|
static struct mdk_personality raid5_personality =
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
|
|
|
.name = "raid5",
|
2006-01-06 09:20:36 +01:00
|
|
|
.level = 5,
|
2005-04-17 00:20:36 +02:00
|
|
|
.owner = THIS_MODULE,
|
|
|
|
.make_request = make_request,
|
|
|
|
.run = run,
|
|
|
|
.stop = stop,
|
|
|
|
.status = status,
|
|
|
|
.error_handler = error,
|
|
|
|
.hot_add_disk = raid5_add_disk,
|
|
|
|
.hot_remove_disk= raid5_remove_disk,
|
|
|
|
.spare_active = raid5_spare_active,
|
|
|
|
.sync_request = sync_request,
|
|
|
|
.resize = raid5_resize,
|
2009-03-18 02:10:40 +01:00
|
|
|
.size = raid5_size,
|
2006-03-27 11:18:13 +02:00
|
|
|
.check_reshape = raid5_check_reshape,
|
|
|
|
.start_reshape = raid5_start_reshape,
|
2009-03-31 06:15:05 +02:00
|
|
|
.finish_reshape = raid5_finish_reshape,
|
2005-09-10 01:23:54 +02:00
|
|
|
.quiesce = raid5_quiesce,
|
2009-03-31 05:39:39 +02:00
|
|
|
.takeover = raid5_takeover,
|
2005-04-17 00:20:36 +02:00
|
|
|
};
|
|
|
|
|
2006-01-06 09:20:36 +01:00
|
|
|
static struct mdk_personality raid4_personality =
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
2006-01-06 09:20:36 +01:00
|
|
|
.name = "raid4",
|
|
|
|
.level = 4,
|
|
|
|
.owner = THIS_MODULE,
|
|
|
|
.make_request = make_request,
|
|
|
|
.run = run,
|
|
|
|
.stop = stop,
|
|
|
|
.status = status,
|
|
|
|
.error_handler = error,
|
|
|
|
.hot_add_disk = raid5_add_disk,
|
|
|
|
.hot_remove_disk= raid5_remove_disk,
|
|
|
|
.spare_active = raid5_spare_active,
|
|
|
|
.sync_request = sync_request,
|
|
|
|
.resize = raid5_resize,
|
2009-03-18 02:10:40 +01:00
|
|
|
.size = raid5_size,
|
2007-03-27 07:32:13 +02:00
|
|
|
.check_reshape = raid5_check_reshape,
|
|
|
|
.start_reshape = raid5_start_reshape,
|
2009-03-31 06:15:05 +02:00
|
|
|
.finish_reshape = raid5_finish_reshape,
|
2006-01-06 09:20:36 +01:00
|
|
|
.quiesce = raid5_quiesce,
|
2010-03-22 06:53:49 +01:00
|
|
|
.takeover = raid4_takeover,
|
2006-01-06 09:20:36 +01:00
|
|
|
};
|
|
|
|
|
|
|
|
static int __init raid5_init(void)
|
|
|
|
{
|
2006-06-26 09:27:38 +02:00
|
|
|
register_md_personality(&raid6_personality);
|
2006-01-06 09:20:36 +01:00
|
|
|
register_md_personality(&raid5_personality);
|
|
|
|
register_md_personality(&raid4_personality);
|
|
|
|
return 0;
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
2006-01-06 09:20:36 +01:00
|
|
|
static void raid5_exit(void)
|
2005-04-17 00:20:36 +02:00
|
|
|
{
|
2006-06-26 09:27:38 +02:00
|
|
|
unregister_md_personality(&raid6_personality);
|
2006-01-06 09:20:36 +01:00
|
|
|
unregister_md_personality(&raid5_personality);
|
|
|
|
unregister_md_personality(&raid4_personality);
|
2005-04-17 00:20:36 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
module_init(raid5_init);
|
|
|
|
module_exit(raid5_exit);
|
|
|
|
MODULE_LICENSE("GPL");
|
2009-12-14 02:49:58 +01:00
|
|
|
MODULE_DESCRIPTION("RAID4/5/6 (striping with parity) personality for MD");
|
2005-04-17 00:20:36 +02:00
|
|
|
MODULE_ALIAS("md-personality-4"); /* RAID5 */
|
2006-01-06 09:20:51 +01:00
|
|
|
MODULE_ALIAS("md-raid5");
|
|
|
|
MODULE_ALIAS("md-raid4");
|
2006-01-06 09:20:36 +01:00
|
|
|
MODULE_ALIAS("md-level-5");
|
|
|
|
MODULE_ALIAS("md-level-4");
|
2006-06-26 09:27:38 +02:00
|
|
|
MODULE_ALIAS("md-personality-8"); /* RAID6 */
|
|
|
|
MODULE_ALIAS("md-raid6");
|
|
|
|
MODULE_ALIAS("md-level-6");
|
|
|
|
|
|
|
|
/* This used to be two separate modules, they were: */
|
|
|
|
MODULE_ALIAS("raid5");
|
|
|
|
MODULE_ALIAS("raid6");
|