2008-10-27 17:27:55 +01:00
|
|
|
/*
|
|
|
|
* Copyright (C) 2005, 2006
|
2009-06-14 16:23:09 +02:00
|
|
|
* Avishay Traeger (avishay@gmail.com)
|
2008-10-27 17:27:55 +01:00
|
|
|
* Copyright (C) 2008, 2009
|
|
|
|
* Boaz Harrosh <bharrosh@panasas.com>
|
|
|
|
*
|
|
|
|
* Copyrights for code taken from ext2:
|
|
|
|
* Copyright (C) 1992, 1993, 1994, 1995
|
|
|
|
* Remy Card (card@masi.ibp.fr)
|
|
|
|
* Laboratoire MASI - Institut Blaise Pascal
|
|
|
|
* Universite Pierre et Marie Curie (Paris VI)
|
|
|
|
* from
|
|
|
|
* linux/fs/minix/inode.c
|
|
|
|
* Copyright (C) 1991, 1992 Linus Torvalds
|
|
|
|
*
|
|
|
|
* This file is part of exofs.
|
|
|
|
*
|
|
|
|
* exofs is free software; you can redistribute it and/or modify
|
|
|
|
* it under the terms of the GNU General Public License as published by
|
|
|
|
* the Free Software Foundation. Since it is based on ext2, and the only
|
|
|
|
* valid version of GPL for the Linux kernel is version 2, the only valid
|
|
|
|
* version of GPL for exofs is version 2.
|
|
|
|
*
|
|
|
|
* exofs is distributed in the hope that it will be useful,
|
|
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
* GNU General Public License for more details.
|
|
|
|
*
|
|
|
|
* You should have received a copy of the GNU General Public License
|
|
|
|
* along with exofs; if not, write to the Free Software
|
|
|
|
* Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
|
|
|
*/
|
exofs: Move all operations to an io_engine
In anticipation for multi-device operations, we separate osd operations
into an abstract I/O API. Currently only one device is used but later
when adding more devices, we will drive all devices in parallel according
to a "data_map" that describes how data is arranged on multiple devices.
The file system level operates, like before, as if there is one object
(inode-number) and an i_size. The io engine will split this to the same
object-number but on multiple device.
At first we introduce Mirror (raid 1) layout. But at the final outcome
we intend to fully implement the pNFS-Objects data-map, including
raid 0,4,5,6 over mirrored devices, over multiple device-groups. And
more. See: http://tools.ietf.org/html/draft-ietf-nfsv4-pnfs-obj-12
* Define an io_state based API for accessing osd storage devices
in an abstract way.
Usage:
First a caller allocates an io state with:
exofs_get_io_state(struct exofs_sb_info *sbi,
struct exofs_io_state** ios);
Then calles one of:
exofs_sbi_create(struct exofs_io_state *ios);
exofs_sbi_remove(struct exofs_io_state *ios);
exofs_sbi_write(struct exofs_io_state *ios);
exofs_sbi_read(struct exofs_io_state *ios);
exofs_oi_truncate(struct exofs_i_info *oi, u64 new_len);
And when done
exofs_put_io_state(struct exofs_io_state *ios);
* Convert all source files to use this new API
* Convert from bio_alloc to bio_kmalloc
* In io engine we make use of the now fixed osd_req_decode_sense
There are no functional changes or on disk additions after this patch.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2009-11-08 13:54:08 +01:00
|
|
|
#ifndef __EXOFS_H__
|
|
|
|
#define __EXOFS_H__
|
2008-10-27 17:27:55 +01:00
|
|
|
|
|
|
|
#include <linux/fs.h>
|
|
|
|
#include <linux/time.h>
|
2010-04-22 12:26:04 +02:00
|
|
|
#include <linux/backing-dev.h>
|
2010-02-02 14:56:53 +01:00
|
|
|
#include <linux/pnfs_osd_xdr.h>
|
2008-10-27 17:27:55 +01:00
|
|
|
#include "common.h"
|
|
|
|
|
|
|
|
#define EXOFS_ERR(fmt, a...) printk(KERN_ERR "exofs: " fmt, ##a)
|
|
|
|
|
|
|
|
#ifdef CONFIG_EXOFS_DEBUG
|
|
|
|
#define EXOFS_DBGMSG(fmt, a...) \
|
|
|
|
printk(KERN_NOTICE "exofs @%s:%d: " fmt, __func__, __LINE__, ##a)
|
|
|
|
#else
|
|
|
|
#define EXOFS_DBGMSG(fmt, a...) \
|
|
|
|
do { if (0) printk(fmt, ##a); } while (0)
|
|
|
|
#endif
|
|
|
|
|
|
|
|
/* u64 has problems with printk this will cast it to unsigned long long */
|
|
|
|
#define _LLU(x) (unsigned long long)(x)
|
|
|
|
|
exofs: ios: Move to a per inode components & device-table
Exofs raid engine was saving on memory space by having a single layout-info,
single pid, and a single device-table, global to the filesystem. Then passing
a credential and object_id info at the io_state level, private for each
inode. It would also devise this contraption of rotating the device table
view for each inode->ino to spread out the device usage.
This is not compatible with the pnfs-objects standard, demanding that
each inode can have it's own layout-info, device-table, and each object
component it's own pid, oid and creds.
So: Bring exofs raid engine to be usable for generic pnfs-objects use by:
* Define an exofs_comp structure that holds obj_id and credential info.
* Break up exofs_layout struct to an exofs_components structure that holds a
possible array of exofs_comp and the array of devices + the size of the
arrays.
* Add a "comps" parameter to get_io_state() that specifies the ids creds
and device array to use for each IO.
This enables to keep the layout global, but the device-table view, creds
and IDs at the inode level. It only adds two 64bit to each inode, since
some of these members already existed in another form.
* ios raid engine now access layout-info and comps-info through the passed
pointers. Everything is pre-prepared by caller for generic access of
these structures and arrays.
At the exofs Level:
* Super block holds an exofs_components struct that holds the device
array, previously in layout. The devices there are in device-table
order. The device-array is twice bigger and repeats the device-table
twice so now each inode's device array can point to a random device
and have a round-robin view of the table, making it compatible to
previous exofs versions.
* Each inode has an exofs_components struct that is initialized at
load time, with it's own view of the device table IDs and creds.
When doing IO this gets passed to the io_state together with the
layout.
While preforming this change. Bugs where found where credentials with the
wrong IDs where used to access the different SB objects (super.c). As well
as some dead code. It was never noticed because the target we use does not
check the credentials.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2011-08-06 00:06:04 +02:00
|
|
|
struct exofs_comp {
|
|
|
|
struct osd_obj_id obj;
|
|
|
|
u8 cred[OSD_CAP_LEN];
|
|
|
|
};
|
2010-01-28 10:46:16 +01:00
|
|
|
|
exofs: ios: Move to a per inode components & device-table
Exofs raid engine was saving on memory space by having a single layout-info,
single pid, and a single device-table, global to the filesystem. Then passing
a credential and object_id info at the io_state level, private for each
inode. It would also devise this contraption of rotating the device table
view for each inode->ino to spread out the device usage.
This is not compatible with the pnfs-objects standard, demanding that
each inode can have it's own layout-info, device-table, and each object
component it's own pid, oid and creds.
So: Bring exofs raid engine to be usable for generic pnfs-objects use by:
* Define an exofs_comp structure that holds obj_id and credential info.
* Break up exofs_layout struct to an exofs_components structure that holds a
possible array of exofs_comp and the array of devices + the size of the
arrays.
* Add a "comps" parameter to get_io_state() that specifies the ids creds
and device array to use for each IO.
This enables to keep the layout global, but the device-table view, creds
and IDs at the inode level. It only adds two 64bit to each inode, since
some of these members already existed in another form.
* ios raid engine now access layout-info and comps-info through the passed
pointers. Everything is pre-prepared by caller for generic access of
these structures and arrays.
At the exofs Level:
* Super block holds an exofs_components struct that holds the device
array, previously in layout. The devices there are in device-table
order. The device-array is twice bigger and repeats the device-table
twice so now each inode's device array can point to a random device
and have a round-robin view of the table, making it compatible to
previous exofs versions.
* Each inode has an exofs_components struct that is initialized at
load time, with it's own view of the device table IDs and creds.
When doing IO this gets passed to the io_state together with the
layout.
While preforming this change. Bugs where found where credentials with the
wrong IDs where used to access the different SB objects (super.c). As well
as some dead code. It was never noticed because the target we use does not
check the credentials.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2011-08-06 00:06:04 +02:00
|
|
|
struct exofs_layout {
|
exofs: RAID0 support
We now support striping over mirror devices. Including variable sized
stripe_unit.
Some limits:
* stripe_unit must be a multiple of PAGE_SIZE
* stripe_unit * stripe_count is maximum upto 32-bit (4Gb)
Tested RAID0 over mirrors, RAID0 only, mirrors only. All check.
Design notes:
* I'm not using a vectored raid-engine mechanism yet. Following the
pnfs-objects-layout data-map structure, "Mirror" is just a private
case of "group_width" == 1, and RAID0 is a private case of
"Mirrors" == 1. The performance lose of the general case over the
particular special case optimization is totally negligible, also
considering the extra code size.
* In general I added a prepare_stripes() stage that divides the
to-be-io pages to the participating devices, the previous
exofs_ios_write/read, now becomes _write/read_mirrors and a new
write/read upper layer loops on all devices calling
_write/read_mirrors. Effectively the prepare_stripes stage is the all
secret.
Also truncate need fixing to accommodate for striping.
* In a RAID0 arrangement, in a regular usage scenario, if all inode
layouts will start at the same device, the small files fill up the
first device and the later devices stay empty, the farther the device
the emptier it is.
To fix that, each inode will start at a different stripe_unit,
according to it's obj_id modulus number-of-stripe-units. And
will then span all stripe-units in the same incrementing order
wrapping back to the beginning of the device table. We call it
a stripe-units moving window.
Special consideration was taken to keep all devices in a mirror
arrangement identical. So a broken osd-device could just be cloned
from one of the mirrors and no FS scrubbing is needed. (We do that
by rotating stripe-unit at a time and not a single device at a time.)
TODO:
We no longer verify object_length == inode->i_size in exofs_iget.
(since i_size is stripped on multiple objects now).
I should introduce a multiple-device attribute reading, and use
it in exofs_iget.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2010-02-01 12:35:51 +01:00
|
|
|
/* Our way of looking at the data_map */
|
|
|
|
unsigned stripe_unit;
|
|
|
|
unsigned mirrors_p1;
|
|
|
|
|
|
|
|
unsigned group_width;
|
2010-02-11 12:01:39 +01:00
|
|
|
u64 group_depth;
|
|
|
|
unsigned group_count;
|
exofs: ios: Move to a per inode components & device-table
Exofs raid engine was saving on memory space by having a single layout-info,
single pid, and a single device-table, global to the filesystem. Then passing
a credential and object_id info at the io_state level, private for each
inode. It would also devise this contraption of rotating the device table
view for each inode->ino to spread out the device usage.
This is not compatible with the pnfs-objects standard, demanding that
each inode can have it's own layout-info, device-table, and each object
component it's own pid, oid and creds.
So: Bring exofs raid engine to be usable for generic pnfs-objects use by:
* Define an exofs_comp structure that holds obj_id and credential info.
* Break up exofs_layout struct to an exofs_components structure that holds a
possible array of exofs_comp and the array of devices + the size of the
arrays.
* Add a "comps" parameter to get_io_state() that specifies the ids creds
and device array to use for each IO.
This enables to keep the layout global, but the device-table view, creds
and IDs at the inode level. It only adds two 64bit to each inode, since
some of these members already existed in another form.
* ios raid engine now access layout-info and comps-info through the passed
pointers. Everything is pre-prepared by caller for generic access of
these structures and arrays.
At the exofs Level:
* Super block holds an exofs_components struct that holds the device
array, previously in layout. The devices there are in device-table
order. The device-array is twice bigger and repeats the device-table
twice so now each inode's device array can point to a random device
and have a round-robin view of the table, making it compatible to
previous exofs versions.
* Each inode has an exofs_components struct that is initialized at
load time, with it's own view of the device table IDs and creds.
When doing IO this gets passed to the io_state together with the
layout.
While preforming this change. Bugs where found where credentials with the
wrong IDs where used to access the different SB objects (super.c). As well
as some dead code. It was never noticed because the target we use does not
check the credentials.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2011-08-06 00:06:04 +02:00
|
|
|
};
|
exofs: RAID0 support
We now support striping over mirror devices. Including variable sized
stripe_unit.
Some limits:
* stripe_unit must be a multiple of PAGE_SIZE
* stripe_unit * stripe_count is maximum upto 32-bit (4Gb)
Tested RAID0 over mirrors, RAID0 only, mirrors only. All check.
Design notes:
* I'm not using a vectored raid-engine mechanism yet. Following the
pnfs-objects-layout data-map structure, "Mirror" is just a private
case of "group_width" == 1, and RAID0 is a private case of
"Mirrors" == 1. The performance lose of the general case over the
particular special case optimization is totally negligible, also
considering the extra code size.
* In general I added a prepare_stripes() stage that divides the
to-be-io pages to the participating devices, the previous
exofs_ios_write/read, now becomes _write/read_mirrors and a new
write/read upper layer loops on all devices calling
_write/read_mirrors. Effectively the prepare_stripes stage is the all
secret.
Also truncate need fixing to accommodate for striping.
* In a RAID0 arrangement, in a regular usage scenario, if all inode
layouts will start at the same device, the small files fill up the
first device and the later devices stay empty, the farther the device
the emptier it is.
To fix that, each inode will start at a different stripe_unit,
according to it's obj_id modulus number-of-stripe-units. And
will then span all stripe-units in the same incrementing order
wrapping back to the beginning of the device table. We call it
a stripe-units moving window.
Special consideration was taken to keep all devices in a mirror
arrangement identical. So a broken osd-device could just be cloned
from one of the mirrors and no FS scrubbing is needed. (We do that
by rotating stripe-unit at a time and not a single device at a time.)
TODO:
We no longer verify object_length == inode->i_size in exofs_iget.
(since i_size is stripped on multiple objects now).
I should introduce a multiple-device attribute reading, and use
it in exofs_iget.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2010-02-01 12:35:51 +01:00
|
|
|
|
exofs: ios: Move to a per inode components & device-table
Exofs raid engine was saving on memory space by having a single layout-info,
single pid, and a single device-table, global to the filesystem. Then passing
a credential and object_id info at the io_state level, private for each
inode. It would also devise this contraption of rotating the device table
view for each inode->ino to spread out the device usage.
This is not compatible with the pnfs-objects standard, demanding that
each inode can have it's own layout-info, device-table, and each object
component it's own pid, oid and creds.
So: Bring exofs raid engine to be usable for generic pnfs-objects use by:
* Define an exofs_comp structure that holds obj_id and credential info.
* Break up exofs_layout struct to an exofs_components structure that holds a
possible array of exofs_comp and the array of devices + the size of the
arrays.
* Add a "comps" parameter to get_io_state() that specifies the ids creds
and device array to use for each IO.
This enables to keep the layout global, but the device-table view, creds
and IDs at the inode level. It only adds two 64bit to each inode, since
some of these members already existed in another form.
* ios raid engine now access layout-info and comps-info through the passed
pointers. Everything is pre-prepared by caller for generic access of
these structures and arrays.
At the exofs Level:
* Super block holds an exofs_components struct that holds the device
array, previously in layout. The devices there are in device-table
order. The device-array is twice bigger and repeats the device-table
twice so now each inode's device array can point to a random device
and have a round-robin view of the table, making it compatible to
previous exofs versions.
* Each inode has an exofs_components struct that is initialized at
load time, with it's own view of the device table IDs and creds.
When doing IO this gets passed to the io_state together with the
layout.
While preforming this change. Bugs where found where credentials with the
wrong IDs where used to access the different SB objects (super.c). As well
as some dead code. It was never noticed because the target we use does not
check the credentials.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2011-08-06 00:06:04 +02:00
|
|
|
struct exofs_components {
|
|
|
|
unsigned numdevs; /* Num of devices in array */
|
|
|
|
/* If @single_comp == EC_SINGLE_COMP, @comps points to a single
|
|
|
|
* component. else there are @numdevs components
|
|
|
|
*/
|
|
|
|
enum EC_COMP_USAGE {
|
|
|
|
EC_SINGLE_COMP = 0, EC_MULTPLE_COMPS = 0xffffffff
|
|
|
|
} single_comp;
|
|
|
|
struct exofs_comp *comps;
|
|
|
|
struct osd_dev **ods; /* osd_dev array */
|
2010-01-28 10:46:16 +01:00
|
|
|
};
|
|
|
|
|
2008-10-27 17:27:55 +01:00
|
|
|
/*
|
|
|
|
* our extension to the in-memory superblock
|
|
|
|
*/
|
|
|
|
struct exofs_sb_info {
|
2011-07-28 02:51:53 +02:00
|
|
|
struct backing_dev_info bdi; /* register our bdi with VFS */
|
2011-02-03 16:53:25 +01:00
|
|
|
struct exofs_sb_stats s_ess; /* Written often, pre-allocate*/
|
2008-10-27 17:27:55 +01:00
|
|
|
int s_timeout; /* timeout for OSD operations */
|
|
|
|
uint64_t s_nextid; /* highest object ID used */
|
|
|
|
uint32_t s_numfiles; /* number of files on fs */
|
|
|
|
spinlock_t s_next_gen_lock; /* spinlock for gen # update */
|
|
|
|
u32 s_next_generation; /* next gen # to use */
|
|
|
|
atomic_t s_curr_pending; /* number of pending commands */
|
exofs: Multi-device mirror support
This patch changes on-disk format, it is accompanied with a parallel
patch to mkfs.exofs that enables multi-device capabilities.
After this patch, old exofs will refuse to mount a new formatted FS and
new exofs will refuse an old format. This is done by moving the magic
field offset inside the FSCB. A new FSCB *version* field was added. In
the future, exofs will refuse to mount unmatched FSCB version. To
up-grade or down-grade an exofs one must use mkfs.exofs --upgrade option
before mounting.
Introduced, a new object that contains a *device-table*. This object
contains the default *data-map* and a linear array of devices
information, which identifies the devices used in the filesystem. This
object is only written to offline by mkfs.exofs. This is why it is kept
separate from the FSCB, since the later is written to while mounted.
Same partition number, same object number is used on all devices only
the device varies.
* define the new format, then load the device table on mount time make
sure every thing is supported.
* Change I/O engine to now support Mirror IO, .i.e write same data
to multiple devices, read from a random device to spread the
read-load from multiple clients (TODO: stripe read)
Implementation notes:
A few points introduced in previous patch should be mentioned here:
* Special care was made so absolutlly all operation that have any chance
of failing are done before any osd-request is executed. This is to
minimize the need for a data consistency recovery, to only real IO
errors.
* Each IO state has a kref. It starts at 1, any osd-request executed
will increment the kref, finally when all are executed the first ref
is dropped. At IO-done, each request completion decrements the kref,
the last one to return executes the internal _last_io() routine.
_last_io() will call the registered io_state_done. On sync mode a
caller does not supply a done method, indicating a synchronous
request, the caller is put to sleep and a special io_state_done is
registered that will awaken the caller. Though also in sync mode all
operations are executed in parallel.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2009-11-16 15:03:05 +01:00
|
|
|
|
2010-01-28 10:46:16 +01:00
|
|
|
struct pnfs_osd_data_map data_map; /* Default raid to use
|
|
|
|
* FIXME: Needed ?
|
|
|
|
*/
|
2011-07-28 02:51:53 +02:00
|
|
|
struct exofs_layout layout; /* Default files layout */
|
exofs: ios: Move to a per inode components & device-table
Exofs raid engine was saving on memory space by having a single layout-info,
single pid, and a single device-table, global to the filesystem. Then passing
a credential and object_id info at the io_state level, private for each
inode. It would also devise this contraption of rotating the device table
view for each inode->ino to spread out the device usage.
This is not compatible with the pnfs-objects standard, demanding that
each inode can have it's own layout-info, device-table, and each object
component it's own pid, oid and creds.
So: Bring exofs raid engine to be usable for generic pnfs-objects use by:
* Define an exofs_comp structure that holds obj_id and credential info.
* Break up exofs_layout struct to an exofs_components structure that holds a
possible array of exofs_comp and the array of devices + the size of the
arrays.
* Add a "comps" parameter to get_io_state() that specifies the ids creds
and device array to use for each IO.
This enables to keep the layout global, but the device-table view, creds
and IDs at the inode level. It only adds two 64bit to each inode, since
some of these members already existed in another form.
* ios raid engine now access layout-info and comps-info through the passed
pointers. Everything is pre-prepared by caller for generic access of
these structures and arrays.
At the exofs Level:
* Super block holds an exofs_components struct that holds the device
array, previously in layout. The devices there are in device-table
order. The device-array is twice bigger and repeats the device-table
twice so now each inode's device array can point to a random device
and have a round-robin view of the table, making it compatible to
previous exofs versions.
* Each inode has an exofs_components struct that is initialized at
load time, with it's own view of the device table IDs and creds.
When doing IO this gets passed to the io_state together with the
layout.
While preforming this change. Bugs where found where credentials with the
wrong IDs where used to access the different SB objects (super.c). As well
as some dead code. It was never noticed because the target we use does not
check the credentials.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2011-08-06 00:06:04 +02:00
|
|
|
struct exofs_comp one_comp; /* id & cred of partition id=0*/
|
|
|
|
struct exofs_components comps; /* comps for the partition */
|
2010-01-28 10:46:16 +01:00
|
|
|
struct osd_dev *_min_one_dev[1]; /* Place holder for one dev */
|
2008-10-27 17:27:55 +01:00
|
|
|
};
|
|
|
|
|
|
|
|
/*
|
|
|
|
* our extension to the in-memory inode
|
|
|
|
*/
|
|
|
|
struct exofs_i_info {
|
2010-01-21 19:00:02 +01:00
|
|
|
struct inode vfs_inode; /* normal in-memory inode */
|
|
|
|
wait_queue_head_t i_wq; /* wait queue for inode */
|
2008-10-27 17:27:55 +01:00
|
|
|
unsigned long i_flags; /* various atomic flags */
|
|
|
|
uint32_t i_data[EXOFS_IDATA];/*short symlink names and device #s*/
|
|
|
|
uint32_t i_dir_start_lookup; /* which page to start lookup */
|
|
|
|
uint64_t i_commit_size; /* the object's written length */
|
exofs: ios: Move to a per inode components & device-table
Exofs raid engine was saving on memory space by having a single layout-info,
single pid, and a single device-table, global to the filesystem. Then passing
a credential and object_id info at the io_state level, private for each
inode. It would also devise this contraption of rotating the device table
view for each inode->ino to spread out the device usage.
This is not compatible with the pnfs-objects standard, demanding that
each inode can have it's own layout-info, device-table, and each object
component it's own pid, oid and creds.
So: Bring exofs raid engine to be usable for generic pnfs-objects use by:
* Define an exofs_comp structure that holds obj_id and credential info.
* Break up exofs_layout struct to an exofs_components structure that holds a
possible array of exofs_comp and the array of devices + the size of the
arrays.
* Add a "comps" parameter to get_io_state() that specifies the ids creds
and device array to use for each IO.
This enables to keep the layout global, but the device-table view, creds
and IDs at the inode level. It only adds two 64bit to each inode, since
some of these members already existed in another form.
* ios raid engine now access layout-info and comps-info through the passed
pointers. Everything is pre-prepared by caller for generic access of
these structures and arrays.
At the exofs Level:
* Super block holds an exofs_components struct that holds the device
array, previously in layout. The devices there are in device-table
order. The device-array is twice bigger and repeats the device-table
twice so now each inode's device array can point to a random device
and have a round-robin view of the table, making it compatible to
previous exofs versions.
* Each inode has an exofs_components struct that is initialized at
load time, with it's own view of the device table IDs and creds.
When doing IO this gets passed to the io_state together with the
layout.
While preforming this change. Bugs where found where credentials with the
wrong IDs where used to access the different SB objects (super.c). As well
as some dead code. It was never noticed because the target we use does not
check the credentials.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2011-08-06 00:06:04 +02:00
|
|
|
struct exofs_comp one_comp; /* same component for all devices */
|
|
|
|
struct exofs_components comps; /* inode view of the device table */
|
2008-10-27 17:27:55 +01:00
|
|
|
};
|
|
|
|
|
exofs: Move all operations to an io_engine
In anticipation for multi-device operations, we separate osd operations
into an abstract I/O API. Currently only one device is used but later
when adding more devices, we will drive all devices in parallel according
to a "data_map" that describes how data is arranged on multiple devices.
The file system level operates, like before, as if there is one object
(inode-number) and an i_size. The io engine will split this to the same
object-number but on multiple device.
At first we introduce Mirror (raid 1) layout. But at the final outcome
we intend to fully implement the pNFS-Objects data-map, including
raid 0,4,5,6 over mirrored devices, over multiple device-groups. And
more. See: http://tools.ietf.org/html/draft-ietf-nfsv4-pnfs-obj-12
* Define an io_state based API for accessing osd storage devices
in an abstract way.
Usage:
First a caller allocates an io state with:
exofs_get_io_state(struct exofs_sb_info *sbi,
struct exofs_io_state** ios);
Then calles one of:
exofs_sbi_create(struct exofs_io_state *ios);
exofs_sbi_remove(struct exofs_io_state *ios);
exofs_sbi_write(struct exofs_io_state *ios);
exofs_sbi_read(struct exofs_io_state *ios);
exofs_oi_truncate(struct exofs_i_info *oi, u64 new_len);
And when done
exofs_put_io_state(struct exofs_io_state *ios);
* Convert all source files to use this new API
* Convert from bio_alloc to bio_kmalloc
* In io engine we make use of the now fixed osd_req_decode_sense
There are no functional changes or on disk additions after this patch.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2009-11-08 13:54:08 +01:00
|
|
|
static inline osd_id exofs_oi_objno(struct exofs_i_info *oi)
|
|
|
|
{
|
|
|
|
return oi->vfs_inode.i_ino + EXOFS_OBJ_OFF;
|
|
|
|
}
|
|
|
|
|
|
|
|
struct exofs_io_state;
|
2010-11-16 19:09:58 +01:00
|
|
|
typedef void (*exofs_io_done_fn)(struct exofs_io_state *ios, void *private);
|
exofs: Move all operations to an io_engine
In anticipation for multi-device operations, we separate osd operations
into an abstract I/O API. Currently only one device is used but later
when adding more devices, we will drive all devices in parallel according
to a "data_map" that describes how data is arranged on multiple devices.
The file system level operates, like before, as if there is one object
(inode-number) and an i_size. The io engine will split this to the same
object-number but on multiple device.
At first we introduce Mirror (raid 1) layout. But at the final outcome
we intend to fully implement the pNFS-Objects data-map, including
raid 0,4,5,6 over mirrored devices, over multiple device-groups. And
more. See: http://tools.ietf.org/html/draft-ietf-nfsv4-pnfs-obj-12
* Define an io_state based API for accessing osd storage devices
in an abstract way.
Usage:
First a caller allocates an io state with:
exofs_get_io_state(struct exofs_sb_info *sbi,
struct exofs_io_state** ios);
Then calles one of:
exofs_sbi_create(struct exofs_io_state *ios);
exofs_sbi_remove(struct exofs_io_state *ios);
exofs_sbi_write(struct exofs_io_state *ios);
exofs_sbi_read(struct exofs_io_state *ios);
exofs_oi_truncate(struct exofs_i_info *oi, u64 new_len);
And when done
exofs_put_io_state(struct exofs_io_state *ios);
* Convert all source files to use this new API
* Convert from bio_alloc to bio_kmalloc
* In io engine we make use of the now fixed osd_req_decode_sense
There are no functional changes or on disk additions after this patch.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2009-11-08 13:54:08 +01:00
|
|
|
|
|
|
|
struct exofs_io_state {
|
|
|
|
struct kref kref;
|
|
|
|
|
|
|
|
void *private;
|
|
|
|
exofs_io_done_fn done;
|
|
|
|
|
2010-01-28 10:46:16 +01:00
|
|
|
struct exofs_layout *layout;
|
exofs: ios: Move to a per inode components & device-table
Exofs raid engine was saving on memory space by having a single layout-info,
single pid, and a single device-table, global to the filesystem. Then passing
a credential and object_id info at the io_state level, private for each
inode. It would also devise this contraption of rotating the device table
view for each inode->ino to spread out the device usage.
This is not compatible with the pnfs-objects standard, demanding that
each inode can have it's own layout-info, device-table, and each object
component it's own pid, oid and creds.
So: Bring exofs raid engine to be usable for generic pnfs-objects use by:
* Define an exofs_comp structure that holds obj_id and credential info.
* Break up exofs_layout struct to an exofs_components structure that holds a
possible array of exofs_comp and the array of devices + the size of the
arrays.
* Add a "comps" parameter to get_io_state() that specifies the ids creds
and device array to use for each IO.
This enables to keep the layout global, but the device-table view, creds
and IDs at the inode level. It only adds two 64bit to each inode, since
some of these members already existed in another form.
* ios raid engine now access layout-info and comps-info through the passed
pointers. Everything is pre-prepared by caller for generic access of
these structures and arrays.
At the exofs Level:
* Super block holds an exofs_components struct that holds the device
array, previously in layout. The devices there are in device-table
order. The device-array is twice bigger and repeats the device-table
twice so now each inode's device array can point to a random device
and have a round-robin view of the table, making it compatible to
previous exofs versions.
* Each inode has an exofs_components struct that is initialized at
load time, with it's own view of the device table IDs and creds.
When doing IO this gets passed to the io_state together with the
layout.
While preforming this change. Bugs where found where credentials with the
wrong IDs where used to access the different SB objects (super.c). As well
as some dead code. It was never noticed because the target we use does not
check the credentials.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2011-08-06 00:06:04 +02:00
|
|
|
struct exofs_components *comps;
|
exofs: Move all operations to an io_engine
In anticipation for multi-device operations, we separate osd operations
into an abstract I/O API. Currently only one device is used but later
when adding more devices, we will drive all devices in parallel according
to a "data_map" that describes how data is arranged on multiple devices.
The file system level operates, like before, as if there is one object
(inode-number) and an i_size. The io engine will split this to the same
object-number but on multiple device.
At first we introduce Mirror (raid 1) layout. But at the final outcome
we intend to fully implement the pNFS-Objects data-map, including
raid 0,4,5,6 over mirrored devices, over multiple device-groups. And
more. See: http://tools.ietf.org/html/draft-ietf-nfsv4-pnfs-obj-12
* Define an io_state based API for accessing osd storage devices
in an abstract way.
Usage:
First a caller allocates an io state with:
exofs_get_io_state(struct exofs_sb_info *sbi,
struct exofs_io_state** ios);
Then calles one of:
exofs_sbi_create(struct exofs_io_state *ios);
exofs_sbi_remove(struct exofs_io_state *ios);
exofs_sbi_write(struct exofs_io_state *ios);
exofs_sbi_read(struct exofs_io_state *ios);
exofs_oi_truncate(struct exofs_i_info *oi, u64 new_len);
And when done
exofs_put_io_state(struct exofs_io_state *ios);
* Convert all source files to use this new API
* Convert from bio_alloc to bio_kmalloc
* In io engine we make use of the now fixed osd_req_decode_sense
There are no functional changes or on disk additions after this patch.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2009-11-08 13:54:08 +01:00
|
|
|
|
|
|
|
/* Global read/write IO*/
|
|
|
|
loff_t offset;
|
|
|
|
unsigned long length;
|
|
|
|
void *kern_buff;
|
2010-01-28 17:24:06 +01:00
|
|
|
|
|
|
|
struct page **pages;
|
|
|
|
unsigned nr_pages;
|
|
|
|
unsigned pgbase;
|
2010-02-11 12:01:39 +01:00
|
|
|
unsigned pages_consumed;
|
exofs: Move all operations to an io_engine
In anticipation for multi-device operations, we separate osd operations
into an abstract I/O API. Currently only one device is used but later
when adding more devices, we will drive all devices in parallel according
to a "data_map" that describes how data is arranged on multiple devices.
The file system level operates, like before, as if there is one object
(inode-number) and an i_size. The io engine will split this to the same
object-number but on multiple device.
At first we introduce Mirror (raid 1) layout. But at the final outcome
we intend to fully implement the pNFS-Objects data-map, including
raid 0,4,5,6 over mirrored devices, over multiple device-groups. And
more. See: http://tools.ietf.org/html/draft-ietf-nfsv4-pnfs-obj-12
* Define an io_state based API for accessing osd storage devices
in an abstract way.
Usage:
First a caller allocates an io state with:
exofs_get_io_state(struct exofs_sb_info *sbi,
struct exofs_io_state** ios);
Then calles one of:
exofs_sbi_create(struct exofs_io_state *ios);
exofs_sbi_remove(struct exofs_io_state *ios);
exofs_sbi_write(struct exofs_io_state *ios);
exofs_sbi_read(struct exofs_io_state *ios);
exofs_oi_truncate(struct exofs_i_info *oi, u64 new_len);
And when done
exofs_put_io_state(struct exofs_io_state *ios);
* Convert all source files to use this new API
* Convert from bio_alloc to bio_kmalloc
* In io engine we make use of the now fixed osd_req_decode_sense
There are no functional changes or on disk additions after this patch.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2009-11-08 13:54:08 +01:00
|
|
|
|
|
|
|
/* Attributes */
|
|
|
|
unsigned in_attr_len;
|
|
|
|
struct osd_attr *in_attr;
|
|
|
|
unsigned out_attr_len;
|
|
|
|
struct osd_attr *out_attr;
|
|
|
|
|
2010-11-16 19:09:58 +01:00
|
|
|
bool reading;
|
|
|
|
|
exofs: Move all operations to an io_engine
In anticipation for multi-device operations, we separate osd operations
into an abstract I/O API. Currently only one device is used but later
when adding more devices, we will drive all devices in parallel according
to a "data_map" that describes how data is arranged on multiple devices.
The file system level operates, like before, as if there is one object
(inode-number) and an i_size. The io engine will split this to the same
object-number but on multiple device.
At first we introduce Mirror (raid 1) layout. But at the final outcome
we intend to fully implement the pNFS-Objects data-map, including
raid 0,4,5,6 over mirrored devices, over multiple device-groups. And
more. See: http://tools.ietf.org/html/draft-ietf-nfsv4-pnfs-obj-12
* Define an io_state based API for accessing osd storage devices
in an abstract way.
Usage:
First a caller allocates an io state with:
exofs_get_io_state(struct exofs_sb_info *sbi,
struct exofs_io_state** ios);
Then calles one of:
exofs_sbi_create(struct exofs_io_state *ios);
exofs_sbi_remove(struct exofs_io_state *ios);
exofs_sbi_write(struct exofs_io_state *ios);
exofs_sbi_read(struct exofs_io_state *ios);
exofs_oi_truncate(struct exofs_i_info *oi, u64 new_len);
And when done
exofs_put_io_state(struct exofs_io_state *ios);
* Convert all source files to use this new API
* Convert from bio_alloc to bio_kmalloc
* In io engine we make use of the now fixed osd_req_decode_sense
There are no functional changes or on disk additions after this patch.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2009-11-08 13:54:08 +01:00
|
|
|
/* Variable array of size numdevs */
|
|
|
|
unsigned numdevs;
|
|
|
|
struct exofs_per_dev_state {
|
|
|
|
struct osd_request *or;
|
|
|
|
struct bio *bio;
|
exofs: RAID0 support
We now support striping over mirror devices. Including variable sized
stripe_unit.
Some limits:
* stripe_unit must be a multiple of PAGE_SIZE
* stripe_unit * stripe_count is maximum upto 32-bit (4Gb)
Tested RAID0 over mirrors, RAID0 only, mirrors only. All check.
Design notes:
* I'm not using a vectored raid-engine mechanism yet. Following the
pnfs-objects-layout data-map structure, "Mirror" is just a private
case of "group_width" == 1, and RAID0 is a private case of
"Mirrors" == 1. The performance lose of the general case over the
particular special case optimization is totally negligible, also
considering the extra code size.
* In general I added a prepare_stripes() stage that divides the
to-be-io pages to the participating devices, the previous
exofs_ios_write/read, now becomes _write/read_mirrors and a new
write/read upper layer loops on all devices calling
_write/read_mirrors. Effectively the prepare_stripes stage is the all
secret.
Also truncate need fixing to accommodate for striping.
* In a RAID0 arrangement, in a regular usage scenario, if all inode
layouts will start at the same device, the small files fill up the
first device and the later devices stay empty, the farther the device
the emptier it is.
To fix that, each inode will start at a different stripe_unit,
according to it's obj_id modulus number-of-stripe-units. And
will then span all stripe-units in the same incrementing order
wrapping back to the beginning of the device table. We call it
a stripe-units moving window.
Special consideration was taken to keep all devices in a mirror
arrangement identical. So a broken osd-device could just be cloned
from one of the mirrors and no FS scrubbing is needed. (We do that
by rotating stripe-unit at a time and not a single device at a time.)
TODO:
We no longer verify object_length == inode->i_size in exofs_iget.
(since i_size is stripped on multiple objects now).
I should introduce a multiple-device attribute reading, and use
it in exofs_iget.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2010-02-01 12:35:51 +01:00
|
|
|
loff_t offset;
|
|
|
|
unsigned length;
|
|
|
|
unsigned dev;
|
exofs: Move all operations to an io_engine
In anticipation for multi-device operations, we separate osd operations
into an abstract I/O API. Currently only one device is used but later
when adding more devices, we will drive all devices in parallel according
to a "data_map" that describes how data is arranged on multiple devices.
The file system level operates, like before, as if there is one object
(inode-number) and an i_size. The io engine will split this to the same
object-number but on multiple device.
At first we introduce Mirror (raid 1) layout. But at the final outcome
we intend to fully implement the pNFS-Objects data-map, including
raid 0,4,5,6 over mirrored devices, over multiple device-groups. And
more. See: http://tools.ietf.org/html/draft-ietf-nfsv4-pnfs-obj-12
* Define an io_state based API for accessing osd storage devices
in an abstract way.
Usage:
First a caller allocates an io state with:
exofs_get_io_state(struct exofs_sb_info *sbi,
struct exofs_io_state** ios);
Then calles one of:
exofs_sbi_create(struct exofs_io_state *ios);
exofs_sbi_remove(struct exofs_io_state *ios);
exofs_sbi_write(struct exofs_io_state *ios);
exofs_sbi_read(struct exofs_io_state *ios);
exofs_oi_truncate(struct exofs_i_info *oi, u64 new_len);
And when done
exofs_put_io_state(struct exofs_io_state *ios);
* Convert all source files to use this new API
* Convert from bio_alloc to bio_kmalloc
* In io engine we make use of the now fixed osd_req_decode_sense
There are no functional changes or on disk additions after this patch.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2009-11-08 13:54:08 +01:00
|
|
|
} per_dev[];
|
|
|
|
};
|
|
|
|
|
|
|
|
static inline unsigned exofs_io_state_size(unsigned numdevs)
|
|
|
|
{
|
|
|
|
return sizeof(struct exofs_io_state) +
|
|
|
|
sizeof(struct exofs_per_dev_state) * numdevs;
|
|
|
|
}
|
|
|
|
|
2008-10-27 17:27:55 +01:00
|
|
|
/*
|
|
|
|
* our inode flags
|
|
|
|
*/
|
|
|
|
#define OBJ_2BCREATED 0 /* object will be created soon*/
|
|
|
|
#define OBJ_CREATED 1 /* object has been created on the osd*/
|
|
|
|
|
|
|
|
static inline int obj_2bcreated(struct exofs_i_info *oi)
|
|
|
|
{
|
|
|
|
return test_bit(OBJ_2BCREATED, &oi->i_flags);
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline void set_obj_2bcreated(struct exofs_i_info *oi)
|
|
|
|
{
|
|
|
|
set_bit(OBJ_2BCREATED, &oi->i_flags);
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline int obj_created(struct exofs_i_info *oi)
|
|
|
|
{
|
|
|
|
return test_bit(OBJ_CREATED, &oi->i_flags);
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline void set_obj_created(struct exofs_i_info *oi)
|
|
|
|
{
|
|
|
|
set_bit(OBJ_CREATED, &oi->i_flags);
|
|
|
|
}
|
|
|
|
|
|
|
|
int __exofs_wait_obj_created(struct exofs_i_info *oi);
|
|
|
|
static inline int wait_obj_created(struct exofs_i_info *oi)
|
|
|
|
{
|
|
|
|
if (likely(obj_created(oi)))
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
return __exofs_wait_obj_created(oi);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* get to our inode from the vfs inode
|
|
|
|
*/
|
|
|
|
static inline struct exofs_i_info *exofs_i(struct inode *inode)
|
|
|
|
{
|
|
|
|
return container_of(inode, struct exofs_i_info, vfs_inode);
|
|
|
|
}
|
|
|
|
|
2008-10-28 14:38:12 +01:00
|
|
|
/*
|
|
|
|
* Maximum count of links to a file
|
|
|
|
*/
|
|
|
|
#define EXOFS_LINK_MAX 32000
|
|
|
|
|
2008-10-27 17:37:02 +01:00
|
|
|
/*************************
|
|
|
|
* function declarations *
|
|
|
|
*************************/
|
exofs: Move all operations to an io_engine
In anticipation for multi-device operations, we separate osd operations
into an abstract I/O API. Currently only one device is used but later
when adding more devices, we will drive all devices in parallel according
to a "data_map" that describes how data is arranged on multiple devices.
The file system level operates, like before, as if there is one object
(inode-number) and an i_size. The io engine will split this to the same
object-number but on multiple device.
At first we introduce Mirror (raid 1) layout. But at the final outcome
we intend to fully implement the pNFS-Objects data-map, including
raid 0,4,5,6 over mirrored devices, over multiple device-groups. And
more. See: http://tools.ietf.org/html/draft-ietf-nfsv4-pnfs-obj-12
* Define an io_state based API for accessing osd storage devices
in an abstract way.
Usage:
First a caller allocates an io state with:
exofs_get_io_state(struct exofs_sb_info *sbi,
struct exofs_io_state** ios);
Then calles one of:
exofs_sbi_create(struct exofs_io_state *ios);
exofs_sbi_remove(struct exofs_io_state *ios);
exofs_sbi_write(struct exofs_io_state *ios);
exofs_sbi_read(struct exofs_io_state *ios);
exofs_oi_truncate(struct exofs_i_info *oi, u64 new_len);
And when done
exofs_put_io_state(struct exofs_io_state *ios);
* Convert all source files to use this new API
* Convert from bio_alloc to bio_kmalloc
* In io engine we make use of the now fixed osd_req_decode_sense
There are no functional changes or on disk additions after this patch.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2009-11-08 13:54:08 +01:00
|
|
|
|
|
|
|
/* ios.c */
|
exofs: ios: Move to a per inode components & device-table
Exofs raid engine was saving on memory space by having a single layout-info,
single pid, and a single device-table, global to the filesystem. Then passing
a credential and object_id info at the io_state level, private for each
inode. It would also devise this contraption of rotating the device table
view for each inode->ino to spread out the device usage.
This is not compatible with the pnfs-objects standard, demanding that
each inode can have it's own layout-info, device-table, and each object
component it's own pid, oid and creds.
So: Bring exofs raid engine to be usable for generic pnfs-objects use by:
* Define an exofs_comp structure that holds obj_id and credential info.
* Break up exofs_layout struct to an exofs_components structure that holds a
possible array of exofs_comp and the array of devices + the size of the
arrays.
* Add a "comps" parameter to get_io_state() that specifies the ids creds
and device array to use for each IO.
This enables to keep the layout global, but the device-table view, creds
and IDs at the inode level. It only adds two 64bit to each inode, since
some of these members already existed in another form.
* ios raid engine now access layout-info and comps-info through the passed
pointers. Everything is pre-prepared by caller for generic access of
these structures and arrays.
At the exofs Level:
* Super block holds an exofs_components struct that holds the device
array, previously in layout. The devices there are in device-table
order. The device-array is twice bigger and repeats the device-table
twice so now each inode's device array can point to a random device
and have a round-robin view of the table, making it compatible to
previous exofs versions.
* Each inode has an exofs_components struct that is initialized at
load time, with it's own view of the device table IDs and creds.
When doing IO this gets passed to the io_state together with the
layout.
While preforming this change. Bugs where found where credentials with the
wrong IDs where used to access the different SB objects (super.c). As well
as some dead code. It was never noticed because the target we use does not
check the credentials.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2011-08-06 00:06:04 +02:00
|
|
|
int exofs_get_rw_state(struct exofs_layout *layout,
|
|
|
|
struct exofs_components *comps,
|
|
|
|
bool is_reading, u64 offset, u64 length,
|
|
|
|
struct exofs_io_state **ios);
|
|
|
|
int exofs_get_io_state(struct exofs_layout *layout,
|
|
|
|
struct exofs_components *comps,
|
|
|
|
struct exofs_io_state **ios);
|
exofs: Move all operations to an io_engine
In anticipation for multi-device operations, we separate osd operations
into an abstract I/O API. Currently only one device is used but later
when adding more devices, we will drive all devices in parallel according
to a "data_map" that describes how data is arranged on multiple devices.
The file system level operates, like before, as if there is one object
(inode-number) and an i_size. The io engine will split this to the same
object-number but on multiple device.
At first we introduce Mirror (raid 1) layout. But at the final outcome
we intend to fully implement the pNFS-Objects data-map, including
raid 0,4,5,6 over mirrored devices, over multiple device-groups. And
more. See: http://tools.ietf.org/html/draft-ietf-nfsv4-pnfs-obj-12
* Define an io_state based API for accessing osd storage devices
in an abstract way.
Usage:
First a caller allocates an io state with:
exofs_get_io_state(struct exofs_sb_info *sbi,
struct exofs_io_state** ios);
Then calles one of:
exofs_sbi_create(struct exofs_io_state *ios);
exofs_sbi_remove(struct exofs_io_state *ios);
exofs_sbi_write(struct exofs_io_state *ios);
exofs_sbi_read(struct exofs_io_state *ios);
exofs_oi_truncate(struct exofs_i_info *oi, u64 new_len);
And when done
exofs_put_io_state(struct exofs_io_state *ios);
* Convert all source files to use this new API
* Convert from bio_alloc to bio_kmalloc
* In io engine we make use of the now fixed osd_req_decode_sense
There are no functional changes or on disk additions after this patch.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2009-11-08 13:54:08 +01:00
|
|
|
void exofs_put_io_state(struct exofs_io_state *ios);
|
|
|
|
|
|
|
|
int exofs_check_io(struct exofs_io_state *ios, u64 *resid);
|
|
|
|
|
|
|
|
int exofs_sbi_create(struct exofs_io_state *ios);
|
|
|
|
int exofs_sbi_remove(struct exofs_io_state *ios);
|
|
|
|
int exofs_sbi_write(struct exofs_io_state *ios);
|
|
|
|
int exofs_sbi_read(struct exofs_io_state *ios);
|
exofs: ios: Move to a per inode components & device-table
Exofs raid engine was saving on memory space by having a single layout-info,
single pid, and a single device-table, global to the filesystem. Then passing
a credential and object_id info at the io_state level, private for each
inode. It would also devise this contraption of rotating the device table
view for each inode->ino to spread out the device usage.
This is not compatible with the pnfs-objects standard, demanding that
each inode can have it's own layout-info, device-table, and each object
component it's own pid, oid and creds.
So: Bring exofs raid engine to be usable for generic pnfs-objects use by:
* Define an exofs_comp structure that holds obj_id and credential info.
* Break up exofs_layout struct to an exofs_components structure that holds a
possible array of exofs_comp and the array of devices + the size of the
arrays.
* Add a "comps" parameter to get_io_state() that specifies the ids creds
and device array to use for each IO.
This enables to keep the layout global, but the device-table view, creds
and IDs at the inode level. It only adds two 64bit to each inode, since
some of these members already existed in another form.
* ios raid engine now access layout-info and comps-info through the passed
pointers. Everything is pre-prepared by caller for generic access of
these structures and arrays.
At the exofs Level:
* Super block holds an exofs_components struct that holds the device
array, previously in layout. The devices there are in device-table
order. The device-array is twice bigger and repeats the device-table
twice so now each inode's device array can point to a random device
and have a round-robin view of the table, making it compatible to
previous exofs versions.
* Each inode has an exofs_components struct that is initialized at
load time, with it's own view of the device table IDs and creds.
When doing IO this gets passed to the io_state together with the
layout.
While preforming this change. Bugs where found where credentials with the
wrong IDs where used to access the different SB objects (super.c). As well
as some dead code. It was never noticed because the target we use does not
check the credentials.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2011-08-06 00:06:04 +02:00
|
|
|
int exofs_truncate(struct exofs_layout *layout, struct exofs_components *comps,
|
|
|
|
u64 size);
|
exofs: Move all operations to an io_engine
In anticipation for multi-device operations, we separate osd operations
into an abstract I/O API. Currently only one device is used but later
when adding more devices, we will drive all devices in parallel according
to a "data_map" that describes how data is arranged on multiple devices.
The file system level operates, like before, as if there is one object
(inode-number) and an i_size. The io engine will split this to the same
object-number but on multiple device.
At first we introduce Mirror (raid 1) layout. But at the final outcome
we intend to fully implement the pNFS-Objects data-map, including
raid 0,4,5,6 over mirrored devices, over multiple device-groups. And
more. See: http://tools.ietf.org/html/draft-ietf-nfsv4-pnfs-obj-12
* Define an io_state based API for accessing osd storage devices
in an abstract way.
Usage:
First a caller allocates an io state with:
exofs_get_io_state(struct exofs_sb_info *sbi,
struct exofs_io_state** ios);
Then calles one of:
exofs_sbi_create(struct exofs_io_state *ios);
exofs_sbi_remove(struct exofs_io_state *ios);
exofs_sbi_write(struct exofs_io_state *ios);
exofs_sbi_read(struct exofs_io_state *ios);
exofs_oi_truncate(struct exofs_i_info *oi, u64 new_len);
And when done
exofs_put_io_state(struct exofs_io_state *ios);
* Convert all source files to use this new API
* Convert from bio_alloc to bio_kmalloc
* In io engine we make use of the now fixed osd_req_decode_sense
There are no functional changes or on disk additions after this patch.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2009-11-08 13:54:08 +01:00
|
|
|
|
|
|
|
int extract_attr_from_ios(struct exofs_io_state *ios, struct osd_attr *attr);
|
2011-05-16 14:26:47 +02:00
|
|
|
extern const struct osd_attr g_attr_logical_length;
|
exofs: Move all operations to an io_engine
In anticipation for multi-device operations, we separate osd operations
into an abstract I/O API. Currently only one device is used but later
when adding more devices, we will drive all devices in parallel according
to a "data_map" that describes how data is arranged on multiple devices.
The file system level operates, like before, as if there is one object
(inode-number) and an i_size. The io engine will split this to the same
object-number but on multiple device.
At first we introduce Mirror (raid 1) layout. But at the final outcome
we intend to fully implement the pNFS-Objects data-map, including
raid 0,4,5,6 over mirrored devices, over multiple device-groups. And
more. See: http://tools.ietf.org/html/draft-ietf-nfsv4-pnfs-obj-12
* Define an io_state based API for accessing osd storage devices
in an abstract way.
Usage:
First a caller allocates an io state with:
exofs_get_io_state(struct exofs_sb_info *sbi,
struct exofs_io_state** ios);
Then calles one of:
exofs_sbi_create(struct exofs_io_state *ios);
exofs_sbi_remove(struct exofs_io_state *ios);
exofs_sbi_write(struct exofs_io_state *ios);
exofs_sbi_read(struct exofs_io_state *ios);
exofs_oi_truncate(struct exofs_i_info *oi, u64 new_len);
And when done
exofs_put_io_state(struct exofs_io_state *ios);
* Convert all source files to use this new API
* Convert from bio_alloc to bio_kmalloc
* In io engine we make use of the now fixed osd_req_decode_sense
There are no functional changes or on disk additions after this patch.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2009-11-08 13:54:08 +01:00
|
|
|
|
2008-10-27 17:37:02 +01:00
|
|
|
/* inode.c */
|
2010-10-07 20:28:18 +02:00
|
|
|
unsigned exofs_max_io_pages(struct exofs_layout *layout,
|
|
|
|
unsigned expected_pages);
|
2008-10-27 17:37:02 +01:00
|
|
|
int exofs_setattr(struct dentry *, struct iattr *);
|
2008-10-27 18:31:34 +01:00
|
|
|
int exofs_write_begin(struct file *file, struct address_space *mapping,
|
|
|
|
loff_t pos, unsigned len, unsigned flags,
|
|
|
|
struct page **pagep, void **fsdata);
|
2008-10-28 14:38:12 +01:00
|
|
|
extern struct inode *exofs_iget(struct super_block *, unsigned long);
|
|
|
|
struct inode *exofs_new_inode(struct inode *, int);
|
2010-03-05 09:21:37 +01:00
|
|
|
extern int exofs_write_inode(struct inode *, struct writeback_control *wbc);
|
2010-06-07 17:42:26 +02:00
|
|
|
extern void exofs_evict_inode(struct inode *);
|
2008-10-28 14:38:12 +01:00
|
|
|
|
|
|
|
/* dir.c: */
|
|
|
|
int exofs_add_link(struct dentry *, struct inode *);
|
|
|
|
ino_t exofs_inode_by_name(struct inode *, struct dentry *);
|
|
|
|
int exofs_delete_entry(struct exofs_dir_entry *, struct page *);
|
|
|
|
int exofs_make_empty(struct inode *, struct inode *);
|
|
|
|
struct exofs_dir_entry *exofs_find_entry(struct inode *, struct dentry *,
|
|
|
|
struct page **);
|
|
|
|
int exofs_empty_dir(struct inode *);
|
|
|
|
struct exofs_dir_entry *exofs_dotdot(struct inode *, struct page **);
|
2009-03-22 11:47:26 +01:00
|
|
|
ino_t exofs_parent_ino(struct dentry *child);
|
2008-10-28 14:38:12 +01:00
|
|
|
int exofs_set_link(struct inode *, struct exofs_dir_entry *, struct page *,
|
|
|
|
struct inode *);
|
2008-10-27 17:37:02 +01:00
|
|
|
|
2009-06-14 15:52:10 +02:00
|
|
|
/* super.c */
|
2011-05-16 14:26:47 +02:00
|
|
|
void exofs_make_credential(u8 cred_a[OSD_CAP_LEN],
|
|
|
|
const struct osd_obj_id *obj);
|
2011-02-03 16:53:25 +01:00
|
|
|
int exofs_sbi_write_stats(struct exofs_sb_info *sbi);
|
2009-06-14 15:52:10 +02:00
|
|
|
|
2008-10-27 17:37:02 +01:00
|
|
|
/*********************
|
|
|
|
* operation vectors *
|
|
|
|
*********************/
|
2008-10-28 14:38:12 +01:00
|
|
|
/* dir.c: */
|
|
|
|
extern const struct file_operations exofs_dir_operations;
|
|
|
|
|
2008-10-27 17:37:02 +01:00
|
|
|
/* file.c */
|
|
|
|
extern const struct inode_operations exofs_file_inode_operations;
|
|
|
|
extern const struct file_operations exofs_file_operations;
|
|
|
|
|
2008-10-27 18:31:34 +01:00
|
|
|
/* inode.c */
|
|
|
|
extern const struct address_space_operations exofs_aops;
|
|
|
|
|
2008-10-28 14:38:12 +01:00
|
|
|
/* namei.c */
|
|
|
|
extern const struct inode_operations exofs_dir_inode_operations;
|
|
|
|
extern const struct inode_operations exofs_special_inode_operations;
|
|
|
|
|
2008-10-27 18:04:34 +01:00
|
|
|
/* symlink.c */
|
|
|
|
extern const struct inode_operations exofs_symlink_inode_operations;
|
|
|
|
extern const struct inode_operations exofs_fast_symlink_inode_operations;
|
|
|
|
|
exofs: ios: Move to a per inode components & device-table
Exofs raid engine was saving on memory space by having a single layout-info,
single pid, and a single device-table, global to the filesystem. Then passing
a credential and object_id info at the io_state level, private for each
inode. It would also devise this contraption of rotating the device table
view for each inode->ino to spread out the device usage.
This is not compatible with the pnfs-objects standard, demanding that
each inode can have it's own layout-info, device-table, and each object
component it's own pid, oid and creds.
So: Bring exofs raid engine to be usable for generic pnfs-objects use by:
* Define an exofs_comp structure that holds obj_id and credential info.
* Break up exofs_layout struct to an exofs_components structure that holds a
possible array of exofs_comp and the array of devices + the size of the
arrays.
* Add a "comps" parameter to get_io_state() that specifies the ids creds
and device array to use for each IO.
This enables to keep the layout global, but the device-table view, creds
and IDs at the inode level. It only adds two 64bit to each inode, since
some of these members already existed in another form.
* ios raid engine now access layout-info and comps-info through the passed
pointers. Everything is pre-prepared by caller for generic access of
these structures and arrays.
At the exofs Level:
* Super block holds an exofs_components struct that holds the device
array, previously in layout. The devices there are in device-table
order. The device-array is twice bigger and repeats the device-table
twice so now each inode's device array can point to a random device
and have a round-robin view of the table, making it compatible to
previous exofs versions.
* Each inode has an exofs_components struct that is initialized at
load time, with it's own view of the device table IDs and creds.
When doing IO this gets passed to the io_state together with the
layout.
While preforming this change. Bugs where found where credentials with the
wrong IDs where used to access the different SB objects (super.c). As well
as some dead code. It was never noticed because the target we use does not
check the credentials.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
2011-08-06 00:06:04 +02:00
|
|
|
/* exofs_init_comps will initialize an exofs_components device array
|
|
|
|
* pointing to a single exofs_comp struct, and a round-robin view
|
|
|
|
* of the device table.
|
|
|
|
* The first device of each inode is the [inode->ino % num_devices]
|
|
|
|
* and the rest of the devices sequentially following where the
|
|
|
|
* first device is after the last device.
|
|
|
|
* It is assumed that the global device array at @sbi is twice
|
|
|
|
* bigger and that the device table repeats twice.
|
|
|
|
* See: exofs_read_lookup_dev_table()
|
|
|
|
*/
|
|
|
|
static inline void exofs_init_comps(struct exofs_components *comps,
|
|
|
|
struct exofs_comp *one_comp,
|
|
|
|
struct exofs_sb_info *sbi, osd_id oid)
|
|
|
|
{
|
|
|
|
unsigned dev_mod = (unsigned)oid, first_dev;
|
|
|
|
|
|
|
|
one_comp->obj.partition = sbi->one_comp.obj.partition;
|
|
|
|
one_comp->obj.id = oid;
|
|
|
|
exofs_make_credential(one_comp->cred, &one_comp->obj);
|
|
|
|
|
|
|
|
comps->numdevs = sbi->comps.numdevs;
|
|
|
|
comps->single_comp = EC_SINGLE_COMP;
|
|
|
|
comps->comps = one_comp;
|
|
|
|
|
|
|
|
/* Round robin device view of the table */
|
|
|
|
first_dev = (dev_mod * sbi->layout.mirrors_p1) % sbi->comps.numdevs;
|
|
|
|
comps->ods = sbi->comps.ods + first_dev;
|
|
|
|
}
|
|
|
|
|
2008-10-27 17:27:55 +01:00
|
|
|
#endif
|