OpenE2K/linux
6
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Btrfs: send, skip backreference walking for extents with many references

Browse Source 
Backreference walking, which is used by send to figure if it can issue
clone operations instead of write operations, can be very slow and use
too much memory when extents have many references. This change simply
skips backreference walking when an extent has more than 64 references,
in which case we fallback to a write operation instead of a clone
operation. This limit is conservative and in practice I observed no
signicant slowdown with up to 100 references and still low memory usage
up to that limit.

This is a temporary workaround until there are speedups in the backref
walking code, and as such it does not attempt to add extra interfaces or
knobs to tweak the threshold.

Reported-by: Atemu <atemu.main@gmail.com>
Link: https://lore.kernel.org/linux-btrfs/CAE4GHgkvqVADtS4AzcQJxo0Q1jKQgKaW3JGp3SGdoinVo=C9eQ@mail.gmail.com/T/#me55dc0987f9cc2acaa54372ce0492c65782be3fa
CC: stable@vger.kernel.org # 4.4+
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This commit is contained in:
Filipe Manana 2019-10-30 12:23:01 +00:00 committed by David Sterba
parent 11f2069c11
commit fd0ddbe250
1 changed files with 24 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
fs/btrfs/send.c
View File

@ -24,6 +24,14 @@
#include "transaction.h" #include "transaction.h"
#include "compression.h" #include "compression.h"
/*
* Maximum number of references an extent can have in order for us to attempt to
* issue clone operations instead of write operations. This currently exists to
* avoid hitting limitations of the backreference walking code (taking a lot of
* time and using too much memory for extents with large number of references).
*/
#define SEND_MAX_EXTENT_REFS 64
/* /*
* A fs_path is a helper to dynamically build path names with unknown size. * A fs_path is a helper to dynamically build path names with unknown size.
* It reallocates the internal buffer on demand. * It reallocates the internal buffer on demand.
@ -1310,6 +1318,7 @@ static int find_extent_clone(struct send_ctx *sctx,
struct clone_root *cur_clone_root; struct clone_root *cur_clone_root;
struct btrfs_key found_key; struct btrfs_key found_key;
struct btrfs_path *tmp_path; struct btrfs_path *tmp_path;
struct btrfs_extent_item *ei;
int compressed; int compressed;
u32 i; u32 i;
@ -1357,7 +1366,6 @@ static int find_extent_clone(struct send_ctx *sctx,
ret = extent_from_logical(fs_info, disk_byte, tmp_path, ret = extent_from_logical(fs_info, disk_byte, tmp_path,
&found_key, &flags); &found_key, &flags);
up_read(&fs_info->commit_root_sem); up_read(&fs_info->commit_root_sem);
btrfs_release_path(tmp_path);
if (ret < 0) if (ret < 0)
goto out; goto out;
@ -1366,6 +1374,21 @@ static int find_extent_clone(struct send_ctx *sctx,
goto out; goto out;
} }
ei = btrfs_item_ptr(tmp_path->nodes[0], tmp_path->slots[0],
struct btrfs_extent_item);
/*
* Backreference walking (iterate_extent_inodes() below) is currently
* too expensive when an extent has a large number of references, both
* in time spent and used memory. So for now just fallback to write
* operations instead of clone operations when an extent has more than
* a certain amount of references.
*/
if (btrfs_extent_refs(tmp_path->nodes[0], ei) > SEND_MAX_EXTENT_REFS) {
ret = -ENOENT;
goto out;
}
btrfs_release_path(tmp_path);
/* /*
* Setup the clone roots. * Setup the clone roots.
*/ */