OpenE2K
/
linux
6
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
linux/fs/xfs
History
Dave Chinner 23647e9878 xfs: log vector rounding leaks log space 
commit 110dc24ad2 upstream.

The addition of direct formatting of log items into the CIL
linear buffer added alignment restrictions that the start of each
vector needed to be 64 bit aligned. Hence padding was added in
xlog_finish_iovec() to round up the vector length to ensure the next
vector started with the correct alignment.

This adds a small number of bytes to the size of
the linear buffer that is otherwise unused. The issue is that we
then use the linear buffer size to determine the log space used by
the log item, and this includes the unused space. Hence when we
account for space used by the log item, it's more than is actually
written into the iclogs, and hence we slowly leak this space.

This results on log hangs when reserving space, with threads getting
stuck with these stack traces:

Call Trace:
[<ffffffff81d15989>] schedule+0x29/0x70
[<ffffffff8150d3a2>] xlog_grant_head_wait+0xa2/0x1a0
[<ffffffff8150d55d>] xlog_grant_head_check+0xbd/0x140
[<ffffffff8150ee33>] xfs_log_reserve+0x103/0x220
[<ffffffff814b7f05>] xfs_trans_reserve+0x2f5/0x310
.....

The 4 bytes is significant. Brain Foster did all the hard work in
tracking down a reproducable leak to inode chunk allocation (it went
away with the ikeep mount option). His rough numbers were that
creating 50,000 inodes leaked 11 log blocks. This turns out to be
roughly 800 inode chunks or 1600 inode cluster buffers. That
works out at roughly 4 bytes per cluster buffer logged, and at that
I started looking for a 4 byte leak in the buffer logging code.

What I found was that a struct xfs_buf_log_format structure for an
inode cluster buffer is 28 bytes in length. This gets rounded up to
32 bytes, but the vector length remains 28 bytes. Hence the CIL
ticket reservation is decremented by 32 bytes (via lv->lv_buf_len)
for that vector rather than 28 bytes which are written into the log.

The fix for this problem is to separately track the bytes used by
the log vectors in the item and use that instead of the buffer
length when accounting for the log space that will be used by the
formatted log item.

Again, thanks to Brian Foster for doing all the hard work and long
hours to isolate this leak and make finding the bug relatively
simple.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
Cc: Bill <billstuff2001@sbcglobal.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
 		2014-08-14 09:38:26 +08:00
..
Kconfig xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
Makefile xfs: abstract the differences in dir2/dir3 via an ops vector 2013-10-30 13:37:38 -05:00
kmem.c xfs: simplify kmem_{zone_}zalloc 2013-11-06 16:31:27 -06:00
kmem.h xfs: simplify kmem_{zone_}zalloc 2013-11-06 16:31:27 -06:00
mrlock.h xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
time.h xfs: remove subdirectories 2011-08-12 16:21:35 -05:00
uuid.c xfs: remove subdirectories 2011-08-12 16:21:35 -05:00
uuid.h xfs: add CRC infrastructure 2012-11-19 20:11:24 -06:00
xfs.h xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
xfs_acl.c xfs: use generic posix ACL infrastructure 2014-01-25 23:58:21 -05:00
xfs_acl.h xfs: use generic posix ACL infrastructure 2014-01-25 23:58:21 -05:00
xfs_ag.h xfs: create a shared header file for format-related information 2013-10-23 14:11:30 -05:00
xfs_alloc.c xfs: add tracepoints to AGF/AGI read operations 2013-11-06 12:42:52 -06:00
xfs_alloc.h xfs: create a shared header file for format-related information 2013-10-23 14:11:30 -05:00
xfs_alloc_btree.c xfs: fix static and extern sparse warnings 2013-10-30 13:59:56 -05:00
xfs_alloc_btree.h xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_aops.c Merge branch 'for-3.14/core' of git://git.kernel.dk/linux-block 2014-01-30 11:19:05 -08:00
xfs_aops.h direct-io: Implement generic deferred AIO completions 2013-09-04 09:23:46 -04:00
xfs_attr.c xfs: use xfs_ilock_attr_map_shared in xfs_attr_get 2013-12-18 16:07:09 -06:00
xfs_attr.h xfs: kill xfs_vnodeops.[ch] 2013-08-12 16:53:39 -05:00
xfs_attr_inactive.c xfs: vectorise encoding/decoding directory headers 2013-10-30 13:47:22 -05:00
xfs_attr_leaf.c xfs: fix static and extern sparse warnings 2013-10-30 13:59:56 -05:00
xfs_attr_leaf.h xfs: unify directory/attribute format definitions 2013-10-23 14:21:40 -05:00
xfs_attr_list.c xfs: use xfs_ilock_attr_map_shared in xfs_attr_list_int 2013-12-18 16:08:04 -06:00
xfs_attr_remote.c xfs: fix off-by-one error in xfs_attr3_rmt_verify 2014-01-10 12:38:41 -06:00
xfs_attr_remote.h xfs: unify directory/attribute format definitions 2013-10-23 14:21:40 -05:00
xfs_attr_sf.h xfs: convert attr to use unsigned names 2010-01-20 10:47:48 +11:00
xfs_bit.c xfs: fix static and extern sparse warnings 2013-10-30 13:59:56 -05:00
xfs_bit.h [XFS] Remove macro-to-function indirections in the mask code 2009-01-09 15:53:54 +11:00
xfs_bmap.c Merge branch 'xfs-extent-list-locking-fixes' into for-next 2014-01-09 16:03:18 -06:00
xfs_bmap.h xfs: remove __KERNEL__ from debug code 2013-08-12 16:58:37 -05:00
xfs_bmap_btree.c xfs: fix static and extern sparse warnings 2013-10-30 13:59:56 -05:00
xfs_bmap_btree.h xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_bmap_util.c xfs: Calling destroy_work_on_stack() to pair with INIT_WORK_ONSTACK() 2014-01-10 12:39:38 -06:00
xfs_bmap_util.h xfs: fold xfs_change_file_space into xfs_ioc_space 2013-10-21 16:57:03 -05:00
xfs_btree.c xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_btree.h xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_buf.c Merge branch 'for-3.14/core' of git://git.kernel.dk/linux-block 2014-01-30 11:19:05 -08:00
xfs_buf.h xfs: allow logical-sector sized O_DIRECT 2014-01-24 11:55:42 -06:00
xfs_buf_item.c Merge branch 'xfs-for-linus-v3.13-rc5' into for-next 2013-12-18 10:36:58 -06:00
xfs_buf_item.h xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_cksum.h xfs: add CRC infrastructure 2012-11-19 20:11:24 -06:00
xfs_da_btree.c xfs: fix directory hash ordering bug 2014-04-26 17:19:05 -07:00
xfs_da_btree.h xfs: abstract the differences in dir2/dir3 via an ops vector 2013-10-30 13:37:38 -05:00
xfs_da_format.c xfs: fix static and extern sparse warnings 2013-10-30 13:59:56 -05:00
xfs_da_format.h xfs: convert directory vector functions to constants 2013-10-30 13:48:41 -05:00
xfs_dinode.h xfs: di_flushiter considered harmful 2013-07-25 10:41:42 -05:00
xfs_dir2.c xfs: convert directory vector functions to constants 2013-10-30 13:49:18 -05:00
xfs_dir2.h xfs: convert directory vector functions to constants 2013-10-30 13:49:18 -05:00
xfs_dir2_block.c xfs: convert directory vector functions to constants 2013-10-30 13:49:18 -05:00
xfs_dir2_data.c xfs: validity check the directory block leaf entry count 2013-10-30 13:57:14 -05:00
xfs_dir2_leaf.c xfs: convert directory vector functions to constants 2013-10-30 13:49:18 -05:00
xfs_dir2_node.c xfs: fix memory leak in xfs_dir2_node_removename 2013-12-17 09:15:12 -06:00
xfs_dir2_priv.h xfs: vectorise encoding/decoding directory headers 2013-10-30 13:47:22 -05:00
xfs_dir2_readdir.c xfs: reinstate the ilock in xfs_readdir 2013-12-18 15:52:36 -06:00
xfs_dir2_sf.c xfs: xfs_dir2_block_to_sf temp buffer allocation fails 2013-12-11 14:59:20 -06:00
xfs_discard.c xfs: don't perform discard if the given range length is less than block size 2013-12-10 10:00:33 -06:00
xfs_discard.h xfs: remove subdirectories 2011-08-12 16:21:35 -05:00
xfs_dquot.c xfs: use xfs_ilock_data_map_shared in xfs_qm_dqtobp 2013-12-18 16:04:18 -06:00
xfs_dquot.h xfs: create a shared header file for format-related information 2013-10-23 14:11:30 -05:00
xfs_dquot_buf.c xfs: fix static and extern sparse warnings 2013-10-30 13:59:56 -05:00
xfs_dquot_item.c xfs: remove the quotaoff log format from the quotaoff log item 2013-12-13 11:34:08 +11:00
xfs_dquot_item.h xfs: remove the quotaoff log format from the quotaoff log item 2013-12-13 11:34:08 +11:00
xfs_error.c xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_error.h xfs: kill support/debug.[ch] 2011-03-07 10:09:35 +11:00
xfs_export.c xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_export.h xfs: remove subdirectories 2011-08-12 16:21:35 -05:00
xfs_extent_busy.c xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_extent_busy.h xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_extfree_item.c xfs: format log items write directly into the linear CIL buffer 2013-12-13 11:34:02 +11:00
xfs_extfree_item.h xfs: split out EFI/EFD log item format definition 2013-08-12 16:07:13 -05:00
xfs_file.c fix O_SYNC|O_APPEND syncing the wrong range on write() 2014-02-09 15:18:09 -05:00
xfs_filestream.c xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_filestream.h xfs: xfs_filestreams.h doesn't need __KERNEL__ 2013-08-12 17:00:11 -05:00
xfs_format.h xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_fs.h xfs: add the inode directory type support to XFS_IOC_FSGEOM 2013-10-08 14:28:09 -05:00
xfs_fsops.c xfs: growfs overruns AGFL buffer on V4 filesystems 2013-12-10 10:04:27 -06:00
xfs_fsops.h xfs: ensure log covering transactions are synchronous 2011-01-11 20:28:17 -06:00
xfs_globals.c xfs: add background scanning to clear eofblocks inodes 2012-11-08 15:34:59 -06:00
xfs_ialloc.c xfs: use xfs_icluster_size_fsb in xfs_imap 2013-12-13 15:51:49 +11:00
xfs_ialloc.h xfs: introduce a common helper xfs_icluster_size_fsb 2013-12-13 15:51:48 +11:00
xfs_ialloc_btree.c xfs: fix static and extern sparse warnings 2013-10-30 13:59:56 -05:00
xfs_ialloc_btree.h xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_icache.c xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_icache.h xfs: update #2 for v3.12-rc1 2013-09-12 16:13:41 -07:00
xfs_icreate_item.c xfs: format log items write directly into the linear CIL buffer 2013-12-13 11:34:02 +11:00
xfs_icreate_item.h xfs: separate icreate log format definitions from xfs_icreate_item.h 2013-08-12 16:10:35 -05:00
xfs_inode.c Merge branch 'xfs-extent-list-locking-fixes' into for-next 2014-01-09 16:03:18 -06:00
xfs_inode.h xfs: add xfs_ilock_attr_map_shared 2013-12-18 15:48:44 -06:00
xfs_inode_buf.c xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_inode_buf.h xfs: create a shared header file for format-related information 2013-10-23 14:11:30 -05:00
xfs_inode_fork.c Merge branch 'xfs-extent-list-locking-fixes' into for-next 2014-01-09 16:03:18 -06:00
xfs_inode_fork.h xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_inode_item.c xfs: remove the inode log format from the inode log item 2013-12-13 11:34:05 +11:00
xfs_inode_item.h xfs: remove the inode log format from the inode log item 2013-12-13 11:34:05 +11:00
xfs_inum.h xfs: move xfsagino_t to xfs_types.h 2012-05-14 16:20:54 -05:00
xfs_ioctl.c fs,userns: Change inode_capable to capable_wrt_inode_uidgid 2014-06-16 13:40:32 -07:00
xfs_ioctl.h xfs: consolidate extent swap code 2013-08-12 16:56:06 -05:00
xfs_ioctl32.c xfs: underflow bug in xfs_attrlist_by_handle() 2013-12-10 09:59:37 -06:00
xfs_ioctl32.h xfs: remove subdirectories 2011-08-12 16:21:35 -05:00
xfs_iomap.c xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_iomap.h xfs: get rid of count from xfs_iomap_write_allocate() 2013-10-01 15:42:34 -05:00
xfs_iops.c xfs: ensure correct timestamp updates from truncate 2014-02-10 10:35:22 +11:00
xfs_iops.h xfs: use generic posix ACL infrastructure 2014-01-25 23:58:21 -05:00
xfs_itable.c xfs: use xfs_icluster_size_fsb in xfs_bulkstat 2013-12-13 15:51:48 +11:00
xfs_itable.h xfs: remove block number from inode lookup code 2010-06-24 11:35:17 +10:00
xfs_linux.h xfs: remove two unused macro definitions in xfs_linux.h 2013-08-20 15:30:23 -05:00
xfs_log.c xfs: trace AIL manipulations 2013-11-06 12:41:51 -06:00
xfs_log.h xfs: log vector rounding leaks log space 2014-08-14 09:38:26 +08:00
xfs_log_cil.c xfs: log vector rounding leaks log space 2014-08-14 09:38:26 +08:00
xfs_log_format.h xfs: create a shared header file for format-related information 2013-10-23 14:11:30 -05:00
xfs_log_priv.h xfs: decouple log and transaction headers 2013-10-23 16:17:44 -05:00
xfs_log_recover.c Merge branch 'xfs-for-linus-v3.13-rc5' into for-next 2013-12-18 10:36:58 -06:00
xfs_log_recover.h xfs: Clean up XFS_BLI_* flag namespace 2010-05-24 10:33:39 -05:00
xfs_log_rlimit.c xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_message.c xfs: decouple log and transaction headers 2013-10-23 16:17:44 -05:00
xfs_message.h xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
xfs_mount.c xfs: xfs_readsb needs to check for magic numbers 2014-07-06 18:57:28 -07:00
xfs_mount.h xfs: increase inode cluster size for v5 filesystems 2013-11-18 09:29:36 -06:00
xfs_mru_cache.c xfs: convert to alloc_workqueue() 2011-02-01 11:42:43 +01:00
xfs_mru_cache.h xfs: Kill filestreams cache flush 2010-01-15 15:34:22 -06:00
xfs_qm.c xfs: use xfs_ilock_data_map_shared in xfs_qm_dqiterate 2013-12-18 16:06:38 -06:00
xfs_qm.h xfs: integrate xfs_quota_priv header file to xfs_qm 2013-12-06 14:16:33 -06:00
xfs_qm_bhv.c xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_qm_syscalls.c xfs: make quota metadata truncation behavior consistent to user space 2013-12-06 14:06:15 -06:00
xfs_quota.h xfs: split dquot buffer operations out 2013-10-23 14:28:35 -05:00
xfs_quota_defs.h xfs: split dquot buffer operations out 2013-10-23 14:28:35 -05:00
xfs_quotaops.c xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_rtalloc.c xfs: split xfs_rtalloc.c for userspace sanity 2013-10-23 17:16:32 -05:00
xfs_rtalloc.h xfs: split xfs_rtalloc.c for userspace sanity 2013-10-23 17:16:32 -05:00
xfs_rtbitmap.c xfs: fix static and extern sparse warnings 2013-10-30 13:59:56 -05:00
xfs_sb.c xfs: limit superblock corruption errors to actual corruption 2014-02-19 15:39:35 +11:00
xfs_sb.h xfs: create a shared header file for format-related information 2013-10-23 14:11:30 -05:00
xfs_shared.h xfs: create a shared header file for format-related information 2013-10-23 14:11:30 -05:00
xfs_stats.c xfs: use common code for quota statistics 2012-03-14 11:09:06 -05:00
xfs_stats.h xfs: use common code for quota statistics 2012-03-14 11:09:06 -05:00
xfs_super.c Revert "writeback: do not sync data dirtied after sync start" 2014-02-22 02:02:28 +01:00
xfs_super.h xfs: xfs_sync_data is redundant. 2012-10-17 12:01:25 -05:00
xfs_symlink.c xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_symlink.h xfs: push down inactive transaction mgmt for remote symlinks 2013-10-08 14:53:02 -05:00
xfs_symlink_remote.c xfs: decouple log and transaction headers 2013-10-23 16:17:44 -05:00
xfs_sysctl.c xfs: Convert use of typedef ctl_table to struct ctl_table 2013-06-17 17:42:25 -05:00
xfs_sysctl.h xfs: add background scanning to clear eofblocks inodes 2012-11-08 15:34:59 -06:00
xfs_trace.c xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_trace.h xfs: add tracepoints to AGF/AGI read operations 2013-11-06 12:42:52 -06:00
xfs_trans.c xfs: decouple inode and bmap btree header files 2013-10-23 16:28:49 -05:00
xfs_trans.h xfs: format log items write directly into the linear CIL buffer 2013-12-13 11:34:02 +11:00
xfs_trans_ail.c xfs: trace AIL manipulations 2013-11-06 12:41:51 -06:00
xfs_trans_buf.c xfs: remove xfsbdstrat error 2013-12-17 09:28:43 -06:00
xfs_trans_dquot.c xfs: fix the comment explaining xfs_trans_dqlockedjoin 2013-12-04 14:26:57 -06:00
xfs_trans_extfree.c xfs: decouple log and transaction headers 2013-10-23 16:17:44 -05:00
xfs_trans_inode.c xfs: open code inc_inode_iversion when logging an inode 2013-11-18 09:42:08 -06:00
xfs_trans_priv.h xfs: decouple log and transaction headers 2013-10-23 16:17:44 -05:00
xfs_trans_resv.c xfs: get rid of XFS_IALLOC_BLOCKS macros 2013-12-13 15:51:48 +11:00
xfs_trans_resv.h xfs: Get rid of all XFS_XXX_LOG_RES() macro 2013-08-12 17:48:08 -05:00
xfs_trans_space.h xfs: get rid of XFS_IALLOC_BLOCKS macros 2013-12-13 15:51:48 +11:00
xfs_types.h xfs: Add read-only support for dirent filetype field 2013-08-22 08:40:24 -05:00
xfs_vnode.h xfs: remove unused FI_ flags 2013-12-04 14:11:05 -06:00
xfs_xattr.c xfs: use generic posix ACL infrastructure 2014-01-25 23:58:21 -05:00