Jonathan Wakely ebf6175464 libstdc++: Fix filesystem::remove_all races [PR104161]
This fixes the remaining filesystem::remove_all race condition by using
POSIX openat to recurse into sub-directories and using POSIX unlinkat to
remove files. This avoids the remaining race where the directory being
removed is replaced with a symlink after the directory has been opened,
so that the filesystem::remove("subdir/file") resolves to "target/file"
instead, because "subdir" has been removed and replaced with a symlink.
The previous patch only fixed the case where the directory was replaced
with a symlink before we tried to open it, but it still used the full
(potentially compromised) path as an argument to filesystem::remove.

The first part of the fix is to use openat when recursing into a
sub-directory with recursive_directory_iterator. This means that opening
"dir/subdir" uses the file descriptor for "dir", and so is sure to open
"dir/subdir" and not "symlink/subdir". (The previous patch to use
O_NOFOLLOW already ensured we won't open "dir/symlink/" here.)

The second part of the fix is to use unlinkat for the remove_all
operation. Previously we used a directory_iterator to get the name of
each file in a directory and then used filesystem::remove(iter->path())
on that name. This meant that any checks (e.g. O_NOFOLLOW) done by the
iterator could be invalidated before the remove operation on that
pathname. The directory iterator contains an open DIR stream, which we
can use to obtain a file descriptor to pass to unlinkat. This ensures
that the file being deleted really is contained within the directory
we're iterating over, rather than using a pathname that could resolve to
some other file.

The filesystem::remove_all function previously used a (non-recursive)
filesystem::directory_iterator for each directory, and called itself
recursively for sub-directories. The new implementation uses a single
filesystem::recursive_directory_iterator object, and calls a new __erase
member function on that iterator. That new __erase member function does
the actual work of removing a file (or a directory after its contents
have been iterated over and removed) using unlinkat. That means we don't
need to expose the DIR stream or its file descriptor to the remove_all
function, it's still encapuslated by the iterator class.

It would be possible to add a __rewind member to directory iterators
too, to call rewinddir after each modification to the directory. That
would make it more likely for filesystem::remove_all to successfully
remove everything even if files are being written to the directory tree
while removing it. It's unclear if that is actually prefereable, or if
it's better to fail and report an error at the first opportunity.

The necessary APIs (openat, unlinkat, fdopendir, dirfd) are defined in
POSIX.1-2008, and in Glibc since 2.10. But if the target doesn't provide
them, the original code (with race conditions) is still used.

This also reduces the number of small memory allocations needed for
std::filesystem::remove_all, because we do not store the full path to
every directory entry that is iterated over. The new filename_only
option means we only store the filename in the directory entry, as that
is all we need in order to use openat or unlinkat.

Finally, rather than duplicating everything for the Filesystem TS, the
std::experimental::filesystem::remove_all implementation now just calls
std::filesystem::remove_all to do the work.

libstdc++-v3/ChangeLog:

	PR libstdc++/104161
	* acinclude.m4 (GLIBCXX_CHECK_FILESYSTEM_DEPS): Check for dirfd
	and unlinkat.
	* config.h.in: Regenerate.
	* configure: Regenerate.
	* include/bits/fs_dir.h (recursive_directory_iterator): Declare
	remove_all overloads as friends.
	(recursive_directory_iterator::__erase): Declare new member
	function.
	* include/bits/fs_fwd.h (remove, remove_all): Declare.
	* src/c++17/fs_dir.cc (_Dir): Add filename_only parameter to
	constructor. Pass file descriptor argument to base constructor.
	(_Dir::dir_and_pathname, _Dir::open_subdir, _Dir::do_unlink)
	(_Dir::unlink, _Dir::rmdir): Define new member functions.
	(directory_iterator): Pass filename_only argument to _Dir
	constructor.
	(recursive_directory_iterator::_Dir_stack): Adjust constructor
	parameters to take a _Dir rvalue instead of creating one.
	(_Dir_stack::orig): Add data member for storing original path.
	(_Dir_stack::report_error): Define new member function.
	(__directory_iterator_nofollow): Move here from dir-common.h and
	fix value to be a power of two.
	(__directory_iterator_filename_only): Define new constant.
	(recursive_directory_iterator): Construct _Dir object and move
	into _M_dirs stack. Pass skip_permission_denied argument to first
	advance call.
	(recursive_directory_iterator::increment): Use _Dir::open_subdir.
	(recursive_directory_iterator::__erase): Define new member
	function.
	* src/c++17/fs_ops.cc (ErrorReporter, do_remove_all): Remove.
	(fs::remove_all): Use new recursive_directory_iterator::__erase
	member function.
	* src/filesystem/dir-common.h (_Dir_base): Add int parameter to
	constructor and use openat to implement nofollow semantics.
	(_Dir_base::fdcwd, _Dir_base::set_close_on_exec, _Dir_base::openat):
	Define new member functions.
	(__directory_iterator_nofollow): Move to fs_dir.cc.
	* src/filesystem/dir.cc (_Dir): Pass file descriptor argument to
	base constructor.
	(_Dir::dir_and_pathname, _Dir::open_subdir): Define new member
	functions.
	(recursive_directory_iterator::_Dir_stack): Adjust constructor
	parameters to take a _Dir rvalue instead of creating one.
	(recursive_directory_iterator): Check for new nofollow option.
	Construct _Dir object and move into _M_dirs stack. Pass
	skip_permission_denied argument to first advance call.
	(recursive_directory_iterator::increment): Use _Dir::open_subdir.
	* src/filesystem/ops.cc (fs::remove_all): Use C++17 remove_all.
2022-02-04 19:51:26 +00:00

585 lines
16 KiB
C++

// Filesystem directory utilities -*- C++ -*-
// Copyright (C) 2014-2022 Free Software Foundation, Inc.
//
// This file is part of the GNU ISO C++ Library. This library is free
// software; you can redistribute it and/or modify it under the
// terms of the GNU General Public License as published by the
// Free Software Foundation; either version 3, or (at your option)
// any later version.
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// Under Section 7 of GPL version 3, you are granted additional
// permissions described in the GCC Runtime Library Exception, version
// 3.1, as published by the Free Software Foundation.
// You should have received a copy of the GNU General Public License and
// a copy of the GCC Runtime Library Exception along with this program;
// see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
// <http://www.gnu.org/licenses/>.
/** @file include/bits/fs_dir.h
* This is an internal header file, included by other library headers.
* Do not attempt to use it directly. @headername{filesystem}
*/
#ifndef _GLIBCXX_FS_DIR_H
#define _GLIBCXX_FS_DIR_H 1
#if __cplusplus >= 201703L
# include <typeinfo>
# include <ext/concurrence.h>
# include <bits/unique_ptr.h>
# include <bits/shared_ptr.h>
#if __cplusplus > 201703L
# include <compare> // std::strong_ordering
#endif
namespace std _GLIBCXX_VISIBILITY(default)
{
_GLIBCXX_BEGIN_NAMESPACE_VERSION
namespace filesystem
{
/** @addtogroup filesystem
* @{
*/
/// Information about a file's type and permissions.
class file_status
{
public:
// constructors and destructor
file_status() noexcept : file_status(file_type::none) {}
explicit
file_status(file_type __ft, perms __prms = perms::unknown) noexcept
: _M_type(__ft), _M_perms(__prms) { }
file_status(const file_status&) noexcept = default;
file_status(file_status&&) noexcept = default;
~file_status() = default;
file_status& operator=(const file_status&) noexcept = default;
file_status& operator=(file_status&&) noexcept = default;
// observers
file_type type() const noexcept { return _M_type; }
perms permissions() const noexcept { return _M_perms; }
// modifiers
void type(file_type __ft) noexcept { _M_type = __ft; }
void permissions(perms __prms) noexcept { _M_perms = __prms; }
#if __cpp_lib_three_way_comparison
friend bool
operator==(const file_status&, const file_status&) noexcept = default;
#endif
private:
file_type _M_type;
perms _M_perms;
};
_GLIBCXX_BEGIN_NAMESPACE_CXX11
struct _Dir;
class directory_iterator;
class recursive_directory_iterator;
/// The value type used by directory iterators
class directory_entry
{
public:
// constructors and destructor
directory_entry() noexcept = default;
directory_entry(const directory_entry&) = default;
directory_entry(directory_entry&&) noexcept = default;
explicit
directory_entry(const filesystem::path& __p)
: _M_path(__p)
{ refresh(); }
directory_entry(const filesystem::path& __p, error_code& __ec)
: _M_path(__p)
{
refresh(__ec);
if (__ec)
_M_path.clear();
}
~directory_entry() = default;
// modifiers
directory_entry& operator=(const directory_entry&) = default;
directory_entry& operator=(directory_entry&&) noexcept = default;
void
assign(const filesystem::path& __p)
{
_M_path = __p;
refresh();
}
void
assign(const filesystem::path& __p, error_code& __ec)
{
_M_path = __p;
refresh(__ec);
}
void
replace_filename(const filesystem::path& __p)
{
_M_path.replace_filename(__p);
refresh();
}
void
replace_filename(const filesystem::path& __p, error_code& __ec)
{
_M_path.replace_filename(__p);
refresh(__ec);
}
void
refresh()
{ _M_type = symlink_status().type(); }
void
refresh(error_code& __ec) noexcept
{ _M_type = symlink_status(__ec).type(); }
// observers
const filesystem::path& path() const noexcept { return _M_path; }
operator const filesystem::path& () const noexcept { return _M_path; }
bool
exists() const
{ return filesystem::exists(file_status{_M_file_type()}); }
bool
exists(error_code& __ec) const noexcept
{ return filesystem::exists(file_status{_M_file_type(__ec)}); }
bool
is_block_file() const
{ return _M_file_type() == file_type::block; }
bool
is_block_file(error_code& __ec) const noexcept
{ return _M_file_type(__ec) == file_type::block; }
bool
is_character_file() const
{ return _M_file_type() == file_type::character; }
bool
is_character_file(error_code& __ec) const noexcept
{ return _M_file_type(__ec) == file_type::character; }
bool
is_directory() const
{ return _M_file_type() == file_type::directory; }
bool
is_directory(error_code& __ec) const noexcept
{ return _M_file_type(__ec) == file_type::directory; }
bool
is_fifo() const
{ return _M_file_type() == file_type::fifo; }
bool
is_fifo(error_code& __ec) const noexcept
{ return _M_file_type(__ec) == file_type::fifo; }
bool
is_other() const
{ return filesystem::is_other(file_status{_M_file_type()}); }
bool
is_other(error_code& __ec) const noexcept
{ return filesystem::is_other(file_status{_M_file_type(__ec)}); }
bool
is_regular_file() const
{ return _M_file_type() == file_type::regular; }
bool
is_regular_file(error_code& __ec) const noexcept
{ return _M_file_type(__ec) == file_type::regular; }
bool
is_socket() const
{ return _M_file_type() == file_type::socket; }
bool
is_socket(error_code& __ec) const noexcept
{ return _M_file_type(__ec) == file_type::socket; }
bool
is_symlink() const
{
if (_M_type != file_type::none)
return _M_type == file_type::symlink;
return symlink_status().type() == file_type::symlink;
}
bool
is_symlink(error_code& __ec) const noexcept
{
if (_M_type != file_type::none)
return _M_type == file_type::symlink;
return symlink_status(__ec).type() == file_type::symlink;
}
uintmax_t
file_size() const
{ return filesystem::file_size(_M_path); }
uintmax_t
file_size(error_code& __ec) const noexcept
{ return filesystem::file_size(_M_path, __ec); }
uintmax_t
hard_link_count() const
{ return filesystem::hard_link_count(_M_path); }
uintmax_t
hard_link_count(error_code& __ec) const noexcept
{ return filesystem::hard_link_count(_M_path, __ec); }
file_time_type
last_write_time() const
{ return filesystem::last_write_time(_M_path); }
file_time_type
last_write_time(error_code& __ec) const noexcept
{ return filesystem::last_write_time(_M_path, __ec); }
file_status
status() const
{ return filesystem::status(_M_path); }
file_status
status(error_code& __ec) const noexcept
{ return filesystem::status(_M_path, __ec); }
file_status
symlink_status() const
{ return filesystem::symlink_status(_M_path); }
file_status
symlink_status(error_code& __ec) const noexcept
{ return filesystem::symlink_status(_M_path, __ec); }
bool
operator==(const directory_entry& __rhs) const noexcept
{ return _M_path == __rhs._M_path; }
#if __cpp_lib_three_way_comparison
strong_ordering
operator<=>(const directory_entry& __rhs) const noexcept
{ return _M_path <=> __rhs._M_path; }
#else
bool
operator!=(const directory_entry& __rhs) const noexcept
{ return _M_path != __rhs._M_path; }
bool
operator< (const directory_entry& __rhs) const noexcept
{ return _M_path < __rhs._M_path; }
bool
operator<=(const directory_entry& __rhs) const noexcept
{ return _M_path <= __rhs._M_path; }
bool
operator> (const directory_entry& __rhs) const noexcept
{ return _M_path > __rhs._M_path; }
bool
operator>=(const directory_entry& __rhs) const noexcept
{ return _M_path >= __rhs._M_path; }
#endif
private:
friend struct _Dir;
friend class directory_iterator;
friend class recursive_directory_iterator;
// _GLIBCXX_RESOLVE_LIB_DEFECTS
// 3171. LWG 2989 breaks directory_entry stream insertion
template<typename _CharT, typename _Traits>
friend basic_ostream<_CharT, _Traits>&
operator<<(basic_ostream<_CharT, _Traits>& __os,
const directory_entry& __d)
{ return __os << __d.path(); }
directory_entry(const filesystem::path& __p, file_type __t)
: _M_path(__p), _M_type(__t)
{ }
// Equivalent to status().type() but uses cached value, if any.
file_type
_M_file_type() const
{
if (_M_type != file_type::none && _M_type != file_type::symlink)
return _M_type;
return status().type();
}
// Equivalent to status(__ec).type() but uses cached value, if any.
file_type
_M_file_type(error_code& __ec) const noexcept
{
if (_M_type != file_type::none && _M_type != file_type::symlink)
{
__ec.clear();
return _M_type;
}
return status(__ec).type();
}
filesystem::path _M_path;
file_type _M_type = file_type::none;
};
/// Proxy returned by post-increment on directory iterators.
struct __directory_iterator_proxy
{
const directory_entry& operator*() const& noexcept { return _M_entry; }
directory_entry operator*() && noexcept { return std::move(_M_entry); }
private:
friend class directory_iterator;
friend class recursive_directory_iterator;
explicit
__directory_iterator_proxy(const directory_entry& __e) : _M_entry(__e) { }
directory_entry _M_entry;
};
/// Iterator type for traversing the entries in a single directory.
class directory_iterator
{
public:
typedef directory_entry value_type;
typedef ptrdiff_t difference_type;
typedef const directory_entry* pointer;
typedef const directory_entry& reference;
typedef input_iterator_tag iterator_category;
directory_iterator() = default;
explicit
directory_iterator(const path& __p)
: directory_iterator(__p, directory_options::none, nullptr) { }
directory_iterator(const path& __p, directory_options __options)
: directory_iterator(__p, __options, nullptr) { }
directory_iterator(const path& __p, error_code& __ec)
: directory_iterator(__p, directory_options::none, __ec) { }
directory_iterator(const path& __p, directory_options __options,
error_code& __ec)
: directory_iterator(__p, __options, &__ec) { }
directory_iterator(const directory_iterator& __rhs) = default;
directory_iterator(directory_iterator&& __rhs) noexcept = default;
~directory_iterator() = default;
directory_iterator&
operator=(const directory_iterator& __rhs) = default;
directory_iterator&
operator=(directory_iterator&& __rhs) noexcept = default;
const directory_entry& operator*() const noexcept;
const directory_entry* operator->() const noexcept { return &**this; }
directory_iterator& operator++();
directory_iterator& increment(error_code& __ec);
__directory_iterator_proxy operator++(int)
{
__directory_iterator_proxy __pr{**this};
++*this;
return __pr;
}
private:
directory_iterator(const path&, directory_options, error_code*);
friend bool
operator==(const directory_iterator& __lhs,
const directory_iterator& __rhs) noexcept
{
return !__rhs._M_dir.owner_before(__lhs._M_dir)
&& !__lhs._M_dir.owner_before(__rhs._M_dir);
}
friend bool
operator!=(const directory_iterator& __lhs,
const directory_iterator& __rhs) noexcept
{ return !(__lhs == __rhs); }
friend class recursive_directory_iterator;
std::__shared_ptr<_Dir> _M_dir;
};
/// @relates std::filesystem::directory_iterator @{
/** @brief Enable range-based `for` using directory_iterator.
*
* e.g. `for (auto& entry : std::filesystem::directory_iterator(".")) ...`
*/
inline directory_iterator
begin(directory_iterator __iter) noexcept
{ return __iter; }
/// Return a past-the-end directory_iterator
inline directory_iterator
end(directory_iterator) noexcept
{ return directory_iterator(); }
/// @}
/// Iterator type for recursively traversing a directory hierarchy.
class recursive_directory_iterator
{
public:
typedef directory_entry value_type;
typedef ptrdiff_t difference_type;
typedef const directory_entry* pointer;
typedef const directory_entry& reference;
typedef input_iterator_tag iterator_category;
recursive_directory_iterator() = default;
explicit
recursive_directory_iterator(const path& __p)
: recursive_directory_iterator(__p, directory_options::none, nullptr) { }
recursive_directory_iterator(const path& __p, directory_options __options)
: recursive_directory_iterator(__p, __options, nullptr) { }
recursive_directory_iterator(const path& __p, directory_options __options,
error_code& __ec)
: recursive_directory_iterator(__p, __options, &__ec) { }
recursive_directory_iterator(const path& __p, error_code& __ec)
: recursive_directory_iterator(__p, directory_options::none, &__ec) { }
recursive_directory_iterator(
const recursive_directory_iterator&) = default;
recursive_directory_iterator(recursive_directory_iterator&&) = default;
~recursive_directory_iterator();
// observers
directory_options options() const noexcept;
int depth() const noexcept;
bool recursion_pending() const noexcept;
const directory_entry& operator*() const noexcept;
const directory_entry* operator->() const noexcept { return &**this; }
// modifiers
recursive_directory_iterator&
operator=(const recursive_directory_iterator& __rhs) noexcept;
recursive_directory_iterator&
operator=(recursive_directory_iterator&& __rhs) noexcept;
recursive_directory_iterator& operator++();
recursive_directory_iterator& increment(error_code& __ec);
__directory_iterator_proxy operator++(int)
{
__directory_iterator_proxy __pr{**this};
++*this;
return __pr;
}
void pop();
void pop(error_code&);
void disable_recursion_pending() noexcept;
private:
recursive_directory_iterator(const path&, directory_options, error_code*);
friend bool
operator==(const recursive_directory_iterator& __lhs,
const recursive_directory_iterator& __rhs) noexcept
{
return !__rhs._M_dirs.owner_before(__lhs._M_dirs)
&& !__lhs._M_dirs.owner_before(__rhs._M_dirs);
}
friend bool
operator!=(const recursive_directory_iterator& __lhs,
const recursive_directory_iterator& __rhs) noexcept
{ return !(__lhs == __rhs); }
struct _Dir_stack;
std::__shared_ptr<_Dir_stack> _M_dirs;
recursive_directory_iterator&
__erase(error_code* = nullptr);
friend uintmax_t
filesystem::remove_all(const path&, error_code&);
friend uintmax_t
filesystem::remove_all(const path&);
};
/// @relates std::filesystem::recursive_directory_iterator @{
/** @brief Enable range-based `for` using recursive_directory_iterator.
*
* e.g. `for (auto& entry : recursive_directory_iterator(".")) ...`
*/
inline recursive_directory_iterator
begin(recursive_directory_iterator __iter) noexcept
{ return __iter; }
/// Return a past-the-end recursive_directory_iterator
inline recursive_directory_iterator
end(recursive_directory_iterator) noexcept
{ return recursive_directory_iterator(); }
/// @}
_GLIBCXX_END_NAMESPACE_CXX11
/// @} group filesystem
} // namespace filesystem
// Use explicit instantiations of these types. Any inconsistency in the
// value of __default_lock_policy between code including this header and
// the library will cause a linker error.
extern template class
__shared_ptr<filesystem::_Dir>;
extern template class
__shared_ptr<filesystem::recursive_directory_iterator::_Dir_stack>;
_GLIBCXX_END_NAMESPACE_VERSION
} // namespace std
#endif // C++17
#endif // _GLIBCXX_FS_DIR_H