b24413180f
Many source files in the tree are missing licensing information, which makes it harder for compliance tools to determine the correct license. By default all files without license information are under the default license of the kernel, which is GPL version 2. Update the files which contain no license information with the 'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally binding shorthand, which can be used instead of the full boiler plate text. This patch is based on work done by Thomas Gleixner and Kate Stewart and Philippe Ombredanne. How this work was done: Patches were generated and checked against linux-4.14-rc6 for a subset of the use cases: - file had no licensing information it it. - file was a */uapi/* one with no licensing information in it, - file was a */uapi/* one with existing licensing information, Further patches will be generated in subsequent months to fix up cases where non-standard license headers were used, and references to license had to be inferred by heuristics based on keywords. The analysis to determine which SPDX License Identifier to be applied to a file was done in a spreadsheet of side by side results from of the output of two independent scanners (ScanCode & Windriver) producing SPDX tag:value files created by Philippe Ombredanne. Philippe prepared the base worksheet, and did an initial spot review of a few 1000 files. The 4.13 kernel was the starting point of the analysis with 60,537 files assessed. Kate Stewart did a file by file comparison of the scanner results in the spreadsheet to determine which SPDX license identifier(s) to be applied to the file. She confirmed any determination that was not immediately clear with lawyers working with the Linux Foundation. Criteria used to select files for SPDX license identifier tagging was: - Files considered eligible had to be source code files. - Make and config files were included as candidates if they contained >5 lines of source - File already had some variant of a license header in it (even if <5 lines). All documentation files were explicitly excluded. The following heuristics were used to determine which SPDX license identifiers to apply. - when both scanners couldn't find any license traces, file was considered to have no license information in it, and the top level COPYING file license applied. For non */uapi/* files that summary was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 11139 and resulted in the first patch in this series. If that file was a */uapi/* path one, it was "GPL-2.0 WITH Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 WITH Linux-syscall-note 930 and resulted in the second patch in this series. - if a file had some form of licensing information in it, and was one of the */uapi/* ones, it was denoted with the Linux-syscall-note if any GPL family license was found in the file or had no licensing in it (per prior point). Results summary: SPDX license identifier # files ---------------------------------------------------|------ GPL-2.0 WITH Linux-syscall-note 270 GPL-2.0+ WITH Linux-syscall-note 169 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17 LGPL-2.1+ WITH Linux-syscall-note 15 GPL-1.0+ WITH Linux-syscall-note 14 ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5 LGPL-2.0+ WITH Linux-syscall-note 4 LGPL-2.1 WITH Linux-syscall-note 3 ((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3 ((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1 and that resulted in the third patch in this series. - when the two scanners agreed on the detected license(s), that became the concluded license(s). - when there was disagreement between the two scanners (one detected a license but the other didn't, or they both detected different licenses) a manual inspection of the file occurred. - In most cases a manual inspection of the information in the file resulted in a clear resolution of the license that should apply (and which scanner probably needed to revisit its heuristics). - When it was not immediately clear, the license identifier was confirmed with lawyers working with the Linux Foundation. - If there was any question as to the appropriate license identifier, the file was flagged for further research and to be revisited later in time. In total, over 70 hours of logged manual review was done on the spreadsheet to determine the SPDX license identifiers to apply to the source files by Kate, Philippe, Thomas and, in some cases, confirmation by lawyers working with the Linux Foundation. Kate also obtained a third independent scan of the 4.13 code base from FOSSology, and compared selected files where the other two scanners disagreed against that SPDX file, to see if there was new insights. The Windriver scanner is based on an older version of FOSSology in part, so they are related. Thomas did random spot checks in about 500 files from the spreadsheets for the uapi headers and agreed with SPDX license identifier in the files he inspected. For the non-uapi files Thomas did random spot checks in about 15000 files. In initial set of patches against 4.14-rc6, 3 files were found to have copy/paste license identifier errors, and have been fixed to reflect the correct identifier. Additionally Philippe spent 10 hours this week doing a detailed manual inspection and review of the 12,461 patched files from the initial patch version early this week with: - a full scancode scan run, collecting the matched texts, detected license ids and scores - reviewing anything where there was a license detected (about 500+ files) to ensure that the applied SPDX license was correct - reviewing anything where there was no detection but the patch license was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied SPDX license was correct This produced a worksheet with 20 files needing minor correction. This worksheet was then exported into 3 different .csv files for the different types of files to be modified. These .csv files were then reviewed by Greg. Thomas wrote a script to parse the csv files and add the proper SPDX tag to the file, in the format that the file expected. This script was further refined by Greg based on the output to detect more types of files automatically and to distinguish between header and source .c files (which need different comment types.) Finally Greg ran the script using the .csv files to generate the patches. Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> |
||
---|---|---|
.. | ||
acl.c | ||
acl.h | ||
background.c | ||
build.c | ||
compr_lzo.c | ||
compr_rtime.c | ||
compr_rubin.c | ||
compr_zlib.c | ||
compr.c | ||
compr.h | ||
debug.c | ||
debug.h | ||
dir.c | ||
erase.c | ||
file.c | ||
fs.c | ||
gc.c | ||
ioctl.c | ||
jffs2_fs_i.h | ||
jffs2_fs_sb.h | ||
Kconfig | ||
LICENCE | ||
Makefile | ||
malloc.c | ||
nodelist.c | ||
nodelist.h | ||
nodemgmt.c | ||
os-linux.h | ||
read.c | ||
readinode.c | ||
README.Locking | ||
scan.c | ||
security.c | ||
summary.c | ||
summary.h | ||
super.c | ||
symlink.c | ||
TODO | ||
wbuf.c | ||
write.c | ||
writev.c | ||
xattr_trusted.c | ||
xattr_user.c | ||
xattr.c | ||
xattr.h |
JFFS2 LOCKING DOCUMENTATION --------------------------- This document attempts to describe the existing locking rules for JFFS2. It is not expected to remain perfectly up to date, but ought to be fairly close. alloc_sem --------- The alloc_sem is a per-filesystem mutex, used primarily to ensure contiguous allocation of space on the medium. It is automatically obtained during space allocations (jffs2_reserve_space()) and freed upon write completion (jffs2_complete_reservation()). Note that the garbage collector will obtain this right at the beginning of jffs2_garbage_collect_pass() and release it at the end, thereby preventing any other write activity on the file system during a garbage collect pass. When writing new nodes, the alloc_sem must be held until the new nodes have been properly linked into the data structures for the inode to which they belong. This is for the benefit of NAND flash - adding new nodes to an inode may obsolete old ones, and by holding the alloc_sem until this happens we ensure that any data in the write-buffer at the time this happens are part of the new node, not just something that was written afterwards. Hence, we can ensure the newly-obsoleted nodes don't actually get erased until the write-buffer has been flushed to the medium. With the introduction of NAND flash support and the write-buffer, the alloc_sem is also used to protect the wbuf-related members of the jffs2_sb_info structure. Atomically reading the wbuf_len member to see if the wbuf is currently holding any data is permitted, though. Ordering constraints: See f->sem. File Mutex f->sem --------------------- This is the JFFS2-internal equivalent of the inode mutex i->i_sem. It protects the contents of the jffs2_inode_info private inode data, including the linked list of node fragments (but see the notes below on erase_completion_lock), etc. The reason that the i_sem itself isn't used for this purpose is to avoid deadlocks with garbage collection -- the VFS will lock the i_sem before calling a function which may need to allocate space. The allocation may trigger garbage-collection, which may need to move a node belonging to the inode which was locked in the first place by the VFS. If the garbage collection code were to attempt to lock the i_sem of the inode from which it's garbage-collecting a physical node, this lead to deadlock, unless we played games with unlocking the i_sem before calling the space allocation functions. Instead of playing such games, we just have an extra internal mutex, which is obtained by the garbage collection code and also by the normal file system code _after_ allocation of space. Ordering constraints: 1. Never attempt to allocate space or lock alloc_sem with any f->sem held. 2. Never attempt to lock two file mutexes in one thread. No ordering rules have been made for doing so. 3. Never lock a page cache page with f->sem held. erase_completion_lock spinlock ------------------------------ This is used to serialise access to the eraseblock lists, to the per-eraseblock lists of physical jffs2_raw_node_ref structures, and (NB) the per-inode list of physical nodes. The latter is a special case - see below. As the MTD API no longer permits erase-completion callback functions to be called from bottom-half (timer) context (on the basis that nobody ever actually implemented such a thing), it's now sufficient to use a simple spin_lock() rather than spin_lock_bh(). Note that the per-inode list of physical nodes (f->nodes) is a special case. Any changes to _valid_ nodes (i.e. ->flash_offset & 1 == 0) in the list are protected by the file mutex f->sem. But the erase code may remove _obsolete_ nodes from the list while holding only the erase_completion_lock. So you can walk the list only while holding the erase_completion_lock, and can drop the lock temporarily mid-walk as long as the pointer you're holding is to a _valid_ node, not an obsolete one. The erase_completion_lock is also used to protect the c->gc_task pointer when the garbage collection thread exits. The code to kill the GC thread locks it, sends the signal, then unlocks it - while the GC thread itself locks it, zeroes c->gc_task, then unlocks on the exit path. inocache_lock spinlock ---------------------- This spinlock protects the hashed list (c->inocache_list) of the in-core jffs2_inode_cache objects (each inode in JFFS2 has the correspondent jffs2_inode_cache object). So, the inocache_lock has to be locked while walking the c->inocache_list hash buckets. This spinlock also covers allocation of new inode numbers, which is currently just '++->highest_ino++', but might one day get more complicated if we need to deal with wrapping after 4 milliard inode numbers are used. Note, the f->sem guarantees that the correspondent jffs2_inode_cache will not be removed. So, it is allowed to access it without locking the inocache_lock spinlock. Ordering constraints: If both erase_completion_lock and inocache_lock are needed, the c->erase_completion has to be acquired first. erase_free_sem -------------- This mutex is only used by the erase code which frees obsolete node references and the jffs2_garbage_collect_deletion_dirent() function. The latter function on NAND flash must read _obsolete_ nodes to determine whether the 'deletion dirent' under consideration can be discarded or whether it is still required to show that an inode has been unlinked. Because reading from the flash may sleep, the erase_completion_lock cannot be held, so an alternative, more heavyweight lock was required to prevent the erase code from freeing the jffs2_raw_node_ref structures in question while the garbage collection code is looking at them. Suggestions for alternative solutions to this problem would be welcomed. wbuf_sem -------- This read/write semaphore protects against concurrent access to the write-behind buffer ('wbuf') used for flash chips where we must write in blocks. It protects both the contents of the wbuf and the metadata which indicates which flash region (if any) is currently covered by the buffer. Ordering constraints: Lock wbuf_sem last, after the alloc_sem or and f->sem. c->xattr_sem ------------ This read/write semaphore protects against concurrent access to the xattr related objects which include stuff in superblock and ic->xref. In read-only path, write-semaphore is too much exclusion. It's enough by read-semaphore. But you must hold write-semaphore when updating, creating or deleting any xattr related object. Once xattr_sem released, there would be no assurance for the existence of those objects. Thus, a series of processes is often required to retry, when updating such a object is necessary under holding read semaphore. For example, do_jffs2_getxattr() holds read-semaphore to scan xref and xdatum at first. But it retries this process with holding write-semaphore after release read-semaphore, if it's necessary to load name/value pair from medium. Ordering constraints: Lock xattr_sem last, after the alloc_sem.