After some discussion in libc-alpha about this POSIX compliance fix, I see
that GLIBC should indeed revert back to previous definition of msghdr and
cmsghdr and implementation of sendmsg, recvmsg, sendmmsg, recvmmsg due some
reasons:
* The possible issue where the syscalls wrapper add the compatibility
layer is quite limited in scope and range. And kernel current
also add some limits to the values on the internal msghdr and
cmsghdr fields:
- msghdr::msg_iovlen larger than UIO_MAXIOV (1024) returns
EMSGSIZE.
- msghdr::msg_controllen larger than INT_MAX returns ENOBUFS.
* There is a small performance hit for recvmsg/sendmsg/recmmsg which
is neglectable, but it is a big hit for sendmmsg since now instead
of calling the syscall for the packed structure, GLIBC is calling
multiple sendmsg. This defeat the very existence of the syscall.
* It currently breaks libsanitizer build on GCC [1] (I fixed on compiler-rt).
However the fix is incomplete because it does add any runtime check
since libsanitizer currently does not have any facility to intercept
symbols with multiple version [2].
This, along with incorret dlsym/dlvsym return for versioned symbol due
another bug [3], makes hard to interpose versioned symbols.
Also, current approach of fixing GCC PR#71445 leads to half-baked
solutions without versioned symbol interposing.
This patch basically reverts commits 2f0dc39029, 222c2d7f43,
af7f7c7ec8. I decided to not revert abf29edd4a (Adjust
kernel-features.h defaults for recvmsg and sendmsg) mainly because it
does not really address the POSIX compliance original issue and also
adds some cleanups.
Tested on x86, i386, s390, s390x, aarch64, and powerpc64le.
[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71445
[2] https://github.com/google/sanitizers/issues/628
[3] https://sourceware.org/bugzilla/show_bug.cgi?id=14932
* conform/data/sys/socket.h-data (msghdr.msg_iovlen): Add xfail-.
(msghdr.msg_controllen): Likewise.
(cmsghdr.cmsg_len): Likewise.
* nptl/Makefile (libpthread-routines): Remove ptw-oldrecvmsg and
ptw-oldsendmsg.
(CFLAGS-oldrecvmsg.c): Remove rule.
(CFLAGS-oldsendmsg.c): Likewise.
(CFLAGS-recvmsg.c): Add rule.
(CFLAGS-sendmsg.c): Likewise.
* sysdeps/unix/sysv/linux/Makefile (sysdep_routines): Remove
oldrecvmsg, oldsendmsg, oldrecvmmsg, oldsendmmsg.
(CFLAGS-recvmsg.c): Remove rule.
(CFLAGS-sendmsg.c): Likewise.
(CFLAGS-oldrecvmsg.c): Likewise.
(CFLAGS-oldsendmsg.c): Likewise.
(CFLAGS-recvmmsg.c): Likewise.
* sysdeps/unix/sysv/linux/bits/socket.h (msghdr.msg_iovlen): Revert
to kernel defined interfaces.
(msghdr.msg_controllen): Likewise.
(cmsghdr.cmsg_len): Likewise.
(msghdr.__glibc_reserved1): Remove member.
(msghdr.__glibc_reserved2): Likewise.
(cmsghdr.__glibc_reserved1): Likewise.
* sysdeps/unix/sysv/linux/oldrecvmmsg.c: Remove file.
* sysdeps/unix/sysv/linux/oldrecvmsg.c: Likewise.
* sysdeps/unix/sysv/linux/oldsendmmsg.c: Likewise.
* sysdeps/unix/sysv/linux/oldsendmsg.c: Likewise.
* sysdeps/unix/sysv/linux/recvmmsg.c: Revert back to previous
version.
* sysdeps/unix/sysv/linux/recvmsg.c: Likewise.
* sysdeps/unix/sysv/linux/sendmmsg.c: Likewise.
* sysdeps/unix/sysv/linux/sendmsg.c: Likewise.
* sysdeps/unix/sysv/linux/aarch64/Versions [libc] (GLIBC_2.24):
Remove recvmsg and sendmsg.
* sysdeps/unix/sysv/linux/alpha/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/hppa/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/i386/Versions [libc] (GLIBC_2.24): Likewise.
* sysdeps/unix/sysv/linux/m68k/Versions [libc] (GLIBC_2.24): Likewise.
* sysdeps/unix/sysv/linux/microblaze/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n32/Versions
[libc] (GLIBC_2.24): Likewise.
* sysdeps/unix/sysv/linux/nios2/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/powerpc/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/Versions
[libc] (GLIBC_2.24): Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/sh/Versions [libc] (GLIBC_2.24): Likewise.
* sysdeps/unix/sysv/linux/sparc/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/tile/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/x86_64/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx64/Versions: Remove file
* sysdeps/unix/sysv/linux/x86_64/64/Versions: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n64/Versions: Likewise.
* sysdeps/unix/sysv/linux/aarch64/libc.abilist: Remove new 2.24
version for {recv,send,recm,sendm}msg.
* sysdeps/unix/sysv/linux/alpha/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/arm/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/hppa/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/i386/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/ia64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/microblaze/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist:
Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc.abilist:
Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/libc-le.abilist: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/nios2/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sh/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilepro/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/x86_64/64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist: Likewise.
This patch fixes various conformtest sys/stat.h expectations that were
incorrect for XPG3 (not based on a full review, so not necessarily an
exhaustive set of such corrections). Most of these corrections fix
spurious failures shown in testing (but that for fchmod introduces a
correct failure, as that function is wrongly declared for XPG3, so
this doesn't eliminate any XFAILs).
Tested for x86_64 and x86.
* conform/data/sys/stat.h-data [XPG3] (S_IFLNK): Do not expect.
[XPG3] (S_IFSOCK): Likewise.
[XPG3] (S_ISVTX): Likewise.
[XPG3] (S_ISLNK): Likewise.
[XPG3] (S_ISSOCK): Likewise.
[XPG3] (fchmod): Likewise.
[XPG3] (lstat): Likewise.
[XPG3] (mknod): Likewise.
The conformtest expectations expect the struct stat fields st_blksize
and st_blocks to be of types blksize_t and blkcnt_t. But XPG4 does
not have those types, using long instead, and XPG3 does not have these
fields at all. This patch adjusts the expectations for those
standards, XFAILing them for XPG4 to allow for systems where the
typedefs don't correspond to long.
Tested for x86_64 and x86.
* conform/data/sys/stat.h-data (st_blksize): Do not expect for
[XPG3]. Expect type long and XFAIL for [XPG4].
(st_blocks): Likewise.
The conformtest tests for sys/types.h for XPG3 and XPG4 fail because
of missing blksize_t. This is a bug in the expectations; that type is
not part of those standards. This patch stops the tests from
expecting it, and some other types that also are not part of XPG3 and
XPG4.
Tested for x86_64 and x86.
* conform/data/sys/types.h-data (blkcnt_t): Do not expect for
[XPG3 || XPG4].
(blksize_t): Likewise.
(clockid_t): Likewise.
* conform/Makefile (test-xfail-XPG3/sys/types.h/conform): Remove
variable.
(test-xfail-XPG4/sys/types.h/conform): Likewise.
conformtest expected pselect for XOPEN2K and POSIX2008 (correctly),
but not for XOPEN2K8 (wrongly). This patch fixes the data to expect
it for XOPEN2K8 as well.
(As for all such fixes based on failures seen from the tests, there is
no guarantee that the data is fully correct for a particular standard
after the fix; headers and data may well have matching bugs. So far
I've only reviewed ISO, ISO99, ISO11 and POSIX data in detail, though
I hope eventually to review the others.)
Tested for x86_64 and x86.
* conform/data/sys/select.h-data [XOPEN2K8] (pselect): Expect.
* conform/Makefile (test-xfail-XOPEN2K8/sys/select.h/conform):
Remove variable.
(test-xfail-XOPEN2K8/sys/time.h/conform): Likewise.
(test-xfail-XOPEN2K8/utmpx.h/conform).
Various conformtest tests fail because of known issues, filed in
Bugzilla, that are hard to fix (requiring new features, kernel
cooperation or involving changing types in ways that may involve care
around padding when interfacing to the kernel). Such an issue has the
effect of making the whole test for the (standard, header) pair fail,
so hiding any other issues with that header for that standard
(possibly regressions or architecture-specific issues).
This patch adds a mechanism for individual conformtest test
expectations to start with xfail-, meaning that a failure of that
particular assertion does not cause the whole test to fail at the
makefile level and so failure at the makefile level can be used to
detect other issues that are likely to be easier to fix. This is
similar to the whitelisted symbols in the linknamespace tests, or the
marking of particular libm tests as allowing spurious or missing
exceptions, for example. The bugs filed in Bugzilla should still be
fixed, and the xfail- markers removed at that point, but xfail-
renders the tests more useful until that happens.
Note that there is no way to add such a marker for the assertion that
a header uses only symbols in the namespace of symbols it's meant to
use. I don't think there's any need for a way to xfail those
namespace tests other than xfailing the whole (standard, header) pair
at the makefile level, since they are generally straightforward to fix
(add appropriate conditionals on the problem definitions).
The xfails in this patch do not necessarily cover all cases of
hard-to-fix header bugs filed in Bugzilla that currently show up in
conformtest failures; there may be more yet to add for existing open
bugs.
Tested for x86_64 and x86.
* conform/conformtest.pl ($xerrors): New variable.
(note_error): New function.
(compiletest): New argument $xfail. Use not_error.
(runtest): Likewise.
(top level): Handle xfail- lines. Update calls to compiletest and
runtest. Handle xfail- and optional- in headers listed with
allow-header.
* conform/data/fcntl.h-data (O_TTY_INIT): Use xfail-.
(O_EXEC): Likewise.
(O_SEARCH): Likewise.
* conform/data/stropts.h-data (ioctl): Likewise.
* conform/data/sys/ipc.h-data (ipc_perm.mode): Likewise.
* conform/data/sys/sem.h-data (semid_ds.sem_nsems): Likewise.
* conform/data/sys/socket.h-data (msghdr.msg_iovlen): Likewise.
(msghdr.msg_controllen): Likewise.
(cmsghdr.cmsg_len): Likewise.
* conform/data/utmpx.h-data (utmpx.ut_tv): Likewise.
* conform/Makefile (test-xfail-XPG3/sys/ipc.h/conform): Remove
variable.
(test-xfail-XPG3/sys/sem.h/conform): Likewise.
(test-xfail-XPG4/stropts.h/conform): Likewise.
(test-xfail-XPG4/sys/ipc.h/conform): Likewise.
(test-xfail-XPG4/sys/sem.h/conform): Likewise.
(test-xfail-XPG4/sys/socket.h/conform): Likewise.
(test-xfail-UNIX98/stropts.h/conform): Likewise.
(test-xfail-UNIX98/sys/ipc.h/conform): Likewise.
(test-xfail-UNIX98/sys/sem.h/conform): Likewise.
(test-xfail-UNIX98/sys/socket.h/conform): Likewise.
(test-xfail-XOPEN2K/stropts.h/conform): Likewise.
(test-xfail-XOPEN2K/sys/ipc.h/conform): Likewise.
(test-xfail-XOPEN2K/sys/sem.h/conform): Likewise.
(test-xfail-XOPEN2K/sys/socket.h/conform): Likewise.
(test-xfail-XOPEN2K/utmpx.h/conform): Likewise.
(test-xfail-POSIX2008/fcntl.h/conform): Likewise.
(test-xfail-POSIX2008/stropts.h/conform): Likewise.
(test-xfail-XOPEN2K8/fcntl.h/conform): Likewise.
(test-xfail-XOPEN2K8/stropts.h/conform): Likewise.
(test-xfail-XOPEN2K8/sys/ipc.h/conform): Likewise.
(test-xfail-XOPEN2K8/sys/sem.h/conform): Likewise.
(test-xfail-XOPEN2K8/sys/socket.h/conform): Likewise.
On 21/05/15 05:29, Siddhesh Poyarekar wrote:
> On Wed, May 20, 2015 at 06:55:02PM +0100, Szabolcs Nagy wrote:
>> i guess it's ok for consistency if i fix struct stat64
>> too to use __USE_XOPEN2K8.
>>
>> i will run some tests and come back with a patch
>
> I also think it would be appropriate to change this code in other
> architectures (microblaze and nacl IIRC) to make all of them
> consistent. It is a mechanical enough change IMO that all arch
> maintainer acks is not necessary.
>
here is the patch with consistent __USE_XOPEN2K8
ok to commit?
2015-05-21 Szabolcs Nagy <szabolcs.nagy@arm.com>
[BZ #18234]
* conform/data/sys/stat.h-data (struct stat): Add tests for st_atim,
st_mtim and st_ctim members.
* sysdeps/nacl/bits/stat.h (struct stat, struct stat64): Make
st_atim, st_ctim, st_mtim visible under __USE_XOPEN2K8 only.
* sysdeps/unix/sysv/linux/generic/bits/stat.h (struct stat,):
(struct stat64): Likewise.
* sysdeps/unix/sysv/linux/ia64/bits/stat.h (struct stat,):
(struct stat64): Likewise.
* sysdeps/unix/sysv/linux/microblaze/bits/stat.h (struct stat,):
(struct stat64): Likewise.
Continuing the series of patches to clean up conformtest expectations
for "POSIX" (1995/6) based on review of the expectations against the
standard, this patch cleans up expectations for sys/utsname.h and
sys/wait.h. Tested x86_64; a new XFAIL for sys/wait.h is added.
* conform/data/sys/utsname.h-data (*_t): Allow.
* conform/data/sys/wait.h-data [POSIX] (uid_t): Do not define.
[POSIX] (WEXITED): Do not expect constant.
[POSIX] (WSTOPPED): Likewise.
[POSIX] (WNOHANG): Likewise.
[POSIX] (WNOWAIT): Likewise.
[POSIX] (siginfo_t): Do not expect type or elements.
[POSIX] (pid_t): Do not expect type.
[POSIX] (signal.h): Do not allow header.
[POSIX] (sys/resource.h): Likewise.
[POSIX] (si_*): Do not allow pattern.
[POSIX] (W*): Likewise.
[POSIX] (P_*): Likewise.
[POSIX] (BUS_*): Likewise.
[POSIX] (CLD_*): Likewise.
[POSIX] (FPE_*): Likewise.
[POSIX] (ILL_*): Likewise.
[POSIX] (POLL_*): Likewise.
[POSIX] (SEGV_*): Likewise.
[POSIX] (SI_*): Likewise.
[POSIX] (TRAP_*): Likewise.
* conform/Makefile (test-xfail-POSIX/sys/wait.h/conform): New
variable.
Continuing the series of patches to clean up conformtest expectations
for "POSIX" (1995/6) based on review of the expectations against the
standard, this patch cleans up expectations for sys/mman.h, sys/stat.h
and sys/types.h. Tested x86_64; no new XFAILs needed.
* conform/data/sys/mman.h-data [POSIX] (size_t): Do not require
type.
[POSIX] (off_t): Likewise.
* conform/data/sys/stat.h-data (S_IRGRP): Require constant.
[POSIX] (S_ISBLK): Require macro.
[POSIX] (S_ISCHR): Likewise.
[POSIX] (S_ISDIR): Likewise.
[POSIX] (S_ISFIFO): Likewise.
[POSIX] (S_ISREG): Likewise.
[POSIX || XPG3 || XPG4 || UNIX98] (S_TYPEISTMO): Do not list
optional-macro.
* conform/data/sys/types.h-data [POSIX] (blkcnt_t): Do not require
type.
[POSIX] (time_t): Likewise.
[POSIX] (timer_t): Likewise.
Continuing the corrections to which headers conformtest tests for
which standards, this patch corrects conformtest mistakes regarding
which headers to test for the remaining standards (XOPEN2K POSIX2008
XOPEN2K8). Tested x86_64.
2014-03-05 Joseph Myers <joseph@codesourcery.com>
* conform/data/netinet/tcp.h-data [POSIX2008 || XOPEN2K8]: Enable
whole file.
* conform/data/sys/timeb.h-data [POSIX2008 || XOPEN2K8]: Disable
whole file.
* conform/data/sys/uio.h-data [POSIX2008]: Likewise.
* conform/data/ucontext.h-data [POSIX2008 || XOPEN2K8]: Likewise.
* time/time.h (struct timespec): Use __time_t for tv_sec element.
* sysdeps/generic/inttypes.h: Define __wchar_t. Use it instead of
wchar_t for function declarations and defintions.
* misc/sys/select.h: Define __need_time_t before including <time.h>.
Define suseconds_t if it hasn't happened yet.
* iconv/gconv.h: Define __need_wchar_t before including <stddef.h>.
* conform/data/sys/uio.h-data: Allow UIO_MAXIOV to be defined.
* sysdeps/generic/stdint.h: Don't get definition of wchar_t from
<stddef.h>.
* conform/data/stddef.h-data: Remove wint_t.
* sysdeps/unix/sysv/linux/bits/statvfs.h: Change type of f_fsid field
to unsigned long. Add __f_unused. Define _STATVFSBUF_F_UNUSED.
* sysdeps/unix/sysv/linux/internal_statvfs.c: Adjust initialization of
f_fsid field after change in struct statvfs.
* sysdeps/unix/sysv/linux/alpha/bits/statvfs.h: New file.
* sysdeps/unix/sysv/linux/ia64/bits/statvfs.h: New file.
* sysdeps/unix/sysv/linux/sparc/sparc64/bits/statvfs.h: New file.
* conform/data/sys/stat.h-data: Remove isfdtype. Use
optional-macro.
* conform/data/sys/socket.h-data: Fix typos. Correct getsockopt,
recvfrom, send, and socketpair prototypes. Add allow lines.
* socket/sys/socket.h: Include <sys/uio.h>.
Don't use fancy __SOCKADDR_ARG definition unless __USE_GNU.
Fix type of second parameter of listen.
Don't declare isfdtype unless __USE_MISC.
* sysdeps/generic/listen.c: Fix type of second parameter of listen.
* sysdeps/mach/hurd/listen.c: Likewise.
* sysdeps/generic/bits/socket.h (struct sockaddr_storage): Define
ss_family and __ss_family.
* sysdeps/unix/sysv/aix/bits/socket.h: Likewise.
* sysdeps/unix/sysv/linux/bits/socket.h: Likewise.
(struct msghdr): Change type of msg_iovlen to int and type of
msg_controllen to socklen_t.
* sysdeps/unix/sysv/linux/alpha/bits/socket.h: New file.
* sysdeps/unix/sysv/linux/ia64/bits/socket.h: New file.
* sysdeps/unix/sysv/linux/sparc/sparc64/bits/socket.h: New file.
2000-02-25 Ulrich Drepper <drepper@redhat.com>
* conform/conformtest.pl (@headers): Add "sys/msg.h", "sys/mman.h",
"sys/ipc.h", "syslog.h", and "stropts.h".
Handle / in header names correctly.
* conform/data/stropts.h-data: New file.
* conform/data/syslog.h-data: New file.
* conform/data/sys/ipc.h-data: New file.
* conform/data/sys/mman.h-data: New file.
* conform/data/sys/msg.h-data: New file.
* misc/sys/mman.h: Define off_t and mode_t if not already done.
* streams/stropts.h: Define gid_t and uid_t if not already done.
* sysdeps/gnu/bits/ipc.h (IPC_INFO): Only define if __USE_GNU.
* sysdeps/unix/sysv/linux/alpha/bits/ipc.h: Likewise.
* sysdeps/unix/sysv/linux/bits/ipc.h: Likewise.
* sysdeps/unix/sysv/linux/mips/bits/ipc.h: Likewise.
* sysdeps/unix/sysv/linux/powerpc/bits/ipc.h: Likewise.
* sysdeps/unix/sysv/linux/sparc/bits/ipc.h: Likewise.
* sysvipc/sys/msg.h: Don't include sys/types.h.
Define time_t, pid_t, and ssize_t if not already done.
(struct msgbuf): Define only if __USE_GNU.
* sysdeps/generic/bits/msq.h: Include bits/types.h, not sys/types.h.
(MSG_EXCEPT): Only define if __USE_GNU.
Define msgqnum_t and msglen_t and use them in struct msqid_ds
definition.
* sysdeps/gnu/bits/msq.h: Likewise.
* sysdeps/unix/sysv/linux/alpha/bits/msq.h: Likewise.
* sysdeps/unix/sysv/linux/bits/msq.h: Likewise.
* sysdeps/unix/sysv/linux/sparc/bits/msq.h: Likewise.
* sysdeps/generic/bits/stropts.h (I_GWROPT): Renamed from I_GWRSET.
(FLUSHBAND): Only define is __USE_GNU.
(RPROTMASK): Likewise.
(SNDPIPE): Likewise.
(MUXID_ALL): Likewise.
(struct strrecvfd): Rename fill to __fill.
Adhemerval Zanella
Joseph Myers
Szabolcs Nagy
Ulrich Drepper
Ulrich Drepper