`ptsname_r' is declared in stdlib.h to only accept a `nonnull'
second argument and therefore GCC may choose to make optimizations
based on the assumption that this argument is NULL. This means
that potentially, GCC can optimize away the NULL check at some
point in the future. Since this is a programming interface, we
might as well remove the NULL check ourselves.
This also warrants a change to the `ptsname_r' manual page that
must be submitted to the corresponding mailing list.
In addition, remove the NULL buffer test in login/tst-ptsname.c.
If the fd refers to a terminal device, but not a pty master, the
TIOCGPTN ioctl returns with ENOTTY. This error is not caught, and the
possibly undefined buffer passed to ptsname_r is sent directly to the
stat64 syscall.
Fix this by using a fallback to the old method only if the TIOCGPTN
ioctl fails with EINVAL. This also fix the return value in that specific
case (it return ENOENT without this patch).
Also add tests to the ptsname_r function (and ptsname at the same time).
Note: this is Debian bug#741482, reported by Jakub Wilk <jwilk@debian.org>
Joseph Myers
Arjun Shankar
Aurelien Jarno