Neither the <dlfcn.h> entry points, nor lazy symbol resolution, nor
initial shared library load-up, are cancellation points, so ld.so
should exclusively use I/O primitives that are not cancellable. We
currently achieve this by having the cancellation hooks compile as
no-ops when IS_IN(rtld); this patch changes to using exclusively
_nocancel primitives in the source code instead, which makes the
intent clearer and significantly reduces the amount of code compiled
under IS_IN(rtld) as well as IS_IN(libc) -- in particular,
elf/Makefile no longer thinks we require a copy of unwind.c in
rtld-libc.a. (The older mechanism is preserved as a backstop.)
The bulk of the change is splitting up the files that define the
_nocancel I/O functions, so they don't also define the variants that
*are* cancellation points; after which, the existing logic for picking
out the bits of libc that need to be recompiled as part of ld.so Just
Works. I did this for all of the _nocancel functions, not just the
ones used by ld.so, for consistency.
fcntl was a little tricky because it's only a cancellation point for
certain opcodes (F_SETLKW(64), which can block), and the existing
__fcntl_nocancel wasn't applying the FCNTL_ADJUST_CMD hook, which
strikes me as asking for trouble, especially as the only nontrivial
definition of FCNTL_ADJUST_CMD (for powerpc64) changes F_*LK* opcodes.
To fix this, fcntl_common moves to fcntl_nocancel.c along with
__fcntl_nocancel, and changes its name to the extern (but hidden)
symbol __fcntl_nocancel_adjusted, so that regular fcntl can continue
calling it. __fcntl_nocancel now applies FCNTL_ADJUST_CMD; so that
both both fcntl.c and fcntl_nocancel.c can see it, the only nontrivial
definition moves from sysdeps/u/s/l/powerpc/powerpc64/fcntl.c to
.../powerpc64/sysdep.h and becomes entirely a macro, instead of a macro
that calls an inline function.
The nptl version of libpthread also changes a little, because its
"compat-routines" formerly included files that defined all the
_nocancel functions it uses; instead of continuing to duplicate them,
I exported the relevant ones from libc.so as GLIBC_PRIVATE. Since the
Linux fcntl.c calls a function defined by fcntl_nocancel.c, it can no
longer be used from libpthread.so; instead, introduce a custom
forwarder, pt-fcntl.c, and export __libc_fcntl from libc.so as
GLIBC_PRIVATE. The nios2-linux ABI doesn't include a copy of vfork()
in libpthread, and it was handling that by manipulating
libpthread-routines in .../linux/nios2/Makefile; it is cleaner to do
what other such ports do, and have a pt-vfork.S that defines no symbols.
Right now, it appears that Hurd does not implement _nocancel I/O, so
sysdeps/generic/not-cancel.h will forward everything back to the
regular functions. This changed the names of some of the functions
that sysdeps/mach/hurd/dl-sysdep.c needs to interpose.
* elf/dl-load.c, elf/dl-misc.c, elf/dl-profile.c, elf/rtld.c
* sysdeps/unix/sysv/linux/dl-sysdep.c
Include not-cancel.h. Use __close_nocancel instead of __close,
__open64_nocancel instead of __open, __read_nocancel instead of
__libc_read, and __write_nocancel instead of __libc_write.
* csu/check_fds.c (check_one_fd)
* sysdeps/posix/fdopendir.c (__fdopendir)
* sysdeps/posix/opendir.c (__alloc_dir): Use __fcntl_nocancel
instead of __fcntl and/or __libc_fcntl.
* sysdeps/unix/sysv/linux/pthread_setname.c (pthread_setname_np)
* sysdeps/unix/sysv/linux/pthread_getname.c (pthread_getname_np)
* sysdeps/unix/sysv/linux/i386/smp.h (is_smp_system):
Use __open64_nocancel instead of __open_nocancel.
* sysdeps/unix/sysv/linux/not-cancel.h: Move all of the
hidden_proto declarations to the end and issue them if either
IS_IN(libc) or IS_IN(rtld).
* sysdeps/unix/sysv/linux/Makefile [subdir=io] (sysdep_routines):
Add close_nocancel, fcntl_nocancel, nanosleep_nocancel,
open_nocancel, open64_nocancel, openat_nocancel, pause_nocancel,
read_nocancel, waitpid_nocancel, write_nocancel.
* io/Versions [GLIBC_PRIVATE]: Add __libc_fcntl,
__fcntl_nocancel, __open64_nocancel, __write_nocancel.
* posix/Versions: Add __nanosleep_nocancel, __pause_nocancel.
* nptl/pt-fcntl.c: New file.
* nptl/Makefile (pthread-compat-wrappers): Remove fcntl.
(libpthread-routines): Add pt-fcntl.
* include/fcntl.h (__fcntl_nocancel_adjusted): New function.
(__libc_fcntl): Remove attribute_hidden.
* sysdeps/unix/sysv/linux/fcntl.c (__libc_fcntl): Call
__fcntl_nocancel_adjusted, not fcntl_common.
(__fcntl_nocancel): Move to new file fcntl_nocancel.c.
(fcntl_common): Rename to __fcntl_nocancel_adjusted; also move
to fcntl_nocancel.c.
* sysdeps/unix/sysv/linux/fcntl_nocancel.c: New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/fcntl.c: Remove file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/sysdep.h:
Define FCNTL_ADJUST_CMD here, as a self-contained macro.
* sysdeps/unix/sysv/linux/close.c: Move __close_nocancel to...
* sysdeps/unix/sysv/linux/close_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/nanosleep.c: Move __nanosleep_nocancel to...
* sysdeps/unix/sysv/linux/nanosleep_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/open.c: Move __open_nocancel to...
* sysdeps/unix/sysv/linux/open_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/open64.c: Move __open64_nocancel to...
* sysdeps/unix/sysv/linux/open64_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/openat.c: Move __openat_nocancel to...
* sysdeps/unix/sysv/linux/openat_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/openat64.c: Move __openat64_nocancel to...
* sysdeps/unix/sysv/linux/openat64_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/pause.c: Move __pause_nocancel to...
* sysdeps/unix/sysv/linux/pause_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/read.c: Move __read_nocancel to...
* sysdeps/unix/sysv/linux/read_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/waitpid.c: Move __waitpid_nocancel to...
* sysdeps/unix/sysv/linux/waitpid_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/write.c: Move __write_nocancel to...
* sysdeps/unix/sysv/linux/write_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/nios2/Makefile: Don't override
libpthread-routines.
* sysdeps/unix/sysv/linux/nios2/pt-vfork.S: New file which
defines nothing.
* sysdeps/mach/hurd/dl-sysdep.c: Define __read instead of
__libc_read, and __write instead of __libc_write. Define
__open64 in addition to __open.
The NO_CANCELLATION macro is used currently on generic headers to
define non cancellable syscalls and on Linux fcntl to implement the non
cancellable variation. Former should be single-handled by not-cancel.h
header and former could be simplified build both cancellable and non
cancellable for default objects and alias the non-cancellable to default
one for rtld ones (since Linux already support cancellation as default).
This patch thus removes the NO_CANCELLATION macro and its usage. The
generic non cancellable fcntl is route to internal fcntl.
Checked on x86_64-linux-gnu and i686-linux-gnu. Also checked with
a build again major ABIs.
* sysdeps/generic/not-cancel.h (NO_CANCELLATION): Remove macro.
* sysdeps/unix/sysv/linux/aarch64/sysdep-cancel.h (NO_CANCELLATION):
Likewise.
* sysdeps/unix/sysv/linux/alpha/sysdep-cancel.h (NO_CANCELLATION):
Likewise.
* sysdeps/unix/sysv/linux/arm/sysdep-cancel.h (NO_CANCELLATION):
Likewise.
* sysdeps/unix/sysv/linux/hppa/sysdep-cancel.h (NO_CANCELLATION):
Likewise.
* sysdeps/unix/sysv/linux/i386/sysdep-cancel.h (NO_CANCELLATION):
Likewise.
* sysdeps/unix/sysv/linux/ia64/sysdep-cancel.h (NO_CANCELLATION):
Likewise.
* sysdeps/unix/sysv/linux/m68k/sysdep-cancel.h (NO_CANCELLATION):
Likewise.
* sysdeps/unix/sysv/linux/microblaze/sysdep-cancel.h
(NO_CANCELLATION): Likewise.
* sysdeps/unix/sysv/linux/mips/sysdep-cancel.h (NO_CANCELLATION):
Likewise.
* sysdeps/unix/sysv/linux/nios2/sysdep-cancel.h (NO_CANCELLATION):
Likewise.
* sysdeps/unix/sysv/linux/powerpc/sysdep-cancel.h (NO_CANCELLATION):
Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/sysdep-cancel.h
(NO_CANCELLATION): Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/sysdep-cancel.h
(NO_CANCELLATION): Likewise.
* sysdeps/unix/sysv/linux/sh/sysdep-cancel.h (NO_CANCELLATION):
Likewise
* sysdeps/unix/sysv/linux/sparc/sysdep-cancel.h (NO_CANCELLATION):
Likewise.
* sysdeps/unix/sysv/linux/tile/sysdep-cancel.h (NO_CANCELLATION):
Likewise.
* sysdeps/unix/sysv/linux/x86_64/sysdep-cancel.h (NO_CANCELLATION):
Likewise.
* sysdeps/unix/sysv/linux/not-cancel.h (__fcntl_nocancel): Add
hidden prototype.
* sysdeps/unix/sysv/linux/fcntl.c (__fcntl_nocancel): Define only
for !IS_IN (rtld) and remove NO_CANCELLATION usage.
This patch consolidate the remaning non cancellable syscall definitions
on not-cancel.h header. They are:
* __fcntl_nocancel: Moved from fcntl.h to not-cancel.h.
* __sigsuspend_nocancel: Removed since 988f991b50 it is not used or
defined anymore.
* __nanosleep_nocancel: Removed since 6f33fd046b it is defined on
not-cancel.h.
Now all non-cancellable syscall definition are defined on not-cancel
(the only exceptions is the stdio symbol __fxprintf_nocancel which
uses non cancellable open and it is used on getopt implementation).
Checked on x86_64-linux-gnu and with build-many-glibc.py.
* include/fcntl.h (__fcntl_nocancel): Remove definition.
* include/signal.h (__sigsuspend_nocancel): Likewise.
* include/time.h (__nanosleep_nocancel): Likewise.
* sysdeps/generic/not-cancel.h (__fcntl_nocancel): New macro.
* login/utmp_file.c: Include non cancellable syscall header.
* sysdeps/unix/sysv/linux/not-cancel.h (__fcntl_nocancel): New
prototype.
There is no current internal usage for non cancellable sigsuspend calls.
Checked on x86_64-linux-gnu, x86_64-linux-gnu-x32, and i686-linux-gnu.
* sysdeps/generic/not-cancel.h (sigsuspend_not_cancel): remove
macro.
* sysdeps/mach/hurd/sigsuspend.c (sigsuspend_not_cancel): remove
alias.
* sysdeps/unix/sysv/linux/not-cancel.h (sigsuspend_not_cancel):
likewise.
This patch consolidates all the non cancellable nanosleep calls to use
the __nanosleep_nocancel identifier. For non cancellable targets it will
be just a macro to call the default respective symbol while on Linux
will be a internal one.
Checked on x86_64-linux-gnu, x86_64-linux-gnu-x32, and i686-linux-gnu.
* nptl/pthread_mutex_timedlock.c (__pthread_mutex_timedlock): Replace
nanosleep_not_cancel with __nanosleep_nocancel.
* sysdeps/generic/not-cancel.h (nanosleep_not_cancel): Remove macro.
(__nanosleep_nocancel): New macro.
* sysdeps/unix/sysv/linux/nanosleep.c (__nanosleep_nocancel): New
function.
* sysdeps/unix/sysv/linux/not-cancel.h (nanosleep_not_cancel): Remove
macro.
(__nanosleep_nocancel): New prototype.
This patch consolidates all the non cancellable pause calls to use
the __pause_nocancel identifier. For non cancellable targets it will
be just a macro to call the default respective symbol while on Linux
will be a internal one.
Checked on x86_64-linux-gnu, x86_64-linux-gnu-x32, and i686-linux-gnu.
* nptl/pthread_mutex_lock.c (__pthread_mutex_lock_full): Replace
pause_not_cancel with __pause_nocancel.
* sysdeps/generic/not-cancel.h (pause_not_cancel): Remove macro.
(__pause_nocancel): New macro.
* sysdeps/unix/sysv/linux/not-cancel.h (pause_not_cancel): Remove
macro.
(__pause_nocancel): New prototype.
* sysdeps/unix/sysv/linux/pause.c (__pause_nocancel): New function.
This patch consolidates all the non cancellable waitpid calls to use
the __waitpid_nocancel identifier. For non cancellable targets it will
be just a macro to call the default respective symbol while on Linux
will be a internal one.
Checked on x86_64-linux-gnu, x86_64-linux-gnu-x32, and i686-linux-gnu.
* libio/ioopen.c (_IO_waitpid): Replace waitpid_not_cancel with
__waitpid_nocancel.
* sysdeps/generic/not-cancel.h (waitpid_not_cancel): Remove macro.
(__waitpid_nocancel): New macro.
* sysdeps/unix/sysv/linux/not-cancel.h (waitpid_not_cancel): Remove
macro.
(__waitpid_nocancel): Replace macro with a function.
* sysdeps/unix/sysv/linux/waitpid.c (__waitpid_nocancel): New
function.
This patch consolidates all the non cancellable fcntl calls to use
the __fcntl_nocancel identifier. For non cancellable targets it will
be just a macro to call the default respective symbol while on Linux
will be a internal one.
Since its prototype is already defined at internal fcntl.h header, it
is removed from not-cancel.h one.
Checked on x86_64-linux-gnu, x86_64-linux-gnu-x32, and i686-linux-gnu.
* login/utmp_file.c (timeout_handler): Replace fcntl_not_cancel with
__fcntl_nocancel.
* sysdeps/generic/not-cancel.h (fcntl_not_cancel): Remove macro.
* sysdeps/unix/sysv/linux/not-cancel.h (fcntl_not_cancel): Likewise.
This patch consolidates all the non cancellable writev calls to use
the __writev_nocancel identifier. For non cancellable targets it will
be just a macro to call the default respective symbol while on Linux
will be a internal one.
Checked on x86_64-linux-gnu, x86_64-linux-gnu-x32, and i686-linux-gnu.
* gmon/gmon.c (write_hist): Replace writev_not_cancel_no_status with
__writev_nocancel_nostatus.
(write_call_graph): Likewise.
(write_bb_counts): Likewise.
* resolv/herror.c (herror): Likewise.
* sysdeps/generic/not-cancel.h (writev_not_cancel_no_status): Remove
macro.
(__writev_nocancel_nostatus): New macro.
* sysdeps/unix/sysv/linux/not-cancel.h (writev_not_cancel_no_status):
Remove macro.
(__writev_nocancel_nostatus): New function.
This patch consolidates all the non cancellable openat{64} calls to use
the __openat{64}_nocancel identifier. For non cancellable targets it will
be just a macro to call the default respective symbol while on Linux
will be a internal one.
Checked on x86_64-linux-gnu, x86_64-linux-gnu-x32, and i686-linux-gnu.
* sysdeps/generic/not-cancel.h (openat_not_cancel): Remove macro.
(openat_not_cancel_3): Likewise.
(openat64_not_cancel_3): Likewise).
(openat_not_cancel_3): Likewise).
* sysdeps/unix/sysv/linux/not-cancel.h (openat_not_cancel): Remove
macro.
(openat_not_cancel_3): Likewise.
(openat64_not_cancel): Likewise.
(openat64_not_cancel_3): Likewise.
* sysdeps/unix/sysv/linux/openat.c (__openat_nocancel): New function.
* sysdeps/unix/sysv/linux/openat64.c (__openat64_nocancel): Likewise.
* io/ftw.c (open_dir_stream): Replace openat{64}_not_cancel{_3} with
__open{64}_nocancel.
* sysdeps/mach/hurd/opendir.c (__opendirat): Likewise.
* sysdeps/posix/getcwd.c (__getcwd): Likewise.
* sysdeps/posix/opendir.c (__opendirat): Likewise.
This patch consolidates all the non cancellable write calls to use
the __write_nocancel identifier. For non cancellable targets it will
be just a macro to call the default respective symbol while on Linux
will be a internal one.
Checked on x86_64-linux-gnu, x86_64-linux-gnu-x32, and i686-linux-gnu.
* sysdeps/generic/not-cancel.h (write_not_cancel): Remove macro.
(__write_nocancel): New macro.
* sysdeps/unix/sysv/linux/not-cancel.h (__write_nocancel):
Rewrite as a function prototype.
(write_not_cancel): Remove macro.
* sysdeps/unix/sysv/linux/write.c (__write_nocancel): New function.
* gmon/gmon.c (ERR): Replace write_not_cancel with __write_nocancel.
(write_gmon): Likewise.
* libio/fileops.c (_IO_new_file_write): Likewise.
* login/utmp_file.c (pututline_file): Likewise.
(updwtmp_file): Likewise.
* stdio-common/psiginfo.c (psiginfo): Likewise.
* sysdeps/posix/spawni.c (__spawni_child): Likewise.
* sysdeps/unix/sysv/linux/gethostid.c (sethostid): Likewise.
* sysdeps/unix/sysv/linux/libc_fatal.c (backtrace_and_maps):
Likewise.
* sysdeps/unix/sysv/linux/pthread_setname.c (pthread_setname_np):
Likewise.
This patch consolidates all the non cancellable read calls to use
the __read_nocancel identifier. For non cancellable targets it will
be just a macro to call the default respective symbol while on Linux
will be a internal one.
Also, since it is used on libcrypto it is also exported in GLIBC_PRIVATE
namespace.
Checked on x86_64-linux-gnu, x86_64-linux-gnu-x32, and i686-linux-gnu.
* sysdeps/generic/not-cancel.h (read_not_cancel): Remove macro.
(__read_nocancel): New macro.
* sysdeps/unix/sysv/linux/Versions (libc) [GLIBC_PRIVATE]: Add
__read_nocancel.
* sysdeps/unix/sysv/linux/not-cancel.h (__read_nocancel): Remove
macro.
(__read_nocancel): New prototype.
* sysdeps/unix/sysv/linux/read.c (__read_nocancel): New function.
* catgets/open_catalog.c (__open_catalog): Replace read_not_cancel
with __read_nocancel.
* intl/loadmsgcat.c (read): Likewise.
* libio/fileops.c (_IO_file_read): Likewise.
* locale/loadlocale.c (_nl_load_locale): Likewise.
* login/utmp_file.c (getutent_r_file): Likewise.
(internal_getut_r): Likewise.
(getutline_r_file): Likewise.
* sysdeps/unix/sysv/linux/fips-private.h (fips_enable_p): Likewise.
* sysdeps/unix/sysv/linux/gethostid.c (gethostid): Likewise.
* sysdeps/unix/sysv/linux/getloadavg.c (getloadavg): Likewise.
* sysdeps/unix/sysv/linux/getlogin_r.c (__getlogin_r_loginuid):
Likewise.
* sysdeps/unix/sysv/linux/getsysstats.c (next_line): Likewise.
* sysdeps/unix/sysv/linux/i386/smp.h (is_smp_system): Likewise.
* sysdeps/unix/sysv/linux/ia64/has_cpuclock.c (has_cpuclock):
Likewise.
* sysdeps/unix/sysv/linux/libc_fatal.c (backtrace_and_maps):
Likewise.
* sysdeps/unix/sysv/linux/malloc-sysdep.h (check_may_shrink_heap):
Likewise.
* sysdeps/unix/sysv/linux/pthread_getname.c (pthread_getname_np):
Likewise.
* sysdeps/unix/sysv/linux/sysconf.c (__sysconf): Likewise.
This patch uses inline calls (through INLINE_SYSCALL macro) to define
the non-cancellable functions macros to avoid use of the
syscall_nocancel entrypoint.
__openat64_not_cancel): Remove prototypes.
(__openat_nocancel, __openat64_nocancel): New prototypes or defines.
(openat_not_cancel, openat_not_cancel_3, openat64_not_cancel,
openat64_not_cancel_3): Use them.
* sysdeps/generic/utmp_file.c: Use not-cancelable syscalls all
over the place. It would be allowed to have these fucntions as
cancellation points but the cleanup would be ugly and a lot of
work.
* sysdeps/generic/not-cancel.h (fcntl_not_cancel): Define.
* sysdeps/unix/sysv/linux/not-cancel.h (fcntl_not_cancel): Define.
* include/fcntl.h (__fcntl_nocancel): Declare.
* sysdeps/unix/sysv/linux/fcntl.c: New file.
* sysdeps/unix/sysv/linux/i386/fcntl.c (__libc_fcntl): Only enable
cancellation if absolutely needed.
(__fcntl_nocancel): Define.
* posix/unistd.h (gethostid): Remove __THROW. Clarify comment.
* sysdeps/unix/sysv/linux/Makefile (CFLAGS-gethostid.c): Add
-fexceptions.
* sysdeps/unix/sysv/linux/gethostid.c (gethostid): Use
extend_alloca.
* resolv/res_init.c (__res_nclose): Use close_not_cancel_no_status
instead of __close.
More cancellation handling fixups.
* sysdeps/unix/sysv/linux/not-cancel.h: Add waitpid_not_cancel.
* sysdeps/generic/not-cancel.h: Likewise.
* catgets/open_catalog.c: Use not-cancelable syscalls.
* time/Makefile (CFLAGS-getdate.c): Add -fexceptions.
* sysdeps/unix/sysv/linux/llseek.c: Must not be cancelable.
* sysdeps/unix/syscalls.list: Don't mark lseek as cancelable.
* dlfcn/dlfcn.h: Mark dlopen with __THROW again.
* io/fcntl.h: Don't mark posix_fallocate with __THROW.
* libio/fileops.c: Use not-cancelable syscalls for fclose.
* libio/iopopen.c: Use no-cancelable syscalls.
* libio/stdio.h: Mark popen and pclose with __THROW again.
* misc/Makefile (CFLAGS-syslog.c): Add -fexceptions.
* misc/syslog.c: Fix locking and cancellation cleanup handling.
* posix/unistd.h: Mark ttyname and ttyname_r again with __THROW.
* stdio-common/Makefile (CFLAGS-tmpfile.c, CFLAGS-tmpfile64.c,
CFLAGS-tempname.c): Add -fexceptions.
* stdlib/Makefile (CFLAGS-mkstemp.c): Add -fexceptions.
* string/string.h: Mark strerror and strerror_r with _THROW again.
* sysdeps/generic/unwind.inc: New file. Copied from gcc.
* sysdeps/generic/unwind-dw2.c: Update from gcc version. Remove
#ifs since we now need all the code compiled.
* sysdeps/posix/spawni.c: Use close_not_cancel instead of close.
* sysdeps/unix/closedir.c: Use not-cancelable syscalls.
* sysdeps/unix/opendir.c: Likewise.
