* tests/sort/sort-NaN-infloop.sh: Up the timeout to about 240x
a "standard" system, as this was seen to timeout on a
mipsel VM using QEMU emulation.
Reported by Bruno Haible.
* tests/dd/no-allocate.sh: Expand memory restriction
(and allocated buffers) by 6MiB, to give more head-room
to the shell.
Reported and tested by Bruno Haible on OpenIndiana (from 2025).
On GNU/Hurd, file descriptor limits set via 'ulimit -n' seem to be
ineffective when set below the default:
$ (ulimit -n; ulimit -n 512; ulimit -n; \
tee $(seq 1024) < /dev/null; echo $?)
1024
512
tee: 1022: Too many open files
tee: 1023: Too many open files
tee: 1024: Too many open files
1
* tests/sort/sort-merge-fdlimit.sh: Don't fail if using 'ulimit -n' to
decrease the file descriptor limit is ineffective.
* tests/stat/stat-mount.sh: LeakSanitizer requires /proc access
for instrumented binaries, so ensure we're checking coreutils
binaries, rather than shell builtins.
* tests/df/skip-rootfs.sh: Test for /init which is listed as rootfs in
/proc/self/mountinfo, but tmpfs according to statfs.
Fixes https://bugs.gnu.org/65331
This adds an ELF note as per the UAPI Group's .note.dlopen spec:
$ readelf -p .note.dlopen src/sort
String dump of section '.note.dlopen':
[ 4] y
[ a] |@FDO
[ 10] [{"feature":"random-sort",
"description":"Support for random sorting",
"priority":"required",
"soname":["libcrypto.so.3"]}]
* bootstrap.conf: Add sd-dlopen.
* src/sort.c (DLOPEN_LIBCRYPTO && HAVE_OPENSSL_MD5): Add an ELF note
with SD_ELF_NOTE_DLOPEN() so that packagers can determine the libcrypto
dependency, needed when --random-sort is specified.
* NEWS: Mention the Build-related change.
Simplify cut_fields_mb_any() used with:
-w in a multi-byte locale, including UTF-8
--w=trimmed in any locale, including C
-f -d CHAR in non utf-8 multi-byte locale
-f -d CHAR in utf-8 with certain invalid CHARs
This is a net reducation of 24 lines,
and is performance neutral.
(cut_fields_mb_any): Use persistent first field and output state.
This allows using the existing field output helpers and
avoiding separate first field and line completion handling.
(cut_fields_bytesearch): Handle blanks explicitly.
(finish_current_line): Remove redundant condition.
There were no significant performance changes from the test cases
in commit a325c9978, but adding additional whitespace delimited cases
shows significant improvement. Below, with -f1, typical space delimited
input shows 4x more throughput, while the long line test shows 17x more.
Performance was tested with GCC 16 on an i7-5600U with
CFLAGS='-march=native -O3 -flto' with the following setup:
$ yes | head -n10M > sl.in
$ yes $(yes eeeaae | head -n10K | paste -s -d' ') | head -n10K > llw.in
$ yes $(yes eeeaae | head -n9 | paste -s -d' ') | head -n1M > asw.in
$ yes $(yes éééááé | head -n9 | paste -s -d$'\xe2\x80\x83') |
head -n1M > mbw.in
for type in sl llw asw mbw; do
cat $type.in >/dev/null;
for imp in '-before' '-after'; do
echo ============ "${imp:-base}" $type ==============;
for d in -w; do
fields='-f1 -f10 -f100'
test "$d" = "-b" && { fields='-b1 -b10 -b100'; d=''; }
test "$d" = "-c" && { fields='-c1 -c10 -c100'; d=''; }
for f in $fields; do
for loc in C.UTF-8; do
# Skip -b for UTF-8 as no different
test "$loc" = C.UTF-8 && echo "$f" | grep -q -- -b \
&& continue
# Skip multi-byte delimiter for C and not allowed
test "$loc" = C && test $(echo -n "$d" | wc -c) -ge 4 \
&& continue
bin=src/cut${imp}
LC_ALL=$loc $bin $f $d /dev/null 2>/dev/null &&
hyperfine --warmup=2 -m2 -M6 \
"LC_ALL=$loc $bin $f $d $type.in >/dev/null" ||
printf 'Benchmark 1: %s\n unsupported\n\n' \
"LC_ALL=$loc $bin $f $d $type.in >/dev/null"
done;
done;
done;
done;
done
After a little post-processing of the results, we get:
-- src/cut-before
| command | sl | llw | asw | mbw |
| --------------- | -------- | -------- | -------- | -------- |
| UTF8 -f1 -w | 96.2 ms | 1.916 s | 190.6 ms | 906.5 ms |
| UTF8 -f10 -w | 104.6 ms | 1.921 s | 187.7 ms | 845.6 ms |
| UTF8 -f100 -w | 104.4 ms | 1.916 s | 187.4 ms | 846.4 ms |
-- src/cut-after
| command | sl | llw | asw | mbw |
| --------------- | -------- | -------- | -------- | -------- |
| UTF8 -f1 -w | 88.7 ms | 112.4 ms | 43.5 ms | 146.3 ms |
| UTF8 -f10 -w | 95.2 ms | 114.4 ms | 193.5 ms | 848.4 ms |
| UTF8 -f100 -w | 94.8 ms | 130.6 ms | 194.4 ms | 848.1 ms |
* src/cut.c (scan_mb_line_end): A new function to skip
to EOL once we have exhausted the field selection.
(cut_fields_mb_any): Only skip after recognizing a field delimiter.
* tests/cut/cut.pl: Add test cases.
* tests/cut/mb-non-utf8.sh: Likewise.
* NEWS: Mention the improvement.
Adding a couple of inlining hints to the compiler
gives a 30% boost in raw character processing performance
in cut_fields_mb_any().
$ yes $(yes eeeaae | head -n10K | paste -s -d,) | head -n10K > ll.in
$ time LC_ALL=C.UTF-8 src/cut-before -f1 -w ll.in >/dev/null
real 0m2.279s
$ time LC_ALL=C.UTF-8 src/cut-after -f1 -w ll.in >/dev/null
real 0m1.619s
* src/cut.c (scan_mb_blank_field): Tag as inline.
(scan_mb_delim_field): Likewise.
Problem reported by Pádraig Brady (bug#81776#16).
* src/copy.c (handle_clone_fail): Be more cautious if
ioctl+FICLONE failed with EIO; do not fall back to read+write,
even if --reflink=auto is in effect.
* src/copy.h: Use errno values for syscall failures, and
negative values otherwise. That way, debugging output can
contain more info when clone or offload ops fail.
All uses changed.
Assume that XFS allocation groups can also fail with EDQUOT etc.,
so treat those failures like ENOSPC.
* src/copy.c (is_terminal_error): Remove; no longer used.
(handle_clone_fail): Remove DEST_DESC arg; no longer needed.
All uses changed. Simplify by acting as if is_terminal_error
always returns false. Change "!= 0" to "< 0" when either will do
for a syscall, as a matter of convention.
reflinking can fail due to failure to allocate metadata,
while a standard copy may succeed.
* src/copy.c (is_terminal_error): Remove ENOSPC from the list.
Fixes https://bugs.gnu.org/81776
It seems that NetBSD requires a fifo to be open, not merely opened,
to ensure a subsequent open proceeds. Immediately closing the opened
fifo caused intermittent failures on NetBSD 10.0. While that seems
like a bug in NetBSD, we can avoid the issue.
* tests/tail/pid-pipe.sh: Hold the fifo open until our writer
has opened it. This also allows specifying a single timeout
(of 12.7s), rather than 2 separate timeouts.
eq_mode(), which is used to determine if tcsetattr() was successful,
failed unnecessarily after speed setting under certain circumstances:
1. the kernel MAY round the speed to the nearest supported value,
so allow a variation of +/- 1/4 bit time per character from
what was originally requested.
2. There MAY be more than one internal binary encoding of a specific
speed (e.g. on Linux). On systems where that information
affects c_cflag, this may introduce inconsistencies;
filter out the known speed-related flags in c_cflag.
3. It is undefined in POSIX if cfsetispeed (..., 0) is retained or
normalized across tcsetattr () ... tcgetattr ().
If cfgetispeed () returns 0, compare the output speed instead.
* src/stty.c (eq_mode): Mask of some of the speed related bits
to avoid checking them for exact equality.
(speed_matches): A new helper to apply the more lenient matching
of requested and resulting speeds.
* NEWS: Mention the bug fix.
* src/printenv.c (main): Make sure quoting is disabled when we have
arguments and standard output is not a terminal.
* tests/misc/printenv.sh: Adjust single argument test cases. Add
multiple argument test cases.
* tests/misc/tty-quoting.sh: Add test cases to check if isatty is used.
* doc/coreutils.texi (printenvAlwaysQuoted): Remove macro.
(printenv invocation): Describe the adjusted quoting behavior.
(env invocation): Move the text from printenvAlwaysQuoted here since it
is no longer used for 'printenv'.
* tests/misc/no-fork.sh: New file.
* tests/local.mk (all_tests): Add the new test.
* tests/misc/stdbuf.sh: Remove a test that is redundant with the new
test.
* tests/printf/printf-cov.pl ($ESCAPE_CHARS, $ESCAPE_OUTPUT): New
variables derived from the existing 'escape-1' test.
(@Tests): Add some additional escaping tests and test %b.
init.cfg (cmd_exe): Provide a function to find the command under test,
whether in the $PATH or in the standard build directory.
* tests/env/env0-from.sh: Use it instead of `command -v`.
* tests/misc/coreutils.sh: Likewise.
* tests/nproc/nproc-quota.sh: Likewise.
* tests/env/env.sh: Likewise. Also remove non robust use of sed.
* tests/stat/stat-printf.pl: The format is interpreted as it is printed,
so whatever precedes an invalid directive is written before the
diagnostic, and --format adds no trailing newline when it stops early.
The existing err-1 and err-2 cases put the directive at the start of the
format, so nothing covered this.
See https://github.com/uutils/coreutils/issues/14101
Link: https://github.com/coreutils/coreutils/pull/333
parse_tab_stops accepts a comma or any blank as a separator, but the
tests only ever used a comma or a space, so a tab separator was
untested. unexpand had no coverage of multi-stop lists at all.
Missing case identified here:
https://github.com/uutils/coreutils/pull/14057
* tests/expand/expand.pl (t4a): New test, using tabs as separators.
* tests/unexpand/unexpand.pl (tabs-3, tabs-4, tabs-5, tabs-6): New
tests, for comma-, space- and tab-separated lists and for a leading
separator.
Link: https://github.com/coreutils/coreutils/pull/332
Avoid unsetenv() on a non standard environment,
as FreeBSD 14 and 15 at least fail when manipulating
such an environment with unsetenv().
$ printf 'A=1\0A=2\0foo\0' |
src/env --env0-from=- src/env foo=bar | grep foo
foo=bar
foo
$ printf 'A=1\0A=2\0foo\0' |
src/env --env0-from=- src/env -u foo
env: environment corrupt; missing value for foo
env: cannot unset ‘foo’: Bad address
$ printf 'A=1\0A=2\0foo\0' |
src/env --env0-from=- src/env -u bar
env: environment corrupt; missing value for foo
env: cannot unset ‘bar’: Bad address
* tests/env/env0-from.sh: Use routines internal to env(1)
to manipulate the environment, only delegating the
executed env(1) to adding any implicit environment variables.
Reported by Bruno Haible.
The following error was seen on a Debian 6.0 virtual machine using perl
5.10.1:
Bareword found where operator expected at \
./tests/join/join.pl line 327, near "tr/_/-/r"
syntax error at ./tests/join/join.pl line 327, near "tr/_/-/r"
Execution of ./tests/join/join.pl aborted due to compilation errors.
FAIL tests/join/join.pl (exit status: 255)
* tests/join/join.pl: Don't use the /r modifier which was not introduced
until perl 5.13.7.