parse_ls_color stores color_indicator[C_LINK].string as a pointer into
the color_buf buffer, then frees color_buf on the unparsable-value path.
The "ln=target" check after the parse loop still read that freed buffer,
e.g. with LS_COLORS='ln=target:x'. Skip it on the failure path, where
color output is disabled anyway and the buffer is gone.
* src/ls.c (parse_ls_color): Return after freeing color_buf on failure,
so the color_indicator[C_LINK] check only runs when the buffer is live.
* tests/ls/color-symlink-target.sh: New test.
* tests/local.mk (all_tests): Add it.
* NEWS: Mention the fix.
Link: https://github.com/coreutils/coreutils/pull/352
On GNU/Hurd, this test would get stuck in an infinite loop like the
following:
+ kill -s INFO 16535
+ sleep .01
+ kill -s INFO 16535
+ sleep .01
+ kill -s INFO 16535
+ sleep .01
[...]
The signals being sent that frequently seem to prevent 'dd' from
performing writes, since it outputs the following during that time:
0+0 records in
0+0 records out
0 bytes copied, 7161.11 s, 0.0 kB/s
0+0 records in
0+0 records out
0 bytes copied, 7161.88 s, 0.0 kB/s
[...]
* tests/dd/stats.sh: Use a larger interval between signals on GNU/Hurd.
`strace -e tee true` will run the tee command on cygwin,
rather than filtering for the tee syscall.
Reported by Bruno Haible on Cygwin 2.9.0
* init.cfg: Use -e trace=foo where foo might be a command,
so that strace probes will fail rather than invoke foo on Cygwin.
* tests/cat/cat-distinct-err.sh: Likewise.
* tests/cat/splice.sh: Likewise.
* tests/ls/no-cap.sh: Likewise.
* tests/misc/yes.sh: Likewise.
* tests/mv/atomic.sh: Likewise.
* tests/mv/atomic2.sh: Likewise.
* tests/nproc/nproc-quota.sh: Likewise.
* tests/pwd/pwd-long.sh: Likewise.
* tests/stty/stty.sh: Likewise.
* tests/tail/debug.sh: Likewise.
* tests/tail/inotify-only-regular.sh: Likewise.
* tests/sort/sort-NaN-infloop.sh: Up the timeout to about 240x
a "standard" system, as this was seen to timeout on a
mipsel VM using QEMU emulation.
Reported by Bruno Haible.
* tests/dd/no-allocate.sh: Expand memory restriction
(and allocated buffers) by 6MiB, to give more head-room
to the shell.
Reported and tested by Bruno Haible on OpenIndiana (from 2025).
On GNU/Hurd, file descriptor limits set via 'ulimit -n' seem to be
ineffective when set below the default:
$ (ulimit -n; ulimit -n 512; ulimit -n; \
tee $(seq 1024) < /dev/null; echo $?)
1024
512
tee: 1022: Too many open files
tee: 1023: Too many open files
tee: 1024: Too many open files
1
* tests/sort/sort-merge-fdlimit.sh: Don't fail if using 'ulimit -n' to
decrease the file descriptor limit is ineffective.
* tests/stat/stat-mount.sh: LeakSanitizer requires /proc access
for instrumented binaries, so ensure we're checking coreutils
binaries, rather than shell builtins.
* tests/df/skip-rootfs.sh: Test for /init which is listed as rootfs in
/proc/self/mountinfo, but tmpfs according to statfs.
Fixes https://bugs.gnu.org/65331
This adds an ELF note as per the UAPI Group's .note.dlopen spec:
$ readelf -p .note.dlopen src/sort
String dump of section '.note.dlopen':
[ 4] y
[ a] |@FDO
[ 10] [{"feature":"random-sort",
"description":"Support for random sorting",
"priority":"required",
"soname":["libcrypto.so.3"]}]
* bootstrap.conf: Add sd-dlopen.
* src/sort.c (DLOPEN_LIBCRYPTO && HAVE_OPENSSL_MD5): Add an ELF note
with SD_ELF_NOTE_DLOPEN() so that packagers can determine the libcrypto
dependency, needed when --random-sort is specified.
* NEWS: Mention the Build-related change.
Simplify cut_fields_mb_any() used with:
-w in a multi-byte locale, including UTF-8
--w=trimmed in any locale, including C
-f -d CHAR in non utf-8 multi-byte locale
-f -d CHAR in utf-8 with certain invalid CHARs
This is a net reducation of 24 lines,
and is performance neutral.
(cut_fields_mb_any): Use persistent first field and output state.
This allows using the existing field output helpers and
avoiding separate first field and line completion handling.
(cut_fields_bytesearch): Handle blanks explicitly.
(finish_current_line): Remove redundant condition.
There were no significant performance changes from the test cases
in commit a325c9978, but adding additional whitespace delimited cases
shows significant improvement. Below, with -f1, typical space delimited
input shows 4x more throughput, while the long line test shows 17x more.
Performance was tested with GCC 16 on an i7-5600U with
CFLAGS='-march=native -O3 -flto' with the following setup:
$ yes | head -n10M > sl.in
$ yes $(yes eeeaae | head -n10K | paste -s -d' ') | head -n10K > llw.in
$ yes $(yes eeeaae | head -n9 | paste -s -d' ') | head -n1M > asw.in
$ yes $(yes éééááé | head -n9 | paste -s -d$'\xe2\x80\x83') |
head -n1M > mbw.in
for type in sl llw asw mbw; do
cat $type.in >/dev/null;
for imp in '-before' '-after'; do
echo ============ "${imp:-base}" $type ==============;
for d in -w; do
fields='-f1 -f10 -f100'
test "$d" = "-b" && { fields='-b1 -b10 -b100'; d=''; }
test "$d" = "-c" && { fields='-c1 -c10 -c100'; d=''; }
for f in $fields; do
for loc in C.UTF-8; do
# Skip -b for UTF-8 as no different
test "$loc" = C.UTF-8 && echo "$f" | grep -q -- -b \
&& continue
# Skip multi-byte delimiter for C and not allowed
test "$loc" = C && test $(echo -n "$d" | wc -c) -ge 4 \
&& continue
bin=src/cut${imp}
LC_ALL=$loc $bin $f $d /dev/null 2>/dev/null &&
hyperfine --warmup=2 -m2 -M6 \
"LC_ALL=$loc $bin $f $d $type.in >/dev/null" ||
printf 'Benchmark 1: %s\n unsupported\n\n' \
"LC_ALL=$loc $bin $f $d $type.in >/dev/null"
done;
done;
done;
done;
done
After a little post-processing of the results, we get:
-- src/cut-before
| command | sl | llw | asw | mbw |
| --------------- | -------- | -------- | -------- | -------- |
| UTF8 -f1 -w | 96.2 ms | 1.916 s | 190.6 ms | 906.5 ms |
| UTF8 -f10 -w | 104.6 ms | 1.921 s | 187.7 ms | 845.6 ms |
| UTF8 -f100 -w | 104.4 ms | 1.916 s | 187.4 ms | 846.4 ms |
-- src/cut-after
| command | sl | llw | asw | mbw |
| --------------- | -------- | -------- | -------- | -------- |
| UTF8 -f1 -w | 88.7 ms | 112.4 ms | 43.5 ms | 146.3 ms |
| UTF8 -f10 -w | 95.2 ms | 114.4 ms | 193.5 ms | 848.4 ms |
| UTF8 -f100 -w | 94.8 ms | 130.6 ms | 194.4 ms | 848.1 ms |
* src/cut.c (scan_mb_line_end): A new function to skip
to EOL once we have exhausted the field selection.
(cut_fields_mb_any): Only skip after recognizing a field delimiter.
* tests/cut/cut.pl: Add test cases.
* tests/cut/mb-non-utf8.sh: Likewise.
* NEWS: Mention the improvement.
Adding a couple of inlining hints to the compiler
gives a 30% boost in raw character processing performance
in cut_fields_mb_any().
$ yes $(yes eeeaae | head -n10K | paste -s -d,) | head -n10K > ll.in
$ time LC_ALL=C.UTF-8 src/cut-before -f1 -w ll.in >/dev/null
real 0m2.279s
$ time LC_ALL=C.UTF-8 src/cut-after -f1 -w ll.in >/dev/null
real 0m1.619s
* src/cut.c (scan_mb_blank_field): Tag as inline.
(scan_mb_delim_field): Likewise.
Problem reported by Pádraig Brady (bug#81776#16).
* src/copy.c (handle_clone_fail): Be more cautious if
ioctl+FICLONE failed with EIO; do not fall back to read+write,
even if --reflink=auto is in effect.
* src/copy.h: Use errno values for syscall failures, and
negative values otherwise. That way, debugging output can
contain more info when clone or offload ops fail.
All uses changed.
Assume that XFS allocation groups can also fail with EDQUOT etc.,
so treat those failures like ENOSPC.
* src/copy.c (is_terminal_error): Remove; no longer used.
(handle_clone_fail): Remove DEST_DESC arg; no longer needed.
All uses changed. Simplify by acting as if is_terminal_error
always returns false. Change "!= 0" to "< 0" when either will do
for a syscall, as a matter of convention.
reflinking can fail due to failure to allocate metadata,
while a standard copy may succeed.
* src/copy.c (is_terminal_error): Remove ENOSPC from the list.
Fixes https://bugs.gnu.org/81776
It seems that NetBSD requires a fifo to be open, not merely opened,
to ensure a subsequent open proceeds. Immediately closing the opened
fifo caused intermittent failures on NetBSD 10.0. While that seems
like a bug in NetBSD, we can avoid the issue.
* tests/tail/pid-pipe.sh: Hold the fifo open until our writer
has opened it. This also allows specifying a single timeout
(of 12.7s), rather than 2 separate timeouts.
eq_mode(), which is used to determine if tcsetattr() was successful,
failed unnecessarily after speed setting under certain circumstances:
1. the kernel MAY round the speed to the nearest supported value,
so allow a variation of +/- 1/4 bit time per character from
what was originally requested.
2. There MAY be more than one internal binary encoding of a specific
speed (e.g. on Linux). On systems where that information
affects c_cflag, this may introduce inconsistencies;
filter out the known speed-related flags in c_cflag.
3. It is undefined in POSIX if cfsetispeed (..., 0) is retained or
normalized across tcsetattr () ... tcgetattr ().
If cfgetispeed () returns 0, compare the output speed instead.
* src/stty.c (eq_mode): Mask of some of the speed related bits
to avoid checking them for exact equality.
(speed_matches): A new helper to apply the more lenient matching
of requested and resulting speeds.
* NEWS: Mention the bug fix.
* src/printenv.c (main): Make sure quoting is disabled when we have
arguments and standard output is not a terminal.
* tests/misc/printenv.sh: Adjust single argument test cases. Add
multiple argument test cases.
* tests/misc/tty-quoting.sh: Add test cases to check if isatty is used.
* doc/coreutils.texi (printenvAlwaysQuoted): Remove macro.
(printenv invocation): Describe the adjusted quoting behavior.
(env invocation): Move the text from printenvAlwaysQuoted here since it
is no longer used for 'printenv'.
* tests/misc/no-fork.sh: New file.
* tests/local.mk (all_tests): Add the new test.
* tests/misc/stdbuf.sh: Remove a test that is redundant with the new
test.
* tests/printf/printf-cov.pl ($ESCAPE_CHARS, $ESCAPE_OUTPUT): New
variables derived from the existing 'escape-1' test.
(@Tests): Add some additional escaping tests and test %b.
init.cfg (cmd_exe): Provide a function to find the command under test,
whether in the $PATH or in the standard build directory.
* tests/env/env0-from.sh: Use it instead of `command -v`.
* tests/misc/coreutils.sh: Likewise.
* tests/nproc/nproc-quota.sh: Likewise.
* tests/env/env.sh: Likewise. Also remove non robust use of sed.
* tests/stat/stat-printf.pl: The format is interpreted as it is printed,
so whatever precedes an invalid directive is written before the
diagnostic, and --format adds no trailing newline when it stops early.
The existing err-1 and err-2 cases put the directive at the start of the
format, so nothing covered this.
See https://github.com/uutils/coreutils/issues/14101
Link: https://github.com/coreutils/coreutils/pull/333
