When only --goal is given, fmt sets the maximum width to the goal plus
ten (src/fmt.c), but nothing exercised that.
identified by https://github.com/uutils/coreutils/pull/14275
* tests/fmt/goal-option.sh: Check that "-g G" formats as "-g G -w G+10"
does, that the derived width is exactly G+10, that a goal of zero is
accepted, and that without -w the goal is still limited to 75.
Link: https://github.com/coreutils/coreutils/pull/358
* tests/cp/capability.sh: Add a getlimits_ invocation. Allow 'cp' to
fail with ENOTSUP, which occurs when trying to copy the security.selinux
extended attribute.
E.g. with podman we get default SELinux context mount options
for /dev/shm and /, resulting in all files getting context like
the following when created:
security.selinux="system_u:object_r:container_file_t:s0:c458,c677"
However you can't explicitly set that xattr:
$ touch file
$ setfattr -n "security.selinux" \
-v "system_u:object_r:container_file_t:s0:c458,c677" file
setfattr: file: Operation not supported
* tests/mv/part-symlink.sh: Avoid the issue. Note this is new
since v9.12 as mv now warns about xattr ENOTSUP failures.
* tests/misc/xattr.sh: Avoid the issue (which would have
existed before v9.12).
Quoting is currently only enabled by default when writing to terminals,
but it is usually desired to constrain the output format for processing.
E.g. for processing by a shell you can now:
env --quoting-style=shell-escape | grep ^FOO
We use an option rather than reyling on $QUOTING_STYLE so
that $QUOTING_STYLE is restricted to be user config for terminals
and does not impact redirected processing.
Also it avoids an extra getenv() call if not outputting to tty.
* src/env.c (main): Parse --quoting-style and use that for the output
format for all outputs.
* src/printenv.c (main): Likewise.
* doc/coreutils.texi (env invocation): Describe --quoting-style.
(printenv invocation): Likewise.
* tests/env/env.sh: Reinstate testing of various output modes.
* tests/misc/printenv.sh: Likewise.
* NEWS: Mention the new feature. Also mention the change from
the previous commit not done there so that builders would have
a clean patch against the previous release.
csplit always creates the split holding the remainder of the input,
even when the patterns have already consumed everything and that
split is therefore empty. Two ways of reaching that state were not
exercised: a regexp with an offset that lands past the last line,
and --suppress-matched with a mix of a line number and a regexp
matching the last line. Only the all-line-numbers case was covered.
identified here: https://github.com/uutils/coreutils/pull/14565
* tests/csplit/csplit.sh: Test an offset reaching past end of input,
with and without -z.
* tests/csplit/csplit-suppress-matched.pl (mixed-1, mixed-2): Test a
line number combined with a regexp matching the last line.
Link: https://github.com/coreutils/coreutils/pull/353
In a long listing, -u and -c only choose which timestamp is shown;
the entries stay sorted by name, and time order is selected by -t
(or by asking for a format that is not long). This was only ever
exercised for short listings, so the rule went untested for every
long listing, including the ones requested with something other
than -l.
Identified here: https://github.com/uutils/coreutils/pull/14573
* tests/ls/ls-time.sh: Check that -u and -c keep name order for each
of -l, -g, -o, -n, --full-time and --dired, and that -t still gives
time order in a long listing.
Link: https://github.com/coreutils/coreutils/pull/354
* src/paste.c (usage): Simplify the description in --help to make it
easier to understand.
Fixes https://bugs.gnu.org/81859
Copyright-paperwork-exempt: Yes
parse_ls_color stores color_indicator[C_LINK].string as a pointer into
the color_buf buffer, then frees color_buf on the unparsable-value path.
The "ln=target" check after the parse loop still read that freed buffer,
e.g. with LS_COLORS='ln=target:x'. Skip it on the failure path, where
color output is disabled anyway and the buffer is gone.
* src/ls.c (parse_ls_color): Return after freeing color_buf on failure,
so the color_indicator[C_LINK] check only runs when the buffer is live.
* tests/ls/color-symlink-target.sh: New test.
* tests/local.mk (all_tests): Add it.
* NEWS: Mention the fix.
Link: https://github.com/coreutils/coreutils/pull/352
On GNU/Hurd, this test would get stuck in an infinite loop like the
following:
+ kill -s INFO 16535
+ sleep .01
+ kill -s INFO 16535
+ sleep .01
+ kill -s INFO 16535
+ sleep .01
[...]
The signals being sent that frequently seem to prevent 'dd' from
performing writes, since it outputs the following during that time:
0+0 records in
0+0 records out
0 bytes copied, 7161.11 s, 0.0 kB/s
0+0 records in
0+0 records out
0 bytes copied, 7161.88 s, 0.0 kB/s
[...]
* tests/dd/stats.sh: Use a larger interval between signals on GNU/Hurd.
`strace -e tee true` will run the tee command on cygwin,
rather than filtering for the tee syscall.
Reported by Bruno Haible on Cygwin 2.9.0
* init.cfg: Use -e trace=foo where foo might be a command,
so that strace probes will fail rather than invoke foo on Cygwin.
* tests/cat/cat-distinct-err.sh: Likewise.
* tests/cat/splice.sh: Likewise.
* tests/ls/no-cap.sh: Likewise.
* tests/misc/yes.sh: Likewise.
* tests/mv/atomic.sh: Likewise.
* tests/mv/atomic2.sh: Likewise.
* tests/nproc/nproc-quota.sh: Likewise.
* tests/pwd/pwd-long.sh: Likewise.
* tests/stty/stty.sh: Likewise.
* tests/tail/debug.sh: Likewise.
* tests/tail/inotify-only-regular.sh: Likewise.
* tests/sort/sort-NaN-infloop.sh: Up the timeout to about 240x
a "standard" system, as this was seen to timeout on a
mipsel VM using QEMU emulation.
Reported by Bruno Haible.
* tests/dd/no-allocate.sh: Expand memory restriction
(and allocated buffers) by 6MiB, to give more head-room
to the shell.
Reported and tested by Bruno Haible on OpenIndiana (from 2025).
On GNU/Hurd, file descriptor limits set via 'ulimit -n' seem to be
ineffective when set below the default:
$ (ulimit -n; ulimit -n 512; ulimit -n; \
tee $(seq 1024) < /dev/null; echo $?)
1024
512
tee: 1022: Too many open files
tee: 1023: Too many open files
tee: 1024: Too many open files
1
* tests/sort/sort-merge-fdlimit.sh: Don't fail if using 'ulimit -n' to
decrease the file descriptor limit is ineffective.
* tests/stat/stat-mount.sh: LeakSanitizer requires /proc access
for instrumented binaries, so ensure we're checking coreutils
binaries, rather than shell builtins.
* tests/df/skip-rootfs.sh: Test for /init which is listed as rootfs in
/proc/self/mountinfo, but tmpfs according to statfs.
Fixes https://bugs.gnu.org/65331
This adds an ELF note as per the UAPI Group's .note.dlopen spec:
$ readelf -p .note.dlopen src/sort
String dump of section '.note.dlopen':
[ 4] y
[ a] |@FDO
[ 10] [{"feature":"random-sort",
"description":"Support for random sorting",
"priority":"required",
"soname":["libcrypto.so.3"]}]
* bootstrap.conf: Add sd-dlopen.
* src/sort.c (DLOPEN_LIBCRYPTO && HAVE_OPENSSL_MD5): Add an ELF note
with SD_ELF_NOTE_DLOPEN() so that packagers can determine the libcrypto
dependency, needed when --random-sort is specified.
* NEWS: Mention the Build-related change.
Simplify cut_fields_mb_any() used with:
-w in a multi-byte locale, including UTF-8
--w=trimmed in any locale, including C
-f -d CHAR in non utf-8 multi-byte locale
-f -d CHAR in utf-8 with certain invalid CHARs
This is a net reducation of 24 lines,
and is performance neutral.
(cut_fields_mb_any): Use persistent first field and output state.
This allows using the existing field output helpers and
avoiding separate first field and line completion handling.
(cut_fields_bytesearch): Handle blanks explicitly.
(finish_current_line): Remove redundant condition.
There were no significant performance changes from the test cases
in commit a325c9978, but adding additional whitespace delimited cases
shows significant improvement. Below, with -f1, typical space delimited
input shows 4x more throughput, while the long line test shows 17x more.
Performance was tested with GCC 16 on an i7-5600U with
CFLAGS='-march=native -O3 -flto' with the following setup:
$ yes | head -n10M > sl.in
$ yes $(yes eeeaae | head -n10K | paste -s -d' ') | head -n10K > llw.in
$ yes $(yes eeeaae | head -n9 | paste -s -d' ') | head -n1M > asw.in
$ yes $(yes éééááé | head -n9 | paste -s -d$'\xe2\x80\x83') |
head -n1M > mbw.in
for type in sl llw asw mbw; do
cat $type.in >/dev/null;
for imp in '-before' '-after'; do
echo ============ "${imp:-base}" $type ==============;
for d in -w; do
fields='-f1 -f10 -f100'
test "$d" = "-b" && { fields='-b1 -b10 -b100'; d=''; }
test "$d" = "-c" && { fields='-c1 -c10 -c100'; d=''; }
for f in $fields; do
for loc in C.UTF-8; do
# Skip -b for UTF-8 as no different
test "$loc" = C.UTF-8 && echo "$f" | grep -q -- -b \
&& continue
# Skip multi-byte delimiter for C and not allowed
test "$loc" = C && test $(echo -n "$d" | wc -c) -ge 4 \
&& continue
bin=src/cut${imp}
LC_ALL=$loc $bin $f $d /dev/null 2>/dev/null &&
hyperfine --warmup=2 -m2 -M6 \
"LC_ALL=$loc $bin $f $d $type.in >/dev/null" ||
printf 'Benchmark 1: %s\n unsupported\n\n' \
"LC_ALL=$loc $bin $f $d $type.in >/dev/null"
done;
done;
done;
done;
done
After a little post-processing of the results, we get:
-- src/cut-before
| command | sl | llw | asw | mbw |
| --------------- | -------- | -------- | -------- | -------- |
| UTF8 -f1 -w | 96.2 ms | 1.916 s | 190.6 ms | 906.5 ms |
| UTF8 -f10 -w | 104.6 ms | 1.921 s | 187.7 ms | 845.6 ms |
| UTF8 -f100 -w | 104.4 ms | 1.916 s | 187.4 ms | 846.4 ms |
-- src/cut-after
| command | sl | llw | asw | mbw |
| --------------- | -------- | -------- | -------- | -------- |
| UTF8 -f1 -w | 88.7 ms | 112.4 ms | 43.5 ms | 146.3 ms |
| UTF8 -f10 -w | 95.2 ms | 114.4 ms | 193.5 ms | 848.4 ms |
| UTF8 -f100 -w | 94.8 ms | 130.6 ms | 194.4 ms | 848.1 ms |
* src/cut.c (scan_mb_line_end): A new function to skip
to EOL once we have exhausted the field selection.
(cut_fields_mb_any): Only skip after recognizing a field delimiter.
* tests/cut/cut.pl: Add test cases.
* tests/cut/mb-non-utf8.sh: Likewise.
* NEWS: Mention the improvement.
Adding a couple of inlining hints to the compiler
gives a 30% boost in raw character processing performance
in cut_fields_mb_any().
$ yes $(yes eeeaae | head -n10K | paste -s -d,) | head -n10K > ll.in
$ time LC_ALL=C.UTF-8 src/cut-before -f1 -w ll.in >/dev/null
real 0m2.279s
$ time LC_ALL=C.UTF-8 src/cut-after -f1 -w ll.in >/dev/null
real 0m1.619s
* src/cut.c (scan_mb_blank_field): Tag as inline.
(scan_mb_delim_field): Likewise.