Commit Graph
31732 Commits
Author SHA1 Message Date
Collin Funk 712fe02df1 tests: df: skip a test on WSL
* tests/df/skip-rootfs.sh: Test for /init which is listed as rootfs in
/proc/self/mountinfo, but tmpfs according to statfs.
Fixes https://bugs.gnu.org/65331
2026-09-07 16:23:49 +01:00
Pádraig Brady bff0e5412d build: sort: explicitly tag libcrypto dependency
This adds an ELF note as per the UAPI Group's .note.dlopen spec:

  $ readelf -p .note.dlopen src/sort
  String dump of section '.note.dlopen':
    [     4]  y
    [     a]  |@FDO
    [    10]  [{"feature":"random-sort",
                "description":"Support for random sorting",
                "priority":"required",
                "soname":["libcrypto.so.3"]}]

* bootstrap.conf: Add sd-dlopen.
* src/sort.c (DLOPEN_LIBCRYPTO && HAVE_OPENSSL_MD5): Add an ELF note
with SD_ELF_NOTE_DLOPEN() so that packagers can determine the libcrypto
dependency, needed when --random-sort is specified.
* NEWS: Mention the Build-related change.
2026-09-06 23:16:55 +01:00
Collin Funk b2fc0baf2c doc: NEWS: fix mismatched quotation marks
* NEWS: Use '...' instead of '...`.
2026-09-04 13:57:55 -07:00
Pádraig Brady 92a8d8fe40 cut: simplify multi-byte field processing
Simplify cut_fields_mb_any() used with:

  -w in a multi-byte locale, including UTF-8
  --w=trimmed in any locale, including C
  -f -d CHAR in non utf-8 multi-byte locale
  -f -d CHAR in utf-8 with certain invalid CHARs

This is a net reducation of 24 lines,
and is performance neutral.

(cut_fields_mb_any): Use persistent first field and output state.
This allows using the existing field output helpers and
avoiding separate first field and line completion handling.
(cut_fields_bytesearch): Handle blanks explicitly.
(finish_current_line): Remove redundant condition.
2026-09-04 21:10:45 +01:00
Pádraig Brady 71419a36f7 cut: -w: optimize extracting the start of a line
There were no significant performance changes from the test cases
in commit a325c9978, but adding additional whitespace delimited cases
shows significant improvement.  Below, with -f1, typical space delimited
input shows 4x more throughput, while the long line test shows 17x more.

Performance was tested with GCC 16 on an i7-5600U with
CFLAGS='-march=native -O3 -flto' with the following setup:

$ yes | head -n10M > sl.in
$ yes $(yes eeeaae | head -n10K | paste -s -d' ') | head -n10K > llw.in
$ yes $(yes eeeaae | head -n9 | paste -s -d' ') | head -n1M > asw.in
$ yes $(yes éééááé | head -n9 | paste -s -d$'\xe2\x80\x83') |
   head -n1M > mbw.in

for type in sl llw asw mbw; do
  cat $type.in >/dev/null;
  for imp in '-before' '-after'; do
    echo ============ "${imp:-base}" $type ==============;
    for d in -w; do
      fields='-f1 -f10 -f100'
      test "$d" = "-b" && { fields='-b1 -b10 -b100'; d=''; }
      test "$d" = "-c" && { fields='-c1 -c10 -c100'; d=''; }
      for f in $fields; do
        for loc in C.UTF-8; do
          # Skip -b for UTF-8 as no different
          test "$loc" = C.UTF-8 && echo "$f" | grep -q -- -b \
           && continue
          # Skip multi-byte delimiter for C and not allowed
          test "$loc" = C && test $(echo -n "$d" | wc -c) -ge 4 \
           && continue
          bin=src/cut${imp}
          LC_ALL=$loc $bin $f $d /dev/null 2>/dev/null &&
          hyperfine --warmup=2 -m2 -M6 \
           "LC_ALL=$loc $bin $f $d $type.in >/dev/null" ||
          printf 'Benchmark 1: %s\n  unsupported\n\n' \
           "LC_ALL=$loc $bin $f $d $type.in >/dev/null"
        done;
      done;
    done;
  done;
done

After a little post-processing of the results, we get:

-- src/cut-before

| command         |       sl |      llw |      asw |      mbw |
| --------------- | -------- | -------- | -------- | -------- |
| UTF8 -f1 -w     |  96.2 ms |  1.916 s | 190.6 ms | 906.5 ms |
| UTF8 -f10 -w    | 104.6 ms |  1.921 s | 187.7 ms | 845.6 ms |
| UTF8 -f100 -w   | 104.4 ms |  1.916 s | 187.4 ms | 846.4 ms |

-- src/cut-after

| command         |       sl |      llw |      asw |      mbw |
| --------------- | -------- | -------- | -------- | -------- |
| UTF8 -f1 -w     |  88.7 ms | 112.4 ms |  43.5 ms | 146.3 ms |
| UTF8 -f10 -w    |  95.2 ms | 114.4 ms | 193.5 ms | 848.4 ms |
| UTF8 -f100 -w   |  94.8 ms | 130.6 ms | 194.4 ms | 848.1 ms |

* src/cut.c (scan_mb_line_end): A new function to skip
to EOL once we have exhausted the field selection.
(cut_fields_mb_any): Only skip after recognizing a field delimiter.
* tests/cut/cut.pl: Add test cases.
* tests/cut/mb-non-utf8.sh: Likewise.
* NEWS: Mention the improvement.
2026-09-04 21:10:45 +01:00
Pádraig Brady f665fdffca cut: -w: avoid per character function calls
Adding a couple of inlining hints to the compiler
gives a 30% boost in raw character processing performance
in cut_fields_mb_any().

  $ yes $(yes eeeaae | head -n10K | paste -s -d,) | head -n10K > ll.in

  $ time LC_ALL=C.UTF-8 src/cut-before -f1 -w ll.in >/dev/null
  real	0m2.279s

  $ time LC_ALL=C.UTF-8 src/cut-after -f1 -w ll.in >/dev/null
  real	0m1.619s

* src/cut.c (scan_mb_blank_field): Tag as inline.
(scan_mb_delim_field): Likewise.
2026-09-04 21:10:45 +01:00
Paul Eggert 1ee46ac2c9 cp: ioctl+FICLONE EIO now fails without fallback
Problem reported by Pádraig Brady (bug#81776#16).
* src/copy.c (handle_clone_fail): Be more cautious if
ioctl+FICLONE failed with EIO; do not fall back to read+write,
even if --reflink=auto is in effect.
2026-09-03 00:57:54 -07:00
Paul Eggert cefa8e8fe8 cp: --debug now reports clone/offload fail errno
* src/copy.h: Use errno values for syscall failures, and
negative values otherwise.  That way, debugging output can
contain more info when clone or offload ops fail.
All uses changed.
2026-09-02 12:40:16 -07:00
Paul Eggert be8600bb24 cp: treat EDQUOT etc. like ENOSPC
Assume that XFS allocation groups can also fail with EDQUOT etc.,
so treat those failures like ENOSPC.
* src/copy.c (is_terminal_error): Remove; no longer used.
(handle_clone_fail): Remove DEST_DESC arg; no longer needed.
All uses changed.  Simplify by acting as if is_terminal_error
always returns false.  Change "!= 0" to "< 0" when either will do
for a syscall, as a matter of convention.
2026-09-02 12:40:16 -07:00
Pádraig Brady 51d40b1ab1 cp,install,mv: try a standard copy if clone fails with ENOSPC
reflinking can fail due to failure to allocate metadata,
while a standard copy may succeed.

* src/copy.c (is_terminal_error): Remove ENOSPC from the list.
Fixes https://bugs.gnu.org/81776
2026-09-02 19:25:42 +01:00
Pádraig Brady 2459c50cde tests: tail: avoid intermittent failure on NetBSD
It seems that NetBSD requires a fifo to be open, not merely opened,
to ensure a subsequent open proceeds. Immediately closing the opened
fifo caused intermittent failures on NetBSD 10.0.  While that seems
like a bug in NetBSD, we can avoid the issue.

* tests/tail/pid-pipe.sh: Hold the fifo open until our writer
has opened it.  This also allows specifying a single timeout
(of 12.7s), rather than 2 separate timeouts.
2026-09-01 19:12:55 +01:00
H. Peter Anvin 409225d662 stty: handle kernel speed encoding variations
eq_mode(), which is used to determine if tcsetattr() was successful,
failed unnecessarily after speed setting under certain circumstances:

1. the kernel MAY round the speed to the nearest supported value,
   so allow a variation of +/- 1/4 bit time per character from
   what was originally requested.

2. There MAY be more than one internal binary encoding of a specific
   speed (e.g. on Linux). On systems where that information
   affects c_cflag, this may introduce inconsistencies;
   filter out the known speed-related flags in c_cflag.

3. It is undefined in POSIX if cfsetispeed (..., 0) is retained or
   normalized across tcsetattr () ... tcgetattr ().
   If cfgetispeed () returns 0, compare the output speed instead.

* src/stty.c (eq_mode): Mask of some of the speed related bits
to avoid checking them for exact equality.
(speed_matches): A new helper to apply the more lenient matching
of requested and resulting speeds.
* NEWS: Mention the bug fix.
2026-09-01 19:12:42 +01:00
Collin Funk ea1c2b1787 printenv: don't quote output when given arguments and redirected
* src/printenv.c (main): Make sure quoting is disabled when we have
arguments and standard output is not a terminal.
* tests/misc/printenv.sh: Adjust single argument test cases. Add
multiple argument test cases.
* tests/misc/tty-quoting.sh: Add test cases to check if isatty is used.
* doc/coreutils.texi (printenvAlwaysQuoted): Remove macro.
(printenv invocation): Describe the adjusted quoting behavior.
(env invocation): Move the text from printenvAlwaysQuoted here since it
is no longer used for 'printenv'.
2026-08-31 18:15:39 -07:00
Collin Funk f188b3cc19 tests: csplit: test the behavior with an empty pattern
* tests/csplit/csplit.sh: Add a test case.
2026-08-31 17:31:08 -07:00
Collin Funk b7575a9d7b tests: mv: test the behavior of symbolic links to "."
* tests/mv/part-hardlink-symlink.sh: Add a test case.
2026-08-31 17:31:08 -07:00
oech3 e4db330242 tests: dd: support referencing of external test command
* tests/dd/stderr.sh: Use env instead of
$abs_top_builddir for external usage of tests.

Link: https://github.com/coreutils/coreutils/pull/340
2026-08-31 12:43:13 +01:00
Sylvestre Ledru 85857f1eaa tests: date: check that composite specifiers ignore the '-' flag
* tests/date/date.pl: Add tests ensuring that %-D, %-F, %-T, %-R, %-r,
%-x and %-X keep the default padding of their sub-fields, while the
other flags and the field width still apply to the whole expansion.
https://github.com/uutils/coreutils/pull/14163

Link: https://github.com/coreutils/coreutils/pull/339
2026-08-31 12:38:01 +01:00
Pádraig Brady 15b8d8ab82 build: update gnulib submodule to latest
Update specifically to get commit 91c129e9db fixing:

  $ date-before -d '1-2-3' "+[%_D]"
  [02/03/ 1]
  $ date-after -d '1-2-3' "+[%_D]"
  [02/03/01]

* gnulib: Update to latest.
* NEWS: Mention the bug fix.
2026-08-31 12:30:00 +01:00
Pádraig Brady 3f3a822de7 doc: printf: resync format examples in texinfo
* doc/coreutils.text (printf invocation): Sync the issues
fixed in the previous commit to the texinfo examples.
2026-08-30 21:56:29 +01:00
Pádraig Brady aea70b28be doc: printf: fix typos in format examples
* src/printf.c (usage): Remove redundant quotes.
Change ambiguous quotes. Fix typos in examples.
Quoting changes suggested by Martin D Kealey.
2026-08-29 14:03:37 +01:00
Collin Funk 29933e8a5c tests: misc: test that various commands don't fork
* tests/misc/no-fork.sh: New file.
* tests/local.mk (all_tests): Add the new test.
* tests/misc/stdbuf.sh: Remove a test that is redundant with the new
test.
2026-08-28 22:10:34 -07:00
Sylvestre Ledru 47dfc77774 tests: mv: check cross-device move of dir with hard links and symlink
* tests/mv/part-hardlink-symlink.sh: New test, ensuring that moving a
directory to another file system preserves the contents and the hard
links of its regular files, as well as the symlinks pointing at them.
* tests/local.mk (all_tests): Add it.
https://github.com/uutils/coreutils/pull/14170

Link: https://github.com/coreutils/coreutils/pull/337
2026-08-27 17:00:39 +01:00
oech3 5e011122e5 tests: misc: support referencing of external commands
* tests/misc/close-stdout.sh: Use env instead of
$abs_top_builddir for external usage of tests.

Link: https://github.com/coreutils/coreutils/pull/338
2026-08-27 16:57:33 +01:00
Pádraig Brady 4e53af6577 env: translate -v messages
* src/env.c: These user facing messages should be translated.
2026-08-27 16:40:53 +01:00
Pádraig Brady d18d3a85be env: ensure all -v messages are quoted
$ env-before -v -u 'B' -S 'A=1 B="\n"' 'true'
split -S:  ‘A=1 B="\\n"’
 into:    ‘A=1’
     &    ‘B=\n’
unset:    B
setenv:   A=1
setenv:   B=

executing: true
   arg[0]= ‘true’

$ src/env-after -v -u 'B' -S 'A=1 B="\n"' 'true'
split -S:  ‘A=1 B="\\n"’
 into:     ‘A=1’
     &     ‘B=\n’
unsetting: ‘B’
setting:   ‘A=1’
setting:   ‘B=\n’
executing: ‘true’
   arg[0]= ‘true’

* src/env.c: -v is often used to debug spacing issues,
so it's important to quote output.
2026-08-27 16:40:49 +01:00
Pádraig Brady b9203a123d env: use more abstract -v messages
* src/env.c: These user facing messages should
be more descriptive and abstract.  Also improve alignment.
2026-08-27 15:29:49 +01:00
Collin Funk 5b70089a7b tests: printf: add tests for the %b specifier
* tests/printf/printf-cov.pl ($ESCAPE_CHARS, $ESCAPE_OUTPUT): New
variables derived from the existing 'escape-1' test.
(@Tests): Add some additional escaping tests and test %b.
2026-08-26 21:23:33 -07:00
Collin Funk 1ab303a901 tests: sort: check that non-blank characters aren't treated as blank
* tests/sort/sort.pl (@Tests): Add a few test cases.
2026-08-26 20:19:56 -07:00
oech3 c717ca569b tests: env: support referencing of external env
* tests/env/env-S-script.sh: Use `cmd_exe` instead of
$abs_top_builddir for external usage of tests.

Link: https://github.com/coreutils/coreutils/pull/336
2026-08-26 18:24:20 +01:00
Pádraig Brady ff91f242fd tests: provide a cmd_exe helper to find a program binary
init.cfg (cmd_exe): Provide a function to find the command under test,
whether in the $PATH or in the standard build directory.
* tests/env/env0-from.sh: Use it instead of `command -v`.
* tests/misc/coreutils.sh: Likewise.
* tests/nproc/nproc-quota.sh: Likewise.
* tests/env/env.sh: Likewise.  Also remove non robust use of sed.
2026-08-26 15:18:08 +01:00
Sylvestre Ledru f1a5961c12 tests: join: cover a repeated -o
* tests/join/join.pl: Test that the field specs of several -o options
accumulate in order, and that 'auto' applies only when every -o
argument is 'auto'.
Identified here: https://github.com/uutils/coreutils/pull/14070

Link: https://github.com/coreutils/coreutils/pull/335
2026-08-26 12:40:26 +01:00
Sylvestre Ledru 0d14766774 tests: cp: exercise copying with a write-denying umask
* tests/cp/preserve-mode.sh: Ensure that with a umask that removes
the owner write bit, cp still fully writes a freshly created
destination file and gives it the expected mode.
  Missing case identified here:
  https://bugs.launchpad.net/ubuntu/+source/rust-coreutils/+bug/2164777

Link: https://github.com/coreutils/coreutils/pull/334
2026-08-26 12:37:43 +01:00
Sylvestre Ledru 31634dbc2e tests: stat: check output preceding an invalid directive
* tests/stat/stat-printf.pl: The format is interpreted as it is printed,
so whatever precedes an invalid directive is written before the
diagnostic, and --format adds no trailing newline when it stops early.
The existing err-1 and err-2 cases put the directive at the start of the
format, so nothing covered this.
See https://github.com/uutils/coreutils/issues/14101

Link: https://github.com/coreutils/coreutils/pull/333
2026-08-26 12:12:46 +01:00
Sylvestre Ledru 6127da8050 tests: expand, unexpand: check blank-separated tab lists
parse_tab_stops accepts a comma or any blank as a separator, but the
tests only ever used a comma or a space, so a tab separator was
untested.  unexpand had no coverage of multi-stop lists at all.
Missing case identified here:
https://github.com/uutils/coreutils/pull/14057

* tests/expand/expand.pl (t4a): New test, using tabs as separators.
* tests/unexpand/unexpand.pl (tabs-3, tabs-4, tabs-5, tabs-6): New
tests, for comma-, space- and tab-separated lists and for a leading
separator.

Link: https://github.com/coreutils/coreutils/pull/332
2026-08-26 12:11:19 +01:00
Pádraig Brady 493ed58f21 tests: env: avoid false failure on FreeBSD
Avoid unsetenv() on a non standard environment,
as FreeBSD 14 and 15 at least fail when manipulating
such an environment with unsetenv().

  $ printf 'A=1\0A=2\0foo\0' |
    src/env --env0-from=- src/env foo=bar | grep foo
  foo=bar
  foo

  $ printf 'A=1\0A=2\0foo\0' |
    src/env --env0-from=- src/env -u foo
  env: environment corrupt; missing value for foo
  env: cannot unset ‘foo’: Bad address

  $ printf 'A=1\0A=2\0foo\0' |
    src/env --env0-from=- src/env -u bar
  env: environment corrupt; missing value for foo
  env: cannot unset ‘bar’: Bad address

* tests/env/env0-from.sh: Use routines internal to env(1)
to manipulate the environment, only delegating the
executed env(1) to adding any implicit environment variables.
Reported by Bruno Haible.
2026-08-26 12:10:22 +01:00
Collin Funk 9c6b27e405 build: update gnulib submodule to latest 2026-08-25 21:51:13 -07:00
Collin Funk 24e485a883 tests: stat: fix a test failure on FreeBSD
* tests/stat/stat-nanoseconds.sh: Don't assume that we can create a file
with timestamps before the epoch.
Reported by Bruno Haible.
2026-08-25 19:34:52 -07:00
Paul Eggert 9068289c95 maint: translate messages from Gnulib
Problem reported by ribbon in:
https://lists.gnu.org/r/coreutils/2026-08/msg00093.html
* configure.ac (GNULIB_TEXT_DOMAIN): Define.
* po/POTFILES.in: Add lib/parse-datetime.y.
2026-08-25 10:24:59 -07:00
Paul Eggert 54120fb722 build: update gnulib submodule to latest
* bootstrap: Update using './bootstrap --bootstrap-sync'.
2026-08-25 10:24:59 -07:00
Collin Funk e92232687a tests: join: support old perl versions
The following error was seen on a Debian 6.0 virtual machine using perl
5.10.1:

    Bareword found where operator expected at \
      ./tests/join/join.pl line 327, near "tr/_/-/r"
    syntax error at ./tests/join/join.pl line 327, near "tr/_/-/r"
    Execution of ./tests/join/join.pl aborted due to compilation errors.
    FAIL tests/join/join.pl (exit status: 255)

* tests/join/join.pl: Don't use the /r modifier which was not introduced
until perl 5.13.7.
2026-08-23 22:47:19 -07:00
Collin Funk 32f109ca8e tests: pathchk: test the output to standard error for empty file names
* tests/misc/pathchk.sh: Move the existing empty file name tests to a
loop to check some additional options. Check the message printed to
standard error.
2026-08-22 19:55:33 -07:00
Collin Funk aa0c0de350 tests: stdbuf: check that a child process isn't created
* tests/misc/stdbuf.sh: Add a test case.
2026-08-20 22:56:49 -07:00
Collin Funk 6ac1d0907a tests: sort: check LC_ALL and LC_COLLATE
* tests/sort/sort-locale.sh: Also check LC_COLLATE.

Link: https://github.com/coreutils/coreutils/pull/269
2026-08-20 17:37:01 +01:00
oech3 cfd13b061e tests: dd: replace LD_PRELOAD of fail-ftruncate-fstat.sh by strace
* tests/dd/fail-ftruncate-fstat.sh: Replace LD_PRELOAD by strace
to support static executables.

Link: https://github.com/coreutils/coreutils/pull/330
2026-08-20 16:25:16 +01:00
Sylvestre Ledru 0b8fff44cb tests: stat: check nanosecond precision away from the Epoch
The existing checks all use a timestamp only 67413 seconds after the
Epoch, which a double can still represent exactly to the nanosecond, so
they do not detect a formatter that goes through floating point.  They
also never look at a timestamp before the Epoch, where the fraction has
to be truncated towards minus infinity.

* tests/stat/stat-nanoseconds.sh: Also check %Y, %.Y, %.3Y and %.9Y on a
2026 timestamp and on a timestamp shortly before the Epoch.
Link: https://github.com/uutils/coreutils/pull/13976

Link: https://github.com/coreutils/coreutils/pull/331
2026-08-20 16:15:02 +01:00
Pádraig Brady 3d3022995b maint: avoid -Werror=maybe-uninitialized with gcc -flto -O3
* src/env.c (main): With GCC 16.1 we need to force initialize
to avoid the maybe uninitialized warning.
2026-08-19 21:09:36 +01:00
Pádraig Brady 3abac0e23a doc: cut: show how to efficiently search ASCII blanks with -w
* doc/coreutils.texi (cut invocation): Mention LC_CTYPE=C
restricts the -w field separator set, and is thus more efficient.
2026-08-19 21:09:29 +01:00
Pádraig Brady 224673eba8 build: fix logname systemd linkage
Note there was worry about redundant linking with -lsystemd
when getlogin() is not replaced, however that is mitigated
in general with the --as-needed linker option (the lib-ignore
gnulib module). Tested with:

  ./configure --with-systemd gl_cv_func_getlogin_works=no

* NEWS: Mention the Build-related fix.
Reported at https://bugs.debian.org/1129960
2026-08-19 13:41:49 +01:00
Collin Funk b98f6d8c53 tests: prefer unified diffs when printing test failures
* tests/Coreutils.pm (_compare_files): Use 'diff -u' instead of
'diff -c'.
2026-08-18 22:12:31 -07:00
Collin Funk ffa26f0c2f tests: df: check that -k and -m are supported
* tests/df/df-output.sh: Check that the block size is correct when using
-k and -m.
2026-08-18 19:41:33 -07:00