Commit Graph
5176 Commits
Author SHA1 Message Date
Pádraig Brady 006dfe4cd4 numfmt: support reading numbers with grouping characters
This does not validate grouping character placement,
and currently just ignores grouping characters.

* src/numfmt.c (simple_strtod_int): Skip grouping chars
that are part of a number.
* tests/misc/numfmt.pl: Add test cases.
* NEWS: Mention the improvement.
2025-10-17 17:26:30 +01:00
Pádraig Brady 8bc11f80a3 numfmt: support reading numbers with NBSP before unit
* src/numfmt.c (simple_strtod_human): Accept (multi-byte)
non-breaking space character between number and unit.
Note we restrict this to a single character between number
and unit, to allow less ambiguous parsing if multiple blanks
are used to delimit fields.
* tests/misc/numfmt.pl: Add test cases.
* doc/coreutils.texi (numfmt invocation): Fix stale description
--delimiter skipping whitespace.
* NEWS: Mention the improvement.
2025-10-17 17:26:25 +01:00
Nicolas Boichat ab5fce879b tests: du/bigtime: try harder to find a suitable filesystem
* tests/du/bigtime.sh: At least on Linux, the ext4 filesystem
doesn't support such large timestamp, while tmpfs does.  Try a bit
harder to look for a filesystem with large timestamp support.
2025-10-16 18:28:52 +01:00
Collin FunkandPádraig Brady 1066d442c2 tests: date: check that the hour format of the current locale is used
* tests/date/date-locale-hour.sh: New file.
* tests/local.mk (all_tests): Add the new test.

Co-authored-by: Pádraig Brady <P@draigBrady.com>
2025-10-13 21:56:43 -07:00
Pádraig Brady b4e02e0ef4 tests: fix false failure in recent memory limit test
* tests/basenc/bounded-memory.sh: Ensure we skip the test
upon failure to determine the memory lower bound.
Reported by Bruno Haible.
2025-10-13 13:22:38 +01:00
Pádraig Brady b880b02c44 numfmt: fix buffer over-read (CWE-126)
* src/numfmt.c (simple_strtod_human): Check for NULL after pointer
adjustment to avoid Out-of-range pointer offset (CWE-823).
* NEWS: Mention the fix.
2025-10-11 12:37:56 +01:00
Pádraig Brady dd3eab13ee tests: basenc: use less redundant naming
Rename to less redundant names, now that we use
a separate test directory per util.

* tests/basenc/basenc-bounded-memory.sh -> .../bounded-memory.sh
* tests/basenc/basenc-large.sh -> .../large-input.sh
* tests/local.mk: Reference new names.
2025-10-11 11:28:22 +01:00
Pádraig Brady 86234ebffa tests: fix memory limit determination in new test
* tests/basenc/basenc-bounded-memory.sh: The passed command
needs to succeed for memory limit determination to work.
2025-10-11 11:27:47 +01:00
Collin Funk f88f0e5d44 tests: basenc: add a test for bounded memory operation
* tests/basenc/basenc-bounded-memory.sh: New file.
* tests/local.mk (all_tests): Add the test.
2025-10-11 01:33:15 -07:00
Sylvestre Ledru ae0fcbba97 tests: ln -f: ensure existing link replaced
Identified here:
<https://github.com/uutils/coreutils/issues/8830>

* tests/ln/misc.sh: Add the check.
2025-10-10 18:19:47 +01:00
Pádraig Brady 51f27a15b0 tests: cksum: add a test case for robust file name parsing
* tests/cksum/cksum-c.sh: Add a test case where the file name
contains tagged format delimiter characters.
2025-10-07 15:58:36 +01:00
Pádraig Brady 4639257610 cksum: fix --check with untagged base64 format with tag matches
* src/digest.c (split_3): Fallback to untagged matching in the
case where -a is specified and we have matched a TAG in
the possibly base64 data.  This might happen in 1 in every 64K files.
Note we remove the modification of string S (and redundant streq) in
the tag matching, as that was not needed since v8.32-223-g217cd278e.
* tests/cksum/cksum-c.sh: Add a test case.
* NEWS: Mention the bug fix.
2025-10-07 15:58:36 +01:00
Pádraig Brady e661c7a525 cksum: fix length validation with SHA2- tagged format
* src/digest.c (sha2_sum_stream): Change from unreachable()
to affirm() so that we have defined behavior unless
we configure with --disable-assert.
(sha3_sum_stream): Likewise.
(split_3): Validate SHA2-lengths before passing on.
* tests/cksum/cksum-c.sh: Add a test case.
* NEWS: Mention the bug fix.
2025-10-07 15:58:36 +01:00
Pádraig Brady cd2a8edd84 cksum: fix --check with --algorithm=sha2
* src/digest.c (split_3): Look up the provided tag with -a sha2
because there is not a 1:1 mapping between them.
* tests/cksum/cksum-c.sh: Add a test case.
* NEWS: Mention the bug fix.
2025-10-07 15:58:34 +01:00
Collin Funk ce8c22d25f cksum: allow -a {blake2b,sha2,sha3} --check to work on base64
* NEWS: Mention the bug.
* src/digest.c (split_3): Check that the base64 digest matches the
length supported by the algorithm.
(digest_check): Check that the read digest matches the base64 length of
the algorithm's digest. The previous condition would not work for
'cksum -a blake2b -l 8 ...'.
* tests/cksum/cksum-base64-untagged.sh: New file.
* tests/local.mk (all_tests): Add the new test.
2025-10-05 15:38:27 -07:00
Pádraig Brady 1b12425f67 tests: factor: add suggested large prime tests
* tests/factor/create-test.sh: Add 2 new large primes from:
https://github.com/coreutils/coreutils/issues/65
* tests/local.mk: Reference the 2 new generated tests.
2025-10-02 17:22:28 +01:00
Pádraig Brady 75e3888bd3 unexpand: fix heap buffer overflow with --tabs=[+/]NUM
This avoids CWE-122: Heap-based Buffer Overflow
where we could write blank characters beyond
the allocated heap buffer.

* src/expand-common.c (set_max_column_width): Refactor function from ...
(add_tab_stop): ... here.
(set_extend_size): Call new function.
(set_increment_size): Likewise.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/79555
2025-10-02 15:26:02 +01:00
Mathieu Bordere 67e9068c5f wc: add AVX512 function for line counting
* configure.ac: Add detection of AVX512 intrinsics for wc.
* src/local.mk: Build AVX512 wc libraries.
* src/wc.c: Add runtime detection of AVX512 intrinsics and call
appropriate function when detected.
* src/wc.h (wc_lines_avx512): Declare function.
* tests/wc/wc-cpu.sh: Add a test that disables AVX512 intrinsics.
* src/wc_avx512.c: New file containing the wc -l implementation using
AVX512. The logic and code is reused from the AVX2 implementation with
slight adaptations. Replaced __builtin_popcount by __builtin_popcountll
and the combination of _mm256_cmpeq_epi8 and _mm256_movemask_epi8 by a
single call to _mm512_cmpeq_epi8_mask.
* NEWS: Mention the improvement.
2025-09-30 14:09:37 +01:00
Pádraig Brady bfcfe9e1c4 tests: test: ensure file operations are covered
A coverage report indicated these weren't tested
(as generally the test shell builtin is used).

* tests/test/test-file.sh: Add a new test.
* tests/local.mk: Reference the new test.
2025-09-26 21:01:04 +01:00
Hannes Braun 914972e80d tail: fix tailing larger number of lines in regular files
* src/tail.c (file_lines): Seek to the previous block instead of the
beginning (or a little before) of the block that was just scanned.
Otherwise, the same block is read and scanned (at least partially)
again. This bug was introduced by commit v9.7-219-g976f8abc1.
* tests/tail/basic-seek.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: mention the bug fix.
2025-09-24 22:37:18 +01:00
Pádraig Brady 0cc3e1bfb6 tests: wc: fix hardware acceleration disabling test
* tests/wc/wc-cpu.sh: The message is only printed with wc -l.
Reported by Mathieu Borderé.
2025-09-24 15:53:34 +01:00
Pádraig Brady 2ef53e5b04 basenc: --base58: fix buffer overflow with input > 15MB
base58_length() operated naively on an int
which resulted in an overflow to a negative number
for any input > 2^31-1/138, i.e. 15,561,475 bytes.

* src/basenc.c (base_length): Change input and output
parameter types from int to idx_t since this needs to
cater for the full input size in the base58 case.
(base58_length): Likewise.  Also reorder the calculation
to be less exact, but doing the division first
to minimize the chance of overflow (which now on 64 bit
would only happen for inputs > around 6 Exa bytes).
* tests/basenc/basenc-large.sh: Add a new test,
that triggers with valgrind or ASAN.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
2025-09-23 20:35:34 +01:00
Pádraig Brady b8b486dc32 tests: ls: avoid alignment check with non printable characters
* tests/ls/block-size.sh: Skip the case where there are
non-printable characters in ls' output, which is the case
with NBSP thousands separators on FreeBSD 11 and 12.
We may drop the MBSW_REJECT_UNPRINTABLE in future from
ls and numfmt, but for now avoid these characters in the test.
Reported by Bruno Haible.
2025-09-22 11:23:50 +01:00
Pádraig Brady f9c6c730eb tests: du: avoid false failure in racy test
* tests/du/move-dir-while-traversing.sh: Expand the work to avoid
a false failure where du completes before the directory is moved.
Also expand the timeout to our more standard 10s to avoid the
"directory mover" being killed before du processes the directory.
This doesn't perceptibly impact the run time of the test.
Reported by Bruno Haible on a CentOS 7 system.
2025-09-22 00:09:03 +01:00
Paul Eggert ebedbe8997 maint: comment spelling fixes 2025-09-21 11:30:40 -07:00
Paul Eggert 6c194dc85b tests: fix OMP_NUM_THREADS misspelling
* tests/nproc/nproc-quota.sh: Fix misspelling.
2025-09-21 11:30:12 -07:00
Pádraig Brady 8a1f434463 tests: avoid false failure on older Linux kernels
* tests/fold/fold-zero-width.sh: Check the shell was able to create
the redirection file, as intermittently on CentOS 5,6,7 this wasn't
the case, with the shell giving an xmalloc failure due to the ulimit.
Reported by William Bader and Bruno Haible.
2025-09-21 16:37:26 +01:00
Pádraig Brady 076e247416 tests: tail: avoid false failure with overlayfs
* tests/tail/inotify-dir-recreate.sh: Add an extra check
that inotify is in use, as it's required for this test.
Inotify is avoided with overlayfs for which the
df --local check is not sufficient exclusion for.
2025-09-21 13:45:54 +01:00
Pádraig Brady 70eb250fd7 tests: fold: avoid false failures on Solaris 10
* tests/fold/fold-characters.sh: Ensure we have independent verification
of the width of characters before testing based on those widths.
* tests/fold/fold-zero-width.sh: Likewise.
* tests/fold/fold.pl: Only compare the exit status,
as the error message can be translated.
2025-09-20 22:12:58 +01:00
Pádraig Brady df30ce36e8 tests: dd: avoid false failure on Solaris 10
* tests/dd/no-allocate.sh: Give some extra space to the
determined vm limit.
2025-09-20 22:12:58 +01:00
Pádraig Brady 7c217bd8ac tests: avoid false failure on CentOS 5,6,7
* tests/fold/fold-zero-width.sh: Increase vm limit to avoid
failures on CentOS 5,6,7.  Match the limit used in write-errors.sh
as per commit v9.5-255-g0bd149403
2025-09-20 16:17:43 +01:00
Collin Funk bbde83899d tests: env: skip a few tests if LD_LIBRARY_PATH is set
* tests/env/env-null.sh: Skip test if LD_LIBRARY_PATH or platform's
equivalent is set, since 'env -i' will unset it which may prevent
programs from running.
* tests/env/env-S.pl: Likewise.
Issue and suggested fix reported by Bruno Haible.
2025-09-19 16:40:09 -07:00
Pádraig Brady 4ca51b1013 tests: tail/overlay-headers.sh: protect against hang
* tests/tail/overlay-headers.sh: Protect tail invocation with timeout,
and extend the possible running period to 60 seconds like other tests.
Reported by Brudno Haible on T2SDE Linux/alpha
2025-09-18 19:04:57 +01:00
Pádraig Brady 0a2a3deb33 tests: tests/tail/wait.sh: protect against hang
* tests/tail/wait.sh: This test was seen to hang occasionally
on an Alpine Linux 3.20 system, so protect the tail(1) call
with `timeout 60` as done in similar tests.
Reported by Bruno Haible.
2025-09-18 17:47:45 +01:00
Pádraig Brady 5662192f39 tests: write-errors.sh: avoid portability issue with dash
* tests/misc/write-errors.sh: Use printf rather than echo
since the echo builtin in dash will interpret backslashes.
* tests/misc/read-errors.sh: Likewise for consistency.
2025-09-17 21:37:35 +01:00
Paul Eggert ee367bd38d maint: STREQ → streq
Use new Gnulib streq function instead of rolling our own macro.
* bootstrap.conf (gnulib_modules): Add stringeq.
* src/rm.c (main): Don’t assume streq is a macro that expands to (...),
as it is now a function.
* src/system.h:
* tests/df/no-mtab-status.sh, tests/df/skip-duplicates.sh:
(STREQ): Remove.  All uses replaced by streq.
2025-09-17 12:20:24 -07:00
Pádraig Brady 78502174a9 tests: fix issues with fold-characters.sh test
This was noticed with:

  trap '' PIPE &&  # Ignore SIGPIPE
  make check TESTS=tests/fold/fold-characters.sh SUBDIRS=.

* tests/fold/fold-characters.sh: Move memory limit test to ...
* tests/misc/write-errors.sh: ... which avoids "write error"
messages on stderr due to the ignored SIGPIPE.  It also protects
the fold invocation with a timeout(1) so that fold implementations
that don't exit promptly upon write error don't hang the test suite
(Like we would have done before commit v9.7-311-gc95c7ee76).
2025-09-17 15:32:47 +01:00
Pádraig Brady c95c7ee76f fold: fix write error checks with invalid multi-byte input
* src/fold.c (write_out): A new helper to check all writes.
(fold-file): Use write_out() for all writes.
* tests/fold/fold-zero-width.sh: Adjust to writing more
data in various patterns, rather than two buffers of NULs.
This is a more robust memory bound check, and the '\303' case
tests this particular logic change.
* NEWS: fold now exits immediately, not just promptly.
2025-09-16 18:41:51 +01:00
Pádraig Brady 10459ca7ed fold: exit promptly upon write errors
* NEWS: Mention the improvement.
* src/fold.c (fold_file): Check for write errors
after each buffer read from stdin.
* tests/misc/write-errors.sh: Add test cases.
2025-09-16 14:10:31 +01:00
Pádraig Brady dda853a4de tests: fold/fold-zero-width.sh: avoid false failure with ENOSPC
* tests/fold/fold-zero-width.sh: Check relatively large test file
is created appropriately.  Also apply more idomatic wc -l usage.
2025-09-16 14:00:44 +01:00
Collin Funk 98e5418a38 fold: fix out of bounds write with zero width characters
* src/fold.c (fold_file): Prefer putchar ('\n') to copying characters.
If we do not have room in the output buffer print it since it is not a
full line of text.
* tests/fold/fold-zero-width.sh: New test case.
* tests/local.mk (all_tests): Add it.
2025-09-15 20:55:56 -07:00
Pádraig Brady 7bd3839cda cksum,wc: support disabling hardware acceleration at runtime
This is useful to give better test coverage at least,
and may be useful for users to tune their environment.

* bootstrap.conf: Reference the cpu-supports gnulib module.
* src/cksum.c: Use cpu_supports() rather than __builtin_cpu_supports().
* src/wc.c: Likewise.
* tests/cksum/cksum.sh: Adjust to testing all implementations.
* tests/wc/wc-cpu.sh: A new test to do likewise.
* tests/local.mk: Reference the new wc test.
2025-09-14 13:43:49 +01:00
Pádraig Brady 4141ae3e49 maint: basenc: refactor all encodings to use finalize
Finalize was required for base58, but it's a more general mechanism
which simplifies the logic for all encodings

* src/basenc.c (do_decode): Always call base_decode_ctx_finalize(),
rather than the awkward double loop at end of buffer.
* tests/basenc/basenc.pl: Add basenc finalization tests.
2025-09-10 23:24:46 +01:00
Pádraig Brady 2ed207cb1f nohup: avoid FORTIFY runtime failure on Bionic libc
The meaning of non-file permission umask bits is implementation defined.
On Bionic libc, attempting to set them triggers a FORTIFY runtime check.

  $ nohup true
  FORTIFY: umask: called with invalid mask -601
  Aborted                    nohup true

* src/nohup.c: (main) Avoid setting non-permission bits in umask.
Just clear the umask to ensure we create nohup.out with u+rw,
as we restore the original umask before the exec().
* tests/misc/nohup.sh: Add a test case.
* NEWS: Mention the bug fix.
2025-09-09 12:21:41 +01:00
Pádraig Brady d9b68362ed basenc: ensure partial padding with newlines induces an error
* src/basenc.c (has_padding): A more robust helper to
identify padding in the presence of trailing newlines.
(do_decode): Use has_padding() rather than just looking
at the last character.
* tests/basenc/base64.pl: Fully test commit v9.4-53-g378dc38f4
by ensuring partially padded data is diagnosed.
baddecode9 is the case fixed in this commit.
* NEWS: Mention the bug fix.
2025-09-08 19:39:47 +01:00
Pádraig Brady 42c4578b49 tests: ensure option aliases are supported
This implicitly tests the previous commit to
adjust how date(1) handles multiple named format options.
Currrently it tests the following are supported:

  chown  --quiet  --silent
  date  --rfc-email  --rfc-822  --rfc-2822
  date  --uct  --utc  --universal
  dircolors  --bourne-shell  --sh
  dircolors  --csh  --c-shell
  head  --quiet  --silent

* tests/misc/option-aliases.sh: A new test to ensure all
option aliases supported by a command are supported.
* Reference the new test.
2025-09-07 13:53:22 +01:00
Collin Funk 24fb014092 tests: fold: check if multi-byte spaces are treated as blank
This avoids a test failure on FreeBSD 14, MacOS 15, and musl.
Fix suggested by Pádraig Brady in:
<https://bugs.gnu.org/79301#32>.
* tests/fold/fold-spaces.sh (isblank): New function. Only run the tests
if the character is treated as blank.
Fixes https://bugs.gnu.org/79301
2025-09-04 19:48:37 -07:00
Pádraig Brady a43a17f216 tests: cksum: check more length variants
* tests/cksum/cksum-raw.sh: Adjust to non legacy naming,
and also check various length variations.
* tests/misc/read-errors.sh: Likewise.
2025-09-04 14:52:21 +01:00
Pádraig Brady aba9800995 cksum: prefer -a sha2 -l ###, to -a sha###
To make the interface more concise and consistent,
while being backwards compatible.

* src/digest.c (main): Continue to support -a "sha###" but
also support -a "sha2" and treat it like "sha3", except in...
(output_file): ... maintain the legacy tags for better compatability.
* doc/coreutils.texi (cksum invocation): Document the -a sha2 option.
* tests/cksum/cksum-base64.pl: Adjust as per modified --help.
* tests/cksum/cksum-c.sh: Add new supported SHA2-### tagged variant.
* NEWS: Mention the new feature.
2025-09-04 14:49:07 +01:00
Collin Funk 403d82a0bf cksum: add support for SHA-3
* src/digest.c: Include sha3.h.
(BLAKE2B_MAX_LEN): Rename to
DIGEST_MAX_LEN since it is also used for SHA-3.
(sha3_sum_stream): New function.
(enum Algorithm, algorithm_args, algorithm_args, algorithm_types)
algorithm_tags, algorithm_bits, cksumfns, cksum_output_fns): Add entries
for SHA-3.
(usage): Mention that SHA-3 is supported. Mention requirements for
--length with SHA-3.
(split_3): Use DIGEST_MAX_LEN instead of BLAKE2B_MAX_LEN. Determine the
length of the digest for SHA-3. Make sure it is 224, 256, 384, or 512.
(digest_file): Set the digest length in bytes. Use DIGEST_MAX_LEN
instead of BLAKE2B_MAX_LEN. Always append the digest length to SHA3 in
the output.
(main): Allow the use of --length with 'cksum -a sha3'.  Use
DIGEST_MAX_LEN instead of BLAKE2B_MAX_LEN. Make sure it is 224, 256,
384, or 512.
* tests/cksum/cksum-base64.pl (@pairs): Add expected sha3 output.
(fmt): Modify the output to use SHA3-512 since that is the default.
(@Tests): Modify arguments for sha3 to use --length=512.
* tests/cksum/cksum-sha3.sh: New test, based on tests/cksum/b2sum.sh.
* tests/local.mk (all_tests): Add the test.
* bootstrap.conf: Add crypto/sha3.
* gnulib: Update to latest commit.
* NEWS: Mention the change.
* doc/coreutils.texi (cksum general options): Mention sha3 as a
supported argument to the -a option. Mention that 'cksum -a sha3'
supports the --length option. Mention that SHA-3 is considered secure.
2025-09-03 22:29:33 -07:00