Commit Graph
5128 Commits
Author SHA1 Message Date
Pádraig Brady aba9800995 cksum: prefer -a sha2 -l ###, to -a sha###
To make the interface more concise and consistent,
while being backwards compatible.

* src/digest.c (main): Continue to support -a "sha###" but
also support -a "sha2" and treat it like "sha3", except in...
(output_file): ... maintain the legacy tags for better compatability.
* doc/coreutils.texi (cksum invocation): Document the -a sha2 option.
* tests/cksum/cksum-base64.pl: Adjust as per modified --help.
* tests/cksum/cksum-c.sh: Add new supported SHA2-### tagged variant.
* NEWS: Mention the new feature.
2025-09-04 14:49:07 +01:00
Collin Funk 403d82a0bf cksum: add support for SHA-3
* src/digest.c: Include sha3.h.
(BLAKE2B_MAX_LEN): Rename to
DIGEST_MAX_LEN since it is also used for SHA-3.
(sha3_sum_stream): New function.
(enum Algorithm, algorithm_args, algorithm_args, algorithm_types)
algorithm_tags, algorithm_bits, cksumfns, cksum_output_fns): Add entries
for SHA-3.
(usage): Mention that SHA-3 is supported. Mention requirements for
--length with SHA-3.
(split_3): Use DIGEST_MAX_LEN instead of BLAKE2B_MAX_LEN. Determine the
length of the digest for SHA-3. Make sure it is 224, 256, 384, or 512.
(digest_file): Set the digest length in bytes. Use DIGEST_MAX_LEN
instead of BLAKE2B_MAX_LEN. Always append the digest length to SHA3 in
the output.
(main): Allow the use of --length with 'cksum -a sha3'.  Use
DIGEST_MAX_LEN instead of BLAKE2B_MAX_LEN. Make sure it is 224, 256,
384, or 512.
* tests/cksum/cksum-base64.pl (@pairs): Add expected sha3 output.
(fmt): Modify the output to use SHA3-512 since that is the default.
(@Tests): Modify arguments for sha3 to use --length=512.
* tests/cksum/cksum-sha3.sh: New test, based on tests/cksum/b2sum.sh.
* tests/local.mk (all_tests): Add the test.
* bootstrap.conf: Add crypto/sha3.
* gnulib: Update to latest commit.
* NEWS: Mention the change.
* doc/coreutils.texi (cksum general options): Mention sha3 as a
supported argument to the -a option. Mention that 'cksum -a sha3'
supports the --length option. Mention that SHA-3 is considered secure.
2025-09-03 22:29:33 -07:00
Collin Funk 8f6430666f maint: avoid syntax-check failure from previous commit
* tests/seq/seq-long-double.sh: Place comma after "I.e.".
2025-09-02 21:14:34 -07:00
Pádraig Brady 701416709d seq: be more accurate with large integer start values
* src/seq.c (main): Avoid possibly innacurate conversion
to long double, for all digit start values.
* tests/seq/seq-long-double.sh: Add a test case.
* NEWS: Mention the improvement.
Fixes https://bugs.gnu.org/79369
2025-09-02 20:51:55 +01:00
Pádraig Brady ebd670e7eb ls: fix alignment with locale formatted --size
Fix allocated size alignment in locales with multi-byte grouping chars.
Tested with: LC_ALL=sv_SE.utf8 ls --size --block-size=\'k

* src/ls.c (print_file_name_and_frills): Don't rely on
printf("%*s", width, string) to pad multi-byte strings appropriately.
Instead work out the padding required and use:
printf("%*s%s", padding, "", string) to pad multi-byte appropriately.
* tests/ls/block-size.sh: Add a test case.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/79347
2025-08-31 19:25:38 +01:00
Pádraig Brady 735a4a27f3 b2sum: --length: fix upper bound check
* src/digest.c (main): Don't saturate -l to BLAKE2B_MAX_LEN,
so that the subsequent bounds check is performed.
* tests/cksum/b2sum.sh: Add a test case.
* NEWS: Mention the fix introduced in commit v9.5-71-gf2c84fe63
2025-08-30 12:44:29 +01:00
Collin Funk 89b9115da6 fold: fix handling of invalid multi-byte characters
* src/fold.c (fold_file): Continue the loop when we have buffered bytes
but nothing left to read from the file.
(adjust_column): Don't assume that the character is printable.
* tests/fold/fold-characters.sh: Add a new test case.
(bad_unicode): New function.
2025-08-28 18:57:13 -07:00
Pádraig Brady 4b35a3b920 tests: fold: add tests for multi-byte width
* tests/fold/fold.pl: The i18n patch didn't actually test folding
of multi-byte characters, so add tests for various multi-byte forms.
2025-08-27 16:41:07 +01:00
Pádraig Brady 0001bbc3e2 tests: fold: copy i18n patch tests
* tests/fold/fold.pl: Copy tests from Fedora,
removing copy & pasted logic that was
extraneous to either the i18n patch or upstream.
2025-08-27 14:36:16 +01:00
Pádraig Brady cbf6a27810 tests: parameterize IO_BUFSIZE
* src/getlimits.c (main): Output IO_BUFSIZE, useful for
sizing data for tests.
* tests/fold/fold-characters.sh: Use it rather than hardcoding.
2025-08-27 12:03:46 +01:00
Collin Funk ae89cd646a fold: don't truncate multibyte characters at the end of the buffer
* src/fold.c (fold_file): Replace invalid characters with the original
byte read. Copy multibyte sequences that may not yet be read to the
start of the buffer before reading more bytes.
* tests/fold/fold-characters.sh: Add a test case.
2025-08-26 16:59:32 -07:00
Pádraig Brady 01646ccd64 tests: fold: consolidate all fold tests in tests/fold
* tests/misc/fold.pl: Move from here to ...
* tests/fold/fold.pl: ... here.
* tests/local.mk: Adjust accordingly.
2025-08-26 16:49:56 +01:00
Pádraig Brady 17be45b952 tests: fold: add a memory constraint test
Enforcing this interface behavior is worthwhile
irrespective of our current implementation,
to ensure future or other implementations conform.

* tests/fold/fold-characters.sh: Ensure the fold implementation
uses bounded memory.
2025-08-26 16:48:31 +01:00
Pádraig Brady 4bfcf62f74 tests: nproc: fix false failure on some systems
* tests/nproc/nproc-quota.sh: Also simulate sched_getscheduler()
as this will not be called on older or non linux, or
may return ENOSYS on Alpine.
Fixes https://bugs.gnu.org/79299
2025-08-24 11:48:52 +01:00
Pádraig Brady 6c668dc133 tests: cp: ensure copy offload is not disabled for sparse files
Related to commits v9.1-109-g879d2180d and v9.7-248-g306de6c26

* tests/cp/sparse-perf.sh: This edge case was missed a couple of times,
so add a test to ensure we attempt copy offload.
2025-08-23 18:55:46 +01:00
Collin Funk 7d0d9658f9 fold: add the --characters option
* src/fold.c: Include mcel.h.
(count_bytes): Remove variable.
(counting_mode, last_character_width): New variables.
(shortopts, long_options): Add the option.
(adjust_column): If --characters is in used account for number of
characters instead of their width.
(fold_file): Use getline and iterate over the result with mcel functions
to handle multibyte characters.
(main): Check for the option.
* src/local.mk (src_fold_LDADD): Add $(LIBC32CONV), $(LIBUNISTRING), and
$(MBRTOWC_LIB).
* tests/fold/fold-characters.sh: New file.
* tests/fold/fold-spaces.sh: New file.
* tests/fold/fold-nbsp.sh: New file.
* tests/local.mk (all_tests): Add the tests.
* NEWS: Mention the new option.
* doc/coreutils.texi (fold invocation): Likewise.
2025-08-22 22:09:50 -07:00
Pádraig Brady 61a7382c8f tests: nproc: add a test for cgroup quotas
* tests/nproc/nproc-quota.sh: New root only test.
* tests/local.mk: Reference the new test.
2025-08-21 00:04:03 +01:00
Collin Funk 238855dd4e maint: prefer https to http
* doc/sort-version.texi (Other version/natural sort implementations):
Use https in documentation link.
* tests/chmod/symlinks.sh: Use https in license text.
2025-08-19 11:41:17 -07:00
Paul Eggert 39eec70f7b tsort: add do-nothing -w option
This is for conformance to POSIX.1-2024
* src/tsort.c: Include getopt.h.
(main): Accept and ignore -w.  Do not bother altering
the usage message, as the option is useless.
* tests/misc/tsort.pl (cycle-3): New test.
2025-08-14 17:35:23 -07:00
Pádraig Brady 2543c0052c maint: use short form bug URLs
* cfg.mk (sc_prohibit-long-form-bug-urls): Disallow long form in code.
* scripts/git-hooks/commit-msg: Disallow long form in commit messages.
* NEWS: Shorten long urls.
* bootstrap.conf: Likewise.
* configure.ac: Likewise.
* scripts/git-hooks/commit-msg: Likewise.
* src/csplit.c: Likewise.
* src/fmt.c: Likewise.
* src/make-prime-list.c: Likewise.
* src/nohup.c: Likewise.
* tests/od/od-float.sh: Likewise.
* tests/rm/r-root.sh: Likewise.
* tests/tail/inotify-race.sh: Likewise.
* tests/tail/inotify-race2.sh: Likewise.
2025-08-12 17:50:40 +01:00
Collin Funk 2a092e80e2 realpath: support the -E option required by POSIX
* src/realpath.c (longopts): Add the option.
(main): Likewise.
(usage): Add the option to the --help message.
* tests/misc/realpath.sh: Add a simple test.
* doc/coreutils.texi (realpath invocation): Mention the new option.
* NEWS: Likewise.
2025-08-10 11:41:56 -07:00
Pádraig Brady 02c3d43e3a basenc: add base58 support
A 58 character encoding that:
 - avoids visually ambiguous 0OIl characters
 - uses only alphanumeric characters
Described at:
 - https://tools.ietf.org/html/draft-msporny-base58-03

This implementation uses GMP (or gnulib's gmp fallback).
Performance is good in comparison to other implementations.
For example when using libgmp on an i7-5600U system,
encoding is 530 times faster, and decoding 830 times faster
than the implementation using arbitrary precision ints in cpython 3.13.

Memory use is proportional to the size of input.

Encoding benchmarks:

  $ time yes | head -c65535 | src/basenc --base58 -w0 >file.enc
  real    0m0.018s

  ./configure --quiet --without-libgmp && make -j $(nproc)
  $ time yes | head -c65535 | src/basenc --base58 -w0 >file.enc
  real    0m3.431s

  # dnf install python3-base58
  $ time yes | head -c65535 | base58 >file.enc  # cpython 3.13
  real    0m9.700s

Decoding benchmarks:

  $ time src/basenc --base58 -d <file.enc >/dev/null
  real    0m0.010s

  $ ./configure --without-libgmp && make  # gnulib gmp
  $ time src/basenc --base58 -d <file.enc >/dev/null
  real    0m0.145s

  $ time base58 -d <file.enc >/dev/null  # cpython 3.13
  real    0m8.302s

* src/basenc.c (base_decode_ctx_finalize, base_encode_ctx_init,
base_encode_ctx, base_encode_ctx_finalize): New functions to
provide more general processing functionality.
(base58_{de,en}code_ctx{_init,,_finalize}): New functions to
accumulate all input before calling ...
(base58_{de,en}code): ... the GMP based encoding/decoding routines.
(do_encode, do_decode): Call the ctx variants if enabled.
* doc/coreutils.texi (basenc invocation): Describe the new option,
and indicate the main use case being interactive user use.
* src/local.mk: Link basenc with GMP.
* tests/basenc/basenc.pl: Add test cases.
* NEWS: Mention the new feature.
2025-08-09 22:10:27 +01:00
Pádraig Brady c480428e37 basenc: fix stripping of '=' chars in some encodings
* src/basenc.c (do_decode): With -i ensure we strip '=' chars
if there is no padding for the chosen encoding.
* tests/basenc/basenc.pl: Add a test case.
* NEWS: Mention the bug fix.
2025-08-09 09:59:00 +01:00
Collin Funk c9a30d6781 maint: use consistent references to standard files in messages
* cfg.mk (sc_standard_outputs): Add a grep command for source files.
* src/du.c (main): Use standard input instead of stdin, standard output
instead of stdout, and standard error instead of stderr in messages.
* src/nohup.c (main): Likewise.
* src/sort.c (main): Likewise.
* src/split.c (main): Likewise.
* src/stdbuf.c (main): Likewise.
* src/wc.c (main): Likewise.
* tests/du/files0-from.pl (@Tests): Adjust test case to new messages.
* tests/sort/sort-files0-from.pl: Likewise.
* tests/wc/wc-files0-from.pl: Likewise.
2025-08-05 18:46:04 -07:00
Paul Eggert 09fc945b37 tail: refactor to skip stat call on failure
* src/tail.c (tail_bytes): New function.
(tail_bytes, tail_lines, tail): Accept struct stat pointer from
caller instead of calling fstat ourselves.  All callers changed.
(tail_file): Skip a call to fstat if fstat already failed.
* tests/tail/follow-stdin.sh: Adjust to match new behavior
on failure, which omits a redundant diagnostic.
2025-08-03 19:48:05 -07:00
Paul Eggert 6439858056 tail: allow >=2**64 in traditional form
This better matches the treatment of POSIX form,
e.g., ‘tail +Nc’ is now like ‘tail -c +N’ even when N is large.
* src/tail.c: Don’t include xstrtol.h.
(parse_obsolete_option): Treat numbers greater than UINTMAX_MAX as
if they are UINTMAX_MAX.  Parse the number by hand with saturating
arithmetic; nowadays that’s simpler than using xstrtoumax.  There
is no need for a diagnostic now, as the error cannot happen any more.
* tests/tail/tail.pl (obs-plus-c3): New test.
2025-08-03 19:48:04 -07:00
Collin Funk ecd803805d readlink: emit errors when POSIXLY_CORRECT is set
* src/readlink.c (main): Set verbose if the POSIXLY_CORRECT environment
variable is set.
* tests/readlink/readlink-posix.sh: New file.
* tests/local.mk (all_tests): Add it.
* NEWS: Mention the change.
* doc/coreutils.texi (readlink invocation): Document the behavior of
POSIXLY_CORRECT.
2025-08-03 10:55:54 -07:00
Pádraig Brady 798a35a7b3 maint: avoid :> pattern in tests
This was seen to trigger the EXIT trap on cygwin

* cfg.mk (sc_prohibit_colon_redirection): Disallow all
cases of :> in tests/
* tests/cksum/md5sum-bsd.sh: Adjust to avoid more
stringent syntax check.
2025-07-26 20:06:42 +01:00
Collin Funk 93304acc10 tests: avoid a false failure on Cygwin
* tests/cksum/md5sum-bsd.sh: Use 'echo' instead of ':' to redirect to
the file. Otherwise 'Exit $fail' is not called on Cygwin.
2025-07-26 11:29:02 -07:00
Collin Funk 56eaa4ffc3 tests: date: add tests for the Ethiopian calendar
* tests/date/date-ethiopia.sh: New file.
* tests/local.mk (all_tests): Add the test.
2025-07-23 07:49:49 -07:00
Collin Funk a9cd174811 tests: date: add tests for the Solar Hijri calendar
* tests/date/date-iran.sh: New file.
* tests/local.mk (all_tests): Add the test.
2025-07-23 07:49:49 -07:00
Pádraig Brady 5a106af2d6 install: with -d, use more accurate diagnostics
* gnulib: Update to latest to get the fix to
propagate the appropriate error message.
* tests/install/basic-1.sh: Add a test case.
* NEWS: Mention the bug fix.
Addresses https://bugs.gnu.org/79072
2025-07-22 21:15:17 +01:00
Collin Funk 5ab05e5a36 tests: date: add tests for the Thai solar calendar
* tests/date/date-thailand.sh: New file.
* tests/local.mk (all_tests): Add the test.
2025-07-21 21:25:58 -07:00
Pádraig Brady 083f89528e tests: sort: add a test for equivalent floats
* tests/sort/sort-float.sh: Ensure equivalent floats
are treated as such.
2025-07-12 19:58:06 +01:00
Paul Eggert 3a9ec02138 tests: fix fraction comparison in sort-float
Problem reported by Cosima Neidahl <https://bugs.gnu.org/78985#13>.
* tests/sort/sort-float.sh: At top level, use C locale at first.
(dbl_minima_order): Assume C locale.
Use string comparison for the fractional parts.

2025-07-10  Paul Eggert  <eggert@cs.ucla.edu>

tests: fix integer overflow in sort-float
Problem reported by Cosima Neidahl <https://bugs.gnu.org/78985>.
* tests/sort/sort-float.sh (dbl_minima_order):
Use expr instead of test, to avoid problems with integers
too large for the shell.
2025-07-11 14:45:04 -07:00
Paul Eggert 8f9fc8f08c tests: fix integer overflow in sort-float
Problem reported by Cosima Neidahl <https://bugs.gnu.org/78985>.
* tests/sort/sort-float.sh (dbl_minima_order):
Use expr instead of test to compare fractions,
to avoid problems with integers too large for the shell.
2025-07-10 10:18:06 -07:00
Paul Eggert 5037ce930c factor: don’t prove primality
Suggested for consideration by Torbjörn Granlund in:
https://lists.gnu.org/r/coreutils/2025-01/msg00000.html
* src/factor.c (PROVE_PRIMALITY): Now defaults to false.
(mp_prime_p): Help the compiler by telling it mpz_prob_prime_p
returns nonnegative.
* tests/factor/create-test.sh (bigprime): Test 2^400 - 593,
since that’s now practical.
* tests/local.mk (factor_tests): Add new test.
2025-07-09 17:12:40 -07:00
Paul Eggert 831623ba66 factor: switch from mp to single when doable
This significantly improves performance when a number exceeds
2**(W_TYPE_SIZE - 1) and is the product of a prime less than
FIRST_OMITTED_PRIME and another prime less than 2**(W_TYPE_SIZE - 1).
On my platform, for example, it doubled the speed of factoring
4999 * (2**128 - 159).
* src/factor.c (mp_size, mp_finish_in_single): New functions.
(mp_factor_using_division, mp_factor_using_pollard_rho):
Finish using single precision when possible.
* tests/factor/factor.pl (lt-5000-times-128-bit): New test.
2025-07-09 17:12:39 -07:00
Paul Eggert badc24c30c factor: fix test case name
* tests/factor/factor.pl (bug-gmp-plus_2_sup_127_plus_1):
Rename from bug-gmp-plus_2_sup_128_plus_1, so that
it accurately reflects the test case.
2025-07-09 17:12:39 -07:00
Pádraig Brady 8771957703 tests: cat: add test for O_APPEND bug fix
* tests/cat/cat-self.sh: Add a test case
for commit v9.6-19-g7386c291b
2025-07-01 23:32:31 +01:00
Pádraig Brady 0512608a44 tests: avoid false failure due to varied ERANGE errors
* tests/od/od.pl: Use the system error string for ERANGE,
rather than hardcoding a particular one.
Fixes https://bugs.gnu.org/78934
2025-07-01 00:03:30 +01:00
Paul Eggert 5c5f069a4c od: add test cases for recent fix
* tests/od/od.pl: New tests for the offset issues
2025-06-29 22:32:05 -07:00
Paul Eggert 66464e61f5 od: fix '+N.' bug
* src/od.c (parse_old_offset): First arg is now char *,
not char const *.  If a decimal number, temporarily
modify the string so that xstrtoumax does not complain
about the '.'.
* tests/od/od.pl: Test for the bug.
2025-06-28 21:00:41 -07:00
Paul Eggert 88f30ee0a5 od: fix some unlikely integer overflows
* src/od.c (print_n_spaces, pad_at, pad_at_overflow):
New static functions.
(struct tspec, PRINT_FIELDS, print_named_ascii, print_ascii)
(decode_one_format, write_block, main):
Use idx_t, not int, for counts that depend on the number
of bytes in an object.
(decode_one_format): Use print_n_spaces to output spaces.
(PRINT_FIELDS, print_named_ascii, print_ascii):
Use pad_at to avoid integer overflow.
(write_block): Do not use %*s to pad, as the total pad might
exceed INT_MAX.  Instead, pad by hand with putchar (' ').
(main): Use pad_at_overflow to report integer overflow due to
oversize -w.  Use better way to tell whether -w is used,
without needing IF_LINT.
* tests/od/big-w.sh: New test.
* tests/local.mk (all_tests): Add it.
2025-06-28 21:00:41 -07:00
Pádraig Brady 2b7f3621f8 od: output standard diagnostics for invalid -w arguments
* src/od.c (main): Don't pass LONGINT_OK to xstrtol_fatal(),
as otherwise it will abort().
* tests/od/od.pl: Add test cases.
* NEWS: Mention the bug fix.

Addresses https://bugs.gnu.org/78879
2025-06-24 16:37:01 +01:00
Pádraig Brady 44809c3379 od: fix various off-by-one issues with --strings with -N
* src/od.c (dump_strings): There are three related issues here
due to not accounting for the terminating NUL char appropriately.

1. Ensure BUF always has enough space for the terminating NUL.
This avoids CWE-122: Heap-based Buffer Overflow,
where we wrote a single NUL byte directly after the allocated buffer.
I.e., there should be no buffer overflow with:
  printf '%100s' | od -N100 -S1

2. Ensure we support -S == -N (END_OFFSET - STRING_MIN == ADDRESS):
I.e., there should be output with:
  printf '%100s' | od -N10 -S10

3. Ensure we always output a valid address by ensuring
the ADDRESS and I variables are kept in sync.
I.e., this should output address 0000000 not 1777777777777777777777:
  printf '%100s' | od -N10 -S1

As well as fixing these we simplify by using a single loop
to read the data, rather than two.

* doc/coreutils.texi (od invocation): Clarify that -N
implicitly NUL terminates strings.
* tests/od/od-N.sh: Add test cases.
* NEWS: Mention the bug fixes.

Fixes https://bugs.gnu.org/78880
2025-06-24 16:21:00 +01:00
Pádraig Brady 3d35b3c0e5 stty: stricter floating point parsing
* src/stty.c (string_to_baud): Disallow extraneous characters
after floating point numbers.
* tests/stty/stty-invalid.sh: Add test cases.
2025-06-22 17:00:02 +01:00
Paul Eggert aec89a3e7d tty: better fix for Bug#26371
* src/tty.c (TTY_USAGE): Rename from TTY_FAILURE, since this
is used only for usage failures.  All uses changed.
(TTY_TTYNAME_FAILURE): New constant.
(main): Remove no-longer-needed assignment of ENOENT to errno.
Make status-setting clearer too.
Report an error if ttyname fails even though stdin is a terminal,
instead of silently pretending that stdin is not a terminal.
* tests/tty/tty.sh: Test for this issue.  This should fix Bug#78244.
2025-06-20 12:15:27 -07:00
Pádraig Brady efaec80781 tests: stty: adjust tests for arbitary speeds
* tests/stty/stty-invalid.sh: Adjust to what is now invalid.
* tests/stty/stty.sh: Add checks for valid speed variants.
* tests/stty/bad-speed.sh: New test to ensure unsupported speeds
are diagnosed.
2025-06-18 14:21:30 +01:00
Paul Eggert 5c5e8b4a98 date: port test to NetBSD 10
Problem reported by Collin Funk in:
https://lists.gnu.org/r/bug-gnulib/2025-06/msg00094.html
* tests/date/date-debug.sh: Also allow NetBSD 10 mktime behavior.
Although NetBSD contradicts POSIX, POSIX is likely wrong here and
I vaguely recall that there’s a POSIX correction in progress
that will allow the NetBSD behavior.
2025-06-10 13:03:40 -07:00