Commit Graph
9265 Commits
Author SHA1 Message Date
Pádraig Brady faf8d8998c doc: shred: clarify what --verbose outputs
* doc/coreutils.texi (shred invocation): Give more details on what
--verbose outputs, to give some indication of its utility.
* src/shred.c (usage): Likewise.  Also since we're changing the string,
split out translations to give translators more granular translations.
2025-11-14 13:23:09 +00:00
Pádraig Brady e8dcdb9932 sort: avoid pthread calls with --parallel=1
Before this change there were the following unneeded pthread calls:

  $ seq 1e6 | ltrace -c -e 'pthread*' sort --parallel=1 | wc -l
  % time     seconds  usecs/call     calls      function
  ------ ----------- ----------- --------- --------------------
   39.13    0.031757          67       468 pthread_mutex_lock
   37.96    0.030811          65       468 pthread_mutex_unlock
   13.17    0.010691          65       162 pthread_cond_signal
    2.15    0.001747          64        27 pthread_mutex_destroy
    2.00    0.001620          60        27 pthread_mutex_init
    0.70    0.000565          62         9 pthread_cond_destroy
    0.64    0.000518          57         9 pthread_cond_init
  ------ ----------- ----------- --------- --------------------
  100.00    0.081159                  1170 total

* src/sort.c (sort): Avoid merge tree overhead when single threaded.
2025-11-12 14:12:43 +00:00
Pádraig Brady 6f265b515e md5sum: fix --text with the MSYS2 runtime
Note the use of "rt" is non-standard, but we're restricting
its use here to systems that define O_BINARY, which should
invariably support "rt" mode.

* src/digest.c (): Where significant, explicitly use "rt" mode
with --text, as MSYS2 defaults to binary mode for fopen'd files
(though not for standard streams).
* NEWS: Mention the bug fix.
Fixes https://github.com/coreutils/coreutils/issues/123
2025-11-12 14:11:27 +00:00
Pádraig Brady 88d05f1516 maint: fix comment re PRLFS' remoteness
* src/stat.c (human_fstype): prlfs is used in a context of
separate host and guest OS, so should be treated as remote always.
2025-11-12 14:07:31 +00:00
Collin Funk b8675fe98c maint: fix a comment in the generated src/fs-is-local.h file
Reported by Bruno Haible in
<https://lists.gnu.org/r/coreutils/2025-11/msg00113.html>.

* src/extract-magic ($magic_comment): Mention that the function returns
1 if the file system is local and 0 if it is remote.
2025-11-11 15:37:44 -08:00
Collin Funk fc93a44bbe nice: clamp the niceness correctly on GNU/Hurd
* NEWS: Mention the bug fix.
* src/nice.c (MIN_ADJUSTMENT): Set to 0 on the Hurd with glibc ≤ 2.42.
(MAX_ADJUSTMENT): Set to (2 * NZERO - 2) on the Hurd with glibc ≤ 2.42.
(main): Clamp the niceness to be greater or equal to MIN_ADJUSTMENT and
less than or equal to MAX_ADJUSTMENT.
* tests/nice/nice.sh: Add some tests for the Hurd's ranges.
2025-11-09 18:48:34 -08:00
Pádraig Brady cc38da00ef chcon: fix memory leak in error path
* src/chcon.c (change_file_context): If compute_context_from_mask fails,
free the previously allocated file_context.
Fixes https://bugs.gnu.org/79780
2025-11-08 10:41:48 +00:00
Pádraig Brady 4c0cf3864a maint: adjust lseek_copy to handle non zero offsets
* src/copy-file-data.c (lseek_copy): hole_start is initialized
only when ext_start == ipos.
(infer_scantype): Update the hole_start initialization to
the more logically correct POS, even though that init
is only needed to suppress a -Wmaybe-uninitialized warning.
Note gcc 15.2 at least doesn't seem to need that suppression.
2025-11-04 16:32:16 +00:00
Collin Funk 0830838609 tests: date: check that write errors are promptly diagnosed
This improvement is due to changes to Gnulib's fprintftime module.

* NEWS: Mention the improvement.
* src/getlimits.c (OFF64_T_MAX, OFF64_T_MIN): New macros.
(main): Print them.
* tests/misc/write-errors.sh: Call getlimits_. Add a date invocation.
2025-11-03 17:05:57 -08:00
Paul Eggert c9425336f4 doc: squashfs issue to be squashed in Linux 6.18 2025-11-03 11:33:38 -08:00
Pádraig Brady cb0ab4ef39 date: avoid a duplicated write error diagnotic
* src/show-date.c (show_date): Only show the fprintftime() diagnostic
if a further diagnostic will not be shown.
2025-11-03 13:00:32 +00:00
Pádraig Brady d870bfbca1 date: translate/clarify recent error message
* src/show-date.c (show_date): Flagged by syntax-check.
2025-11-03 12:55:35 +00:00
Pádraig Brady 7d17e1d36c date: promptly diagnose write errors with --file
* src/date.c (batch_convert): Check error state of stdout
after each date is processed.
* tests/misc/write-errors.sh: Add a test case.
2025-11-03 12:51:04 +00:00
Paul Eggert fdf88157c6 date: diagnose fprintftime failure
* src/show-date.c (show_date): Diagnose fprintftime failure,
as it need not be an output error.
2025-11-01 18:00:32 -06:00
Paul Eggert 70bb8fa34f pr: improve nstrftime failure check
* src/pr.c (init_header): Do not report an nstrftime EOVERFLOW
error as memory exhaustion.  Instead, output the time as an
integer.  Also, work even if nstrftime (nullptr, SIZE_MAX, ...)
would return PTRDIFF_MAX which means adding 1 would overflow..
2025-11-01 18:00:32 -06:00
Paul Eggert b427c185f2 ls: better nstrftime failure check
* src/ls.c (print_long_format): Streamline the checking
for failure of nstrftime.
2025-11-01 18:00:32 -06:00
Pádraig Brady cf08d2d3b1 build: reduce explicit dependencies on macOS CoreFoundation
* src/local.mk: Revert v9.7-322-gc2e1816a5, instead relying
on the more focused v9.8-79-g532cd66af.  When built with
--disable-nls on macOS this will result in only some commands
being linked with INTL_MACOSX_LIBS, thus resulting in env(1)
at least not setting a __CF_USER_TEXT_ENCODING envirnoment variable.
2025-11-01 15:07:15 +00:00
Grisha Levit 8b111d0274 build: fix macOS build without libintl
A more precise fix than commit v9.7-322-gc2e1816a5

* configure.ac (USE_NLS): new AM_CONDITIONAL.
* src/local.mk: add @INTL_MACOSX_LIBS@ to <prog>_LDADD that need it.
2025-11-01 15:07:13 +00:00
Bruno Haible d5cc0c3b8a maint: ln: eliminate a clang -Wformat-extra-args warning
* src/ln.c (do_link): Don't pass unused arguments to error().
Don't use "%.0s" to consume a string argument without printing it.
2025-11-01 14:15:24 +00:00
Bruno Haible 8db63075a5 maint: ls: adjust to gnulib nstrftime changes
* src/ls.c (align_nstrftime): Change return type to ptrdiff_t.
(print_long_format): Treat a negative return value from align_nstrftime
as failure.
2025-11-01 11:38:15 +00:00
Pádraig Brady 2c5754649e copy: be more defensive/restrictive with posix_fadvise
* src/copy-file-data.c (copy_file_data): Only give the
POSIX_FADV_SEQUENTIAL hint when we _know_ we'll definitely
use a read/write loop to copy the data.  Also only apply
the hint to the whole file, as we've seen OpenZFS at least
special case that.
(sparse_copy): Update stale comment.
2025-11-01 11:24:59 +00:00
Pádraig Brady 64b8fdb5b4 copy: don't avoid copy-offload upon SEEK_HOLE indicating non-sparse
* src/copy-file-data.c (infer_scantype): Fall back to a plain copy
if SEEK_HOLE indicates non-sparse, as zero copy avoids copy offload.
This was seen with transparently compressed files on OpenZFS.
* tests/cp/sparse-perf.sh: Add a test case even though it might
only trigger on compressed file systems that don't support reflink.
* NEWS: Mention the bug fix.
Addresses https://github.com/coreutils/coreutils/issues/122
2025-11-01 11:24:59 +00:00
Pádraig Brady 231cc20195 copy: avoid posix_fadvise bypassing copy offload behavior
* src/copy-file-data.c (): pass 0 to posix_fadvise to indicate to EOF.
coreutils 9.8 used OFF_T_MAX instead, which triggered OpenZFS 2.2.2
at least to synchronously (decompress and) populate the page cache.
Addresses https://github.com/coreutils/coreutils/issues/122
2025-11-01 11:24:59 +00:00
Collin Funk 58a66cf3fd timeout: use fork and execvp instead of posix_spawn
* NEWS: Remove timeout from the list of programs that use posix_spawn.
* bootstrap.conf (gnulib_modules): Remove posix_spawnattr_setsigmask.
* src/timeout.c: Don't include spawn.h.
(main): Use fork and execvp instead of posix_spawn.

This reverts commit dac96ce3e3.
2025-10-31 19:57:36 -07:00
Bruno Haible f287a2940c sort: optimize the lookups of the --compress-program
* src/sort.c (get_resolved_compress_program): New function.
(pipe_child): Use it instead of calling find_in_given_path each time.
2025-10-31 13:37:22 +00:00
Collin Funk cb53662552 maint: avoid syntax-check failure from previous commit
* src/sort.c (pipe_child): Use nullptr instead of NULL.
2025-10-30 22:05:20 -07:00
Bruno Haible 1cfc35ab4c sort: consistently diagnose access issues to --compress-program
* bootstrap.conf (gnulib_modules): Add findprog-in.
* src/sort.c: Include findprog.h.
(pipe_child): Look up the compress_program in $PATH and report errors
such as ENOENT or EACCES before invoking posix_spawnp.
This avoids inconsistency on systems that emulate posix_spawn through
fork/exec, as they would otherwise treat such a failure as a generic
failure and fail the sort, rather than continuing without compression.
2025-10-30 18:34:49 +00:00
Pádraig Brady b294aff3fe sort: fix silent exit upon SIGPIPE from --compress-program
* src/sort.c (main): Ignore SIGPIPE so we've more control over
how we handle for stdout and compression programs.
(sort_die): Handle EPIPE from stdout and mimic a standard SIGPIPE,
otherwise reverting to a standard exit(SORT_FAILURE);
* tests/sort/sort-compress-proc.sh: Add a test case.
* NEWS: Mention the bug fix.
2025-10-29 19:03:23 +00:00
Pádraig Brady 8177cc5b5d numfmt: ensure fields don't split on nbsp
* src/numfmt.c (newline_or_blank): Explicitly ensure
we don't match NBSP as on platforms like NetBSD 10 or Solaris 11,
NBSP is considered a blank character.
This should have been part of commit v9.8-39-g8bc11f80a
Solaris 11 test failure reported by Bruno Haible.
2025-10-28 12:23:28 +00:00
Collin Funk dfcf92e96c maint: prefer FTS_ROOTLEVEL when checking the fts_level of an FTSENT
* src/chcon.c (process_file): Use FTS_ROOTLEVEL instead of 0.
* src/chmod.c (process_file): Likewise.
* src/chown-core.c (change_file_owner): Likewise.
2025-10-27 16:56:01 -07:00
Collin Funk dac96ce3e3 timeout: use the more efficient posix_spawn to invoke the command
* NEWS: Mention the improvement. Consolidate the posix_spawn
improvements into one item.
* bootstrap.conf (gnulib_modules): Add posix_spawnattr_setsigmask.
* src/timeout.c: Include spawn.h.
(main): Setup signals using a posix_spawnattr_t object. Use posix_spawn
instead of fork and execvp.
2025-10-27 16:54:45 -07:00
Collin Funk ffa2632dcd maint: prefer ckd_add for overflow checks
* src/cksum.c (cksum_slice8, crc32b_sum_stream): Use ckd_add to check
for overflow.
* src/cksum_avx2.c (cksum_avx2): Likewise.
* src/cksum_avx512.c (cksum_avx512): Likewise.
* src/cksum_pclmul.c (cksum_pclmul): Likewise.
* src/cksum_vmull.c (cksum_vmull): Likewise.
2025-10-27 16:52:16 -07:00
Collin Funk 2b3eb795c8 sort: use the more efficient posix_spawn to invoke --compress-program
* NEWS: Mention the improvement. Mention that 'sort' will continue
without compressing temporary files if the program specified by
--compress-program cannot be executed.
* doc/coreutils.texi (sort invocation): Document the behavior when the
program specified by --compress-program cannot be executed.
* src/sort.c: Include spawn.h.
(MAX_FORK_TRIES_COMPRESS, MAX_FORK_TRIES_DECOMPRESS): Remove definition.
(MAX_TRIES_COMPRESS, MAX_TRIES_DECOMPRESS): New definitions based on
MAX_FORK_TRIES_COMPRESS and MAX_FORK_TRIES_DECOMPRESS.
(async_safe_die): Remove function.
(posix_spawn_file_actions_move_fd): New function.
(pipe_fork): Remove function.
(pipe_child): New function based on pipe_fork. Return an error number
instead of a pid. Use posix_spawnp instead of calling fork and expecting
the caller to exec.
(maybe_create_temp): Call pipe_child instead of pipe_fork. Print a
warning to standard error if --compress-program cannot be executed and
the error is different than the previous call. Remove code for the child
process.
(open_temp): Remove code for the child process. Improve error message.
* tests/sort/sort-compress.sh: Add a test case for when the program
specified by --compress-program does not exist.
2025-10-25 12:59:08 -07:00
Collin Funk a2a2f22efe maint: remove unnecessary ignore_value usage
* src/install.c: Remove ignore-value.h include.
(strip): Don't use ignore_value on posix_spawnattr_destroy since it
isn't declared with the warn_unused_result attribute. Pass the checked
pointer to posix_spawnattr_destroy instead of the variable it points to.
2025-10-23 14:40:39 -07:00
Bernhard Voelker 8b60d7a5aa install: enhance error diagnostic when running strip fails
* src/install.c (strip): Mention explicitly that the strip program
failed in the case the user specified a custom strip program name.
2025-10-23 22:37:16 +02:00
Collin Funk 2213542e40 split: cleanup after posix_spawn
* boostrap.conf (gnulib_modules): Add posix_spawn_file_actions_destroy.
* src/split.c (cleanup): Call posix_spawnattr_destroy and
posix_spawn_file_actions_destroy after a successful posix_spawn.
2025-10-23 12:42:01 -07:00
Collin Funk ca8b928665 split: prefer posix_spawn to fork and execl
* NEWS: Mention the change.
* bootstrap.conf (gnulib_modules): Add posix_spawn,
posix_spawnattr_setsigdefault, posix_spawn_file_actions_addclose,
posix_spawn_file_actions_adddup2, and posix_spawn_file_actions_init.
* src/split.c: Include spawn.h.
(create): Use posix_spawn instead of fork and execl.
2025-10-23 11:24:06 -07:00
Pádraig Brady c5b147fabf pr: promptly diagnose write errors
* src/pr.c (print_page): Exit promptly for `yes | pr`.
(print_clump): Exit promptly for `pr < /dev/zero`.
* tests/misc/write-errors.sh: Enable test cases.
* NEWS: Mention the improvement.
2025-10-22 19:11:37 +01:00
Pádraig Brady 9415cf01c6 nl: promptly diagnose write errors
* NEWS: Mention the improvement.
* src/nl.c (process_file): Exit if error outputting line.
* tests/misc/write-errors.sh: Enable the test case.
2025-10-22 16:32:34 +01:00
Pádraig Brady c12a2ecb53 fmt: promptly diagnose write errors
* NEWS: Mention the improvement.
* src/fmt.c (put_line): Exit if any error writing line.
(flush_paragraph): Exit if any error writing buffer.
* tests/misc/write-errors.sh: Enable the (flush_paragraph) test case,
and add another to check the put_line() case.
2025-10-22 16:18:06 +01:00
Pádraig Brady 5971f450b5 numfmt: promptly diagnose write errors
* src/numfmt.c (process line): Inspect the stdio error state when
outputting each line so that we don't have to check each output function
but do eventually exit upon write error, while also remaining buffered.
(main): Also check when outputting a header for the edge case
of very long headers.
* tests/misc/write-errors.sh: Enable the numfmt test case.
* NEWS: Mention the improvement, and reorganize all numfmt improvements.
2025-10-22 15:53:06 +01:00
Collin Funk 0c1334eb68 install: prefer posix_spawnp to fork and execlp
* NEWS: Mention the change.
* bootstrap.conf (gnulib_modules): Add posix_spawnattr_destroy,
posix_spawnattr_init, posix_spawnattr_setflags, and posix_spawnp.
* src/install.c (strip): Use posix_spawnp instead of fork and execlp.
2025-10-21 20:26:59 -07:00
Pádraig Brady e58f9390f1 numfmt: optimize multi-byte --delimiter search
* src/numfmt.c (is_utf8_charset): A new function to efficiently
determine if running with a UTF-8 charset.
(mbsmbchr): A new function to efficiently search for
a (multi-byte) character in a multi-byte string.
(next-field): Use mbsmbchr() rather than mbstr() directly.
2025-10-20 13:13:25 +01:00
Pádraig Brady 09af58f4f9 numfmt: support multi-byte --delimiter
* bootstrap.conf: Depend on mbsstr() to robustly search for a
multi-byte delimiter character (string) within a multi-byte string.
* src/numfmt.c (main): Accept a valid multi-byte delimiter character.
(next_field): Adjust delimiter search from single byte
to multi-byte aware.  Use mbsstr to find the first match.
* tests/misc/numfmt.pl: Add test case.
* NEWS: Mention the improvement.
2025-10-20 13:13:24 +01:00
Pádraig Brady e5d7764386 numfmt: use multi-byte aware suffix matching
* src/numfmt.c (process_suffixed_number): Use gnulib's
mbs_endswith() helper, which is more robust in non UTF-8 locales.
Also always output a devmsg if a suffix is specified.
2025-10-18 20:03:57 +01:00
Pádraig Brady 770078e315 numfmt: fix issues with multi-byte blanks
* src/numfmt.c (process_line): Restore byte overwritten with NUL,
as it may be part of a multi-byte blank.
(process_suffixed_number): Skip multi-byte blanks,
and correctly determine width with mbswidth().
(parse_format_string): Use c_isblank() to explicitly
indicate that's all the format spec supports.
* tests/misc/numfmt.pl: Add test cases.
* NEWS: Mention the bug fix.
2025-10-18 18:37:10 +01:00
Pádraig Brady 0f7babad38 numfmt: add --unit-separator
Output, accept, or disallow a string between the number and unit
as recommended in <https://physics.nist.gov/cuu/Units/checklist.html>
I.e. support outputting numbers of the form: "1234 M"

* src/numfmt.c (simple_strtod_human): Skip unit separator if present,
or disallow a unit separator if empty.
(double_to_human): Output unit separator if specified.
(main): Accept --unit-separator.
* tests/misc/numfmt.pl: Add test cases.
* doc/coreutils.texi: Describe the new option,
giving examples of interaction with --delimiter.
* NEWS: Mention the new feature.
* THANKS.in: Add Johannes Schauer Marin Rodrigues,
who provided a preliminary patch.
2025-10-17 19:16:26 +01:00
Pádraig Brady 006dfe4cd4 numfmt: support reading numbers with grouping characters
This does not validate grouping character placement,
and currently just ignores grouping characters.

* src/numfmt.c (simple_strtod_int): Skip grouping chars
that are part of a number.
* tests/misc/numfmt.pl: Add test cases.
* NEWS: Mention the improvement.
2025-10-17 17:26:30 +01:00
Pádraig Brady 8bc11f80a3 numfmt: support reading numbers with NBSP before unit
* src/numfmt.c (simple_strtod_human): Accept (multi-byte)
non-breaking space character between number and unit.
Note we restrict this to a single character between number
and unit, to allow less ambiguous parsing if multiple blanks
are used to delimit fields.
* tests/misc/numfmt.pl: Add test cases.
* doc/coreutils.texi (numfmt invocation): Fix stale description
--delimiter skipping whitespace.
* NEWS: Mention the improvement.
2025-10-17 17:26:25 +01:00
Pádraig Brady b880b02c44 numfmt: fix buffer over-read (CWE-126)
* src/numfmt.c (simple_strtod_human): Check for NULL after pointer
adjustment to avoid Out-of-range pointer offset (CWE-823).
* NEWS: Mention the fix.
2025-10-11 12:37:56 +01:00