* NEWS: Mention the improvement.
* src/fmt.c (put_line): Exit if any error writing line.
(flush_paragraph): Exit if any error writing buffer.
* tests/misc/write-errors.sh: Enable the (flush_paragraph) test case,
and add another to check the put_line() case.
* src/numfmt.c (process line): Inspect the stdio error state when
outputting each line so that we don't have to check each output function
but do eventually exit upon write error, while also remaining buffered.
(main): Also check when outputting a header for the edge case
of very long headers.
* tests/misc/write-errors.sh: Enable the numfmt test case.
* NEWS: Mention the improvement, and reorganize all numfmt improvements.
* NEWS: Mention the change.
* bootstrap.conf (gnulib_modules): Add posix_spawnattr_destroy,
posix_spawnattr_init, posix_spawnattr_setflags, and posix_spawnp.
* src/install.c (strip): Use posix_spawnp instead of fork and execlp.
* src/numfmt.c (is_utf8_charset): A new function to efficiently
determine if running with a UTF-8 charset.
(mbsmbchr): A new function to efficiently search for
a (multi-byte) character in a multi-byte string.
(next-field): Use mbsmbchr() rather than mbstr() directly.
* bootstrap.conf: Depend on mbsstr() to robustly search for a
multi-byte delimiter character (string) within a multi-byte string.
* src/numfmt.c (main): Accept a valid multi-byte delimiter character.
(next_field): Adjust delimiter search from single byte
to multi-byte aware. Use mbsstr to find the first match.
* tests/misc/numfmt.pl: Add test case.
* NEWS: Mention the improvement.
* src/numfmt.c (process_suffixed_number): Use gnulib's
mbs_endswith() helper, which is more robust in non UTF-8 locales.
Also always output a devmsg if a suffix is specified.
* src/numfmt.c (process_line): Restore byte overwritten with NUL,
as it may be part of a multi-byte blank.
(process_suffixed_number): Skip multi-byte blanks,
and correctly determine width with mbswidth().
(parse_format_string): Use c_isblank() to explicitly
indicate that's all the format spec supports.
* tests/misc/numfmt.pl: Add test cases.
* NEWS: Mention the bug fix.
Output, accept, or disallow a string between the number and unit
as recommended in <https://physics.nist.gov/cuu/Units/checklist.html>
I.e. support outputting numbers of the form: "1234 M"
* src/numfmt.c (simple_strtod_human): Skip unit separator if present,
or disallow a unit separator if empty.
(double_to_human): Output unit separator if specified.
(main): Accept --unit-separator.
* tests/misc/numfmt.pl: Add test cases.
* doc/coreutils.texi: Describe the new option,
giving examples of interaction with --delimiter.
* NEWS: Mention the new feature.
* THANKS.in: Add Johannes Schauer Marin Rodrigues,
who provided a preliminary patch.
This does not validate grouping character placement,
and currently just ignores grouping characters.
* src/numfmt.c (simple_strtod_int): Skip grouping chars
that are part of a number.
* tests/misc/numfmt.pl: Add test cases.
* NEWS: Mention the improvement.
* src/numfmt.c (simple_strtod_human): Accept (multi-byte)
non-breaking space character between number and unit.
Note we restrict this to a single character between number
and unit, to allow less ambiguous parsing if multiple blanks
are used to delimit fields.
* tests/misc/numfmt.pl: Add test cases.
* doc/coreutils.texi (numfmt invocation): Fix stale description
--delimiter skipping whitespace.
* NEWS: Mention the improvement.
* src/digest.c (split_3): Mention the ambiguity in misinterpreting
base64 characters as hex is not a practical consideration.
Also add an example of both tagged formats which makes it
easier to interpret the parsing logic.
* src/digest.c (split_3): Fallback to untagged matching in the
case where -a is specified and we have matched a TAG in
the possibly base64 data. This might happen in 1 in every 64K files.
Note we remove the modification of string S (and redundant streq) in
the tag matching, as that was not needed since v8.32-223-g217cd278e.
* tests/cksum/cksum-c.sh: Add a test case.
* NEWS: Mention the bug fix.
* src/digest.c (sha2_sum_stream): Change from unreachable()
to affirm() so that we have defined behavior unless
we configure with --disable-assert.
(sha3_sum_stream): Likewise.
(split_3): Validate SHA2-lengths before passing on.
* tests/cksum/cksum-c.sh: Add a test case.
* NEWS: Mention the bug fix.
* src/digest.c (split_3): Look up the provided tag with -a sha2
because there is not a 1:1 mapping between them.
* tests/cksum/cksum-c.sh: Add a test case.
* NEWS: Mention the bug fix.
* src/remove.c (rm_fts): When not recursive,
arrange for ‘rm -d DIR’ to behave more like ‘rmdir DIR’.
This works better for Ceph snapshot directories.
Problem reported by Yannick Le Pennec (bug#78245).
* NEWS: Mention the bug.
* src/digest.c (split_3): Check that the base64 digest matches the
length supported by the algorithm.
(digest_check): Check that the read digest matches the base64 length of
the algorithm's digest. The previous condition would not work for
'cksum -a blake2b -l 8 ...'.
* tests/cksum/cksum-base64-untagged.sh: New file.
* tests/local.mk (all_tests): Add the new test.
This avoids CWE-122: Heap-based Buffer Overflow
where we could write blank characters beyond
the allocated heap buffer.
* src/expand-common.c (set_max_column_width): Refactor function from ...
(add_tab_stop): ... here.
(set_extend_size): Call new function.
(set_increment_size): Likewise.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/79555
* gl/modules/mbbuf: New file.
* gl/lib/mbbuf.c: Likewise.
* gl/lib/mbbuf.h: Likewise.
* gl/local.mk (EXTRA_DIST): Add the new files.
* bootstrap.conf (gnulib_modules): Add mbbuf.
* src/fold.c: Include mbbuf.h.
(fold_file): Use the mbbuf functions instead of calling fread and
handling the input buffer ourselves.
* cfg.mk (exclude_file_name_regexp--sc_preprocessor_indentation)
(exclude_file_name_regexp--sc_GPL_version): Match gl/lib/mbbuf.c and
gl/lib/mbbuf.h.
* configure.ac: Add detection of AVX512 intrinsics for wc.
* src/local.mk: Build AVX512 wc libraries.
* src/wc.c: Add runtime detection of AVX512 intrinsics and call
appropriate function when detected.
* src/wc.h (wc_lines_avx512): Declare function.
* tests/wc/wc-cpu.sh: Add a test that disables AVX512 intrinsics.
* src/wc_avx512.c: New file containing the wc -l implementation using
AVX512. The logic and code is reused from the AVX2 implementation with
slight adaptations. Replaced __builtin_popcount by __builtin_popcountll
and the combination of _mm256_cmpeq_epi8 and _mm256_movemask_epi8 by a
single call to _mm512_cmpeq_epi8_mask.
* NEWS: Mention the improvement.
Prompted by the syntax-check failure:
maint.mk: the above files include argmatch.h but don't use it
make: *** [maint.mk:741: sc_prohibit_argmatch_without_use] Error 1
* src/join.c: Remove include, as the previous commit changed from using
ARRAY_CARDINALITY to countof - for which system.h includes the header.
* src/tail.c (file_lines): Seek to the previous block instead of the
beginning (or a little before) of the block that was just scanned.
Otherwise, the same block is read and scanned (at least partially)
again. This bug was introduced by commit v9.7-219-g976f8abc1.
* tests/tail/basic-seek.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: mention the bug fix.
base58_length() operated naively on an int
which resulted in an overflow to a negative number
for any input > 2^31-1/138, i.e. 15,561,475 bytes.
* src/basenc.c (base_length): Change input and output
parameter types from int to idx_t since this needs to
cater for the full input size in the base58 case.
(base58_length): Likewise. Also reorder the calculation
to be less exact, but doing the division first
to minimize the chance of overflow (which now on 64 bit
would only happen for inputs > around 6 Exa bytes).
* tests/basenc/basenc-large.sh: Add a new test,
that triggers with valgrind or ASAN.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
* src/cksum.c (pclmul_supported): In the case where
gnulib has pclmul enabled but coreutils does not,
ensure we don't reference the coreutils pclmul function.
Addresses https://bugs.gnu.org/79491
* src/dd.c (v): Add the O_EXCL flag to the set of bits that we do not
want to use for our definitions.
* cfg.mk (sc_dd_O_FLAGS): Adjust to pass syntax-check.
* NEWS: Mention the fix.
Reported by Bruno Haible.
* src/local.mk: Explicitly depend on @INTL_MACOS_LIBS@
which may be not implicitly referenced (in LIBINTL) without gettext.
This is a new transitive dependency through localename-unsafe.
We add this globally to ease future maintenance as currently
6 commands require the localename-unsafe dependency:
date, du through show-date() (fprintftime), and
ls, pr, stat, uptime through strftime().
Use new Gnulib streq function instead of rolling our own macro.
* bootstrap.conf (gnulib_modules): Add stringeq.
* src/rm.c (main): Don’t assume streq is a macro that expands to (...),
as it is now a function.
* src/system.h:
* tests/df/no-mtab-status.sh, tests/df/skip-duplicates.sh:
(STREQ): Remove. All uses replaced by streq.
* src/comm.c (compare_files): Use the "%ju" printf directive instead of
"%s" and printing the result of umaxtostr.
* src/df.c (get_header): Likewise.
* src/ls.c (print_long_format): Likewise.
* src/sort.c (check): Likewise.
* src/fold.c (write_out): A new helper to check all writes.
(fold-file): Use write_out() for all writes.
* tests/fold/fold-zero-width.sh: Adjust to writing more
data in various patterns, rather than two buffers of NULs.
This is a more robust memory bound check, and the '\303' case
tests this particular logic change.
* NEWS: fold now exits immediately, not just promptly.
* NEWS: Mention the improvement.
* src/fold.c (fold_file): Check for write errors
after each buffer read from stdin.
* tests/misc/write-errors.sh: Add test cases.
* src/copy-file-data.c (sparse_copy, lseek_copy)
(copy_file_data): Prefer idx_t to size_t.
(copy_file_data): Defend against st_size < 0, which can happen on
ancient buggy platforms. Check for overflow; the old code was
wrong on theoretical-but-valid hosts where SIZE_MAX <= INT_MAX.
* src/copy-file-data.c (write_zeros): New args abuf, buf_size.
Use the lazily-allocated buffer, which most likely already exists,
rather than allocating a separate buffer just for zeros.
This makes the code more reentrant as there is no longer
a need for static storage here. Although there is some CPU
overhead due to the need to zero out the buffer for each file,
the overhead is relatively small as the buffer is smallish
and should be cached. All callers changed.
This doesn’t change behavior; it simplifies future changes.
* src/copy-file-data.c (sparse_copy, lseek_copy, copy_file_data):
Return the number of bytes copied, or -1 on failure,
instead of merely returning a success indication.
All callers changed.
This does not affect current coreutils behavior;
it is merely to help make copy_file_data more useful in the future.
* src/copy-file-data.c (lseek_copy): New arg ibytes.
Caller changed.
(copy_file_data): Implement the ibytes arg; formerly
it was always treated as COUNT_MAX, though all callers
currently pass COUNT_MAX so there was no problem in practice.