Commit Graph
1648 Commits
Author SHA1 Message Date
Paul Eggert d9dc02b09a doc: mention -lcrypto change in NEWS 2024-02-25 21:24:04 -08:00
Daan De Meyer a357cfb021 cp: add --keep-directory-symlink option
When recursively copying files into OS trees, it often happens that
some subdirectory of the source directory is a symlink in the target
directory. Currently, cp will fail in that scenario with the error:

"cannot overwrite non-directory %s with directory %s"

However, we'd like cp in this scenario to follow the destination
directory symlink and copy the files into the symlinked directory
instead. Let's support this by adding a new option
--keep-directory-symlink that makes cp follow destination directory
symlinks.

We name the option --keep-directory-symlink to keep consistent with
tar which has the same option with the same effect.

* doc/coreutils.texi (cp invocation): Describe the new option.
* src/copy.h: Add the new setting.
* src/copy.h: Adjust to follow symlinks if setting enabled.
* src/cp.c (usage): Describe the new option.
(main): Accept the new option.
* tests/cp/keep-directory-symlink.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
2024-02-22 14:37:47 +00:00
Pádraig Brady 76604db7d2 od: support half precision floating point
Rely on compiler support for _Float16 and __bf16
to support -fH and -fB formats respectively.
I.e. IEEE 16 bit, and brain 16 bit floats respectively.
Modern GCC and LLVM compilers support both types.

clang-sect=half-precision-floating-point
https://gcc.gnu.org/onlinedocs/gcc/Half-Precision.html
https://clang.llvm.org/docs/LanguageExtensions.html#$clang-sect
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0192r4.html
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p1467r9.html

This was tested on:
gcc 13, clang 17 x86 (Both types supported)
gcc 7 aarch64 (Only -fH supported)
gcc 13 ppc(be) (Neither supported. Both will be with GCC 14)

* src/od.c: Support -tf2 or -tfH to print IEEE 16 bit floating point,
or -tfB to print Brain 16 bit floating point.
* configure.ac: Check for _Float16 and __bf16 types.
* doc/coreutils.texi (od invocation): Mention the new -f types.
* tests/od/od-float.sh: Add test cases.
* NEWS: Mention the new feature.
Addresses https://bugs.gnu.org/68871
2024-02-05 13:30:45 +00:00
Pádraig Brady c01ee18934 doc: split -C: test and document a heap overflow
This was introduced in coreutils 9.2 through commit v9.1-184-g40bf1591b,
and was fixed in coreutils 9.5 through commit v9.4-111-gc4c5ed8f4.
This issue has been assigned CVE-2024-0684.

* NEWS: Mention the bug fix.
* tests/split/line-bytes.sh: Add a test case.
Reported by Valentin Metz.
2024-01-18 16:52:36 +00:00
Pádraig Brady a966dcdb69 maint: update all copyright year number ranges
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...

* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Manually update copyright year,
until we fully sync with gnulib at a later stage.
* tests/sample-test: Adjust to use the single most recent year.
2024-01-01 13:27:23 +00:00
Pádraig Brady 9e67ccd745 sort: fix thousands grouping handling on single byte locales
* gl/lib/strnumcmp-in.h (numcompare): After commit v9.0-8-g6cafb122f,
we need to treat characters as signed to avoid invalid comparisons
between negative integers and unsigned characters.
* NEWS: Mention the bug fix.
2023-12-29 18:04:36 +00:00
Christian Göttsche 98d463ef5d copy,install: avoid unnecessary security context translations
Do not perform SELinux context translation for operations not involving
user input or output.  Context translation converts MCS/MLS labels into
human readable form, which is useful for user facing applications like
ls(1) or the --context=CTX argument of cp(1).

* src/copy.c (set_process_security_ctx): Use raw selinux variants.
* src/install.c (need_copy): Likewise.
(setdefaultfilecon): Likewise.
* src/selinux.c (computecon): Likewise.
(defaultcon): Likewise.
* tests/cp/no-ctx.sh: Add raw variants to preload lib.
* NEWS: Mention the improvement.
2023-12-19 17:34:33 +00:00
Pádraig Brady da091b3ab3 chgrp: add --from parameter similar to chown
* doc/coreutils.texi (chown invocation): Convert --from option
description to a macro and call from ...
(chgrp description): ... here.
* src/chown-core.h (emit_from_option_description): A new function
refactored from ...
* src/chown.c (usage): ... here, and called from ...
* src/chgrp.c (usage): ... here.
(main): Accept the --from option as chown(1) does.
* po/POTFILES.in: Add chown-core.h as now translated.
* tests/chown/basic.sh: Decouple the root user from id 0.
* tests/chgrp/from.sh: A new test largely based on chown/basic.sh.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
Suggested by Ed Neville.
2023-12-15 13:19:27 +00:00
dann frazier 73d119f4f8 tail: fix tailing sysfs files where PAGE_SIZE > BUFSIZ
* src/tail.c (file_lines): Ensure we use a buffer size >= PAGE_SIZE when
searching backwards to avoid seeking within a file,
which on sysfs files is accepted but also returns no data.
* tests/tail/tail-sysfs.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/67490
2023-12-01 23:01:32 +00:00
Pádraig Brady 615167cc4d numfmt: support lowercase 'k' for Kilo and Kibi
For consistency with the "SI" standard, and with other coreutils
which output a lowercase 'k' in "SI" mode.

* src/numfmt.c (suffix_power): Treat 'k' like 'K' on input.
(double_to_human): Output lowercase 'k' in SI mode.
(usage): Adjust accordingly.
* doc/coreutils.texi: Mention 'k' accepted, and printed in SI mode.
* tests/misc/numfmt.pl: Adjust accordingly.
* NEWS: Mention the change in behavior.
Fixes https://bugs.gnu.org/47103
2023-11-27 19:41:49 +00:00
Paul Eggert 11b01fc21f join,uniq: support multi-byte separators
* NEWS: Mention this.
* bootstrap.conf (gnulib_modules): Remove cu-ctype, as this module
is now more trouble than it’s worth.  All uses removed.
Add skipchars.
* gl/lib/cu-ctype.c, gl/lib/cu-ctype.h, gl/modules/cu-ctype:
Remove.
* gl/lib/skipchars.c, gl/lib/skipchars.h, gl/modules/skipchars:
* tests/misc/join-utf8.sh:
New files.
* src/join.c: Include skipchars.h and mcel.h instead of cu-ctype.h.
(tab): Now mcel_t, not int.  All uses changed.
(output_separator, output_seplen): New static vars.
(eq_tab, newline_or_blank, comma_or_blank): New functions.
(xfields, prfields, prjoin, add_field_list, main):
Support multi-byte characters.
* src/numfmt.c: Include ctype.h, skipchars.h.
Do not include cu-ctype.h.
(newline_or_blank): New function.
(next_field): Support multi-byte characters.
* src/sort.c: Include ctype.h instead of cu-ctype.h.
(inittables): Open-code field_sep since it no longer exists.
‘sort’ is not multi-byte safe yet, but when it is this code
will need revamping anyway.
* src/uniq.c: Include mcel.h and skipchars.h instead of cu-ctype.h.
(newline_or_blank): New function.
(find_field): Support multi-byte characters.
* tests/local.mk (all_tests): Add tests/misc/join-utf8.sh
2023-10-30 00:58:04 -07:00
Pádraig Brady 8c735f6585 base32,base64: disallow non-canonical encodings
This will make decoding more resilient to corruption
whether due to transmission errors or nefarious adjustment.
See https://eprint.iacr.org/2022/361.pdf

* gnulib: Update to commit 3f463202bd enforcing canonical encoding.
* tests/basenc/base64.pl: Add test cases, and adjust existing cases.
* NEWS: Mention the change in behavior.
2023-10-28 13:13:34 +01:00
Pádraig Brady d733f2ec26 basenc: --base16: support lower case hex digits
* src/basenc.c (base16_decode_ctx): Convert to uppercase
before converting from hex.
* tests/basenc/basenc.pl: Add a test case.
* NEWS: Mention the change in behavior.
Addresses https://bugs.gnu.org/66698
2023-10-23 14:04:38 +01:00
Pádraig Brady 378dc38f48 basenc: auto pad base32 and base64 inputs when decoding
Padding of encoded data is useful in cases where
base64 encoded data is concatenated / streamed.
I.e. where there are padding chars _within_ the stream.
In other cases padding is optional and can be inferred.
Note we continue to treat partial padding as invalid,
as that would be indicative of truncation.

* src/basenc.c (do_decode): Auto pad the end of the input.
* NEWS: Mention the change in behavior.
* tests/misc/base64.pl: Adjust to not fail for missing padding.
Addresses https://bugs.gnu.org/66265
2023-10-06 18:21:12 +01:00
Paul Eggert 17a9e79023 wc: 3× speedup in C locale
The 3× speedup was measured by invoking 'wc $(find * -type f)'
on the coreutils sources etc. on an Ubuntu 23.04 x86-64.
These changes also speed up wc 20% in UTF-8 locales.
* src/wc.c (wc_isprint, wc_isspace): New static vars.
(wc): Use them for speed.
(main): Initialize them if needed.
(isnbspace): Remove; no longer used.
2023-09-23 00:28:27 -07:00
Paul Eggert bee39b93f5 wc: treat encoding errors as non white space
* src/wc.c (wc): Treat encoding errors like non white space
characters.
2023-09-23 00:28:27 -07:00
Paul Eggert 31076e8689 wc: fix word count bug
* bootstrap.conf (gnulib_modules): Remove c32isprint.
* src/wc.c (wc): Consider all non-white-space characters
to be word constituents, even if they are not printable.
POSIX requires this, and it is what BSD does.
Partly do this by simplifying the check for a word,
by counting word starts rather than word ends.
* tests/wc/wc.pl: Test for the bug.
2023-09-23 00:28:27 -07:00
Stephen Kitt d24a117707 tail: allow multiple PIDs
tail can watch multiple files, but currently only a single writer. It
can be useful to watch files from multiple writers, or even processes
not directly related to the files (e.g. watch log files written by a
server process, for the duration of a test driven by a separate
client).

* src/tail.c (writers_are_dead): New function.
(tail_forever): Use it to wait for writers.
(tail_forever_inotify): As above.
(parse_options): Manage --pid options in an array.
* doc/coreutils.texi: Update documentation.
* tests/tail/pid.sh: Add a variant with two PIDs.
* News: Mention the new feature.
2023-09-20 15:53:34 +01:00
Sylvestre Ledru 8367b95a13 ls: --dired now implies long format with hyperlinks disabled
Currently --dired is silently ignored
with conflicting output formats

* src/ls.c (decode_switches): Set default format and hyperlink mode
when the --dired option is specified.
* tests/ls/dired.sh: Check that formats are implied / overridden.
* NEWS: Mention the change in behavior.
* doc/coreutils.texi (ls invocation): Adjust --dired description.
2023-09-17 18:47:50 +01:00
Paul Eggert 9cd52bd999 cp,mv,install: fix chmod on Linux CIFS
This bug occurs only when temporarily setting the mode to the
intersection of old and new modes when changing ownership.
* src/copy.c (owner_failure_ok): Treat EACCES like EPERM.
2023-09-02 13:28:23 -07:00
Paul Eggert 5f97136160 cp,mv,install: fix chown on Linux CIFS
* src/copy.c (chown_failure_ok): Also treat EACCES as OK.
2023-09-01 15:10:45 -07:00
Paul Eggert 8463f41e4d maint: remove rename macro
* src/copy.h (rename) [RENAME_TRAILING_SLASH_BUG]:
Remove: unused, now that Gnulib takes care of this.
2023-08-29 23:57:40 -07:00
Pádraig Brady 632beba630 maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2023-08-29 15:23:14 +01:00
Pádraig Brady 9530a14420 version 9.4
* NEWS: Record release date.
2023-08-29 14:36:22 +01:00
Paul Eggert 2dddc87214 maint: spelling fixes, including author names
Most of this just affects commentary and documentations.  The only
significant behavior change is translating author names via
proper_name_lite rather than proper_name_utf8, or not translating
them at all.  proper_name_lite is good enough for coreutils and
avoids the bloat that had coreutils not using Gnulib proper_name.
* bootstrap.conf (gnulib_modules): Use propername-lite instead
of propername.
(XGETTEXT_OPTIONS): Look for proper_name_lite instead of for
proper_name_utf8.
* cfg.mk (local-checks-to-skip): Remove
sc_proper_name_utf8_requires_ICONV, since we no longer use
proper_name_utf8.
(old_NEWS_hash): Update.
(sc_check-I18N-AUTHORS): Remove; no longer needed.
2023-08-28 14:06:43 -07:00
Pádraig Brady 13f439ce56 shred: fix operation on Solaris with 64 bit builds
* NEWS: Mention the bug fix.
* gl/lib/randread.c (get_nonce): Limit getrandom() <= 1024 bytes.
2023-08-23 15:23:42 +01:00
Pádraig Brady 4f92de5822 cp: with --sparse=never, avoid COW and copy offload
* src/cp.c (main): Set default reflink mode appropriately
with --sparse=never.
* src/copy.c (infer_scantype): Add a comment to related code.
* tests/cp/sparse-2.sh: Add a test case.
* NEWS: Mention the bug.
2023-08-21 14:28:14 +01:00
Pádraig Brady 20b9a015e5 maint: remove extraneous line in NEWS
* NEWS: Remove extraneous line.
2023-08-16 11:47:54 +01:00
Paul Eggert f2c2643fc5 maint: update uptime NEWS
* NEWS: Update as per Bruno Haible <https://bugs.gnu.org/65255#14>.
2023-08-15 15:22:27 -07:00
Bernhard Voelker 0cb8332196 maint: fix typo in NEWS
* NEWS: s|/who/log/wtmp|/var/log/tmp|, introduced in commit 85edb4afbd.
2023-08-15 13:35:13 +02:00
Paul Eggert 85edb4afbd doc: improve NEWS discussion of systemd 2023-08-12 23:51:59 -07:00
Pádraig Brady 5420604332 doc: separate out description of 2038 time stamp change
* NEWS: Separate out the description of the _existing_ issues
with outputting timestamps on 32 bit systems, from _future_ issues
outputting timestamps on all systems.  Also move this to the
"improvement" section, since it's not really a coreutils
specific issue, and also is a build time configurable option.
2023-08-11 14:34:49 +01:00
Paul Eggert 0e58c95000 maint: Update after gnulib module 'readutmp' changed
(This patch is coauthored with Bruno Haible,
with original version at <https://bugs.gnu.org/64937#>.)
This updates the gnulib submodule to latest.
For year-2038 safety on Linux/{x86,arm},
this adds an --enable-systemd option to ‘configure’.
The idea is that this sort of thing will become the default
after it has been tested more.
* configure.ac: Don't test whether struct utmp and struct utmpx
have the ut_host field; this is now done in gnulib's readutmp module.
* src/local.mk: Link the programs 'pinky', 'uptime', 'users',
'who' with $(READUTMP_LIB).
* src/pinky.c, src/who.c:
Test HAVE_STRUCT_XTMP_UT_HOST instead of HAVE_UT_HOST.
* src/pinky.c (print_entry):
* src/who.c (print_user, print_deadprocs, print_login)
(print_initspawn, scan_entries):
Support the situation where ut_line is a 'char *' rather than a
'char[]' of fixed size.  Likewise for ut_user and ut_host.
(make_id_equals_comment): Likewise for ut_id.
* src/pinky.c (print_entry):
* src/who.c (print_user):
Open /dev to simplify looking up its entries.
Don’t use printf if the output might in theory be longer than INT_MAX.
* src/pinky.c (scan_entries, short_pinky):
* src/uptime.c (print_uptime, uptime):
* src/users.c (list_entries_users, users):
* src/who.c (who):
Use idx_t where new read_utmp needs it.
* src/system.h (STREQ_LEN): Add comment that last arg can be -1.
2023-08-03 23:16:00 -07:00
Pádraig Brady 779f34e180 tac: handle short reads on input
This can be reproduced by getting the read() above 2G,
which induces a short read, thus triggering the erroneous failure.

  $ truncate -s 5G 5G

  $ cat 5G | TMPDIR=$PWD tac | wc -c
  tac: /tmp/tacFt7txA: read error: Illegal seek
  0

With the fix in place we now get:

  $ cat 5G | TMPDIR=$PWD src/tac | wc -c
  5368709120

* src/tac.c (tac_seekable): Use full_read() to handle short reads.
* NEWS: Mention the bug fix.
Reported at https://bugs.debian.org/1042546
2023-07-31 18:55:19 +01:00
Bruno Haible a102826245 uptime: output correct user count on OpenBSD
* src/uptime.c (print_uptime, uptime): Always call read_utmp
and count the result.
* NEWS: Mention the fix (text by Bruno Haible).
2023-07-31 09:28:04 -07:00
Paul Eggert 39f5c3f92e build: update gnulib submodule to latest
* NEWS: Mention a bug that this fixes.
2023-07-29 17:23:15 -07:00
Paul Eggert 3cb862ce5f mv: better diagnostic for 'mv dir x' failure
Problem reported by Nir Oren <https://bugs.gnu.org/64785>.
* src/copy.c (copy_internal): Use a more-specific diagnostic when
a rename fails due to a problem that must be due to the
destination, avoiding user confusion in cases like 'mv dir x'
where x is a nonempty directory.
* tests/mv/dir2dir.sh: Adjust to match.
2023-07-22 13:41:07 -07:00
Pádraig Brady 464be62df6 split: honor $TMPDIR for temp files
* bootstrap.conf: Depend on tmpdir rather than tmpfile,
as the standard tmpfile() doesn't honor $TMPDIR.
* src/split.c (copy_to_tmpfile): Adjust to call temp_stream() rather
than tmpfile();
* NEWS: Mention the improvement.
2023-07-18 23:11:24 +01:00
Pádraig Brady 1b86b70dd5 tac: fall back to /tmp if $TMPDIR is unavailable
This also refactors temp_stream() to its own module,
in preparation for use by split.

* src/tac.c: Refactor temp_stream() out to ...
* src/temp-stream.c: ... A new module mostly refactored from tac,
but uses tmpdir to more robustly support $TMPDIR,
while falling back to /tmp if not available.
* src/temp-stream.h: The new module interface.
* src/local.mk: Reference the new module from tac.
* tests/tac/tac.pl: Adjust to non failing missing $TMPDIR.
* po/POTFILES.in: Reference the new module with translatable strings.
* NEWS: Mention the user visible improvements to tac TMPDIR handling.
2023-07-18 23:10:40 +01:00
Pádraig Brady 02a4ebd6c4 join: promptly diagnose write errors
* src/join.c (prjoin): Check for write errors after each line.
* tests/misc/write-errors.sh: enable the test for join.
* NEWS: Mention the improvement.
2023-07-17 11:28:36 +01:00
Pádraig Brady b1df1d557e comm: promptly diagnose write errors
* src/comm.c (writeline): Simplify by removing the unneeded STREAM
parameter.  Call write_error() upon ferror().
(compare_files): Adjust to simplified writeline().
* tests/misc/write-errors.sh: Enable comm test.
* NEWS: Mention the improvement.
2023-07-17 11:28:36 +01:00
Pádraig Brady e0a4a60af5 cut: promptly diagnose write errors
* src/cut.c (cut_bytes): Diagnose errors from fwrite() and putchar().
(cut_fields): Likewise.
* tests/misc/write-errors.sh: Enable the test for cut,
and augment to cover both cut_bytes() and cut_fields().
* NEWS: Mention the improvement.
2023-07-17 11:28:36 +01:00
Pádraig Brady ca7711456f uniq: promptly diagnose write errors
* src/uniq.c (write_line): Check the output from fwrite() immediately.
(check_file): Likewise.
* tests/misc/write-errors.sh: Enable the test case.
* NEWS: Mention the improvement.
2023-07-17 11:28:36 +01:00
Pádraig Brady a03c00023a od: promptly diagnose write errors
* src/od.c (dump): Check for write errors after each block written,
to exit early even with large / unbounded inputs.
* tests/misc/write-errors.sh: enable od check.
* NEWS: Mention the improvement.
Fixes https://bugs.gnu.org/64540
2023-07-17 11:28:36 +01:00
Pádraig Brady 86614ba1c2 cksum: escape filenames with a leading '\' in --check status
* src/digest.c (digest_check): Also escape in the case that the
file name contains '\'.
* tests/cksum/md5sum-bsd.sh: Add a test case.
* doc/coreutils.texi (md5um invocation): Clarify escaping operation.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/64392
2023-07-11 12:16:41 +01:00
Pádraig Brady 5e1e0993b5 cksum: support transparent emulation of older utils
Support -b, --binary, and -t, --text
to allow full emulation of older utilities with:
  exec cksum -a $algo --untagged "$@"
Note this would diverge from OpenBSD's support of cksum -b.

* src/digest.c: Change -b to mean --binary, not --base64 in all cases.
Accept -b and -t in all cases.  Keep --binary and --text undocumented
for cksum.
* tests/cksum/cksum-base64.pl: s/-b/--base64/.
* tests/cksum/cksum-a.sh: Ensure cksum supports -b and -t appropriately.
* NEWS: Mention the change in behavior.
2023-07-09 14:33:14 +01:00
Pádraig Brady e8858f1515 b2sum: fix UAR with --check with malformed checksum lines
* src/digest.c (split_3): Reinstate the check for whitespace after the
digest portion of the line, so that we exit early before inspecting
the file name which would be outside the passed buffer in the case
where the input does not contain a newline.
* tests/cksum/b2sum.sh: Add a test case.
* NEWS: Mention the bug fix.
* THANKS.in: Add Frank Busse who has reported multiple bugs using KLEE.
Fixes https://bugs.gnu.org/64229
2023-06-22 23:22:07 +01:00
Paul Eggert 3638944ff5 doc: mention fix for bug#64123 2023-06-19 23:24:02 -07:00
Pádraig Brady d53190ed46 doc: mention cksum error fix with cpu feature checks changes
* NEWS: Mention the error message to aid those searching
for solutions to the issue, and mention cksum also
as that was confirmed to fix the error with the adjusted
cpu feature detection, as discussed at https://bugs.debian.org/1037264
* src/cksum.c: Cleanup syntax-check failure from previous commit.
2023-06-15 01:00:05 +01:00
Paul Eggert 91a74d3614 wc: port to kernels that disable XSAVE YMM
Problem reported by Dave Hansen <https://bugs.gnu.org/64058>.
Apply similar change to cksum and pclmul, too.
* NEWS: Mention wc fix.
* configure.ac (cpuid_exists, get_cpuid_count_exists):
Remove.  All uses removed, since we no longer use __get_cpuid or
__get_cpuid_count.
(pclmul_intrinsic_exists, avx2_intrinsic_exists): Set to no if
__builtin_cpu_supports calls cannot be compiled.
(HAVE_PCLMUL_INTRINSIC, HAVE_AVX2_INTRINSIC): Remove; unused.
Simplify surrounding code because of this.
* src/cksum.c (pclmul_supported):
* src/wc.c (avx2_supported):
Use __builtin_cpu_supports instead of doing it by hand.
Simplify surrounding code because of this.
2023-06-13 21:14:29 -07:00