When recursively copying files into OS trees, it often happens that
some subdirectory of the source directory is a symlink in the target
directory. Currently, cp will fail in that scenario with the error:
"cannot overwrite non-directory %s with directory %s"
However, we'd like cp in this scenario to follow the destination
directory symlink and copy the files into the symlinked directory
instead. Let's support this by adding a new option
--keep-directory-symlink that makes cp follow destination directory
symlinks.
We name the option --keep-directory-symlink to keep consistent with
tar which has the same option with the same effect.
* doc/coreutils.texi (cp invocation): Describe the new option.
* src/copy.h: Add the new setting.
* src/copy.h: Adjust to follow symlinks if setting enabled.
* src/cp.c (usage): Describe the new option.
(main): Accept the new option.
* tests/cp/keep-directory-symlink.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
This was introduced in coreutils 9.2 through commit v9.1-184-g40bf1591b,
and was fixed in coreutils 9.5 through commit v9.4-111-gc4c5ed8f4.
This issue has been assigned CVE-2024-0684.
* NEWS: Mention the bug fix.
* tests/split/line-bytes.sh: Add a test case.
Reported by Valentin Metz.
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...
* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Manually update copyright year,
until we fully sync with gnulib at a later stage.
* tests/sample-test: Adjust to use the single most recent year.
* gl/lib/strnumcmp-in.h (numcompare): After commit v9.0-8-g6cafb122f,
we need to treat characters as signed to avoid invalid comparisons
between negative integers and unsigned characters.
* NEWS: Mention the bug fix.
Do not perform SELinux context translation for operations not involving
user input or output. Context translation converts MCS/MLS labels into
human readable form, which is useful for user facing applications like
ls(1) or the --context=CTX argument of cp(1).
* src/copy.c (set_process_security_ctx): Use raw selinux variants.
* src/install.c (need_copy): Likewise.
(setdefaultfilecon): Likewise.
* src/selinux.c (computecon): Likewise.
(defaultcon): Likewise.
* tests/cp/no-ctx.sh: Add raw variants to preload lib.
* NEWS: Mention the improvement.
* doc/coreutils.texi (chown invocation): Convert --from option
description to a macro and call from ...
(chgrp description): ... here.
* src/chown-core.h (emit_from_option_description): A new function
refactored from ...
* src/chown.c (usage): ... here, and called from ...
* src/chgrp.c (usage): ... here.
(main): Accept the --from option as chown(1) does.
* po/POTFILES.in: Add chown-core.h as now translated.
* tests/chown/basic.sh: Decouple the root user from id 0.
* tests/chgrp/from.sh: A new test largely based on chown/basic.sh.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
Suggested by Ed Neville.
* src/tail.c (file_lines): Ensure we use a buffer size >= PAGE_SIZE when
searching backwards to avoid seeking within a file,
which on sysfs files is accepted but also returns no data.
* tests/tail/tail-sysfs.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/67490
For consistency with the "SI" standard, and with other coreutils
which output a lowercase 'k' in "SI" mode.
* src/numfmt.c (suffix_power): Treat 'k' like 'K' on input.
(double_to_human): Output lowercase 'k' in SI mode.
(usage): Adjust accordingly.
* doc/coreutils.texi: Mention 'k' accepted, and printed in SI mode.
* tests/misc/numfmt.pl: Adjust accordingly.
* NEWS: Mention the change in behavior.
Fixes https://bugs.gnu.org/47103
* NEWS: Mention this.
* bootstrap.conf (gnulib_modules): Remove cu-ctype, as this module
is now more trouble than it’s worth. All uses removed.
Add skipchars.
* gl/lib/cu-ctype.c, gl/lib/cu-ctype.h, gl/modules/cu-ctype:
Remove.
* gl/lib/skipchars.c, gl/lib/skipchars.h, gl/modules/skipchars:
* tests/misc/join-utf8.sh:
New files.
* src/join.c: Include skipchars.h and mcel.h instead of cu-ctype.h.
(tab): Now mcel_t, not int. All uses changed.
(output_separator, output_seplen): New static vars.
(eq_tab, newline_or_blank, comma_or_blank): New functions.
(xfields, prfields, prjoin, add_field_list, main):
Support multi-byte characters.
* src/numfmt.c: Include ctype.h, skipchars.h.
Do not include cu-ctype.h.
(newline_or_blank): New function.
(next_field): Support multi-byte characters.
* src/sort.c: Include ctype.h instead of cu-ctype.h.
(inittables): Open-code field_sep since it no longer exists.
‘sort’ is not multi-byte safe yet, but when it is this code
will need revamping anyway.
* src/uniq.c: Include mcel.h and skipchars.h instead of cu-ctype.h.
(newline_or_blank): New function.
(find_field): Support multi-byte characters.
* tests/local.mk (all_tests): Add tests/misc/join-utf8.sh
This will make decoding more resilient to corruption
whether due to transmission errors or nefarious adjustment.
See https://eprint.iacr.org/2022/361.pdf
* gnulib: Update to commit 3f463202bd enforcing canonical encoding.
* tests/basenc/base64.pl: Add test cases, and adjust existing cases.
* NEWS: Mention the change in behavior.
* src/basenc.c (base16_decode_ctx): Convert to uppercase
before converting from hex.
* tests/basenc/basenc.pl: Add a test case.
* NEWS: Mention the change in behavior.
Addresses https://bugs.gnu.org/66698
Padding of encoded data is useful in cases where
base64 encoded data is concatenated / streamed.
I.e. where there are padding chars _within_ the stream.
In other cases padding is optional and can be inferred.
Note we continue to treat partial padding as invalid,
as that would be indicative of truncation.
* src/basenc.c (do_decode): Auto pad the end of the input.
* NEWS: Mention the change in behavior.
* tests/misc/base64.pl: Adjust to not fail for missing padding.
Addresses https://bugs.gnu.org/66265
The 3× speedup was measured by invoking 'wc $(find * -type f)'
on the coreutils sources etc. on an Ubuntu 23.04 x86-64.
These changes also speed up wc 20% in UTF-8 locales.
* src/wc.c (wc_isprint, wc_isspace): New static vars.
(wc): Use them for speed.
(main): Initialize them if needed.
(isnbspace): Remove; no longer used.
* bootstrap.conf (gnulib_modules): Remove c32isprint.
* src/wc.c (wc): Consider all non-white-space characters
to be word constituents, even if they are not printable.
POSIX requires this, and it is what BSD does.
Partly do this by simplifying the check for a word,
by counting word starts rather than word ends.
* tests/wc/wc.pl: Test for the bug.
tail can watch multiple files, but currently only a single writer. It
can be useful to watch files from multiple writers, or even processes
not directly related to the files (e.g. watch log files written by a
server process, for the duration of a test driven by a separate
client).
* src/tail.c (writers_are_dead): New function.
(tail_forever): Use it to wait for writers.
(tail_forever_inotify): As above.
(parse_options): Manage --pid options in an array.
* doc/coreutils.texi: Update documentation.
* tests/tail/pid.sh: Add a variant with two PIDs.
* News: Mention the new feature.
Currently --dired is silently ignored
with conflicting output formats
* src/ls.c (decode_switches): Set default format and hyperlink mode
when the --dired option is specified.
* tests/ls/dired.sh: Check that formats are implied / overridden.
* NEWS: Mention the change in behavior.
* doc/coreutils.texi (ls invocation): Adjust --dired description.
This bug occurs only when temporarily setting the mode to the
intersection of old and new modes when changing ownership.
* src/copy.c (owner_failure_ok): Treat EACCES like EPERM.
Most of this just affects commentary and documentations. The only
significant behavior change is translating author names via
proper_name_lite rather than proper_name_utf8, or not translating
them at all. proper_name_lite is good enough for coreutils and
avoids the bloat that had coreutils not using Gnulib proper_name.
* bootstrap.conf (gnulib_modules): Use propername-lite instead
of propername.
(XGETTEXT_OPTIONS): Look for proper_name_lite instead of for
proper_name_utf8.
* cfg.mk (local-checks-to-skip): Remove
sc_proper_name_utf8_requires_ICONV, since we no longer use
proper_name_utf8.
(old_NEWS_hash): Update.
(sc_check-I18N-AUTHORS): Remove; no longer needed.
* src/cp.c (main): Set default reflink mode appropriately
with --sparse=never.
* src/copy.c (infer_scantype): Add a comment to related code.
* tests/cp/sparse-2.sh: Add a test case.
* NEWS: Mention the bug.
* NEWS: Separate out the description of the _existing_ issues
with outputting timestamps on 32 bit systems, from _future_ issues
outputting timestamps on all systems. Also move this to the
"improvement" section, since it's not really a coreutils
specific issue, and also is a build time configurable option.
(This patch is coauthored with Bruno Haible,
with original version at <https://bugs.gnu.org/64937#>.)
This updates the gnulib submodule to latest.
For year-2038 safety on Linux/{x86,arm},
this adds an --enable-systemd option to ‘configure’.
The idea is that this sort of thing will become the default
after it has been tested more.
* configure.ac: Don't test whether struct utmp and struct utmpx
have the ut_host field; this is now done in gnulib's readutmp module.
* src/local.mk: Link the programs 'pinky', 'uptime', 'users',
'who' with $(READUTMP_LIB).
* src/pinky.c, src/who.c:
Test HAVE_STRUCT_XTMP_UT_HOST instead of HAVE_UT_HOST.
* src/pinky.c (print_entry):
* src/who.c (print_user, print_deadprocs, print_login)
(print_initspawn, scan_entries):
Support the situation where ut_line is a 'char *' rather than a
'char[]' of fixed size. Likewise for ut_user and ut_host.
(make_id_equals_comment): Likewise for ut_id.
* src/pinky.c (print_entry):
* src/who.c (print_user):
Open /dev to simplify looking up its entries.
Don’t use printf if the output might in theory be longer than INT_MAX.
* src/pinky.c (scan_entries, short_pinky):
* src/uptime.c (print_uptime, uptime):
* src/users.c (list_entries_users, users):
* src/who.c (who):
Use idx_t where new read_utmp needs it.
* src/system.h (STREQ_LEN): Add comment that last arg can be -1.
This can be reproduced by getting the read() above 2G,
which induces a short read, thus triggering the erroneous failure.
$ truncate -s 5G 5G
$ cat 5G | TMPDIR=$PWD tac | wc -c
tac: /tmp/tacFt7txA: read error: Illegal seek
0
With the fix in place we now get:
$ cat 5G | TMPDIR=$PWD src/tac | wc -c
5368709120
* src/tac.c (tac_seekable): Use full_read() to handle short reads.
* NEWS: Mention the bug fix.
Reported at https://bugs.debian.org/1042546
Problem reported by Nir Oren <https://bugs.gnu.org/64785>.
* src/copy.c (copy_internal): Use a more-specific diagnostic when
a rename fails due to a problem that must be due to the
destination, avoiding user confusion in cases like 'mv dir x'
where x is a nonempty directory.
* tests/mv/dir2dir.sh: Adjust to match.
* bootstrap.conf: Depend on tmpdir rather than tmpfile,
as the standard tmpfile() doesn't honor $TMPDIR.
* src/split.c (copy_to_tmpfile): Adjust to call temp_stream() rather
than tmpfile();
* NEWS: Mention the improvement.
This also refactors temp_stream() to its own module,
in preparation for use by split.
* src/tac.c: Refactor temp_stream() out to ...
* src/temp-stream.c: ... A new module mostly refactored from tac,
but uses tmpdir to more robustly support $TMPDIR,
while falling back to /tmp if not available.
* src/temp-stream.h: The new module interface.
* src/local.mk: Reference the new module from tac.
* tests/tac/tac.pl: Adjust to non failing missing $TMPDIR.
* po/POTFILES.in: Reference the new module with translatable strings.
* NEWS: Mention the user visible improvements to tac TMPDIR handling.
* src/join.c (prjoin): Check for write errors after each line.
* tests/misc/write-errors.sh: enable the test for join.
* NEWS: Mention the improvement.
* src/comm.c (writeline): Simplify by removing the unneeded STREAM
parameter. Call write_error() upon ferror().
(compare_files): Adjust to simplified writeline().
* tests/misc/write-errors.sh: Enable comm test.
* NEWS: Mention the improvement.
* src/cut.c (cut_bytes): Diagnose errors from fwrite() and putchar().
(cut_fields): Likewise.
* tests/misc/write-errors.sh: Enable the test for cut,
and augment to cover both cut_bytes() and cut_fields().
* NEWS: Mention the improvement.
* src/uniq.c (write_line): Check the output from fwrite() immediately.
(check_file): Likewise.
* tests/misc/write-errors.sh: Enable the test case.
* NEWS: Mention the improvement.
* src/od.c (dump): Check for write errors after each block written,
to exit early even with large / unbounded inputs.
* tests/misc/write-errors.sh: enable od check.
* NEWS: Mention the improvement.
Fixes https://bugs.gnu.org/64540
* src/digest.c (digest_check): Also escape in the case that the
file name contains '\'.
* tests/cksum/md5sum-bsd.sh: Add a test case.
* doc/coreutils.texi (md5um invocation): Clarify escaping operation.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/64392
Support -b, --binary, and -t, --text
to allow full emulation of older utilities with:
exec cksum -a $algo --untagged "$@"
Note this would diverge from OpenBSD's support of cksum -b.
* src/digest.c: Change -b to mean --binary, not --base64 in all cases.
Accept -b and -t in all cases. Keep --binary and --text undocumented
for cksum.
* tests/cksum/cksum-base64.pl: s/-b/--base64/.
* tests/cksum/cksum-a.sh: Ensure cksum supports -b and -t appropriately.
* NEWS: Mention the change in behavior.
* src/digest.c (split_3): Reinstate the check for whitespace after the
digest portion of the line, so that we exit early before inspecting
the file name which would be outside the passed buffer in the case
where the input does not contain a newline.
* tests/cksum/b2sum.sh: Add a test case.
* NEWS: Mention the bug fix.
* THANKS.in: Add Frank Busse who has reported multiple bugs using KLEE.
Fixes https://bugs.gnu.org/64229
* NEWS: Mention the error message to aid those searching
for solutions to the issue, and mention cksum also
as that was confirmed to fix the error with the adjusted
cpu feature detection, as discussed at https://bugs.debian.org/1037264
* src/cksum.c: Cleanup syntax-check failure from previous commit.
Problem reported by Dave Hansen <https://bugs.gnu.org/64058>.
Apply similar change to cksum and pclmul, too.
* NEWS: Mention wc fix.
* configure.ac (cpuid_exists, get_cpuid_count_exists):
Remove. All uses removed, since we no longer use __get_cpuid or
__get_cpuid_count.
(pclmul_intrinsic_exists, avx2_intrinsic_exists): Set to no if
__builtin_cpu_supports calls cannot be compiled.
(HAVE_PCLMUL_INTRINSIC, HAVE_AVX2_INTRINSIC): Remove; unused.
Simplify surrounding code because of this.
* src/cksum.c (pclmul_supported):
* src/wc.c (avx2_supported):
Use __builtin_cpu_supports instead of doing it by hand.
Simplify surrounding code because of this.