strcoll() is only significant to uniq(1) if it returns 0,
and it generally only does so with buggy locales or mismatched
locales and data. Some systems may have strcoll()
return 0 for equivalent normalized unicode forms,
but for consistency across platforms strcoll() is avoided.
The various cases are defined in the new test.
This is consistent with newer POSIX standards as discussed at:
https://www.austingroupbugs.net/view.php?id=963
* src/uniq.c: s/xstrcoll/memcmp/.
* tests/local.mk: Reference the new test.
* tests/misc/uniq-collate.sh: Add a new test.
* NEWS: Mention the change in behavior.
Fixes https://bugs.gnu.org/38627
* src/stat.c (usage): Mention permission bits rather than
"access" so there is no confusion with ACLs etc.
Also indicate we output the file type with '%A'.
* doc/coreutils.texi (stat invocation): Likewise.
Also indicate '%A' is similar to `ls -ld` output.
Addresses https://bugs.gnu.org/39613
Pick up recent build fixes to avoid sysctl.h inclusion on glibc systems,
restrict the max file size supported by read-file to PTRDIFF_MAX,
and to avoid a -Werror=unused failure in test-canonicalize.
* tests/cp/proc-short-read.sh: Switch to using /proc/cpuinfo,
rather than /proc/kallsyms which was seen to vary in some cases.
Fixes https://bugs.gnu.org/39357
Since v6.10-21-ged5c4e7 `rmdir --ignore-fail-on-non-empty`
had reversed the failure status for directories that failed
to be removed for permissions reasons. I.E. it would have
returned a failure status for such non empty dirs, and vice versa.
* src/rmdir.c (errno_may_be_non_empty): Rename from the
more confusing errno_may_be_empty(), and remove the EEXIST
case (specific to Solaris), which is moot here since
handled in errno_rmdir_non_empty().
(ignorable_failure): Fix the logic error so that
_non_ empty dirs are deemed to have ignorable failures.
(main): Fix clobbering of errno by is_empty_dir().
(remove_parents): Likewise.
* tests/rmdir/ignore.sh: Add a test case.
* THANKS.in: Add reporter who fixed the errno handling.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/39364
* configure.ac: Add -Wno-vector-operation-performance to suppress the
following gcc-9.2 error in gl/lib/randperm.c:
error: vector operation will be expanded piecewise
* src/ls.c (usage): Reorganize help for --time,
and add description for --time=birth.
(do_statx): Store btime in mtime if available.
(get_stat_btime): A new function to read the creation time
from the appropriate stat structure member.
(cmp_btime): A new function to compare birth time.
(print_long_format): Output '?' when birth time unavailable.
* doc/coreutils.texi: Document --time={birth,creation}.
* tests/local.mk: Reference the new test.
* tests/ls/birthtime.sh: Add a new test.
* NEWS: Mention the new feature.
* src/yes.c (main): Convert for loop to do-while in order to indicate
that the loop will be run at least once.
This avoids the following warning after the second loop:
src/yes.c:110:20: error: writing 1 byte into a region of size 0
* configure.ac: Set --with-openssl=auto-gpl-compat as the default,
so that openssl is used for md5sum etc., with openssl >= 3,
which is newly licensed under ASL v2.
* gnulib: Update to include "auto-gpl-compat" support.
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
* doc/coreutils.texi (sleep invocation): Add an example to demonstrate
how to use the floating-point and the scientific notation to sleep
for sub-second times, e.g. milli-, micro- and nanoseconds.
Inspired by Stephane Chazelas in:
https://lists.gnu.org/r/coreutils/2019-12/msg00005.html
* src/chcon.c (main): Skip call of security_check_context()
in case SELinux is disabled to avoid unnecessary failure.
Bug: https://bugzilla.redhat.com/1777831
* doc/sort-version.texi (Minus/Hyphen and Colon characters):
Rename from “Minus/Hyphen @samp{-} and Colon @samp{:} characters”,
as texi2any 6.6 complains about colons in node names.
* doc/coreutils.texi (shred invocation):
Modernize discussion to today’s technology (Bug#38168).
* src/shred.c (usage): Omit lengthy duplication of the manual’s
discussion of file systems and storage devices, as that became out
of sync with the manual. Instead, just cite the manual.
This addresses a longstanding "update all callers" FIXME in
lib/xstrtol.c, by having programs check that numbers do not
have unknown suffixes. The problem was also reported for
'shuf' by my student Maggie Huang while reimplementing a shuf
subset in Python as an exercise in UCLA Computer Science 35L:
https://web.cs.ucla.edu/classes/fall19/cs35L/assign/assign3.html
This patch also improves the portability of the code to unusual
platforms where ULONG_MAX < SIZE_MAX.
* NEWS: Mention user-visible changes.
* src/chgrp.c (parse_group):
* src/chroot.c (parse_additional_groups):
* src/du.c (main):
* src/install.c (get_ids):
* src/join.c (string_to_join_field):
* src/ls.c (decode_switches):
* src/md5sum.c (split_3):
* src/shuf.c (main):
* src/sort.c (specify_nthreads):
* src/uniq.c (size_opt, main):
Use uintmax_t instead of unsigned long, for portability
to oddball platforms where unsigned long is not wide enough.
* src/du.c (main):
* src/expr.c (mpz_init_set_str) [!HAVE_GMP]:
* src/install.c (get_ids):
* src/ls.c (decode_switches):
* src/mknod.c (main):
* src/ptx.c (main):
* src/shuf.c (main):
* src/sort.c (specify_nmerge, specify_nthreads):
Reject numbers with suffixes.
* src/md5sum.c (split_3): Simplify.
* gl/lib/randperm.c: Include randperm.h first, since it’s the API.
Include stdint.h, count-leading-zeros.h, verify.h.
(floor_lg): Rename from ceil_log (which was not actually
implementing the ceiling!) and implement the floor using
count_leading_zeros.
(randperm_bound): Use floor_lg, not ceil_log. Use uintmax_t
instead of size_t in case the size gets large on a 32-bit host.
* gl/modules/randperm (Depends-on): Add count-leading-zeros, stdint.
* configure.ac: When --enable-gcc-warnings is used, omit
-Wno-type-limits. The need for -Wno-type-limits has passed, now
that intprops.h uses builtin primitives for GCC 5 and later, given
that recent GCCs issue type-limits warnings only for non-constant
expressions. --enable-gcc-warnings is not intended for use with
old compilers, so we can drop -Wno-type-limits now.
‘shuf -r -n 0 file’ would mistakenly read from standard input.
Problem reported by my student Jingnong Qu while reimplementing a
shuf subset in Python as an exercise in UCLA Computer Science 35L:
https://web.cs.ucla.edu/classes/fall19/cs35L/assign/assign3.html
* NEWS: Mention the fix. Also, ASCIIfy a previous item.
* src/shuf.c (main): Fix bug.
* tests/misc/shuf.sh: Add a test case for the bug.
statx allows ls to indicate interest in only certain inode metadata.
This is potentially a win on networked/clustered/distributed
file systems. In cases where we'd have to do a full, heavyweight stat()
call we can now do a much lighter statx() call.
As a real-world example, consider a file system like CephFS where one
client is actively writing to a file and another client does an
ls --color in the same directory. --color means that we need to fetch
the mode of the file.
Doing that with a stat() call means that we have to fetch the size and
mtime in addition to the mode. The MDS in that situation will have to
revoke caps in order to ensure that it has up-to-date values to report,
which disrupts the writer.
This has a measurable affect on performance. I ran a fio sequential
write test on one cephfs client and had a second client do "ls --color"
in a tight loop on the directory that held the file:
Baseline -- no activity on the second client:
WRITE: bw=76.7MiB/s (80.4MB/s), 76.7MiB/s-76.7MiB/s (80.4MB/s-80.4MB/s),
io=4600MiB (4824MB), run=60016-60016msec
Without this patch series, we see a noticable performance hit:
WRITE: bw=70.4MiB/s (73.9MB/s), 70.4MiB/s-70.4MiB/s (73.9MB/s-73.9MB/s),
io=4228MiB (4433MB), run=60012-60012msec
With this patch series, we gain most of that ground back:
WRITE: bw=75.9MiB/s (79.6MB/s), 75.9MiB/s-75.9MiB/s (79.6MB/s-79.6MB/s),
io=4555MiB (4776MB), run=60019-60019msec
* src/stat.c: move statx to stat struct conversion to new header...
* src/statx.h: ...here.
* src/ls.c: Add wrapper functions for stat/lstat/fstat calls,
and add variants for when we are only interested in specific info.
Add statx-enabled functions and set the request mask based on the
output format and what values are needed.
* NEWS: Mention the Improvement.
* src/truncate.c (do_ftruncate): Simplify overflow checking,
and don’t rely on theoretically-nonportable assumptions
like assuming that OFF_MAX < UINTMAX_MAX.
* src/seq.c: (seq_fast): Accept STEP as a parameter and use that
to skip the output of generated numbers.
(main): Relax to using seq_fast for integer steps between 1 and 200.
For larger steps the throughput was faster using the standard
incrementing procedure.
(cmp): Use the equivalent but faster memcmp for equal len strings.
* tests/misc/seq.pl: Update fast path cases.
Addresses https://bugs.gnu.org/37241
The previous commit introduced a bug into the following syntax-check,
and thus effectively turned it off:
$ make sc_prohibit_test_calls_print_ver_with_irrelevant_argument; \
echo $?
prohibit_test_calls_print_ver_with_irrelevant_argument
fatal: cannot change to 'grep': No such file or directory
0
* cfg.mk (sc_prohibit_test_calls_print_ver_with_irrelevant_argument):
Remove changing directory, and pass $(srcdir) as argument to 'git -C'.
* cfg.mk (sc_prohibit_colon_redirection): Don't expect `|` to denote
the pipe character in git grep.
(sc_tests_executable)
(sc_case_insensitive_file_names)
(sc_some_programs_must_avoid_exit_failure)
(sc_prohibit_test_background_without_cleanup_)
(sc_prohibit_test_calls_print_ver_with_irrelevant_argument)
(sc_prohibit_test_ulimit_without_require_)
(sc_prohibit_test_background_without_cleanup_)
(sc_THANKS_in_duplicates)
*sc_prohibit_test_calls_print_ver_with_irrelevant_argument):
Don't expect builddir to be a descendant of srcdir.
(sc_strftime_check): Don't check file size against 0 when "N\nq\n" was
already put in the file.
* THANKS.in: Remove me.
Under certain circumstances seq prints an extra line when the output
format has custom format with characters following the printed numbers:
$ seq -f "%g " 1000000 1000000
1e+06
1e+06
This is due to the "print_extra_number" logic using strings to determine
whether a 'extra number' is needed, but only one string was trimmed
when using a custom printf format.
Prompted by https://lists.gnu.org/r/coreutils/2019-08/msg00001.html
* NEWS: Mention fix.
* src/seq.c (print_numbers): Trim the 'x0_str' string before comparing
it to the previous 'x_str' string.
* tests/misc/seq-extra-number.sh: Add this scenario.
* tests/local.mk (all_tests): Add new test.
* doc/sort-version.texi: Fix some typos, avoid overly long lines in
the generated PDF, enclose some sample strings in @samp{...} for better
readability, etc. This also avoids an sc-avoid-builtin error:
s/builtin/built-in/
* doc/sort-version.texi: New file.
* doc/local.mk (doc_coreutils_TEXINFOS): Add new file.
* doc/coreutils.texi: @include new file, replace previous "Details about
version sort" section.
When calling 'stat -c %N' to print the filename, don't explicitly
request the size of the file via statx(), as it may add overhead on
some filesystems. The size is only needed to optimize an allocation
for the relatively rare case of reading a symlink name, and the worst
effect is a somewhat-too-large temporary buffer may be allocated for
areadlink_with_size(), or internal retries if buffer is too small.
The file size will be returned by statx() on most filesystems, even
if not requested, unless the filesystem considers this to be too
expensive for that file, in which case the tradeoff is worthwhile.
* src/stat.c: Don't explicitly request STATX_SIZE for filenames.