Commit Graph
29718 Commits
Author SHA1 Message Date
Pádraig Brady a4525de1ef tests: add a test case for the previous date fix
* NEWS: Also mention this bug fix.
* tests/misc/date-f.sh: Add a new test.
* tests/local.mk: Reference the new test.
2023-03-28 13:40:43 +01:00
Paul Eggert 9c5e542fd1 date: diagnose -f read errors
* src/date.c (batch_convert): Diagnose read errors, fixing Bug#62497.
2023-03-28 01:53:06 -07:00
Paul Eggert 6272817de0 cp: clarify commentary
* src/copy.c: Make comments a bit clearer.
2023-03-25 13:20:16 -07:00
Pádraig Brady 093a8b4bfa copy: fix --reflink=auto to fallback in more cases
On restricted systems like android or some containers,
FICLONE could return EPERM, EACCES, or ENOTTY,
which would have induced the command to fail to copy
rather than falling back to a more standard copy.

* src/copy.c (is_terminal_failure): A new function refactored
from handle_clone_fail().
(is_CLONENOTSUP): Merge in the handling of EACCES, ENOTTY, EPERM
as they also pertain to determination of whether cloning is supported
if we ever use this function in that context.
(handle_clone_fail): Use is_terminal_failure() in all cases,
so that we assume a terminal failure in less errno cases.
* NEWS: Mention the bug fix.
Addresses https://bugs.gnu.org/62404
2023-03-24 13:12:51 +00:00
Pádraig Brady 55456b95d8 doc: add a NEWS entry for the previous fix
* NEWS: Mention the previous cksum --check fix.
2023-03-24 12:52:13 +00:00
Pádraig Brady 76f2fb6271 cksum: fix reporting of failed checks
This applies to all checksumming utilities,
where we incorrectly report all subsequent files as checking 'OK'
once any file has passed a digest check.
The exit status was not impacted, only the printed status.

* src/digest.c (digest_check): Use the correct state variable
to determine if the _current_ file has passed or not.
* tests/misc/md5sum.pl: Add a test case.
Fixes https://bugs.gnu.org/62403
2023-03-23 12:36:53 +00:00
Nick Alcock eeabb11eb6 tests: skip some parts of tests/misc/tee.sh if run as root
Similarly to the fix to tests/rmdir/ignore.sh in c0e5f8c59,
tee should not be expected to fail when run with read-only outputs
when run as root.

* tests/misc/tee.sh: Add uid_is_privileged_ guard around test for
read-only outputs.
2023-03-21 16:15:18 +00:00
Pádraig Brady bdda9adb38 maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2023-03-20 14:08:46 +00:00
Pádraig Brady df4e4fbc7d version 9.2
* NEWS: Record release date.
v9.2
2023-03-20 13:44:36 +00:00
Pádraig Brady 445f502839 tests: avoid false failure on systems without statx
* tests/ls/stat-free-symlinks.sh: Filter out syscalls that
return ENOSYS, as that was seen with statx() on Debian 10.13
on mips64, and resulted in overcounting of stat calls.
2023-03-19 23:44:51 +00:00
Pádraig Brady eb745e3b95 stty: ensure arbitrary data is not displayed
* src/stty.c (main): Use static structures to ensure
they're initialized (to zero), so that random data is
not displayed, or compared resulting in a inaccurate
failure reported to users.  This was seen on musl libc
where some parts of the termios c_cc array were
not initialized by tcgetattr().
Reported by Bruno Haible.
2023-03-19 22:25:50 +00:00
Pádraig Brady 82bb131c55 tests: df: avoid false failure on systems with multiple roots
* tests/df/df-symlink.sh: Skip the test on systems
with multiple file system roots.
2023-03-19 12:23:05 +00:00
Pádraig Brady 11f4d59138 doc: uniq: revert previous man page adjustment
* src/uniq.c (usage): -D doesn't take an argument
so should be separated in the description.
2023-03-18 14:58:20 +00:00
Pádraig Brady 8a759fdc88 doc: uniq: use more standard description of -D long option
* src/uniq.c: Describe -D and --all-repeated together,
as with all other long options.
Fixes https://bugs.gnu.org/62249
2023-03-18 11:44:24 +00:00
Pádraig Brady a109106eb8 tests: aix: avoid unsupported functionality
* tests/misc/tee.sh: AIX doesn't support detecting
closed outputs either with poll() or select() so avoid
testing that functionality.
* tests/tail-2/pipe-f.sh: Likewise.
2023-03-18 11:44:24 +00:00
Bernhard Voelker f346fc0144 maint: avoid discarded-qualifiers warnings with SELinux 3.5
Since SELinux version 3.5, the return value of context_str(3) is
declared as const; see:
  https://github.com/SELinuxProject/selinux/commit/dd98fa322766
Therefore, GCC complains (here with -Werror):
  src/selinux.c: In function 'defaultcon':
  src/selinux.c:152:16: error: assignment discards 'const' qualifier \
    from pointer target type [-Werror=discarded-qualifiers]
    152 |   if (!(constr = context_str (tcontext)))
        |                ^
  src/selinux.c: In function 'restorecon_private':
  src/selinux.c:252:16: error: assignment discards 'const' qualifier \
    from pointer target type [-Werror=discarded-qualifiers]
    252 |   if (!(constr = context_str (tcontext)))
      |                ^

* src/selinux.c (defaultcon): Define CONSTR as const.
(restorecon_private): Likewise.
2023-03-17 00:12:18 +01:00
Pádraig Brady 1d03781daa test: avoid a test hang on Hurd
* tests/cp/sparse-to-pipe.sh: Protect the cp call seen to
hang on Hurd/i686 with a timeout.
Reported By: Bruno Haible
2023-03-15 18:11:00 +00:00
Pádraig Brady a8e6e627f1 cksum: fix --raw on 64 bit big endian systems
* src/sum.c (output_bsd): On sparc64 for example,
a crc of 0 was output due to casting an int variable
to uint16_t and thus operating on the wrong end of the variable.
Instead use explicit assignment to the narrower type
to ensure we get the appropriate data.
(output_sysv): Likewise.
Reported by Bruno Haible.
2023-03-15 14:05:38 +00:00
Bruno Haible f1ff021247 build: avoid -Wsometimes-uninitialized on macOS 12
* src/pr.c (integer_overflow): Mark as _Noreturn.
* src/test.c (beyond): Likewise.
2023-03-15 13:15:11 +00:00
Pádraig Brady e68b15a3ef doc: remove older ChangeLog items
* Makefile.am: Update the oldest documented version
to 8.29 which is now about 5 years old.
2023-03-13 21:45:50 +00:00
Pádraig Brady 73d372dcc1 tee: avoid undefined behavior after fclose()
* iopoll.c (fclose_wait): Rename from confusing fclose_nonblock name.
Also adjust to do no operations on the stream after fclose()
as this is undefined. Instead use fflush() to determine EAGAIN status.
(fwrite_wait): Renamed from confusing fwrite_nonblock name.
2023-03-13 21:26:21 +00:00
Pádraig Brady f26af59833 doc: dircolors: better separate the sections in the database
* src/dircolors.hin: Make the separate sections of the self
documenting dircolors database more apparent,
by adding heading comments, and appropriate separation.
2023-03-13 16:20:38 +00:00
Pádraig Brady 47988fad88 ls: --color: honor separate sequences for extension cases
Following on from commit v8.29-45-g24053fbd8 which unconditionally
used case insensitive extension matching, support selective
case sensitive matching when there are separate extension cases
defined with different display sequences.

* src/dircolors.hin: Document how file name suffixes are matched.
Note this is displayed with `dircolors --print-database` which
the texi info recommends to use for details.
* src/ls.c (parse_ls_color): Postprocess the list to
mark entries for case sensitive matching,
and also adjust so that unmatchable entries are more quickly ignored.
(get_color_indicator): Use exact matching rather than
case insensitive matching if so marked.
* tests/ls/color-ext.sh: Add test cases.
* NEWS: Mention the change in behavior.
Addresses https://bugs.gnu.org/33123
2023-03-13 16:20:38 +00:00
Pádraig Brady 1ac1d6def6 tests: adjust du test for recent apparent size change
* tests/du/threshold.sh: Directories are assumed to be
of size 0 with --apparent since commit v9.1-187-g110bcd283
so remove --apparent cases from this test.
2023-03-13 16:20:38 +00:00
Pádraig Brady 16000805eb tee: support non blocking outputs
Non blocking outputs can be seen for example
when piping telnet through tee to a terminal.
In that case telnet sets its input to nonblocking mode,
which results in tee's output being nonblocking,
in which case in may receive an EAGAIN error upon write().
The same issue was seen with mpirun.
The following can be used to reproduce this
locally at a terminal (in most invocations):

  $ { dd iflag=nonblock count=0 status=none;
      dd bs=10K count=10 if=/dev/zero status=none; } |
    tee || echo fail >/dev/tty

* src/iopoll.c (iopoll_internal): A new function refactored from
iopoll(), to also support a mode where we check the output
descriptor is writeable.
(iopoll): Now refactored to just call iopoll_internal().
(fwait_for_nonblocking_write): A new internal function which
uses iopoll_internal() to wait for writeable output
if an EAGAIN or EWOULDBLOCK was received.
(fwrite_nonblock): An fwrite() wrapper which uses
fwait_for_nonblocking_write() to handle EAGAIN.
(fclose_nonblock): Likewise.
src/iopoll.h: Add fclose_nonblock, fwrite_nonblock.
src/tee.c: Call fclose_nonblock() and fwrite_nonblock wrappers,
instead of the standard functions.
* tests/misc/tee.sh: Add a test case.
* NEWS: Mention the improvement.

The idea was suggested by Kamil Dudka in
https://bugzilla.redhat.com/1615467
2023-03-08 18:32:05 +00:00
Pádraig Brady 176b74f0e7 doc: remove redundant NEWS entry
* NEWS: There was no bug on macOS,
and the split -n new feature is documented already.
2023-03-08 18:28:06 +00:00
Paul Eggert bb9dbcbbfd split: support split -n on larger pipe input
* bootstrap.conf (gnulib_modules): Add free-posix, tmpfile.
* src/split.c (copy_to_tmpfile): New function.
(input_file_size): Use it to split larger files when sizes cannot
easily be determined via fstat or lseek.  See Bug#61386#235.
* tests/split/l-chunk.sh: Mark tests of /dev/zero as
very expensive since they exhaust /tmp.
2023-03-07 13:41:46 -08:00
Pádraig Brady 35ac97e0d6 tee: fix a crash with unwriteable files
This was introduced recently with commit v9.1-166-g6b12e62d9

* src/tee.c (tee_files): Check the return from fopen()
before passing to fileno() etc.
* tests/misc/tee.sh: Add a test case.
2023-03-07 01:14:00 +00:00
Paul Eggert a4778006c8 maint: pacify ‘make syntax-check’
Problem reported by Pádraig Brady (Bug#61386#226).
* src/split.c (parse_chunk): Use die instead of error.
(main): Quote a string.
* tests/local.mk (all_root_tests): Move du/apparent.sh from here ...
(all_tests): ... to here.
2023-03-06 15:39:07 -08:00
Pádraig Brady e30af1e584 tail,tee: avoid issues with many files on systems without poll
* src/iopoll.c (iopoll): Protect the call to select against
passing in a descriptor larger than FD_SETSIZE.
2023-03-06 13:04:40 +00:00
Paul Eggert 110bcd2838 du: --apparent counts only symlinks and regular
Problem reported by Christoph Anton Mitterer (Bug#61884).
* src/du.c (process_file): When counting apparent sizes, count
only usable st_size members.
* tests/du/apparent.sh: New file.
* tests/local.mk (all_root_tests): Add it.
2023-03-04 16:57:16 -08:00
Paul Eggert 8022874d12 split: tune for when creating output files
* src/split.c (create): Avoid fstat + ftruncate in the usual case
where the output file does not already exist, by trying
to create it with O_EXCL first.  This costs a failed open
in the unusual case where the output file already exists,
but that’s OK.
2023-03-04 14:49:46 -08:00
Paul Eggert 788654dd82 split: style fix
* src/split.c (ofile_open): Avoid ‘if (! (a = b))’ style.
2023-03-04 14:49:46 -08:00
Paul Eggert 40bf1591bb split: prefer signed integers to size_t
This allows for better runtime checking with gcc
-fsanitize=undefined.
* src/split.c: Include idx.h.
(open_pipes_alloc, n_open_pipes, suffix_length)
(set_suffix_length, input_file_size, sufindex, outbase_length)
(outfile_length, addsuf_length, create, cwrite, bytes_split)
(lines_split, line_bytes_split, lines_chunk_split)
(bytes_chunk_extract, ofile_open, lines_rr, main):
Prefer signed integers (typically idx_t) to size_t.
2023-03-04 14:49:46 -08:00
Paul Eggert 3434cdcec1 split: handle large numbers better
Prefer signed types to uintmax_t, as this allows for better
runtime checking with gcc -fsanitize=undefined.
Also, when an integer overflows just use the maximal value
when the code will do the right thing anyway.
* src/split.c (set_suffix_length, bytes_split, lines_split)
(line_bytes_split, lines_chunk_split, bytes_chunk_extract)
(lines_rr, parse_chunk, main):
Prefer a signed type (typically intmax_t) to uintmax_t.
(strtoint_die): New function.
(OVERFLOW_OK): New macro.  Use it elsewhere, where we now allow
LONGINT_OVERFLOW because the code then does the right thing on all
practical platforms (they have int wide enough so that it cannot
be practically exhausted).  We can do this now that we can safely
assume intmax_t has at least 64 bits.
(parse_n_units): New function.
(parse_chunk, main): Use it.
(main): Do not worry about integer overflow when the code
will do the right thing anyway with the extreme value.
Just use the extreme value.
* tests/split/fail.sh: Adjust to match new behavior.
2023-03-04 14:49:46 -08:00
Paul Eggert 1ebee5b1a3 split: prefer ssize_t for read result
* src/split.c (bytes_split, lines_chunk_split)
(bytes_chunk_extract, main): Prefer ssize_t to size_t when
representing the return value of ‘read’.  Use a negative value
instead of SIZE_MAX to indicate a missing value.
2023-03-04 14:49:46 -08:00
Paul Eggert e19a59141b split: be more careful about buffer sizes
* src/split.c: Include sys-limits.h, not safe-read.h.
(input_file_size, bytes_split, lines_split, line_bytes_split)
(lines_chunk_split, bytes_chunk_extract, lines_rr): Call read, not
safe_read, since safe_read no longer buys us anything.
(main): Reject outlandish buffer sizes right away,
rather than allocating huge buffers and never using them.
2023-03-04 14:49:46 -08:00
Paul Eggert 0450987853 split: minor -1 / 0 refactor
* src/split.c (create, bytes_split, ofile_open):
Prefer comparing to 0 to comparing to -1.
2023-03-04 14:49:46 -08:00
Paul Eggert a110ce4ce3 split: don’t worry about ECHILD
* src/split.c (closeout): There should be no need for a special
case for ECHILD, since we never wait for the same child twice.
Simplify with this in mind.
2023-03-04 14:49:46 -08:00
Paul Eggert 41615f0f8f split: don’t assume pid_t fits in int
* src/split.c (filter_pid): Now pid_t, not int.
(of_t): opid member is now pid_t, not int.
2023-03-04 14:49:45 -08:00
Paul Eggert 99fcde22ce split: simplify SIGPIPE handling
Ignore and default SIGPIPE, rather than blocking and unblocking it.
* src/split.c (default_SIGPIPE):
New static var, replacing oldblocked and newblocked.
(create): Use it.
(main): Set it.
2023-03-04 14:49:45 -08:00
Paul Eggert aa266f1b3d split: port ‘split -n N /dev/null’ better to macOS
* src/split.c (input_file_size): Do not bother with lseek if the
initial read probe reaches EOF, since the file size is known then.
This works better on macOS, which doesn’t allow lseek on /dev/null.
Do not special-case size-zero files, as the issue can occur
with any size file (though /proc files are the most common).
If the current position is past end of file, treat this as
size zero regardless of whether the file has a usable st_size.
Pass through lseek -1 return values rather than using ‘return -1’;
this makes the code a bit easier to analyze (and a bit faster).
Avoid undefined behavior if the size calculation overflows.
(lines_chunk_split): Do not bother with lseek if it would have
no effect if successful.  This works better on macOS, which
doesn’t allow lseek on /dev/null.
* tests/split/l-chunk.sh: Adjust to match fixed behavior.
2023-03-04 14:49:45 -08:00
Paul Eggert fe64f8be01 maint: fix NEWS quoting
* NEWS: Use English-language quoting in recent news.
2023-03-04 14:49:45 -08:00
Paul Eggert fb6fc7f3ce split: split more evenly with -n
* src/split.c (bytes_split): New arg REM_BYTES.
Use this to split more evenly.  All callers changed.
(lines_chunk_split, bytes_chunk_extract):
Be consistent with new byte_split.
* tests/split/b-chunk.sh, tests/split/l-chunk.sh: Test new behavior.
2023-03-04 14:49:45 -08:00
Paul Eggert 0d997e18b9 split: small -n lines simplification
* src/split.c (lines_chunk_split):
Rewrite while as if-while for clarity.
2023-03-04 14:49:45 -08:00
Paul Eggert f749449e5c split: refactor lines_chunk_split
* src/split.c (lines_chunk_split): Simplify by having chunk_end
point to the first byte after the chunk, rather than to the last
byte of the chunk.  This will reduce confusion once we allow
chunks to be empty.
2023-03-04 14:49:45 -08:00
Paul Eggert 0789451237 tee: tune when later -p overrides earlier
* src/tee.c (pipe_check): Make this a local var instead
of a static var.  This suppresses a -Wmaybe-uninitialized
diagnostic with gcc 12.2.1 20221121 (Red Hat 12.2.1-4).
(main): Don’t set pipe_check unnecessarily if a later
-p option overrides an earlier one that wants pipe_check.
Problem discovered when I investigated the GCC warning.
2023-03-04 14:49:45 -08:00
Pádraig Brady 76b5843ff9 maint: refactor tail.c to use iopoll
* src/tail.c (check_output_alive): Reuse iopoll()
rather than directly calling poll() or select().
* src/iopoll.c (iopoll): Refactor to support non blocking operation,
or ignoring descriptors by passing a negative value.
* src/iopoll.h (iopoll): Adjust to support a BLOCK parameter.
* src/tee.c (tee_files): Adjust iopoll() call to explicitly block.
* src/local.mk: Have tail depend on iopoll.c.
2023-03-01 19:27:36 +00:00
Pádraig Brady 5bcc2912e7 tail: avoid undefined behavior when polling outputs
* src/tail.c (check_output_alive): Only check the returned
events from poll() when it indicates there are events to check.
2023-02-28 14:04:16 +00:00
Pádraig Brady fb757fd7d9 doc: tee -p: clarify operation
* src/tee.c (usage): Change from describing one (non pipe) aspect
to the more general point of being the option to use if working with
pipes, and referencing the more detailed info below.
* doc/coreutils.texi (tee invocation): s/standard/appropriate/ since
the standard operation with pipes is to exit immediately upon write
error.  s/early/immediately/ as it's ambiguous as to what "early"
is in relation to.
2023-02-28 14:02:42 +00:00