Commit Graph
28284 Commits
Author SHA1 Message Date
Pádraig Brady 00eb7af8ea doc: fix texinfo for short options taking a parameter
* doc/coreutils.texi: (tail invocation): Add missing -s,
along with the existing --sleep-interval description.
(csplit invocation): s/--suffix/--suffix-format/.
(head invocation): Use same variable (COUNT) for -n and --head-count.
(seq invocation): Add opindex items for all options.
(ptx invocation): Likewise.
Fix typo s/--flac-truncation/--flag-truncation/.
(touch invocation): State explicitly that -d takes a parameter,
which also indicates that an '=' is not to be used
for the short option syntax.
(ls invocation): Likewise for the -w option.
Fixes http://bugs.gnu.org/21809
2015-11-02 10:12:12 +00:00
Pádraig Brady 3f5e0453e1 maint: allow 'all:' tag in commit summaries
To tag changes that are user visible
and affect all (or many) commands.
2015-10-29 00:18:13 +00:00
Pádraig Brady b6db36b07f doc: clarify iso-8601 formats used by ls and du
* doc/coreutils.texi (du invocation): Remove 'like' from
mentions of ISO-8601 as the components are individually conformant.
(ls invocation): Likewise, except for --time-style=iso for recent
files, where the MM-DD component is not a valid ISO-8601 timestamp.
2015-10-28 13:58:14 +00:00
Pádraig Brady 17bbf6ce44 date: use extended format timezone for --iso-8601
* src/date.c (main): Use %:z rather than %z with --iso-8601
as the standard states to consistently use extended format.
Note either format can be parsed by date.
* tests/misc/date.pl: Adjust accordingly.
* doc/coreutils.texi (du invocation): Clarify that "iso"
time styles are only similar to ISO-8601.
(ls invocation): Likewise.
(date invocation): Adjust the comment stating
that only --rfc-3339 output can be parsed by date(1).
* NEWS: Mention the change in behavior.
Reported at http://bugs.debian.org/799479
2015-10-27 17:25:12 +00:00
Pádraig Brady badff99041 doc: reference related commands from users(1) and groups(1)
* man/users.x: See also getent, who
* man/groups.x: See also getent
Addresses http://bugs.gnu.org/21735
2015-10-27 17:25:12 +00:00
Pádraig Brady 9459d9d811 copy,dd: simplify and optimize NUL bytes detection
* src/factor.c: Move LIKELY() definition to...
* src/system.h: ...here.
(is_nul): Reimplement with a version that doesn't
require a sentinel after the buffer, and which calls
down to (the system optimized) memcmp.
Performance analyzed at http://rusty.ozlabs.org/?p=560
* src/dd.c (alloc_obuf): Simplify the is_nul() call by
not needing to write the sentinel.
* src/copy.c (sparse_copy): Likewise.
(copy_reg): Simplify the buffer allocation by avoiding
consideration of the sentinel in the buffer size calculation.
2015-10-27 17:25:12 +00:00
Pádraig Brady 6796698c99 all: quote string arguments in error messages
These strings are often file names or other user specified
parameters, which can give confusing errors in
the presence of unexpected characters for example.

* cfg.mk (sc_error_quotes): A new syntax check rule.
* src/*.c: Wrap error() string arguments with quote().
* tests/: Adjust accordingly.
* NEWS: Mention the improvement.
2015-10-27 17:24:54 +00:00
Pádraig Brady 106d4bf159 md5sum: quote all printed file names
This is especially significant when using --check
with files generated on a windows system, where the \r
characters produce corrupted and confusing error messages.
This also ensures status messages are output on a single line.

* src/md5sum.c: Use quote() for printed file names.
* tests/misc/md5sum.pl: Adjust accordingly.
* NEWS: Mention the change in behavior.
Fixes http://bugs.gnu.org/21757
2015-10-27 13:20:50 +00:00
Pádraig Brady e71be1292b ls: fix off by one error when determining max display columns
* src/ls.c (main): Account for the first column not including
a separator when calculating max_idx.
* tests/ls/w-option.sh: Add a test case.
* NEWS: Mention the bug fix.
2015-10-21 16:13:57 +01:00
Pádraig Brady 0e997681d4 ls: allow -w0 to mean no limit on line length
* src/ls.c (print_with_separator): Renamed from print_with_commas,
and parameterized to accept the separator to print.
Also fix an edge case where '\n' not printed when
the POS variable overflows SIZE_MAX.
(print_current_files): Degenerate -x and -C to using the
cheaper print_with_separator() in the -w0 case.
* doc/coreutils.texi (ls invocation): Document the new feature.
* tests/ls/w-option.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the improvement.
Fixes http://bugs.gnu.org/21325
2015-10-21 01:40:02 +01:00
Pádraig Brady 069961924e ls: detect terminal color support using glob patterns
* src/ls.c (know_term_type): Corresponding to commit v8.24-48-gc249a5a,
use fnmatch to inspect the dircolors database.  Noticed due to
failing tests/ls/color-{dtype-dir,term}.sh tests.
2015-10-19 12:59:09 +01:00
Pádraig Brady 31b3db53da tests: adjust recent change to csplit VM limit
* tests/misc/csplit-heap.sh: A little more memory is required
for the full run case.  Noticed with --enable-single-binary.
2015-10-19 11:25:42 +01:00
Pádraig Brady b6757e83c4 maint: fixes to support improved sc_tight_scope
The gnulib provided sc_tight_scope target was ineffective,
as it was checking against an invalid blank regular expression,
and thus ignoring any extern function issues.  This is now
fixed up in gnulib, and so we need to fix our scoping issues
before the next gnulib update.

* cfg.mk: Setup and document the tight_scope config variables
appropriately.
* src/selinux.h: Since declared in *_SOURCES, use the two line
form for the extern function declarations.
* src/set-fields.h: Add the extern declarations, and since declared
in noinst_HEADERS use the single line form.
2015-10-19 10:25:14 +01:00
Pádraig Brady 66504d41aa tests: avoid failure when auto selecting factor tests
* tests/factor/run.sh: If this template is found through
`grep -El "print_ver_.* factor"` for example, then just skip it.
2015-10-19 10:24:12 +01:00
Pádraig Brady 07b73c689d factor: remove unreachable SQUFOF code at compile time
It was a little confusing as to whether the SQUFOF algorithm was
enabled, and in fact there were no options available to enable it.
Therefore clarify the 3 configurable behaviors for the code to
3 defines at the top of the program, and only include the SQUFOF
code if enabled at compile time.

$ size src/factor-before
   text    data     bss
  93997    1412    2504
$ size src/factor-after
   text    data     bss
  87885    1404    2504

* src/factor.c: Only include the SQUFOF factor code
when enabled via the USE_SQUFOF define.
* doc/coreutils.texi (factor invocation): Update note about
factor limits, as we can factor 128 bit numbers without GMP.
2015-10-19 10:24:12 +01:00
Pádraig Brady e50f5273aa doc: reference shuf(1) from the sort (-R) man page
* man/sort.x: Cross reference with shuf(1).
* src/sort.c (usage): Mention shuf(1) with -R option.
Suggested in http://bugs.debian.org/641166
2015-10-19 00:05:18 +01:00
Bernhard Voelker 8802eb16e6 maint: avoid duplicate test runs
* tests/local.mk (all_tests): Remove the tests 'tests/id/setgid.sh' and
'tests/mkdir/smack-root.sh' because they are mentioned in the
'all_root_tests' list; these tests are skipped anyway during a non-root
run because flagged with 'require_root_'.
2015-10-18 18:48:23 +02:00
Paul Eggert 0cb602e33a doc: remove obsolete performance comment
sha512sum can be faster than sha256sum.
E.g., ‘dd if=/dev/zero bs=1024k count=1024 | time sha256sum’
reports 8.16 user CPU seconds on my host, whereas sha512sum
consumes 5.45 seconds (Fedora x86-64 on an AMD Phenom II X4 910e).
Although sha512sum is still considerably slower on x86, a good
chunk of uses are on 64-bit hosts and anyway there’s little point
to scaring people away from sha512sum nowadays.
* doc/coreutils.texi (sha2 utilities): Remove obsolete comment.
2015-10-16 12:07:07 -07:00
Jim Meyering e8d7e20530 maint: avoid uniq.c warning from bleeding-edge gcc's -Wstrict-overflow
* src/uniq.c (main): Make the type of "nfiles" unsigned,
to avoid a brand new warning from a gcc I built from today's
sources (gcc version 6.0.0 20151015 (experimental) (GCC)):
src/uniq.c:523:14: error: assuming signed overflow does not occur \
  when simplifying conditional to constant [-Werror=strict-overflow]
           if (nfiles == 2)
              ^
2015-10-16 08:02:51 -07:00
Pádraig Brady c249a5afc4 dircolors: support globbing of TERM entries
* src/dircolors.c (dc_parse_stream): Support globbing of
TERM entries, to allow entries like "TERM *256color*" for example.
* src/dircolors.hin: Reduce the internal list with globbing.
* tests/misc/dircolors.pl: New test cases.
* NEWS: Mention the improvement.
2015-10-16 12:23:30 +01:00
Pádraig Brady 3a68c3e990 tests: avoid false failure in rm/r-root.sh with gdb warnings
* tests/rm/r-root.sh: Skip the test if there are gdb warnings
that will impact further stderr checks.  For example some
buggy gdb versions may report "Got object file from memory
but can't read symbols: File truncated".  Also fix an incorrect
stderr check from the previous change.
Reported by Bernhard Voelker.
2015-10-15 11:55:18 +01:00
Pádraig Brady 2366daf725 tests: avoid false failure in rm/r-root.sh under load
* tests/rm/r-root.sh: Use gdb rather than timeout(1) as the
last resort protection against unlinkat() calls.  The timeout
of 2s was susceptible to false positives under load, and
gdb is stronger protection in any case.  We remove the
"expensive" tag on this test also since it should be robust.
Reported by Jim Meyering.
2015-10-15 08:44:22 +01:00
Pádraig Brady 0458701041 tests: avoid false failure in a tail test under load
* tests/tail-2/follow-stdin.sh: Use the standard tail
testing framework to avoid the race seen under very high load,
and also test the non inotify case.
Reported by Jim Meyering
2015-10-15 08:44:09 +01:00
Pádraig Brady f9fb9acb0d tests: avoid false failures with default ACLs
To reproduce:
  setfacl -dm group::rx .
  setfacl -dm other::rx .
  make check

* init.cfg (require_no_default_acl_): A new function to skip
when default ACLs are detected, or if the getfacl utility is
not available then skip if any non LSM ACLs detected.
* tests/cp/existing-perm-race.sh: Call require_no_default_acl_.
* tests/mkdir/parents.sh: Likewise.
* tests/mkdir/perm.sh: Likewise.
2015-10-14 15:23:14 +01:00
Pádraig Brady e6b3af647d tail: no longer warn about unrecognized file systems
* src/tail.c (fremote): No longer prompt the user to email
with the unrecognized file system constant, since we have
process in place to sync periodically with the latest Linux
constants, and the fall back polling mode is still fully functional.
2015-10-12 01:40:27 +01:00
Pádraig Brady 88eb88ee23 tests: avoid false failure when restorecon is ineffective
* tests/cp/cp-a-selinux.sh: Ensure we skip the portion of the test
depending on restorecon to be effective.  I.E. also skip when restorecon
warns, as it doesn't exit with error status when matchpathcon fails to
find a match for a file.  This is the case in /tmp on Fedora for
example, in which case the new destination that cp creates will have the
default security context of the process, rather than the explicit
context we set on the source file.
Details at: http://bugzilla.redhat.com/1247641
2015-10-10 17:05:37 +01:00
Pádraig Brady 934940b561 tests: make a long running test responsive to Ctrl-C
* tests/misc/sort-compress-hang.sh: Use --foreground with the
timeout(1) command (noting the caveats), to run the sort command
in the foreground program group, and thus be responsive to Ctrl-C.
This very_expensive_ test takes over a minute on a i3-2310M,
with RAM backed /tmp.
2015-10-10 11:21:34 +01:00
Pádraig Brady c1dba59a79 tests: adjust recent changes to virtual memory limits
* tests/dd/no-allocate.sh: Account for timeout(1) when
determining the required mem, as timeout has additional shared libs.
This avoids the need for the hardcoded 4M addition to the limit.
* tests/misc/head-c.sh: Increase the base limit, to account for
the fact that head(1) will allocate some additional mem in this case.
* tests/misc/cut-huge-range.sh: Remove mention of specific limits.
* tests/misc/printf-surprise.sh: Likewise.
Reported by Dmitry V. Levin
2015-10-08 03:54:02 +01:00
Pádraig Brady ce501787d4 csplit: remove erroneous mention of -m in --help
* src/csplit.c (usage): -m is not accepted, only --suppress-matched.
* tests/misc/csplit-suppress-matched.pl: Spelling fix.

Reported by Ondrej Oprala
2015-10-06 11:24:27 +01:00
Paul Eggert 88a7228bb2 tee: simplify argv handling
* src/tee.c (tee_files): Last arg is now char ** instead of char
const **, as that is a bit simpler.  All callers changed.  Modify
files[-1], not files[nfiles], as that is a bit faster and simpler.
Latter problem pointed out by Rainer Deyke in:
http://bugs.gnu.org/21611
2015-10-03 17:02:43 -07:00
Paul Eggert ac81caaf77 build: update gnulib submodule to latest 2015-10-03 00:32:34 -07:00
Paul Eggert f689642bac build: Pacify GCC 5 on 32-bit hosts
This pacifies GCC 5 in a better way, without disabling diagnostics.
* src/df.c (main): Tell compiler that optind is positive.
* src/shred.c (known): New function.
(dopass): Go back to off_t for file sizes.
Avoid integer overflow if we run off the end of the file.
Tell compiler that a write cannot write more bytes than requested.
2015-10-03 00:32:34 -07:00
Pádraig Brady 3ba68f9e64 tail: handle kernel dentry unlink race
Avoid the intermittent loss of "... has become inaccessible" messages.
That would cause tests/tail-2/assert.sh to fail sometimes,
mainly on uniprocessor systems.

* src/tail.c (tail_forever_inotify): Also monitor IN_DELETE
events on the directory, to avoid a dentry unlink()..open() race,
where the open() on the deleted file was seen to succeed after an,
unlink() and a subsequent IN_ATTRIB, was sent to tail.  Note an
IN_ATTRIB is sent on the monitored file to indicate the change in
number of links, and we can't just use a decrease in the number of
links to determine the file being unlinked, due to the possibility
of the file having multiple links.

Reported by Assaf Gordon and Ludovic Courtès.
Fixes http://bugs.gnu.org/21460
2015-10-03 03:03:24 +01:00
Pádraig Brady 00ac178893 build: avoid -Wstrict-overflow warnings with GCC 5.1 on 32 bit
* src/shred.c (dopass): With -O2, GCC 5.1 gives "assuming signed
overflow does not occur when simplifying conditional to constant"
warnings, in regard to the signed (off_t) variables.  Therefore
use unsigned (uintmax_t) instead, and a separate boolean to cater
for the special meaning of the negative part of the integer range.
Noticed at http://hydra.nixos.org/build/24983447
2015-10-03 02:54:51 +01:00
Pádraig Brady 8fe6f70f35 build: update gnulib submodule to latest
Includes a change to xalloc.h to avoid -Wstrict-overflow warnings
with GCC 5.1 on 32 bit with optimization enabled.  A subsequent
commit will fix similar issues in shred.
2015-10-03 02:54:44 +01:00
Dario Giovannetti a960c31c7d dircolors: add xterm-termite entry
* src/dircolors.hin: Add "xterm-termite" as this VTE based terminal
emulator is quite different from xterm, despite the name.
For example "Termite supports italic text and it won't work if TERM
is set to xterm. Even the backspace key won't work properly anymore
for applications relying on terminfo".
Reported also by Lukas Sabota and Sven-Hendrik Haase.
2015-10-02 16:59:50 +01:00
Dave Chiluk 3babaf8387 df: prioritize mounts nearer the device root
In the presence of bind mounts of a device, the 4th "mount root" field
from /proc/self/mountinfo is now considered, so as to prefer mount
points closer to the root of the device.  Note on older systems with
an /etc/mtab file, the source device was listed as the originating
directory, and so this was not an issue.
Details at http://pad.lv/1432871

* src/df.c (filter_mount_list): When deduplicating mount entries,
only prefer sources nearer or at the root of the device, when the
target is nearer the root of the device.
* NEWS: Mention the change in behavior.
2015-09-23 01:02:44 +01:00
Pádraig Brady 982c79fa14 build: update gnulib submodule to latest
This includes a change to propagate the 4th "mount root"
field from /proc/self/mountinfo from the mountlist module,
which is needed in a subsequent commit in df.

* gl/lib/regcomp.c.diff: Regenerate against latest gnulib.
* gl/lib/regex_internal.c.diff: Likewise.
* gl/lib/regex_internal.h.diff: Likewise.
* cfg.mk: Exclude diffs from trailing whitespace check,
which is generally correct, and now needed.
2015-09-23 01:02:44 +01:00
Bernhard Voelker 9db234ad09 maint: use adaptive approach for ulimit -v based tests
When configured with either 'symlinks' or 'shebangs' as value for
the --enable-single-binary option, tests based on `ulimit -v` are
skipped.  The reason is that the multicall 'coreutils' binary requires
much more memory due to shared libraries being loaded, and the size of
the 'date' binary (~290KiB) compared to the multicall binary (~5MiB),
of course.  Finally, in the case of 'shebangs', the starting shell
requires more memory, too

Instead of using hard-coded values for the memory limit, use an
adaptive approach: first determine the amount of memory for a similar,
yet more trivial invocation of the command, and then do the real test
run using that limit (plus some buffer in some cases).

* init.cfg (require_ulimit_v_): Remove function.
(get_min_ulimit_v_): Add function to determine the minimum memory limit
required for a given command in an adaptive way.
* cfg.mk (sc_prohibit_test_ulimit_without_require_): Change the name
of the above function in the syntax-check rule.
* tests/cp/link-heap.sh: Use the above function to determine the
minimum memory required to run a command simpler than in the real test
run.  Use that limit plus a buffer there.  While at it, change to list
of commands in the subshell to fail also if the beginning `ulimit -v`
fails.
* tests/dd/no-allocate.sh: Likewise.
* tests/misc/csplit-heap.sh: Likewise.
* tests/misc/cut-huge-range.sh: Likewise.
* tests/misc/head-c.sh: Likewise.
* tests/misc/printf-surprise.sh: Likewise.
* tests/split/line-bytes.sh: Likewise.
* tests/rm/many-dir-entries-vs-OOM.sh: Likewise - doing it separately
for each program under test.
2015-09-22 23:23:26 +02:00
Pádraig Brady 50e61bfdb9 maint: avoid deprecation warning with <selinux/flask.h>
* src/runcon.c (main): As per the compile time warning from
libselinux-2.4-3, lookup the class with string_to_security_class(),
rather than using defines from flask.h.
2015-09-22 19:09:32 +01:00
Pádraig Brady dee6b76010 sort,numfmt: with --debug, diagnose failure to set locale
* src/sort.c (main): With --debug, warn upon setlocale() failure,
which can happen due to incorrectly specified environment variables,
or due to memory exhaustion (simulated with ulimit -v), etc.
* tests/misc/sort-debug-warn.sh: Add a test case.
See also http://savannah.gnu.org/bugs/11004
2015-09-22 02:30:30 +01:00
Bernhard Voelker d4a22dff9b du: avoid to stat all mount points if possible
du calls stat for each mount point at startup.  This would block or
even make du fail if stat for an unrelated mount point hangs.
The result is not needed in the normal case anyway and therefore
should be avoided.  Issue introduced in commit v8.19-2-gcf7e1b5.

* src/du.c (fill_mount_table): Move function up as it's not used ...
(mount_point_in_fts_cycle): ... here, i.e., the DI_MNT set is
initialized and filled only iff FTS has detected a directory cycle.
(main): Remove the initialization and filling of the DI_MNT set here,
and free the DI_MNT set only if it was used.
2015-09-18 13:29:17 +02:00
Paul Eggert e4d8c99436 shred: don’t document -NUMBER option
The -NUMBER option was removed from ‘shred’ in 1999, but the
manual wasn’t updated to match.  Problem reported by Nick Rose in:
http://bugs.gnu.org/21502
* doc/coreutils.texi (shred invocation):
Remove documentation for -NUMBER option.
2015-09-17 00:38:13 -07:00
Assaf Gordon ba24c42e13 numfmt: use new set-fields module to parse --field
numfmt --field=LIST can accept the same options as cut.

* bootstrap.conf: remove xlist, linked-list
* src/local.mk: link numfmt with set-fields
* src/numfmt.c: use set-fields.c instead of custom field parsing code.
  (include_field): adapt to new code.
* tests/misc/numfmt.pl: add new tests, adapt current tests to new
  error message wording from set-fields.c
2015-09-12 02:27:32 +00:00
Assaf Gordon bd1bc2b3fe cut: refactor into set-fields module
Extract the functionality of parsing --field=LIST into a separate
module, to be used by other programs.

* src/cut.c: move field parsing code from here ...
* src/set-fields.{c,h}: ... to here.
  (set_fields): generalize by supporting multiple parsing/reporting
  options.
  (struct range_pair): rename to field_range_pair.
* src/local.mk: link cut with set-field.
* po/POTFILES.in: add set-field.c
* tests/misc/cut.pl: update wording of error messages
2015-09-12 02:27:32 +00:00
Pádraig Brady 5bbdcd2a02 maint: fix heap manipulations in previous commit
* src/sort.c (main): Ensure we don't free() and invalid
pointer when reading implicit stdin.  Also avoid
"definitely lost" valgrind warnings in the --files0-from case.
2015-09-08 12:56:04 +01:00
Pádraig Brady a4b640549d maint: avoid "definitely lost" valgrind warnings
Since commit v8.23-19-g8defcee, main() will return,
rather than call exit(), this inducing "definitely lost"
warnings in valgrind's leak checker.  That precludes using
the following to flag memory leaks:

  valgrind --leak-check=full --error-exitcode=1 \
           --errors-for-leak-kinds=definite

* src/pr.c (main): In dev builds, explicitly free memory allocated.
* src/sort.c (main): Likewise.
* src/tail.c (main): Likewise.
* src/tsort.c (tsort): Likewise.
2015-09-04 17:04:03 +01:00
Pádraig Brady 26c11773a4 tests: test numfmt stdin behavior
* tests/misc/tty-eof.pl: Add numfmt to the list of programs
that accept input on stdin.
2015-09-03 01:11:39 +01:00
Pádraig Brady 895a56c531 ls,ptx: restrict quotearg use to file name output
* src/ls.c (getenv_quoting_style, decode_switches, parse_ls_color):
Use quote() rather than quotearg(), as the latter defaults to
outputting the input unquoted.
* src/ptx.c (main): Likewise.
2015-09-03 01:11:32 +01:00
Pádraig Brady df3b9120b4 base64: no longer support hex or oct --wrap params
* src/base64.c (main): Support decimal numbers with leading zeros,
by disabling the auto detection of octal and hex.  It's not
envisaged that base conversion is needed for --wrap parameters,
and in the edge case it is, $((0x0)) shell constructs can be used.
* tests/misc/base64.pl: Adjust accordingly.
* NEWS: Mention the change in behavior.
2015-09-03 00:34:12 +01:00