Commit Graph
7826 Commits
Author SHA1 Message Date
Pádraig Brady ea94589e9e b2sum: a new checksum utility with md5sum like interface
Note we don't support the --algorithm option of the
b2sum command in the external BLAKE2 project, as that
was deemed too confusing for users.
"BLAKE2b" was chosen as the default algorithm to use,
which is single threaded but performs well on 64 bit.

* src/blake2: CC0 source copied from external project.
* cfg.mk[VC_LIST_ALWAYS_EXCLUDE_REGEX]: Exclude blake2/
from syntax checks, make update-copyright, etc.
* src/local.mk: Reference the sources for b2sum,
and set the compilation flags.
* doc/coreutils.texi (b2sum invocation): Reference the
md5sum invocation node, and add descriptions of -l.
* tests/misc/b2sum.sh: Add new test.
* tests/local.mk: Reference new test.
* AUTHORS: Add new binary.
* README: Likewise.
* build-aux/gen-lists-of-programs.sh: Likewise.
* man/.gitignore: Likewise.
* scripts/git-hooks/commit-msg: Likewise.
* man/b2sum.x: New man page template.
* man/local.mk: Reference new template.
* src/.gitignore: Ignore new binaries.
* src/blake2/.gitignore: Ignore new build atrifacts.
* src/md5sum.c (usage): Describe the new -l option.
* NEWS: Mention the new program.
2016-11-22 20:04:24 +00:00
Pádraig Brady f179919848 cp: improve status message when omitting directories
* src/copy.h (cp_options): Add a new flag for install(1).
* src/copy.c (copy_internal): For cp, also output remediation
advice which also indicates why directories aren't copied by default.
The message is unchanged for install(1).
* src/cp.c (cp_option_init): Init install_mode to false.
* src/mv.c (cp_option_init): Likewise.
* src/install.c (cp_option_init): Init install_mode to true.
* tests/install/basic-1.sh: Add a test case.
* tests/cp/link-deref.sh: Adjust test case.
Fixes http://bugs.gnu.org/24958
2016-11-19 17:05:33 +00:00
Antonio Ospite 1ba4cdb3e1 dircolors: highlight Motion JPEG multimedia files
* src/dircolors.hin: Add .mjpg and .mjpeg multimedia files.
This resolves http://bugs.gnu.org/24436.
2016-11-15 12:35:37 -08:00
Pádraig Brady eba871cd32 all: reduce usage of proper_name_utf8()
This reduces a standard coreutils install size by about 160K.

* src/cat.c: Change to proper_name() which removes about 18K text.
* src/cp.c: Likewise.
* src/df.c: Likewise.
* src/du.c: Likewise.
* src/getlimits.c: Likewise.
* src/realpath.c: Likewise.
* src/split.c: Likewise.
* src/stdbuf.c: Likewise.
* src/timeout.c: Likewise.
* src/truncate.c: Likewise.
* src/local.mk: Remove -llibiconv from the above programs.
* cfg.mk (sc_check-AUTHORS): Adjust to use factor(1).
* AUTHORS: Adjust to use ASCII to satisfy sc_check-AUTHORS.
2016-11-14 00:34:39 +00:00
Pádraig Brady 4d45722d11 tail: only retry file open if --retry specifed
* src/tail.c (tail_file): On failure to open a file,
set ignore=true when --retry is not specified.
* tests/tail-2/assert-2.sh: Adjust to the new behavior.
* tests/tail-2/retry.sh: Add a test case.  Also change
from `tail ... && fail=1` to the more robust `returns_ 1 ...`
construct which detects segfaults etc.
* NEWS: Document the fix.
2016-11-10 12:43:42 +00:00
Pádraig Brady 659060233f tail: ensure -f --retry 'missing' handles truncation
* src/tail.c (tail_forever): The BLOCKING optimization is only
enabled for non regular files (which can't be truncated), so ensure
we don't enable that unless we've a valid file descriptor.
* tests/tail-2/retry.sh: Add a test case.
* NEWS: Mention the bug fix.
2016-11-09 21:38:33 +00:00
Pádraig Brady 7fc7206b03 tail: avoid outputting repeated data with remote files
* src/tail.c (tail_forever): Only read up to st_size on network
file systems to avoid the issue with a stale attribute cache
returning a smaller st_size than we have already read().
The was seen with glusterfs at least and caused the complete
file to be repeatedly output due to assuming the file was
truncated in this case.
* NEWS: Mention the fix.
2016-11-09 19:36:42 +00:00
Pádraig Brady 2a80912529 tail: terminate when following pipes and untailable non pipes
* src/tail.c (ignore_pipe_or_fifo): Mark the descriptor as -1
for pipes so that any_live_files() detects correctly that
the entry is no longer live.
* tests/tail-2/pipe-f.sh: Add a test case.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/24903 which was detected
using Symbolic Execution techniques developed in
the course of the SYMBIOSYS research project at
COMSYS, RWTH Aachen University.
2016-11-08 23:19:08 +00:00
Pádraig Brady 7ce2589b77 maint: refactor printing of backup suffix --help
* src/system.h (emit_backup_suffix_note): A new function to
output the backup suffix info.  The strings are unchanged,
so translations are not impacted.
* src/cp.c (usage): Use the new function.
* src/ln.c (usage): Likewise.
* src/mv.c (usage): Likewise.
* src/install.c (usage): Likewise.
2016-11-07 17:09:35 +00:00
Rishabh Dave ec946718ae maint: simplify handling of backup --suffix in various tools
* src/cp.c (main): Avoid the getenv("SIMPLE_BACKUP_SUFFIX") call,
which is now done if needed in the gnulib backupfile module.
Also avoid the redundant strdup, as we don't modify this suffix.
* src/install.c (main): Likewise.
* src/ln.c (main): Likewise.
* src/mv.c (main): Likewise.
Fixes http://bugs.gnu.org/23153
2016-11-07 17:09:35 +00:00
Pádraig Brady 30012b290f date: add %q to output the quarter of the year
* doc/coreutils.texi (date invocation): Document %q.
* src/date.c (usage): Likewise.
* tests/misc/date.pl: Add a test case.
* cfg.mk (sc_strftime_check): Adjust to allow %q.
* NEWS: Mention the new feature.
2016-11-07 17:09:15 +00:00
Pádraig Brady 9ad400f1ad stat: make --format=%N honor the QUOTING_STYLE env var
* doc/coreutils.texi (stat invocation): Describe the
QUOTING_STYLE values now supported.
* src/stat.c (getenv_quoting_style): A new function called
from main, that sets the default quoting style for quotearg.
(main): Call getenv_quoting_style() when %N specified.
* tests/misc/stat-fmt.sh: Add a test case.
* NEWS: Mention the improvement.
Fixes http://bugs.gnu.org/23422
2016-11-05 00:44:06 +00:00
Pádraig Brady 6f64cb64f7 doc: improve the --help description of ls -N
* src/ls.c (usage): The existing description was inaccurate
as -N does not directly impact whether control characters
are displayed or not.
2016-11-05 00:42:23 +00:00
Pádraig Brady c6656eed6a dd: warn about counts specified with confusing 0x prefix
* src/dd.c (parse_integer): Suggest to use "00x" instead of "0x",
which is significant for the "count", "seek", and "skip" operands.
* tests/dd/misc.sh: Add a test case.
Fixes http://bugs.gnu.org/24874
2016-11-04 20:38:32 +00:00
Pádraig Brady d0ddfadfb2 md5sum,sha*sum: fix --ignore-missing with checksums starting with 00
* NEWS: Mention the fix.
* src/md5sum.c (digest_file): Add a new MISSING parameter to
return whether the file was missing, separately from the digest.
* tests/misc/md5sum.pl: Add a test case.
Fixes http://bugs.gnu.org/24795
2016-10-26 20:42:30 +01:00
Jim Meyering f8b8bbc1b9 maint: we use die; remove now-unnecessary uses of abort
* src/expr.c (integer_overflow): Remove an abort-after-die.
* src/paste.c (write_error): Likewise.
* src/sort.c (badfieldspec): Likewise.
* src/tr.c (string2_extend): Likewise. Also remove a few
break statements, each after an abort.
2016-10-16 11:28:39 -07:00
Pádraig Brady 492dcb2eb1 all: use die() rather than error(EXIT_FAILURE)
die() has the advantage of being apparent to the compiler
that it doesn't return, which will avoid warnings in some cases,
and possibly generate better code.
* cfg.mk (sc_die_EXIT_FAILURE): A new syntax check rule to
catch any new uses of error (CONSTANT, ...);
2016-10-16 12:23:55 +01:00
Jim Meyering d035eacfde build: also distribute new file, src/die.h
* src/local.mk (noinst_HEADERS): Add die.h.
2016-10-15 16:52:14 -07:00
Jim Meyering dad7ab0b7b build: add die.h; avoid new warnings from GCC 7
* src/die.h (die): New file/function from grep.
Note: we expect this file to migrate to gnulib.
* src/csplit.c: Include die.h.
(check_format_conv_type): Use die in place of error-nonzero;break;
* src/install.c (strip): Likewise.
* src/nl.c (proc_text): Likewise.  This also suppresses a new warning
from GCC 7's -Werror=strict-overflow.
* src/tail.c (parse_options): Likewise.
* src/basename.c (main): Adjust "fall through" comment
so that GCC 7's -Wimplicit-fallthrough honors it.
* src/cp.c (main): Add a "fall through" comment.
* src/ls.c (gobble_file): Likewise.
(get_funky_string): Adjust a "fall through" comment so it is
recognized.
* cfg.mk (exclude_file_name_regexp--sc_system_h_headers): Add die.h
to this list of exempt src/*.h files.
2016-10-15 08:41:42 -07:00
Jim Meyering 0d9323ec55 maint: pr: avoid new GCC 7 warning
* src/pr.c (main): Avoid this warning from GCC 7:
src/pr.c:1119:6: error: assuming signed overflow does not occur when \
  simplifying conditional to constant [-Werror=strict-overflow]
   if (n_files == 0)
2016-10-15 08:41:42 -07:00
Pádraig Brady 0e060d9fef rm: disallow --n alias for --no-preserve-root
* src/rm.c (main): Ensure the full --no-preserve-root
option is specified, rather than allowing --n etc.
* tests/rm/r-root.sh: Add a test case.
* NEWS: Mention the change in behavior.

Improved by Jim Meyering.
Fixes http://bugs.gnu.org/24604
2016-10-04 23:35:59 +01:00
Pádraig Brady f04daf570b tail: -F now always processes initially untailable files
which was not the case when inotify was not available.

* src/tail.c (any_live_files): Simplify, since the IGNORE
flag is now only set when a file should be ignored indefinitely.
(recheck): Only output the "giving up on name" message
when that's actually the case.  Only set the IGNORE flag
when ignoring a file indefinitely.
(tail_file): Likewise.
* tests/tail-2/retry.sh: Add a test case.  Also run
all existing test cases with and without inotify.
NEWS: Mention the fix.
THANKS.in: Add the reporter.
Fixes http://bugs.gnu.org/24495 which was detected
using Symbolic Execution techniques developed in
the course of the SYMBIOSYS research project at
COMSYS, RWTH Aachen University.
2016-09-28 23:40:47 +01:00
Jim Meyering eb406b2caf who: avoid new warning from upcoming gcc-7
* src/who.c (idle_string): This function would fail to compile
with -Werror and today's built-from-git gcc due to this warning:
src/who.c: In function 'print_user':
src/who.c:201:36: error: may write format character ':' at offset 4 \
  past the end of the destination [-Werror=format-length=]
           sprintf (idle_hhmm, "%02d:%02d",
                                    ^~~~~
The fix is to use an assertion to inform gcc of the existing
invariant that guarantees the number of hours is less than 24.
2016-09-22 10:17:07 -07:00
Paul Eggert 1db94ee969 ls: fix %%b format and precompute more
The old code mishandled --time-spec='+%%b', as it misinterpreted
the '%b' as being the month abbreviation.  Also, it mishandled
the extremely unlikely case of a month abbreviation containing '%'.
The performance part of this patch sped up 'ls' by about 1% on my
little benchmark of 'ls -lR' on the source directory in the
en_US.UTF-8 locale (Fedora 24 x86-64).
* NEWS: Document the bug fix.
* src/ls.c (first_percent_b, abformat_init): New static functions.
(ABFORMAT_SIZE): New constant.
(use_abformat): New static var.
(abmon, required_mon_width): Remove these static vars.
(abmon_init): Now accepts a pointer to abmon, and returns a boolean.
All callers changed.  Reject month abbrs containing '%', as these
would mess up strftime.  Simplify mbsalign result checking,
since (size_t) -1 exceeds ABFORMAT_SIZE.
(abformat_init, align_nstrftime): Precompute all 24 formats at
startup, rather than computing a format for each time stamp.
(decode_switches): Call abformat_init instead of abmon_init.
(align_nstrftime): Accept recentness bool instead of format.
All callers changed.
* tests/misc/time-style.sh: Test for format with '%%b'.
2016-09-19 12:48:32 -07:00
George Burgess IV e095378302 pinky,who: remove redeclarations of ttyname
Given that `ttyname` is already conditionally declared in src/system.h,
other declarations are redundant and problematic for example in ChromeOS
which has a new FORTIFY implementation that, as an artifact of how it's
implemented, causes the compiler to complain if certain standard library
functions are redeclared without special compiler-specific attributes.

* src/pinky.c: Remove declaration (which was unused anyway).
* src/who.c: Remove declaration.
* src/system.h (getlogin, getuid, geteuid, getgrgid, getpwuid, ttyname):
Add the parameter type to the declarations to avoid warnings when these
backup declarations are used.
2016-09-11 12:46:02 +01:00
Paul Eggert 89e1fef97c cp: use FICLONE instead of BTRFS_IOC_CLONE
This doesn't affect the generated code on my system; it's merely
a cleaner way to use the recently-introduced Linux-specific API.
* m4/jm-macros.m4 (gl_CHECK_ALL_HEADERS): Check for linux/fs.h.
* src/copy.c: Include <linux.fs.h> if available.
(FICLONE) [__linux__]: Define if not already defined.
(clone_file): Use FICLONE instead of the older BTRFS_IOC_CLONE,
as this ioctl is no longer btrfs-specific.
2016-09-09 15:29:07 -07:00
Assaf Gordon c6389a1340 date: add '--debug' option
Usage example:
    date --debug -d 'now + 3 days'

Print parsing and debugging information to stderr when using date's
-d/--date option.
See: http://lists.gnu.org/archive/html/coreutils/2016-01/msg00076.html

* src/date.c (main): Add '--debug' option, enable debugging
in gnulib's parse-datetime.y module.
* tests/misc/date.pl: Repeat tests with '--debug' enable, ensure
no regression.
* tests/misc/date-debug.sh: Test output of '--debug' option.
* tests/local.mk: Add above test.
* NEWS: Mention new option.
* doc/coreutils.texi: Likewise.
2016-09-08 01:00:23 -04:00
Paul Eggert d47090d964 sort: port to strict C + valgrind
* bootstrap.conf (gnulib_modules): Add flexmember.
* src/sort.c: Include flexmember.h.
(struct tempnode): Make the last member flexible.
(create_temp_file): Port to strict C11/C99 rules for
allocation alignment with flexible array members.
2016-09-07 19:51:12 -07:00
Pádraig Brady 6643842bad factor: reinstate immediate output with interactive use
* src/factor.c (lbuf_putc): Only buffer more than a line
when not using the tool interactively.
* NEWS: Mention the bug fix.
Fixes http://pad.lv/1620139
2016-09-07 10:20:39 +01:00
Kamil Dudka 5445f7811f ls: allow interruption when reading slow directories
Postpone installation of signal handlers until they're needed.
That is right before the first escape sequence is printed.

* src/ls.c (signal_setup): A new function refactored from main()
to set and restore signal handlers.
(main): Move signal handler setup to put_indicator()
so that the default signal handling is untouched as long as possible.
Adjusted condition for restoring signal handlers to reflect the change.
(put_indicator): Install signal handlers if called for the very first
time.  It uses the same code that was in main() prior to this commit.
* NEWS: Mention the improvement.

See https://bugzilla.redhat.com/1365933
Fixes http://bugs.gnu.org/24232
2016-09-07 10:16:59 +01:00
Jim Meyering 48ea99e1fa dircolors: recognize .zst and .tzst suffixes
* src/dircolors.hin: Add .zst and .tzst suffixes for the
Zstandard/zstd compression tools.  See http://zStd.net and
https://code.facebook.com/posts/1658392934479273/
2016-08-31 10:39:30 -07:00
Jim Meyering d7ee241c23 maint: tr: tweak/make-consistent a pair of loop indices
* src/tr.c (unquote): Use indices of type "unsigned int", rather
than a mix or "size_t" and "int", presuming that no command-line-
specified string will have length longer than UINT_MAX.
2016-08-30 07:57:21 -07:00
Jim Meyering def5cd571b tr: move decls "down" 2016-08-30 07:57:21 -07:00
Jim Meyering d591776d81 ptx: avoid new warning/error from upcoming gcc-7.x
* src/ptx.c (fix_output_parameters): Switch to an unsigned type that
matches the OCCURS.file_index type. This avoids the following error
from gcc-7.0.0 20160829 (experimental):
  src/ptx.c:1220:14: error: assuming signed overflow does not occur \
    when simplifying conditional to constant [-Werror=strict-overflow]
          if (file_index > 0)
2016-08-29 08:37:08 -07:00
Pádraig Brady bd75118c16 cp: with --parents --no-preserve=mode don't copy dir perms
* src/cp.c (make_dir_parents_private): Use default permissions for
created directories when --no-preserve=mode is specified.
* tests/cp/cp-parents.sh: Add a test case.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/24251
2016-08-18 13:54:48 +01:00
Pádraig Brady 3e71b06577 doc: fix ambiguous multiplier info in dd man page
* src/dd.c (usage): Add a comma which is significant for
the rewrapping done in man pages.
2016-08-18 11:58:39 +01:00
Assaf Gordon ff2178bf30 maint: refactor common expand(1) and unexpand(1) code
* src/expand.c, src/unexpand.c: Move global variables from here...
* src/expand-common.h, src/expand-common.c: ... to here.
* src/expand.c, src/unexpand.c: (parse_tab_stops, validate_tab_stops,
next_file): Move identical functions to new module.
(add_tab_stop): Move to new module, including additional code from
'unexpand' (keeping max_column_width) which will have no effect in when
used in 'expand'. Refactor common next-column calculation code
into a new function 'get_next_tab_column'.
* src/local.mk: (src_expand_SOURCES, src_unexpand_SOURCES): Add
'expand-common.c'; (noinst_HEADERS): Add 'expand-common.h'.
* po/POTFILES.in: Add 'expand-common.c'.
* tests/misc/expand.pl: Add more tests.
* tests/misc/unexpand.pl: Likewise.
* TODO: Move conclusions to above test after investigation.
2016-08-09 11:31:00 +01:00
Philipp Thomas 1c17f61ef9 df: improve performance with many mount points
Use hash table for seaching in filter_mount_list() and get_dev()

This improves performance for 20K mount entries from:
  real    0m1.731s
  user    0m0.532s
  sys     0m1.188s
to:
  real    0m1.066s
  user    0m0.028s
  sys     0m1.032s

* src/df.c (devlist_table): Define hash table.
(devlist_hash): Add hash function.
(devlist_compare): Add hash comparison function.
(devlist_for_dev): Add lookup function.
(devlist_free): Add cleanup function.
(filter_mount_list): Use the above hash table.
While at it, rename the variable 'devlist' to 'seen_dev' for
better readability.
(me_for_dev): Use the above lookup function.
NEWS: Mention the improvement.
THANKS.in: Remove the committer; add original submitter Josef Cejka.
2016-08-03 12:35:47 +01:00
Pádraig Brady 5bbce2d2d7 doc: revert recent date(1) synopsis change
* src/date.c (usage): This was thought to introduce other ambiguities,
and was inconsistent with the format presented in the touch(1) docs.
* doc/coreutils.texi (date invocation): Likewise.
See http://bugs.gnu.org/24077
2016-07-28 12:38:20 +01:00
Bernhard Voelker 558ce3044d maint: pacify GCC 6 with -Wnull-dereference
src/id.c:249:29: error: potential null pointer dereference \
[-Werror=null-dereference]
       pw_name = xstrdup (pwd->pw_name);
                          ~~~^~~~~~~~~
src/whoami.c:89:11: error: potential null pointer dereference \
[-Werror=null-dereference]
   puts (pw->pw_name);
         ~~^~~~~~~~~

* src/id.c (main): Explicitly exit with EXIT_FAILURE after an eror to
help gcc-6 to detect that the dereferenced pointer is valid.
* src/whoami.c (main): Likewise.
2016-07-27 21:28:49 +02:00
Pádraig Brady 8d1b163080 doc: make date(1) synopsis consistent with output +FORMAT
* src/date.c (usage): Make the characters used to summarize
the input format, match the output +FORMAT characters.
* doc/coreutils.texi (date invocation): Likewise.
Fixes http://bugs.gnu.org/24077
2016-07-27 17:08:24 +01:00
Kamil Dudka 0f9fc7b3b9 sort: with -h, disallow thousands separator between number and unit
* src/sort.c (traverse_raw_number): Accept thousands separator only
if it is immediately followed by a digit.
* tests/misc/sort-h-thousands-sep.sh: Cover the fix for this bug.

Suggested by Pádraig Brady in http://bugs.gnu.org/24015
2016-07-18 22:24:26 +01:00
Kamil Dudka dfbc945a56 sort: make -h work with -k and blank used as thousands separator
* src/sort.c (traverse_raw_number): Allow to skip only one occurrence
of thousands_sep to avoid finding the unit in the next column in case
thousands_sep matches as blank and is used as column delimiter.
* tests/misc/sort-h-thousands-sep.sh: Add regression test for this bug.
* tests/local.mk: Reference the test.
* NEWS: Mention the bug fix.
Reported at https://bugzilla.redhat.com/1355780
Fixes http://bugs.gnu.org/24015
2016-07-18 22:12:32 +01:00
Kamil Dudka 5cc804337b maint: sort.c: deduplicate code for traversing numbers
* src/sort.c (traverse_raw_number): New function for traversing numbers.
(find_unit_order): Use traverse_raw_number() instead of open-coding it.
(debug_key): Likewise.
2016-07-18 22:09:07 +01:00
Kamil Dudka 502518b440 install: with -Z, set default SELinux context for created directories
* doc/coreutils.texi (install invocation): Update -Z documentation.
* src/install.c (make_ancestor): Set default security context before
calling mkdir() if the -Z option is given.
(process_dir): Call restorecon() on the destination directory if the
-Z option is given.
(usage): Update -Z documentation.
* tests/install/install-Z-selinux.sh: A new test for 'install -Z -D'
and 'install -Z -d' based on tests/mkdir/restorecon.sh.
* tests/local.mk: Reference the test.
* NEWS: Mention the improvement.
Reported at https://bugzilla.redhat.com/1339135
Fixes http://bugs.gnu.org/23868
2016-07-08 18:40:23 +01:00
Pádraig Brady dfae782200 stty: fix sane setting of susp to ^z on Solaris
* src/stty.c: Disable setting of "swtch" to ^z as that
conflicts with and disables using ^z as "susp".
* NEWS: Mention the bug fix.
Reported and tested by Rich Burridge at:
http://bugs.gnu.org/23866
2016-06-29 10:00:05 +01:00
Paul Eggert 4cdb1703af maint: simplify partial writes etc. in 'yes'
* src/yes.c: Include full-write.h.
(main): Use full_write, not write, to simplify handling of partial
writes.  Don't bother using stdio to output data; just use
full_write with a buffer as large as needed.  Reuse operand
strings if possible, and if the buffer would otherwise be large.
2016-06-25 01:02:08 +02:00
Pádraig Brady 5845664c8c yes: handle short writes
* src/yes.c (main): Loop over the write buffer to
handle the case where write may write less than requested.
* NEWS: Mention the bug fix.
Reported by Paul Eggert.
2016-06-24 01:13:33 +01:00
Pádraig Brady ba5767c2b8 yes: fix copy and paste issue with previous commit
* src/yes.c (main): Output 'y' not '-'.
* tests/misc/yes.sh: Add a test for default output.
2016-06-24 01:09:28 +01:00
Paul Eggert 818b2f4897 maint: work even if argc == INT_MAX
GCC 7 warned about undefined behavior in this unlikely case.
Problem reported by Jim Meyering in: http://bugs.gnu.org/23825
* src/md5sum.c (main):
* src/paste.c (main):
* src/yes.c (main):
Avoid undefined behavior when argc == INT_MAX.
2016-06-23 20:18:03 +02:00