Commit Graph
8009 Commits
Author SHA1 Message Date
Chih-Hsuan Yen 437555061c df: avoid multibyte character corruption on macOS
* src/df.c (hide_problematic_chars): Use c_iscntrl() as
passing 8 bit characters to iscntrl() is not supported on macOS.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/32236
2018-07-26 10:24:41 -07:00
Wodry 46d6bd0c0a doc: improve documentation of binary prefixes
* doc/coreutils.texi (Common options):
* src/dd.c, src/head.c, src/od.c, src/stdbuf.c, src/tail.c (usage):
* src/system.h (emit_size_note):
Mention binary prefixes (Bug#32242).
Copyright-paperwork-exempt: Yes
2018-07-22 08:37:15 -07:00
Paul Eggert 4397410532 build: update gnulib submodule to latest
* bootstrap.conf, src/copy.c, src/mv.c, src/shred.c:
Adjust to renaming of renameat2 to renameatu.
2018-07-05 09:52:14 -07:00
Pádraig Brady 3246bc08a6 maint: copy: avoid new static analyzer warnings
* src/copy.c (copy_internal): Use the lint protected src_mode,
rather than accessing the src_sb again.  Also unconditionally
populate src_sb when !x->move_mode and in lint mode.
Reported by Kamil Dudka with coverity and clang analyzer.
2018-06-30 18:57:43 -07:00
Pádraig Brady 68cf7ad5ad maint: fix recent stale comments and spelling mistakes
* doc/coreutils.texi: s/seperator/separator/.
* tests/misc/env-S.pl: Likewise.
* src/env.c: Fix stale comment.
2018-06-30 18:56:39 -07:00
Pádraig Brady a68efd1283 env: adjust diagnostics provided for shebang usage
* src/env.c (main): Don't process '-' specially since
that causes an issue on the openbsd getopt implementation
where a lone '-' is now processed as an option, and anyway
it doesn't particuarly help diagnosing common shebang
usage issues.  Also don't restrict the extra diagnostics
for shebang usage to the case with 3 arguments, as
further arguments can be passed to a script.
* tests/misc/env-S.pl: Adjust accordingly.
2018-06-27 02:38:59 -07:00
Carlos Santos 2ed7c28679 maint: fix -Werror=suggest-attribute=malloc in expr.c
Add attribute 'malloc' to mpz_get_str to prevent
the following on GCC 8.1.1

  src/expr.c:117:1: error: function might be candidate for attribute
    'malloc' if it is known to return normally
      [-Werror=suggest-attribute=malloc]
   mpz_get_str (char const *str, int base, mpz_t z)
   ^~~~~~~~~~~
  cc1: all warnings being treated as errors

* src/expr.c (mpz_get_str): Add _GL_ATTRIBUTE_MALLOC.
2018-06-24 21:23:05 -07:00
Pádraig Brady da07736952 maint: sync longlong.h from gmp repo
* src/longlong.h: Sync changes.  No functional change.
2018-06-24 17:43:17 -07:00
Pádraig Brady 07df260421 maint: avoid false positive in src/fs-magic-compare
* src/local.mk (fs_normalize_perl_subst): `make src/fs-magic-compare`
was reporting incorrectly that AFS was not being handled.
Add a mapping to our KAFS identifier.
* .gitignore: Add intermediate files from `make src/fs-magic-compare`
2018-06-24 17:43:17 -07:00
Pádraig Brady c932e5c9ec rm: add --preserve-root=all to protect mounts
* src/remove.c (rm_fts): With the --preserve-root=all extension,
reject command line arguments that are mount points.
* src/remove.h (rm_options): Add preserve_all_root to store config.
* src/mv.c (rm_option_init): Init preserve_all_root to false.
* src/rm.c (main): Init preserve_all_root as per option.
(usage): Describe the new option.
* src/remove.c (rm_fts): Lookup the parent device id,
and reject the cli argument if a separate file system.
* tests/rm/one-file-system.sh: Add a test case.
* NEWS: Mention the new feature.
2018-06-20 21:18:32 -07:00
Adam Borowski 163df41d45 cp: add --reflink=never to force standard copy mode
This mode is currently the default, but most if not all users of
reflink-capable filesystems want --reflink=auto, which is often
encapsulated into an alias.  Adding --reflink=never allows overriding
such an alias.

* doc/coreutils.texi (cp invocation): Describe the new option.
* src/cp.c: Support --reflink=never.
* tests/cp/reflink-auto.sh: Add a test case.
* NEWS: Mention the new feature.
2018-06-20 21:17:19 -07:00
Assaf Gordon 668306ed86 env: add -S/--split-string option
Adopted from FreeBSD's env(1), useful for specifing multiple
parameters on a shebang (#!) script line, e.g:

   #!/usr/bin/env -S perl -w -T

Discussed in https://lists.gnu.org/r/coreutils/2018-04/msg00011.html

* src/env.c (valid_escape_sequence,escape_char,scan_varname,
  extract_varname,validate_split_str,build_argv,
  parse_split_string): New functions.
  (main): Process new option and call parse_split_string.
  (usage): Mention new option.
* tests/misc/env-S.pl: Test new option from the command line.
* tests/misc/env-S-script.sh: Test new option from shebang scripts.
* tests/local.mk (all_tests): Add new tests.
* man/env.x (OPTIONS): Show a brief example of -S usage and point to
the full documentation for more information.
* doc/coreutils.texi (env invocation): Detail usage of -S/--split-string
option.
* NEWS: Mention new option.
2018-06-20 21:15:31 -07:00
Assaf Gordon a0118a71bb env: add -v/--debug option
Prints verbose information about each step:

  $ env -v -uFOO  -C /tmp BAR=BAZ date -u
  env: unset:    FOO
  env: setenv:   BAR=BAZ
  env: chdir:    '/tmp'
  env: executing: date
  env:    arg[0]= ‘date’
  env:    arg[1]= ‘-u’
  Sun Apr 22 08:52:30 UTC 2018

Inspired by FreeBSD's env(1).

* src/env.c (usage): Mention new option.
  (main): Print debug information if requested.
* NEWS: Mention new option.
* doc/coreutils.texi (env invocation): Mention -v/--debug.
2018-06-20 21:14:47 -07:00
Assaf Gordon 068b46e73b maint: refactor unsetenv call in env
Keep unset envvars (-uFOO) in an array for later deletion,
instead of reiterating over argv.  Done in preparation for
'-S string' feature. Related to '-u' discussion in
https://lists.gnu.org/r/coreutils/2018-04/msg00013.html

* src/env.c (append_unset_var,unset_envvars): New functions.
 (main): Use new functions.
2018-06-20 21:14:10 -07:00
Kaxandra Labat 24053fbd8f ls: ignore case when coloring file extensions
* src/ls.c (get_color_indicator): s/STREQ_LEN/c_strncasecmp/
* src/dircolors.hin: Remove a now redundant entry.
* tests/ls/color-ext.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the change in behavior.
2018-06-20 21:02:30 -07:00
Pádraig Brady c744c65f50 md5sum,b2sum,sha*sum: support -z,--zero option
* doc/coreutils.texi (md5sum invocation): Describe the new option,
and how it's not supported by --check, and how it disables escaping.
* src/md5sum.c (delim): A new global to parmeterize the out delimiter.
(main): Don't enable file name escaping with -z, and output '\0'.
* tests/misc/md5sum-newline.pl: Add a test case.
* NEWS: Mention the new feature.
2018-06-20 21:01:14 -07:00
Pádraig Brady 77517a9917 wc: optimize processing of ASCII in multi byte locales
===== Benchmark setup (on GNU/Linux) ====
$ yes áááááááááááááááááááá | head -n100000 > mbc.txt
$ yes 12345678901234567890 | head -n100000 > num.txt

===== Before ====
$ time src/wc -Lm < mbc.txt
real    0m0.186s
$ time src/wc -m < mbc.txt
real    0m0.186s
$ time src/wc -Lm < num.txt
real    0m0.055s
$ time src/wc -m < num.txt
real    0m0.056s

==== After ====
$ time src/wc -Lm < mbc.txt
real    0m0.196s
$ time src/wc -m < mbc.txt
real    0m0.173s
$ time src/wc -Lm < num.txt
real    0m0.031s
$ time src/wc -m < num.txt
real    0m0.028s

* src/wc.c (wc): Only call wide variant functions like
iswprint() and wcwidth() for non is_basic() characters.
I.E. non ISO C "basic character set" characters.
This is especially significant on OSX where wcwidth()
is very expensive (about 10x in tests).
* NEWS: Mention the improvement.
Suggested by Eric Fischer.
2018-06-20 21:00:39 -07:00
Paul Eggert de73c801f3 doc: port man pages to doclifter
Problem reported by Eric S. Raymond (Bug#31803).
* man/test.x: Add SYNOPSIS section, since help2man
understandably gets confused by the square brackets.
* src/ln.c (usage): Omit parenthetical "(Nth form)" in usage,
as it confuses doclifter.
2018-06-13 16:01:14 -07:00
Pádraig Brady 5e7b8928b7 cp: preserve existing permissions with --no-preserve=mode
This issue was introduced in commit v8.19-145-g24ebca6

* src/copy.c (copy_internal): With --no-preserve=mode,
only reset permissions for newly created files.
(copy_reg): Likewise.
* NEWS: Mention the fix.
* tests/cp/preserve-mode.sh: Add a test case.
Fixes https://bugs.gnu.org/31675
2018-06-03 16:27:37 -07:00
Pádraig Brady c732388fa1 cp: with --force; replace self referential symlinks
* src/copy.c (copy_internal): Don't fail immediately upon
getting ELOOP when running stat() on the destination,
rather proceeding if -f specified, allowing the link
to be removed.  If the loop is not in the final component
of the destination path, we still fail but at the
subsequent unlink() stage.
* doc/coreutils.texi (cp invocation): Adjust wording to say
that --force doesn't work with dangling links, rather than
all links that can't be traversed.
* tests/cp/thru-dangling.sh: Add a test case.
* NEWS: Mention the change in behavior.
Discussed in https://bugs.gnu.org/31335
2018-05-17 20:52:09 -07:00
Pádraig Brady d3daa95096 cp: fix symlink checks when overwriting files
Ensure this _does_ recreate the symlink
  Given "path1" and "path2" are on different devices.
  $ touch "path1/file"
  $ cd path2/; ln -s path1/file
  $ cp -dsf path1/file .

Ensure this does _not_ overwrite file
  $ touch file
  $ ln -s file l1
  $ cp -sf l1 file

* src/copy.c (same_file_ok): Remove device ids from consideration,
instead deferring to future EXDEV with --link or allowing
the first case above to work.
Also ensure that we do not exist this function too early,
when the destination file is not a symlink, which protects
against the second case.
* tests/cp/cross-dev-symlink.sh: Add a test for the first case.
* tests/cp/same-file.sh: Add a test for the second case above.
* NEWS: Mention the bug fixes.
* THANKS.in: Mention the reporters who also analyzed the code.
Fixes https://bugs.gnu.org/31364
2018-05-15 09:55:19 -07:00
Pádraig Brady a391007511 cp: ensure --remove-destination doesn't traverse symlinks
* src/cp.c (target_directory_operand): Allow through inaccessible
arguments with -f or --remove.
* doc/coreutils.texi (cp invocation): Clarify that -f doesn't directly
impact the removal of non-traversable symlinks.
* tests/cp/dir-rm-dest.sh: Test the new behavior.
* tests/cp/thru-dangling.sh: Enforce -f behavior wrt symlinks.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/31335
2018-05-15 09:52:44 -07:00
Pádraig Brady 694d10b71e maint: make chmod/chgrp/chown leak free under valgrind
* src/chmod.c: Deallocate the mode change array in dev mode.
* src/chown.c: Make chopt_free() actually deallocate, but
only call in dev mode.
* src/chgrp.c: Likewise.
2018-05-14 23:51:13 -07:00
Pádraig Brady a5d158ab8f doc: improve formatting of nl --help
* src/nl.c (usage): Better delineate the information.
2018-05-14 21:33:37 -07:00
Paul Eggert f6cb50cc99 who: simplify port to GCC 8
* src/who.c (make_id_equals_comment): Use simpler workaround
for GCC bug 85602.  Suggested by Martin Sebor in:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85602#c3
2018-05-14 13:49:58 -07:00
Pádraig Brady ea7ae28b10 build: make GCC 8 adjustments more portable
* src/chown-core.h (chopt_free): Just define away this noop.
* src/chown-core.c (chopt_free): Remove the empty implementation.
2018-05-03 18:59:53 -07:00
Paul Eggert 9781fcd532 maint: port to GCC 8
* src/chown-core.h (chopt_free, gid_to_name, uid_to_name):
No longer const.
* src/make-prime-list.c (xalloc): Add malloc attribute.
* src/who.c (make_id_equals_comment): Work around GCC bug 85602
by using mempcpy rather than strncat.  Although the old code
was correct, strncat raises so many hackles that it’s not
worth maintaining its use here.
2018-05-03 13:04:15 -07:00
Pádraig Brady 3968f28dfd doc: timeout --help: mention 0 DURATION disables timeout
* src/timeout.c (usage): Mention that a duration of 0 disables
the associated timeout, which is both concise info and useful
functionality as timeouts are frequently configured.
2018-04-18 20:11:23 -07:00
Tobias Stoeckmann d1a754c827 cut: improve large file support on 32 bit
Increase max range from SIZE_MAX to UINTMAX_MAX, which will
allow cut to support line lengths up to the max file size
on all systems. The inherent SIZE_MAX limitation in cut was
removed with the enhancements in https://bugs.gnu.org/13127.
Also numfmt gets similarly increased --field ranges due to
shared code.

* src/cut.c: s/size_t/uintmax_t/.
* src/numfmt.c: Likewise.
* src/set-fields.c: Likewise.
* src/set-fields.h: Likewise.
* tests/misc/cut-huge-range.sh: Adjust accordingly.
* tests/misc/numfmt.pl: Likewise.
* NEWS: Mention the improvement.
2018-03-28 00:00:09 -07:00
Paul Eggert 8d6acfd853 ls: -A now overrides -a
Problem reported by Karl Berry (Bug#30963).
* NEWS: Mention this.
* src/ls.c (decode_switches): Implement this.
* tests/ls/a-option.sh: New file.
* tests/local.mk (all_tests): Add it.
2018-03-27 15:05:22 -07:00
Pádraig Brady 5ed2018360 ls: increase the allowed abmon width from 5 to 12
This will impact relatively few languages,
and will make Arabic or Catalan etc.
output unambiguous abbreviated month names.

* src/ls.c (MAX_MON_WIDTH): Increase from 5 to 12.
* NEWS: Mention the bug fix.
* tests/ls/abmon-align.sh: Augment to check for ambiguous output.
Fixes https://bugs.gnu.org/30814
2018-03-16 03:16:43 -07:00
Brent Petit 0753c3f6dc stat,tail: add support for the EXFS file system
Enhanced XFS (EXFS) is a version of XFS maintained by HPE.
EXFS uses a unique magic number to allow the use of community
XFS, and EXFS filesystems at the same time.

* src/stat.c (human_fstype): Add file system ID definition,
and use "exfs" as the name.
* NEWS: Mention the Improvement.
2018-03-14 09:15:59 -07:00
Paul Eggert 2eaf3a14a5 stat: work around IBM xlC bug
Problem reported by John Wiersba (Bug#30718)
* src/stat.c (human_time): Avoid giving an integer constant
expression a name, as it runs afoul of a bug in IBM XL C/C++ for
AIX 12.01.0000.0002.
2018-03-05 15:51:06 -08:00
Pádraig Brady 0a85449dde cp: set appropriate default permissions for special files
This issue was introduced in commit v8.19-145-g24ebca6

* src/copy.c (copy_internal): When setting default permissions
to use with --no-preserve=mode, only set executable bits for
directories or sockets.
* NEWS: Mention the fix.
* tests/cp/preserve-mode.sh: Add a test case.
Fixes https://bugs.gnu.org/30534
2018-02-24 18:15:26 -08:00
Paul Eggert bedd9bb652 mv: fewer syscalls for ‘mv a b’
This builds on a previous patch for mv atomicity (Bug#29961).
It merely improves performance; it does not fix bugs.
* src/copy.h (struct cp_options): New members last_file, rename_errno.
* src/copy.c (copy_internal): Support new rename_errno member
for the copy options.  Avoid calling stat when new members
suggest it’s not needed.
(cp_options_default): Initialize new members.
* src/mv.c: Include renameat2.h.
(main): With two arguments, first call ‘renamat2 (AT_FDCWD, "a",
AT_FDCWD, "b", RENAME_NOREPLACE)’.  Use its results to skip
remaining processing if possible; for example, if it succeeds
there is no need to stat either "a" or "b".  Also, set
x.last_file when it is the last file to rename.
2018-01-10 00:51:54 -08:00
Paul Eggert 29baf25aa0 mv: improve -n atomicity
Problem reported by Kamil Dudka (Bug#29961).
* NEWS: Mention this.
* src/copy.c: Include renameat2.h.
(copy_internal): If mv, try renameat2 first thing, with
RENAME_NOREPLACE.  If this works, skip most of the remaining code.
Also, fail quickly if it fails with EEXIST, and we are using -n.
2018-01-10 00:51:54 -08:00
Paul Eggert b88f8d1101 cp: remove ASSIGN_BASENAME_STRDUPA
* src/cp.c (do_copy): Just use ASSIGN_STRDUPA, as this simplifies
the code and uses less memory.
2018-01-05 22:57:21 -08:00
Paul Eggert 7e244891b0 mv: -n overrides -u
* NEWS: Mention these fixes.
* doc/coreutils.texi (cp invocation, mv invocation):
Mention that -n is silent, and that it overrides -u.
* src/cp.c, src/mv.c (main): -n overrides -u.
2018-01-04 13:57:58 -08:00
Paul Eggert 3f9b1b86b1 tr: add -A, for compatibility with AIX tr
Problem reported by Michael (Bug#29946).
* src/tr.c (main): Add undocumented -A option.
2018-01-03 11:04:03 -08:00
Pádraig Brady ece7157933 maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2018-01-01 15:57:22 +00:00
Pádraig Brady a1300c60ef build: avoid a signed overflow warning in ptx
* src/ptx.c (fix_output_parameters): GCC 6.3.1 with
./configure --enable-single-binary would give:
  error: assuming signed overflow does not occur
  when simplifying conditional to constant [-Werror=strict-overflow]
    if (file_index > 0)
So change the type of file_index to signed (size_t).
2017-12-16 13:08:43 -08:00
Pádraig Brady e44df9d3f5 tail: fix tailing non seekable files on certain systems
* src/tail.c (tail_bytes): On systems were blksize_t is unsigned
and the same size or wider than off_t (android for example),
our initialized (off_t) -1 would be promoted to unsigned before
comparison, and thus fail to follow the appropriate path.
* tests/tail-2/tail-c.sh: Add a test case.
* NEWS: Mention the fix.
This issue was introduced in commit v8.23-47-g2662702
Reported at https://github.com/termux/termux-app/issues/233
2017-12-10 17:13:18 -08:00
Pádraig Brady 8fb32039bc build: avoid build failure without sys/mtio.h
* m4/jm-macros.m4: Check for the header.
* src/dd.c: Avoid the workaround where the header
is not available (on non glibc systems).
* src/shred.c: Likewise.
2017-12-10 17:13:18 -08:00
Pádraig Brady b697bac049 doc: reorganize ls -k and --time-style help
* src/ls.c (usage): Clarify -k only applies to -s usage
and directory 'total' lines.  Move the description
of TIME_STYLE out of the option section as it was awkward
to read and write there within 80 columns.
2017-12-10 17:12:49 -08:00
Pádraig Brady 6150b10698 doc: shred: change 'truncate' to the more descriptive 'deallocate'
* doc/coreutils.texi (shred invocation): s/truncate/deallocate/.
* src/shred.c (usage): Likewise.
Fixes https://bugs.gnu.org/29317
2017-12-09 16:54:03 -08:00
Pádraig Brady ba9c7449b9 readlink: remove superfluous comma from usage output
* src/readlink.c (usage): Remove ',' after --quiet option.
2017-11-29 13:27:51 +01:00
Bernhard Voelker 5782a362e1 all: use consistent diagnostics for unknown long options
Previously, e.g. cksum failed to output the offending unknown long
option:
  $ cksum --unknown-opt
  cksum: invalid option -- '-'
  Try 'cksum --help' for more information.
i.e., it tried to diagnose '-' as short option.
Instead, it should diagnose the unknown long option:
  $ cksum --unknown-opt
  cksum: unrecognized option '--unknown-opt'
  Try 'cksum --help' for more information.

* src/cksum.c (long_options): Add struct with null entry only.
(main): Use it in the getopt_long call.
* src/dd.c: Likewise.
* src/hostid.c: Likewise.
* src/hostname.c: Likewise.
* src/link.c: Likewise.
* src/logname.c: Likewise.
* src/nohup.c: Likewise.
* src/sleep.c: Likewise.
* src/tsort.c: Likewise.
* src/unlink.c: Likewise.
* src/uptime.c: Likewise.
* src/users.c: Likewise.
* src/whoami.c: Likewise.
* src/yes.c: Likewise.
* NEWS (Improvements): Mention the fix.
2017-11-29 13:27:02 +01:00
Bernhard Voelker a72abc9304 timeout: also support short -v option
* src/timeout.c (main): Add short option character 'v' to getopt_long
call.
* tests/misc/timeout.sh: Run the test both for the long and the short
option.
2017-11-27 22:05:56 +01:00
Pádraig Brady 5f02cddc18 dd: support iflag=direct with arbitrary sized files
* src/dd.c (iread): Handle read error with a non-aligned
file offset in the O_DIRECT case.  This is not an issue
on XFS at least, but on EXT4 the final read will return
EINVAL rather than the expected 0 to indicate EOF.
* tests/dd/direct.sh: Test the iflag=direct case also.
* NEWS: Mention the improvement.
2017-11-24 16:28:57 -08:00
Pádraig Brady 559c7a57c7 timeout: add --verbose to diagnose timeouts
This is useful as handling in shell is complicated
with the varying exit status in the --kill-after case.

* src/timeout.c (main): Handle '-v' and store
COMMAND for the diagnostic.
(cleanup): Diagnose the signal name before sending.
(usage): Document -v, --verbose.
* doc/coreutils.texi (timeout invocation): Likewise.
* tests/misc/timeout.sh: Add a test case.
* NEWS: Mention the new feature
Fixes https://bugs.gnu.org/21760
2017-11-23 17:46:55 -08:00