Commit Graph
7612 Commits
Author SHA1 Message Date
Pádraig Brady bf11fd17a9 doc: clarify the date standard output formats
* src/date.c (usage): Use FMT rather than TIMESPEC as the parameter,
since it's simpler to understand and can be better aligned.
Give an example for the --iso-8601 output format.
Adjust the example used for the 3 standard formats to be unambiguous
with respect to day/mon ordering and use of leading zeros in the time.
Reorder the options descriptions slightly, so that the
3 standards options are together.
Indent the multi-line descriptions so that grouping is obvious.
Remove a redundant description of the --rfc-3339 format,
which is obvious in the existing example.
Separate these 3 standards options to their own translatable string
to simplify translation.
Change 'date and time' to 'date/time' in the --iso-8601 description
to be consistent with --rfc-3339 and to help avoid the implication
that the time is always output or even output by default.
Fixes http://bugs.gnu.org/20203
2015-03-26 08:40:31 +00:00
Pádraig Brady 88f170e0c0 doc: clarify the uniq -D man page description
* src/uniq.c (usage): The description was very confusing in the man page
due to the stripped newlines.  Add punctuation for clarification.
2015-03-24 10:25:48 +00:00
Pádraig Brady 56ff7a6760 wc: use a more adaptive wc -l implementation
* src/wc.c (wc): Allow any block to select the count implementation,
rather than just using the first 10 lines.  This also simplifies
the code from 3 loops to 2.
2015-03-24 10:23:29 +00:00
Dan Jacobson 01fb984887 doc: clarify default order for ls --sort=size
* src/ls.c (usage): Mention that default order is largest first.
Fixes http://bugs.gnu.org/20172
2015-03-23 09:03:58 +00:00
Kristoffer Brånemyr 1025243b6a wc: speedup counting of short lines
Using a test file generated with:
  yes | head -n100M > 2x100M.txt

before> time wc -l 2x100M.txt
  real 0.842s
  user 0.810s
  sys  0.033s

after> time wc -l 2x100M.txt
  real 0.142s
  user 0.111s
  sys  0.031s

* src/wc.c (wc): Split the loop that deals with -l into 3.
The first is used at the start of the input to determine if
the average line length is < 15, and if so the second loop is
used to look for '\n' internally to wc.  For longer lines,
memchr is used as before to take advantage of system specific
optimizations which any outweigh function call overhead.
Note the first 2 loops could be combined, though in testing,
GCC 4.9.2 at least, wasn't sophisticated enough to separate
the loops based on the "check_len" invariant.
Note also __builtin_memchr() isn't significant here as
GCC currently only applies constant folding with that.
* NEWS: Mention the improvement.
2015-03-20 00:48:52 +00:00
Giuseppe Scrivano e2e11119e0 yes: improve efficiency when all args aren't buffered
* src/yes.c (main): Even when the internal buffer isn't large enough,
output what we've buffered already, and interate over the rest.
This improves the performance in the edge case where there are
many small arguments that overflow the buffer.
* tests/misc/yes.sh: Add a test case for the many small arguments case.
2015-03-10 12:30:33 +00:00
Pádraig Brady 35217221c2 yes: output data more efficiently
yes(1) may be used to generate repeating patterns of text
for test inputs etc., so adjust to be more efficient.

Profiling the case where yes(1) is outputting small items
through stdio (which was the default case), shows the overhead
of continuously processing small items in main() and in stdio:

    $ yes >/dev/null & perf top -p $!
    31.02%  yes           [.] main
    27.36%  libc-2.20.so  [.] _IO_file_xsputn@@GLIBC_2.2.5
    14.51%  libc-2.20.so  [.] fputs_unlocked
    13.50%  libc-2.20.so  [.] strlen
    10.66%  libc-2.20.so  [.] __GI___mempcpy
     1.98%  yes           [.] fputs_unlocked@plta

Sending more data per stdio call improves the situation,
but still, there is significant stdio overhead due to memory copies,
and the repeated string length checking:

    $ yes "`echo {1..1000}`" >/dev/null & perf top -p $!
    42.26%  libc-2.20.so  [.] __GI___mempcpy
    17.38%  libc-2.20.so  [.] strlen
     5.21%  [kernel]      [k] __srcu_read_lock
     4.58%  [kernel]      [k] __srcu_read_unlock
     4.27%  libc-2.20.so  [.] _IO_file_xsputn@@GLIBC_2.2.5
     2.50%  libc-2.20.so  [.] __GI___libc_write
     2.45%  [kernel]      [k] system_call
     2.40%  [kernel]      [k] system_call_after_swapgs
     2.27%  [kernel]      [k] vfs_write
     2.09%  libc-2.20.so  [.] _IO_do_write@@GLIBC_2.2.5
     2.01%  [kernel]      [k] fsnotify
     1.95%  libc-2.20.so  [.] _IO_file_write@@GLIBC_2.2.5
     1.44%  yes           [.] main

We can avoid all stdio overhead by building up the buffer
_once_ and outputting that, and the profile below shows
the bottleneck moved to the kernel:

    $ src/yes >/dev/null & perf top -p $!
    15.42%  [kernel]      [k] __srcu_read_lock
    12.98%  [kernel]      [k] __srcu_read_unlock
     9.41%  libc-2.20.so  [.] __GI___libc_write
     9.11%  [kernel]      [k] vfs_write
     8.35%  [kernel]      [k] fsnotify
     8.02%  [kernel]      [k] system_call
     5.84%  [kernel]      [k] system_call_after_swapgs
     4.54%  [kernel]      [k] __fget_light
     3.98%  [kernel]      [k] sys_write
     3.65%  [kernel]      [k] selinux_file_permission
     3.44%  [kernel]      [k] rw_verify_area
     2.94%  [kernel]      [k] __fsnotify_parent
     2.76%  [kernel]      [k] security_file_permission
     2.39%  yes           [.] main
     2.17%  [kernel]      [k] __fdget_pos
     2.13%  [kernel]      [k] sysret_check
     0.81%  [kernel]      [k] write_null
     0.36%  yes           [.] write@plt

Note this change also ensures that yes(1) will only write
complete lines for lines shorter than BUFSIZ.

* src/yes.c (main): Build up a BUFSIZ buffer of lines,
and output that, rather than having stdio process each item.
* tests/misc/yes.sh: Add a new test for various buffer sizes.
* tests/local.mk: Reference the new test.
Fixes http://bugs.gnu.org/20029
2015-03-10 00:12:30 +00:00
Pádraig Brady 5e9c277f99 tee: generalize the --write-error option to --output-error
Adjust commit v8.23-140-gfdd6ebf to add the --output-error option
instead of --write-error, and treat open() errors like write() errors.

* doc/coreutils.texi (tee invocation): s/write-error/output-error/.
* src/tee.c (main): Exit on open() error if appropriate.
* tests/misc/tee.sh: Add a case to test open() errors.
* NEWS: Adjust for the more general output error behavior.

Suggested by Bernhard Voelker.
2015-03-04 13:24:21 +00:00
Jarosław Gruca 69410690b8 maint: update stale comment about ls color sequences
* src/ls.c (color_indicator[C_END]): Comment with the correct
sequence, which was used since commit v6.10-61-g483297d
Fixes http://bugs.gnu.org/19992
2015-03-04 11:47:59 +00:00
Shane M Seymour e60547db04 tail,stat: improve support for the IBRIX file system
Note that IBRIX used to have a different magic number 0x013111A7
instead of the current 0x013111A8.  However, the former is no longer
used and the version of IBRIX it was used in is really ancient, so
it's extremely unlikely anyone is still using it.  Therefore, just
add the newer magic number.
Mark IBRIX as a 'remote' file system type as inotify support had
never been officially tested with it.

* src/stat.c (human_fstype): Add file system ID definition.
* NEWS: Mention the improvement.

Fixes http://bugs.gnu.org/19951
2015-02-27 18:54:01 +01:00
Pádraig Brady fdd6ebf0ae tee: add --write-error to control handling of closed pipes
tee is very often used with pipes and this gives better control
when writing to them.  There are 3 classes of file descriptors
that tee can write to: files(1), pipes(2), and early close pipes(3).
Handling write errors to 1 & 2 is supported at present with the caveat
that failure writing to any pipe will terminate tee immediately.
Handling write errors to type 3 is not currently supported.
To improve the supported combinations we add these options:

 --write-error=warn
   Warn if error writing any output including pipes.
   Allows continued writing to still open files/pipes.
   Exit status is failure if any output had error.
 --write-error=warn-nopipe, -p
   Warn if error writing any output except pipes.
   Allows continued writing to still open files/pipes.
   Exit status is failure if any non pipe output had error.
 --write-error=exit
   Exit if error writing any output including pipes.
 --write-error=exit-nopipe
   Exit if error writing any output except pipes.

Use the "nopipe" variants when files are of types 1 and 3, otherwise
use the standard variants with types 1 and 2.  A caveat with the above
scheme is that a combination of pipe types (2 & 3) is not supported
robustly.  I.e. if you use the "nopipe" variants when using both type
2 and 3 pipes, then any "real" errors on type 2 pipes will not be
diagnosed.
  Note also a general issue with type 3 pipes that are not on tee's
stdout, is that shell constructs don't allow to distinguish early
close from real failures.  For example `tee >(head -n1) | grep -m1 ..`
can't distinguish between an error or an early close in "head" pipe,
while the fail on the grep part of the pipe is distinguished
independently from the resulting pipe errors.  This is a general
issue with the >() construct, rather than with tee itself.

* NEWS: Mention the new feature.
* doc/coreutils.texi (tee invocation): Describe the new option.
* src/tee.c (usage): Likewise.
(main): With --write-error ignore SIGPIPE, and handle
the various exit, diagnostics combinations.
* tests/misc/tee.sh: Tess all the new options.
Fixes http://bugs.gnu.org/11540
2015-02-24 18:12:22 +00:00
Bernhard Voelker 19d34d244b tee: close "-" file
This is a cleanup to the previous commit v8.23-138-g7ceaf1d.

* src/tee.c (tee_files): Do not exempt the "-" file from being closed,
as this is no longer stdout but a normal file.
2015-02-23 11:39:08 +01:00
Bernhard Voelker 7ceaf1d975 tee: treat '-' operand as file name as mandated by POSIX
Since v5.2.1-1247-g8dafbe5, tee(1) treated '-' as stdout while POSIX
explicitly requires to treat this as a file name.  Revert this change,
as the interleaved output - due to sending another copy of input to
stdout - is not considered to be useful.  Discussed in
http://lists.gnu.org/archive/html/coreutils/2015-02/msg00085.html

* src/tee.c (tee_files): Remove the special handling for "-" operands.
(usage): Remove the corresponding sentence.
* doc/coreutils.texi (common options): Remove the "tee -" example.
(tee invocation): Document that tee(1) now treats "-" as a file name.
* tests/misc/tee.sh: Add a test case for "tee -".
While at it, re-indent the above multi-argument processing case and
extend that to 13 operands, as POSIX mandates that, too.
* tests/misc/tee-dash.sh: Remove now-obsolete test.
* tests/local.mk (all_tests): Remove the above test.
* NEWS (Changes in behavior): Mention the change.
2015-02-20 14:00:41 +01:00
Pádraig Brady 3f2f05f067 tee: exit early if no more writable outputs
* src/tee.c (main): Don't continue reading if we can't
output anywhere.
* tests/misc/tee.sh: Ensure we exit when no more outputs.
* NEWS: Mention the change in behavior.
2015-02-18 23:41:27 +00:00
Bernhard Voelker 9f5dce8070 maint: prefer STREQ_LEN and STRPREFIX over strncmp in all cases
* cfg.mk (sc_prohibit_strncmp): Improve the search pattern: use
_sc_search_regexp to find all invocations of strncmp except when
used on a macro definition line; just match the function name with
an opening parenthesis.  Before, the expression missed places where
the comparison against 0 was in a subsequent line.
* src/system.h (STRNCMP_LIT): Shorten 'literal' to 'lit' to move
the whole definition of the macro into one line - thus making
sc_prohibit_strncmp pass.
(STRPREFIX): Add space before parenthesis.
* src/du.c (main): Prefer STREQ_LEN over strncmp.
* src/pinky.c (scan_entries): Likewise.
* src/tac.c (tac_seekable): Likewise.
* src/who.c (scan_entries): Likewise.
2015-02-17 08:50:34 +01:00
Pádraig Brady 321bd11352 stty: fix setting of 'extproc' on BSD
This setting is unusual on BSD as it's read normally in the local
flags returned by tcgetattr(), but can only be set with an ioctl.
Setting with tcsetattr() is ignored.

* src/stty.c (NO_SETATTR): A new flag to indicate the setting
is read and displayed like a normal termios flag, but is set
in some other manner.
(main): Skip tcsetattr() for this setting when this flag is set.
Also fixup the exiting 'extproc' processing to handle the
'-extproc' case correctly.
(sane_mode): Skip setting '-extproc' for 'sane' to avoid the error.
This isn't ideal but matches the operation of the BSD native stty.
2015-02-17 03:03:23 +00:00
Pádraig Brady 50820f8470 maint: avoid arbitrary memory access with buggy localtime()
* src/ls.c (align_nstrftime): Be defensive and validate the tm_mon
index before using to access the abmon array.  This was _not_ seen
to be an issue any system.  See https://bugzilla.redhat.com/1190454
2015-02-11 00:35:08 +00:00
Pádraig Brady 0f7ca66bc5 maint: document a caveat in the tail inotify implementation
* src/tail.c (main): Document another caveat with the
inotify implementation wrt multiple hardlinked files.
2015-02-06 10:42:11 +00:00
Pádraig Brady 235d52c3ea tail: return inotify resources where possible
Each user has a maximum number of inotify watches,
so handle the cases where we exhaust these resources.

* src/tail.c (tail_forever_inotify): Ensure we inotify_rm_watch()
the watch for an inode, when replacing with a new watch for a name.
Return all used inotify resources when reverting to polling.
Revert to polling upon first indication of inotify resource exhaustion.
Revert to polling on any inotify resource exhaustion.
Diagnose resource exhaustion correctly in all cases.
Avoid redundant reinsertion in the hash for unchanged watches
(where only attributes of the file are changed).
* tests/tail-2/retry.sh: Avoid false failure when reverting to polling.
* tests/tail-2/inotify-rotate.sh: Likewise.
* tests/tail-2/symlink.sh: Likewise.
* tests/tail-2/inotify-rotate-resources.sh: New test to check
that we're calling inotify_rm_watch() for replaced files.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
* THANKS.in: Thanks for reporting and problem identification.
2015-02-06 10:41:06 +00:00
Yury Usishchev 1d0f1b7ce1 build: ensure make-prime-list doesn't access out of bounds memory
The -fsanitize=address run associated with v8.22-75-gf940fec
failed to check make-prime-list, as src/primes.h is not
regenerated with `make clean`.  Running with -fsanitize=address
indicates a read 1 byte beyond the allocated buffer.

  $ rm src/make-prime-list.o
  $ make AM_CFLAGS=-fsanitize=address src/make-prime-list
  $ src/make-prime-list 5000

=================================================================
==13913==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x61e00000fa43 at pc 0x4016f5 bp 0x7fff9d9840e0 sp 0x7fff9d9840d0
READ of size 1 at 0x61e00000fa43 thread T0
    #0 0x4016f4 in main src/make-prime-list.c:214
    #1 0x7f98892c5fdf in __libc_start_main (/lib64/libc.so.6+0x1ffdf)
    #2 0x401774 (src/make-prime-list+0x401774)

0x61e00000fa43 is located 0 bytes to the right of 2499-byte
region [0x61e00000f080,0x61e00000fa43) allocated by thread T0 here:
    #0 0x7f98896ba7b7 in malloc (/lib64/libasan.so.1+0x577b7)
    #1 0x400f3f in xalloc src/make-prime-list.c:163
    #2 0x400f3f in main src/make-prime-list.c:198

SUMMARY: AddressSanitizer: heap-buffer-overflow
src/make-prime-list.c:214 main
Shadow bytes around the buggy address:
  0x0c3c7fff9ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c3c7fff9f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c3c7fff9f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c3c7fff9f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c3c7fff9f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c3c7fff9f40: 00 00 00 00 00 00 00 00[03]fa fa fa fa fa fa fa
  0x0c3c7fff9f50: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c3c7fff9f60: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c3c7fff9f70: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c3c7fff9f80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c3c7fff9f90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:     fa
  ...
==13913==ABORTING

* src/make-prime-list.c (main): Bounds check the incremented index,
before using to access the buffer.
Fixes http://bugs.gnu.org/19784
2015-02-05 23:22:46 +00:00
Pádraig Brady b3ecf3b1b2 dircolors: sync with Xiph file extensions
* src/dircolors.hin: Remove deprecated anx, axa, axv.  Add opus.
Suggested by Scott Teal.
2015-02-02 17:09:13 +00:00
Bernhard Voelker a5c8cdff03 doc,maint: fix use of "i.e." in documentation and comments
To align with all other places (and correct grammar), change all
upper-case "I.E." to "I.e.".  Furthermore, ensure that "i.e." is
followed by a comma.  Finally, ensure to use a double-space before
"I.e.," at the beginning of a sentence.

The following was used to change all offending uses (apart from
old ChangeLog files):

  $ git grep -liF 'i.e.' \
      | xargs sed -i \
            -e 's/I\.E\./I.e./g' \
            -e 's/\. \(I\.e\.\)/.  \1/g' \
            -e 's/\([Ii]\.e\.\)\( \)/\1,\2/g' \
            -e 's/\([Ii]\.e\.\)$/\1,/g'

* cfg.mk (sc_prohibit_uppercase_id_est): Add new rule.
(sc_ensure_double_space_after_dot_before_id_est): Likewise.
(sc_ensure_comma_after_id_est): Likewise.
(old_NEWS_hash): Refresh hash via "make update-NEWS-hash".
* NEWS: Change use of "id est" abbreviation via the above command.
* README: Likewise.
* README-prereq: Likewise.
* doc/coreutils.texi: Likewise.
* gl/lib/rand-isaac.c: Likewise.
* gl/lib/tempname.c.diff: Likewise.
* man/stdbuf.x: Likewise.
* src/cat.c: Likewise.
* src/copy.c: Likewise.
* src/copy.h: Likewise.
* src/cp.c: Likewise.
* src/cut.c: Likewise.
* src/dd.c: Likewise.
* src/df.c: Likewise.
* src/fiemap.h: Likewise.
* src/longlong.h: Likewise.
* src/ls.c: Likewise.
* src/numfmt.c: Likewise.
* src/pr.c: Likewise.
* src/shred.c: Likewise.
* src/shuf.c: Likewise.
* src/split.c: Likewise.
* tests/Coreutils.pm: Likewise.
* tests/df/df-symlink.sh: Likewise.
* tests/df/skip-rootfs.sh: Likewise.
* tests/init.sh: Likewise.
* tests/ls/color-norm.sh: Likewise.
* tests/misc/basename.pl: Likewise.
* tests/misc/ls-misc.pl: Likewise.
* tests/misc/md5sum-bsd.sh: Likewise.
* tests/misc/shred-exact.sh: Likewise.
* tests/misc/sort.pl: Likewise.
* tests/misc/stdbuf.sh: Likewise.
* tests/misc/tac-continue.sh: Likewise.
* tests/rm/r-root.sh: Likewise.
* tests/tail-2/symlink.sh: Likewise.
2015-01-31 17:39:04 +01:00
Pádraig Brady ecc8f81048 doc: clarify the output format for the *sum utilities
* src/md5sum.c (usage): Detail the reasons for the default
double space between checksum and file name.
* doc/coreutils.texi (md5sum invocation): Likewise.
Explicitly mention the 3 formats that --check supports.

Fixes http://bugs.gnu.org/19725
2015-01-29 19:06:57 +00:00
Giuseppe Scrivano 8b2bf5295f sync: support syncing specified arguments
* m4/jm-macros.m4 (coreutils_MACROS): Check for syncfs().
* man/sync.x: Add references to syncfs, fsync and fdatasync.
* doc/coreutils.texi (sync invocation): Document the new feature.
* src/sync.c: Include "quote.h".
(AUTHORS): Include myself.
(MODE_FILE, MODE_DATA, MODE_FILE_SYSTEM, MODE_SYNC): New enum values.
(long_options): Define.
(sync_arg): New function.
(usage): Describe that arguments are now accepted.
(main): Add arguments parsing and add support for fsync(2),
fdatasync(2) and syncfs(2).
* tests/misc/sync.sh: New (and only) test for sync.
* tests/local.mk: Reference the new test.
* AUTHORS: Add myself to sync's authors.
* NEWS: Mention the new feature.
2015-01-28 10:50:45 +00:00
Pádraig Brady 564f84a91a stty: only print supported options for combined options
* src/stty.c (usage): Don't reference unsupported options,
in the combined options descriptions.
* doc/coreutils.texi (stty invocation): Adjust for the
new order of the 'sane' and 'raw' combined options.
Also add -iutf8 to the 'sane' list.
2015-01-24 01:07:00 +00:00
Pádraig Brady c6a722946d stty: document the 'status' character where supported
* src/stty.c (usage): On systems that support this setting (BSD),
display 'status' in the list of adjustable special characters.
* doc/coreutils.texi (stty invocation): Mention the option, and that
it's not currently supported on Linux.
2015-01-24 01:07:00 +00:00
Pádraig Brady 81e5b59d8c stty: document the 'discard' character setting
The equivalent of this is 'flush', but that was never documented
as an option (though was output with stty -a).  Therefore use
the more descriptive name, also generally used on BSD systems.
Note even though this setting seems ineffective on Linux, supporting
the setting is useful to allow terminal programs to receive
the default ^O character code.

* doc/coreutils.texi (stty invocation): Document the 'discard' option.
* src/stty.c (struct control_info): Add 'discard'; same as 'flush'.
(display_all): Show 'discard' rather than 'flush' char.
(display_changed): Likewise.
(usage): Document the 'discard' option.
2015-01-24 01:06:59 +00:00
Pádraig Brady ed38d9d1ce stty: add support for extproc/LINEMODE
Add support for the "extproc" option which is well described at:
http://lists.gnu.org/archive/html/bug-readline/2011-01/msg00004.html

* src/stty.c (usage): Describe the extproc option if either the
Linux EXTPROC local option is defined, or the equivalent
BSD TIOCEXT ioctl is defined.
(main): Make the separate ioctl call for extproc on BSD.
* doc/coreutils.texi (stty invocation): Describe the option,
and reference the related RFC 1116.
* NEWS: Mention the new feature.
2015-01-24 01:06:59 +00:00
Assaf Gordon 4c795d5439 split: new -t option to select record separator
* src/split.c (eolchar): A new variable to hold
the separator character (unibyte for now).
This is reference throughout rather than hardcoding '\n'.
(usage): Describe the new --separator option, and
mention records along with lines so there is no ambiguity
that all options treat lines and records equivalently.
(main): Have -t update eolchar, or default to '\n'.
* tests/split/record-sep.sh: New test case.
* tests/local.mk: Reference the new test.
* doc/coreutils.texi (split invocation): Document the new option.
Adjust --lines, --line-bytes, --number=[lr]/... to mention
they pertain to records if --separator is specified.
* NEWS: Mention the new feature.
2015-01-19 23:22:37 +00:00
Pádraig Brady 4c77bd23e8 doc: clarify that du operands are interdependent
Following on from http://bugs.gnu.org/17546
make it more obvious that du may elide specified operands
to avoid double counting in the set.

* src/du.c (usage): Specify that du operates on the set of
operands, rather than each independently.
* doc/coreutils.texi (du invocation): Likewise.  Also state
that the number of entries printed may change due to the
order specified.  Currently, deeper items specified earlier
will result in them being displayed, but don't mention that
implementation detail in the documentation.
* THANKS.in: Add reporter.
Reported by Stephen Shirley
2015-01-19 11:10:02 +00:00
Paul Eggert 63bc65fbe7 shuf: do not mishandle 'shuf -i0-0 1'
Problem reported by Daiki Ueno in: http://bugs.gnu.org/19520
* src/shuf.c (main): Avoid core dump if !input_range.
* tests/misc/shuf.sh: Test for this bug.
2015-01-05 19:40:21 -08:00
Pádraig Brady 272aa7cad6 doc: fix man page formatting for split CHUNKS options
* src/split.c (usage): Indent the info on CHUNKS so that
help2man can match it and align appropriately in its own section.
Fixes http://bugs.gnu.org/19228
2015-01-01 18:12:29 +00:00
Pádraig Brady e0afeb0099 maint: update all copyright year number ranges
Run "make update-copyright" and then...

* tests/sample-test: Adjust to use the single most recent year.
* tests/du/bind-mount-dir-cycle-v2.sh: Fix case in copyright message,
so that year is updated automatically in future.
2015-01-01 04:52:17 +00:00
Pádraig Brady 974c355c59 build: update to latest gnulib
Included in this are gnulib changes 3ea43e02 2768ceb7
which make the device IDs from /proc/self/mountinfo
available to df.  This can be leveraged by a subsequent
change to df to present a more accurate list of file systems.

* bootstrap: Merge from gnulib.
* src/ls.c (dev_ino_pop): s/obstack_blank/obstack_blank_fast/
as this API/ABI has changed, giving memory exhausted errors
if negative (large positive) numbers are passed to obstack_blank().
* tests/df/skip-duplicates.sh: Adjust as the new gnulib code
requires a non NULL mnt_opts even when mnt_type is not "none".
2014-12-30 02:21:47 +00:00
Pádraig Brady 6894816c65 diagnose too-large numbers better
Following on from commit v8.23-82-gaddae94, consistently diagnose
numbers that are too large, so as to distinguish from other errors,
and make the limits obvious.

* gl/modules/xdectoint: A new module implementing xdecto[iu]max(),
which handles the common case of parsing a bounded integer and
exiting with a diagnostic on error.
* gl/lib/xdectoimax.c: The signed variant.
* gl/lib/xdectoint.c: The parameterized implementation.
* gl/lib/xdectoint.h: The interface.
* gl/lib/xdectoumax.c: The unsigned variant.
* bootstrap.conf: Reference the new module.
* cfg.mk (exclude_file_name_regexp--sc_require_config_h_first):
Exclude the parameterized templates.
* src/csplit.c: Output EOVERFLOW or ERANGE errors if appropriate.
* src/fmt.c: Likewise.
* src/fold.c: Likewise.
* src/head.c: Likewise.
* src/ls.c: Likewise.
* src/nl.c: Likewise.
* src/nproc.c: Likewise.
* src/shred.c: Likewise.
* src/shuf.c: Likewise.
* src/stdbuf.c: Likewise.
* src/stty.c: Likewise.
* src/tail.c: Likewise.
* src/truncate.c: Likewise.
* src/split.c: Likewise.
* src/pr.c: Likewise.
* tests/pr/pr-tests.pl: Adjust to avoid matching errno diagnostic.
* tests/fmt/base.pl: Likewise.
* tests/split/l-chunk.sh: Likewise.
* tests/misc/shred-negative.sh: Likewise.
* tests/misc/tail.pl: Likewise.  Also remove the redundant
existing ERR_SUBST from test err-6.
* tests/ls/hex-option.sh: Check HEX/OCT options.
* tests/misc/shred-size.sh: Likewise.
* tests/misc/stty-row-col.sh: Likewise.
2014-12-19 11:56:20 +00:00
KO Myung-Hun 70d8d0f874 build: don't call OS/2 routines on all systems
* src/system.h: Add a missing __OS2__ ifdef guard.
Also adjust spacing around () to avoid a syntax-check failure.
2014-12-19 01:57:21 +00:00
KO Myung-Hun 92519ce058 build: expand a response file and a wildcard on OS/2
OS/2 traditional shells(cmd) do not expand a response file(@file)
or a wildcard.  Expand them in each utility itself.

* src/system.h (initialize_main): Define on OS/2. Expand a response
file and a wildcard.
2014-12-18 07:21:21 -08:00
Paul Eggert baff828afa dd: fix typo in previous change
Reported by Bernhard Voelker in:
http://lists.gnu.org/archive/html/bug-gnulib/2014-12/msg00191.html
* src/dd.c (scanargs): s/IN/OUT/.
2014-12-16 00:41:22 -08:00
Paul Eggert addae94b3b dd: diagnose too-large numbers better
Reported by Isabella Parakiss in:
http://lists.gnu.org/archive/html/bug-gnulib/2014-12/msg00184.html
* src/dd.c (parse_integer): Return strtol_error code, not bool.
All callers changed.
(scanargs): Improve quality of diagnostic when a number is too large.
2014-12-15 23:44:17 -08:00
Pádraig Brady a6d9792997 maint: avoid signed overflow warning with -O3
Prompted by the implicit -O3 added by american-fuzzy-lop,
seen with GCC 4.9.2 on x86_64.

  src/pr.c: In function 'print_files.part.5':
  src/pr.c:1781:6: error: assuming signed overflow does not occur
  when simplifying conditional to constant [-Werror=strict-overflow]
     if (cols_ready_to_print () == 0)

This happens because cols_ready_to_print() is inlined
thus reducing the comparison to the N variable in print_page().
Now this can't overflow due to the protection when parsing the
specified column, but use an unsigned type to avoid the
apparent signed overflow.

* src/pr.c (cols_ready_to_print): Increment an unsigned type to
avoid the subsequent signed overflow warning.
2014-12-02 01:20:39 +00:00
Boris Ranto dc1c0523a6 du: handle sub-bind-mount cycles gracefully
This patch fixes the handling of sub-bind-mount cycles which are
incorrectly detected as the file system errors.  If you bind mount the
directory 'a' to its subdirectory 'a/b/c' and then run 'du a/b' you
will get the circular dependency warning even though nothing is wrong
with the file system.  This happens because the first directory that is
traversed twice in this case is not a bind mount but a child of bind
mount.  The solution is to traverse all the directories in the cycle
that fts detected and check whether they are not a (bind) mount.

* src/du.c (mount_point_in_fts_cycle): New function that checks whether
any of the directories in the cycle that fts detected is a mount point.
* src/du.c (process_file): Update the function to use the new function
that looks up all the directories in the fts cycle instead of only the
last one.
* tests/du/bind-mount-dir-cycle-v2.sh: New test case that exhibits the
described behavior.
* tests/local.mk: Reference the new root test.
* NEWS: Mention the bug fix.
2014-12-02 01:01:35 +00:00
Pádraig Brady 458c5cbcff build: fix missing casts from recent change
* src/dd.c (alloc_[io]buf): I committed a stale patch that omitted
the casts needed on 32 bit.
Identified by http://hydra.nixos.org/build/17610188
2014-11-28 10:25:22 +00:00
Pádraig Brady 7d1fe8868e rm: fix prompted number of arguments to remove on some platforms
"zu" was output on solaris 8 for example rather than the number,
since coreutils-8.22.

* cfg.mk: Disallow %z, since we don't currently use the gnulib
fprintf module, so any usage with it is non portable.  Also
our usage with error() currently works only through an ancillary
dependency on the vfprintf gnulib module.
* src/rm.c (main): Use %PRIuMAX rather than %zu for portability.
* src/dd.c (alloc_[io]buf): Likewise for consistency.
* src/od.c (main): Likewise.
* src/split.c (set_suffix_length): Likewise.
* NEWS: Mention the rm bug fix.
Reported in http://bugs.gnu.org/19184
2014-11-27 14:25:00 +00:00
Paul Eggert d07d9f2434 build: port new rule for coreutils.h to old Bash
Reported by Ted Carr in: http://bugs.gnu.org/19184
* src/local.mk (src/coreutils.h):
Don't assume single_binary_progs is nonempty.
2014-11-25 21:16:48 -08:00
Tobias Stoeckmann 16e2347bd5 paste: fix possible truncated output with large files
If '\n' was present at the size_t boundary of a file,
then that and subsequent data would be discarded.

* src/paste.c (paste_parallel): Avoid the overflow issue
by changing the flag to a boolean rather than a count.
* NEWS: Mention the bug fix.
2014-11-24 09:31:49 +00:00
Pádraig Brady 2e81e62243 df: only suppress remote mounts of separate exports with --total
* src/df.c (filter_mount_list): Separate remote locations are
generally explicitly mounted, so list each even if they share
the same remote device and thus storage.  However with --total
keep the suppression to give a more accurate value for the
total storage available.
(usage): Expand on the new implications of --total and move
it in the options list according to alphabetic order.
doc/coreutils.texi (df invocation): Mention that --total impacts
on deduplication of remote file systems and also move location
according to alphabetic order.
* tests/df/skip-duplicates.sh: Add remote test cases.
* NEWS: Mention the change in behavior.

Reported in http://bugs.debian.org/737399
Reported in http://bugzilla.redhat.com/920806
Reported in http://bugzilla.opensuse.org/866010
Reported in http://bugzilla.opensuse.org/901905
2014-11-24 09:04:54 +00:00
Pádraig Brady 3ab3c7bb24 df: ensure -a shows all remote file system entries
commit v8.22-125-g9d736f8 printed placeholder "-" values
for device names that didn't match the preferred device name
for a particular mount point.  However that was seen to erroneously
suppress values for aliased host names or exports, common with
remote file systems.

* src/df.c (me_for_dev): Rename from devname_for_dev() so that
we can determine the remoteness as well as the name for the
preferred mount entry.
(get_dev): Don't output place holder values when both
current and preferred mount entries are remote.

Reported in http://bugs.debian.org/737399
2014-11-23 19:46:38 +00:00
Pádraig Brady a78819af28 tests: add a case verifying mv on case insensitive file systems
* NEWS: Update the recent entry to also mention the avoidance
of incorrectly unlinking a multi-hardlinked "source" file when
presented with source and dest that only differ in case.
* src/copy.c (same_file_ok): Mention the case issue with same_name().
* tests/mv/hardlink-case.sh: Test the issue on HFS+.
* tests/local.mk: Reference the new test case.
* tests/mv/vfat: Remove an old related but unused test case.
2014-11-22 03:46:34 +00:00
Pádraig Brady 57c8196ef6 doc: mention how to avoid newlines impacting ls -1
* src/ls.c (usage): Mention the -b and -q options
in the -1 description.
* doc/coreutils.texi (ls invocation): Likewise.
2014-11-21 16:25:42 +00:00
Boris Ranto 222d7ac0c4 mv: fail when moving a file to a hardlink
We may run into a race condition if we treat hard links to the same file
as distinct files.  If we do 'mv a b' and 'mv b a' in parallel, both a
and b can disappear from the file system.  The reason is that in this
case the unlink on src is called and the system calls can end up being
run in the order where unlink(a) and unlink(b) are the last two system
calls.  Therefore exit with an error code so that we avoid the potential
data loss.

* src/copy.c (same_file_ok): Don't set unlink_src that was used by mv,
and return false for two hardlinks to a file in move_mode.
*src/copy.c (copy_internal): No longer honor the unlink_src option,
used only by mv.
NEWS: Mention the change in behavior.
* tests/cp/same-file.sh: Augment to cover the `cp -a hlsl1 sl1` case.
* tests/mv/hard-verbose.sh: Remove no longer needed test.
* tests/local.mk: Remove the reference to hard-verbose.sh.
* tests/mv/hard-4.sh: Adjust so we fail in this case.
* tests/mv/i-4.sh: Likewise.
* tests/mv/symlink-onto-hardlink-to-self.sh: Likewise.
2014-11-21 02:48:45 +00:00