Commit Graph
991 Commits
Author SHA1 Message Date
Assaf Gordon ba24c42e13 numfmt: use new set-fields module to parse --field
numfmt --field=LIST can accept the same options as cut.

* bootstrap.conf: remove xlist, linked-list
* src/local.mk: link numfmt with set-fields
* src/numfmt.c: use set-fields.c instead of custom field parsing code.
  (include_field): adapt to new code.
* tests/misc/numfmt.pl: add new tests, adapt current tests to new
  error message wording from set-fields.c
2015-09-12 02:27:32 +00:00
Assaf Gordon bd1bc2b3fe cut: refactor into set-fields module
Extract the functionality of parsing --field=LIST into a separate
module, to be used by other programs.

* src/cut.c: move field parsing code from here ...
* src/set-fields.{c,h}: ... to here.
  (set_fields): generalize by supporting multiple parsing/reporting
  options.
  (struct range_pair): rename to field_range_pair.
* src/local.mk: link cut with set-field.
* po/POTFILES.in: add set-field.c
* tests/misc/cut.pl: update wording of error messages
2015-09-12 02:27:32 +00:00
Pádraig Brady 26c11773a4 tests: test numfmt stdin behavior
* tests/misc/tty-eof.pl: Add numfmt to the list of programs
that accept input on stdin.
2015-09-03 01:11:39 +01:00
Pádraig Brady df3b9120b4 base64: no longer support hex or oct --wrap params
* src/base64.c (main): Support decimal numbers with leading zeros,
by disabling the auto detection of octal and hex.  It's not
envisaged that base conversion is needed for --wrap parameters,
and in the edge case it is, $((0x0)) shell constructs can be used.
* tests/misc/base64.pl: Adjust accordingly.
* NEWS: Mention the change in behavior.
2015-09-03 00:34:12 +01:00
Pádraig Brady a8cc9ce3f2 base64: use stricter validation on wrap column
* src/base64.c (main): Use the higher level xnumtoumax()
rather than xstrtoumax(), which is simpler and improves
validation of input.  Also pass the _empty_ rather than NULL
string as the suffixes parameter so that invalid trailing
characters are not allowed.  For example -w08 is now
flagged as an error, rather than being interpreted as 0.
A subsequent commit will further improve verification
of numbers with leading zeros by dropping backwards compatibility
wrt auto parsing oct and hex numbers.
* tests/misc/base64.pl: Add tests for invalid wrap values.
2015-09-03 00:34:12 +01:00
Pádraig Brady 89c517d9e2 base32: A new program similar to base64
Suggested in https://bugzilla.redhat.com/1250113

* AUTHORS: Add base32.
* THANKS.in: Add suggester.
* README: Reference the new program.
* NEWS: Mention the new program.
* src/.gitignore: Ignore the new binary.
* bootstrap.conf: Reference the gnulib base32 module.
* build-aux/gen-lists-of-programs.sh: Add base32.
* man/base32.x: A new template.
* man/.gitignore: Ignore the new man page.
* man/local.mk: Reference the new man page.
* doc/coreutils.texi (base32 invocation): Document the new command.
* src/local.mk: Adjust to build base32 based on base64.c.
* src/base64.c: Parameterize to use the correct headers,
functions and buffer sizes, depending on which binary
is being built.
* tests/misc/base64.pl: Adjust to test both base32 and base64.
* tests/misc/tty-eof.pl: Add base32 as a program that
accepts input on stdin without any options specified.
* scripts/git-hooks/commit-msg: Add base32 to the template.
2015-09-03 00:33:19 +01:00
Pádraig Brady 5e5d454037 shred: fix pattern selection for certain iteration counts
This was detected in about 25% of runs with gcc -fsanitize=address

  ERROR: AddressSanitizer: global-buffer-overflow on address ...
  READ of size 4 at 0x000000416628 thread T0
    #0 0x40479f in genpattern src/shred.c:782
    #1 0x4050d9 in do_wipefd src/shred.c:921
    #2 0x406203 in wipefile src/shred.c:1175
    #3 0x406b84 in main src/shred.c:1316
    #4 0x7f3454a1ef9f in __libc_start_main (/lib64/libc.so.6+0x1ff9f)
    #5 0x4025d8 (/tmp/coreutils-8.23/src/shred+0x4025d8)
  0x000000416628 is located 56 bytes to the left of
  global variable '*.LC49' from 'src/shred.c' (0x416660) of size 17
  0x000000416628 is located 12 bytes to the right of
  global variable 'patterns' from 'src/shred.c' (0x416540) of size 220
  SUMMARY: AddressSanitizer: global-buffer-overflow src/shred.c:782

* src/shred.c (gen_patterns): Restrict pattern selection
to the K available, which regressed due to v5.92-1462-g65533e1.
* tests/misc/shred-passes.sh: Add a deterministic test case.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/20998
2015-07-07 03:26:58 +01:00
Pádraig Brady 1d4289b3c1 tests: avoid side effects of $SHELL environment variable
Since non interactive shells don't generally set $SHELL,
its value is propagated through the tests and may cause issues;
for example if $SHELL implicitly adjusts $PATH when run.
Instead we set $SHELL to that determined by the posix-shell module,
and use that consistently for all test sub scripts,
including those created thorugh the `split --filter` command.

* tests/local.mk: Explicitly set $SHELL to $(PREFERABLY_POSIX_SHELL)
which defaults to $CONFIG_SHELL and thus usually /bin/sh.
* tests/envvar-check: Remove bash environment variables with
side effects, in case /bin/bash was selected for $SHELL.
* tests/misc/help-version.sh: Remove redundant initialization of $SHELL.
* tests/install/strip-program.sh: Use $SHELL for sub script.
* tests/misc/sort-compress-hang.sh: Likewise.
* tests/misc/sort-compress-proc.sh: Likewise.
* tests/misc/sort-compress.sh: Likewise.
* tests/misc/timeout-group.sh: Likewise.
* tests/rm/fail-eperm.xpl: Remove redundant elision of bash env vars.
* tests/misc/pwd-long.sh: Likewise.
2015-07-03 13:02:21 +01:00
Pádraig Brady 970b891702 tests: avoid false failures on OpenBSD 5.7
* tests/du/threshold.sh: Homogenize getopt error messages.
* tests/misc/numfmt.pl: Likewise.
* tests/mv/i-3.sh: Skip on *BSD not just FreeBSD.
2015-07-02 19:15:08 +01:00
Pádraig Brady 491f54fb5a tests: avoid false failure on FreeBSD systems
* tests/misc/stty.sh: FreeBSD returns ENOTTY for
the TIOCEXT ioctl, so just avoid this option for now.
2015-07-01 12:27:29 +01:00
Pádraig Brady cacd9bf9c2 tests: avoid false failure when running as root
* tests/misc/sync.sh: Ensure dir is unreadable before
including the permission check.
2015-06-25 14:10:47 +01:00
Pádraig Brady 4d2d6c5b7c factor: avoid interspersed lines for parallel runs
* src/factor.c (n_out): A new global variable to track
how much data has been written to stdout.
(print_factors_single): Use n_out to determine whether
to flush the current (and previous) lines.
* tests/misc/factor-parallel.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
2015-06-24 17:04:01 +01:00
Pádraig Brady 32c97093db seq: handle exponents more consistently
src/seq.c (scan_arg): Set precision and width _after_ exponentiation.
For example, this will make '1.1e1 12' and '11 1.2e1' equivalent.
One can still set the precision by specifying extra precision on
the start value, or more naturally with a precision on a step value.
* tests/misc/seq-precision.sh: Add new cases.
2015-06-24 17:04:01 +01:00
Pádraig Brady 60c8e31d92 seq: use consistent output format with hex integers
* src/seq.c (scan_arg): Set precision to 0 for hex constants
(while avoiding hex floats).  This will use then use the
fast path for these arguments.  Note we also set the precision
of inf to 0 here, which ensures we use consistent precision
on output where possible.
* tests/misc/seq-precision.sh: Add corresponding test cases.
2015-06-24 17:03:59 +01:00
Pádraig Brady fc6825b231 seq: support inf last item more generally/efficiently
* src/seq.c (main): Call seq_fast for infinite last value.
This implicitly avoids format conversion on the
999999 -> 1000000 transition.
* src/seq.c (seq_fast): Generalize the buffer handling,
and adjust to handle the "inf" last value specifics.
* tests/misc/seq-precision.sh: A new test.
* tests/local.mk: Reference the new test.
2015-06-24 17:03:35 +01:00
Pádraig Brady 4f824b3f36 numfmt: don't hardcode floating point limits
* src/numfmt.c (MAX_UNSCALED_DIGITS): Set this to LDBL_DIG
rather than hardcoding at 18 for better portability.
* tests/misc/numfmt.pl: Restrict limit tests to supported platforms.
2015-06-22 17:22:04 +01:00
Pádraig Brady bc9d16b79f numfmt: handle leading zeros correctly
* src/numfmt.c (simple_strtod_int): Don't count leading zeros
as significant digits.  Also have leading zeros as optional
for floating point numbers.
* tests/misc/numfmt.pl: Add test cases.
* NEWS: Mention the fix.
2015-06-22 02:21:02 +01:00
Pádraig Brady a3c3e1e9e6 numfmt: avoid integer overflow when rounding
Due to existing limits this is usually triggered
with an increased precision.  We also add further
restrictions to the output of increased precision numbers.

* src/numfmt.c (simple_round): Avoid intmax_t overflow.
(simple_strtod_int): Count digits consistently
for precision loss and overflow detection.
(prepare_padded_number): Include the precision
when excluding numbers to output, since the precision
determines the ultimate values used in the rounding scheme
in double_to_human().
* tests/misc/numfmt.pl: Add previously failing test cases.
* NEWS: Mention the fix.
2015-06-22 02:02:05 +01:00
Pádraig Brady 0a279f6190 numfmt: support user specified output precision
* src/numfmt.c (usage): Update the --format description
to indicate precision is allowed.
(parse_format_string): Parse a precision specification
like the standard printf does.
(double_to_human): Honor the precision in --to mode.
* tests/misc/numfmt.pl: New tests.
* doc/coreutils.texi (numfmt invocation): Mention the new feature.
* NEWS: Likewise.
2015-06-21 04:16:29 +01:00
Dylan Cali 71063bc858 numfmt: implement support for field ranges
* src/numfmt.c: Replace field handling code with logic that understands
field range specifiers.  Instead of processing a single field and
printing line prefix/suffix around it, process each field in the line
checking whether it has been included for conversion.  If so convert and
print, otherwise just print the unaltered field.
(extract_fields): Removed.
(skip_fields): Removed.
(process_line): Gutted and heavily reworked.
(process_suffixed_number): FIELD is now passed as an arg instead of
using a global.
(parse_field_arg): New function that parses field range specifiers.
(next_field): New function that returns pointers to the next field in
a line.
(process_field): New function that wraps the field conversion logic
(include_field): New function that checks whether a field should be
converted
(compare_field): New function used for field value comparisons in a
gl_list.
(free_field): New function used for freeing field values in a gl_list.
Global variable FIELD removed.
New global variable all_fields indicates whether all fields should be
processed.
New global variable all_fields_after stores the first field of a N-
style range.
New global variable all_fields_before stores the last field of a -M
style range.
New global variable field_list stores explicitly specified fields to
process (N N,M or N-M style specifiers).
(usage): Document newly supported field range specifiers.
* bootstrap.conf: Include xlist and linked-list modules.  numfmt now
uses the gl_linked_list implementation to store the field ranges.
* tests/misc/numfmt.pl: Add tests for 'cut style' field ranges.
Adjust existing tests as partial output can occur before an error
Remove test for the 'invalid' field -5.. this is now a valid range.
* gnulib: update to avoid compiler warnings in linked-list.
* NEWS: Mention the new feature.
2015-06-19 19:59:21 +01:00
Pádraig Brady 6fadd46acd numfmt: handle suffixes consistently with --{from,to}-unit
* src/numfmt.c (unit_to_umax): Support SI (power of 10) suffixes
with the --from-unit and --to-unit options.  Treat suffixes like
is done with --from=auto, which for example will change the meaning
of --to-unit=G to that of --to-unit=Gi.  The suffix support was
previously undocumented and it's better to avoid the traditional
coreutils suffix handling in numfmt by default.
* doc/coreutils.texi: Document the new behavior.  Also fix a typo
mentioning {from,to}=units=.
* tests/misc/numfmt.pl: Adjust accordingly.
* NEWS: Mention the change in behavior.
2015-06-19 14:50:34 +01:00
Pádraig Brady 3150ec900c tests: fix false failure in recent test adjustment
* configure.ac: Comment on why we link rather than run the test,
and remove the moot __ELF__ check since we never ran it anyway,
and the new CFLAGS and LDFLAGS are a more direct test of support.
* tests/misc/wc-parallel.sh: Fix a syntax error in the previous change.
* tests/misc/md5sum-parallel.sh: Use better error checking, consistent
with that used in wc-parallel.sh.
2015-06-06 14:27:54 +01:00
Paul Eggert 079652ee6b build: port to AIX
Problems reported by Michael Felt, and and part of this fix taken
from code suggested by Pádraig Brady in:
http://bugs.gnu.org/20733#112
* configure.ac (stdbuf_supported): Check for warnings, and
for -fPIC and -shared, for AIX.
* src/stat.c (STRUCT_STATVFS): Define to struct statvfs64 if
STATFS is statvfs64.
* src/sync.c (sync_arg) [_AIX]: Open in write mode,
since AIX fsync doesn't work on read-only file descriptors.
* tests/misc/wc-parallel.sh: Skip test if xargs -P does not work.
2015-06-06 00:46:13 -07:00
Pádraig Brady b09f34b80f tests: fix false failure on loaded systems
* tests/misc/uniq-perf.sh: Use our standard 10s timeout,
which is sufficient to trigger the failure and also
avoids a false failure on slow/loaded systems.
Noticed at http://hydra.nixos.org/build/22766288
2015-06-01 15:38:23 +01:00
Pádraig Brady 9069f4ec85 tests: cleanup background processes upon interruption
Reap background processes so that:
- Stray processes aren't left on the system
- Files aren't held open causing deletion issues on NFS
- Partitions used to run the tests from can be unmounted

* tests/tail-2/F-vs-missing.sh: Add the `kill && wait` of the
background $pid(s) to cleanup_().
* tests/tail-2/F-vs-rename.sh: Likewise.
* tests/tail-2/f-vs-rename.sh: Likewise.
* tests/tail-2/append-only.sh: Likewise.
* tests/tail-2/assert-2.sh: Likewise.
* tests/tail-2/assert.sh: Likewise.
* tests/tail-2/flush-initial.sh: Likewise.
* tests/tail-2/inotify-hash-abuse.sh: Likewise.
* tests/tail-2/inotify-hash-abuse2.sh: Likewise.
* tests/tail-2/inotify-race.sh: Likewise.
* tests/tail-2/inotify-rotate-resources.sh: Likewise.
* tests/tail-2/inotify-rotate.sh: Likewise.
* tests/tail-2/pid.sh: Likewise.
* tests/tail-2/pipe-f2.sh: Likewise.
* tests/tail-2/retry.sh: Likewise.
* tests/tail-2/symlink.sh: Likewise.
* tests/tail-2/tail-n0f.sh: Likewise.
* tests/tail-2/wait.sh: Likewise.
* tests/cp/existing-perm-race.sh: Likewise.
* tests/cp/file-perm-race.sh: Likewise.
* tests/cp/parent-perm-race.sh: Likewise.
* tests/cp/sparse-to-pipe.sh: Likewise.
* tests/dd/stats.sh: Likewise.
* tests/du/move-dir-while-traversing.sh: Likewise.
* tests/misc/cat-buf.sh: Likewise.
* tests/misc/help-version.sh: Likewise.
* tests/misc/printf-surprise.sh: Likewise.
* tests/misc/sort-compress-proc.sh: Likewise.
* tests/misc/sort-spinlock-abuse.sh: Likewise.
* tests/misc/stdbuf.sh: Likewise.
* tests/misc/tac-continue.sh: Likewise.
* tests/misc/timeout-group.sh: Likewise.
* tests/mv/i-3.sh: Likewise.
* tests/rm/dangling-symlink.sh: Likewise.
* tests/rm/isatty.sh: Likewise.
* cfg.mk (sc_prohibit_test_background_without_cleanup_):
A new syntax-check to ensure cleanup_() is defined
when background tasks are created in a test.
2015-05-11 23:02:16 +01:00
Giuseppe Scrivano e2e11119e0 yes: improve efficiency when all args aren't buffered
* src/yes.c (main): Even when the internal buffer isn't large enough,
output what we've buffered already, and interate over the rest.
This improves the performance in the edge case where there are
many small arguments that overflow the buffer.
* tests/misc/yes.sh: Add a test case for the many small arguments case.
2015-03-10 12:30:33 +00:00
Pádraig Brady 35217221c2 yes: output data more efficiently
yes(1) may be used to generate repeating patterns of text
for test inputs etc., so adjust to be more efficient.

Profiling the case where yes(1) is outputting small items
through stdio (which was the default case), shows the overhead
of continuously processing small items in main() and in stdio:

    $ yes >/dev/null & perf top -p $!
    31.02%  yes           [.] main
    27.36%  libc-2.20.so  [.] _IO_file_xsputn@@GLIBC_2.2.5
    14.51%  libc-2.20.so  [.] fputs_unlocked
    13.50%  libc-2.20.so  [.] strlen
    10.66%  libc-2.20.so  [.] __GI___mempcpy
     1.98%  yes           [.] fputs_unlocked@plta

Sending more data per stdio call improves the situation,
but still, there is significant stdio overhead due to memory copies,
and the repeated string length checking:

    $ yes "`echo {1..1000}`" >/dev/null & perf top -p $!
    42.26%  libc-2.20.so  [.] __GI___mempcpy
    17.38%  libc-2.20.so  [.] strlen
     5.21%  [kernel]      [k] __srcu_read_lock
     4.58%  [kernel]      [k] __srcu_read_unlock
     4.27%  libc-2.20.so  [.] _IO_file_xsputn@@GLIBC_2.2.5
     2.50%  libc-2.20.so  [.] __GI___libc_write
     2.45%  [kernel]      [k] system_call
     2.40%  [kernel]      [k] system_call_after_swapgs
     2.27%  [kernel]      [k] vfs_write
     2.09%  libc-2.20.so  [.] _IO_do_write@@GLIBC_2.2.5
     2.01%  [kernel]      [k] fsnotify
     1.95%  libc-2.20.so  [.] _IO_file_write@@GLIBC_2.2.5
     1.44%  yes           [.] main

We can avoid all stdio overhead by building up the buffer
_once_ and outputting that, and the profile below shows
the bottleneck moved to the kernel:

    $ src/yes >/dev/null & perf top -p $!
    15.42%  [kernel]      [k] __srcu_read_lock
    12.98%  [kernel]      [k] __srcu_read_unlock
     9.41%  libc-2.20.so  [.] __GI___libc_write
     9.11%  [kernel]      [k] vfs_write
     8.35%  [kernel]      [k] fsnotify
     8.02%  [kernel]      [k] system_call
     5.84%  [kernel]      [k] system_call_after_swapgs
     4.54%  [kernel]      [k] __fget_light
     3.98%  [kernel]      [k] sys_write
     3.65%  [kernel]      [k] selinux_file_permission
     3.44%  [kernel]      [k] rw_verify_area
     2.94%  [kernel]      [k] __fsnotify_parent
     2.76%  [kernel]      [k] security_file_permission
     2.39%  yes           [.] main
     2.17%  [kernel]      [k] __fdget_pos
     2.13%  [kernel]      [k] sysret_check
     0.81%  [kernel]      [k] write_null
     0.36%  yes           [.] write@plt

Note this change also ensures that yes(1) will only write
complete lines for lines shorter than BUFSIZ.

* src/yes.c (main): Build up a BUFSIZ buffer of lines,
and output that, rather than having stdio process each item.
* tests/misc/yes.sh: Add a new test for various buffer sizes.
* tests/local.mk: Reference the new test.
Fixes http://bugs.gnu.org/20029
2015-03-10 00:12:30 +00:00
Pádraig Brady 5e9c277f99 tee: generalize the --write-error option to --output-error
Adjust commit v8.23-140-gfdd6ebf to add the --output-error option
instead of --write-error, and treat open() errors like write() errors.

* doc/coreutils.texi (tee invocation): s/write-error/output-error/.
* src/tee.c (main): Exit on open() error if appropriate.
* tests/misc/tee.sh: Add a case to test open() errors.
* NEWS: Adjust for the more general output error behavior.

Suggested by Bernhard Voelker.
2015-03-04 13:24:21 +00:00
Pádraig Brady fdd6ebf0ae tee: add --write-error to control handling of closed pipes
tee is very often used with pipes and this gives better control
when writing to them.  There are 3 classes of file descriptors
that tee can write to: files(1), pipes(2), and early close pipes(3).
Handling write errors to 1 & 2 is supported at present with the caveat
that failure writing to any pipe will terminate tee immediately.
Handling write errors to type 3 is not currently supported.
To improve the supported combinations we add these options:

 --write-error=warn
   Warn if error writing any output including pipes.
   Allows continued writing to still open files/pipes.
   Exit status is failure if any output had error.
 --write-error=warn-nopipe, -p
   Warn if error writing any output except pipes.
   Allows continued writing to still open files/pipes.
   Exit status is failure if any non pipe output had error.
 --write-error=exit
   Exit if error writing any output including pipes.
 --write-error=exit-nopipe
   Exit if error writing any output except pipes.

Use the "nopipe" variants when files are of types 1 and 3, otherwise
use the standard variants with types 1 and 2.  A caveat with the above
scheme is that a combination of pipe types (2 & 3) is not supported
robustly.  I.e. if you use the "nopipe" variants when using both type
2 and 3 pipes, then any "real" errors on type 2 pipes will not be
diagnosed.
  Note also a general issue with type 3 pipes that are not on tee's
stdout, is that shell constructs don't allow to distinguish early
close from real failures.  For example `tee >(head -n1) | grep -m1 ..`
can't distinguish between an error or an early close in "head" pipe,
while the fail on the grep part of the pipe is distinguished
independently from the resulting pipe errors.  This is a general
issue with the >() construct, rather than with tee itself.

* NEWS: Mention the new feature.
* doc/coreutils.texi (tee invocation): Describe the new option.
* src/tee.c (usage): Likewise.
(main): With --write-error ignore SIGPIPE, and handle
the various exit, diagnostics combinations.
* tests/misc/tee.sh: Tess all the new options.
Fixes http://bugs.gnu.org/11540
2015-02-24 18:12:22 +00:00
Bernhard Voelker 7ceaf1d975 tee: treat '-' operand as file name as mandated by POSIX
Since v5.2.1-1247-g8dafbe5, tee(1) treated '-' as stdout while POSIX
explicitly requires to treat this as a file name.  Revert this change,
as the interleaved output - due to sending another copy of input to
stdout - is not considered to be useful.  Discussed in
http://lists.gnu.org/archive/html/coreutils/2015-02/msg00085.html

* src/tee.c (tee_files): Remove the special handling for "-" operands.
(usage): Remove the corresponding sentence.
* doc/coreutils.texi (common options): Remove the "tee -" example.
(tee invocation): Document that tee(1) now treats "-" as a file name.
* tests/misc/tee.sh: Add a test case for "tee -".
While at it, re-indent the above multi-argument processing case and
extend that to 13 operands, as POSIX mandates that, too.
* tests/misc/tee-dash.sh: Remove now-obsolete test.
* tests/local.mk (all_tests): Remove the above test.
* NEWS (Changes in behavior): Mention the change.
2015-02-20 14:00:41 +01:00
Pádraig Brady 3f2f05f067 tee: exit early if no more writable outputs
* src/tee.c (main): Don't continue reading if we can't
output anywhere.
* tests/misc/tee.sh: Ensure we exit when no more outputs.
* NEWS: Mention the change in behavior.
2015-02-18 23:41:27 +00:00
Bernhard Voelker bfb4f5c8b7 tests: avoid a FP on aarch64 when comparing env/printenv output
The following test fails on aarch64 on openSUSE's OpenBuildService
due to glibc's execvp reversing the pointers of 'environ', i.e.,
the output of "env|tac" equals "env env" on that platform.

* tests/misc/printenv.sh: Use 'env env' to work around the behavior
on that platform.
While at it, fix the grep pattern which suppressed all environment
variables starting with an underscore "_" instead of "$_" (and
"$LD_PRELOAD") only.
2015-02-03 02:19:54 +01:00
Bernhard Voelker a5c8cdff03 doc,maint: fix use of "i.e." in documentation and comments
To align with all other places (and correct grammar), change all
upper-case "I.E." to "I.e.".  Furthermore, ensure that "i.e." is
followed by a comma.  Finally, ensure to use a double-space before
"I.e.," at the beginning of a sentence.

The following was used to change all offending uses (apart from
old ChangeLog files):

  $ git grep -liF 'i.e.' \
      | xargs sed -i \
            -e 's/I\.E\./I.e./g' \
            -e 's/\. \(I\.e\.\)/.  \1/g' \
            -e 's/\([Ii]\.e\.\)\( \)/\1,\2/g' \
            -e 's/\([Ii]\.e\.\)$/\1,/g'

* cfg.mk (sc_prohibit_uppercase_id_est): Add new rule.
(sc_ensure_double_space_after_dot_before_id_est): Likewise.
(sc_ensure_comma_after_id_est): Likewise.
(old_NEWS_hash): Refresh hash via "make update-NEWS-hash".
* NEWS: Change use of "id est" abbreviation via the above command.
* README: Likewise.
* README-prereq: Likewise.
* doc/coreutils.texi: Likewise.
* gl/lib/rand-isaac.c: Likewise.
* gl/lib/tempname.c.diff: Likewise.
* man/stdbuf.x: Likewise.
* src/cat.c: Likewise.
* src/copy.c: Likewise.
* src/copy.h: Likewise.
* src/cp.c: Likewise.
* src/cut.c: Likewise.
* src/dd.c: Likewise.
* src/df.c: Likewise.
* src/fiemap.h: Likewise.
* src/longlong.h: Likewise.
* src/ls.c: Likewise.
* src/numfmt.c: Likewise.
* src/pr.c: Likewise.
* src/shred.c: Likewise.
* src/shuf.c: Likewise.
* src/split.c: Likewise.
* tests/Coreutils.pm: Likewise.
* tests/df/df-symlink.sh: Likewise.
* tests/df/skip-rootfs.sh: Likewise.
* tests/init.sh: Likewise.
* tests/ls/color-norm.sh: Likewise.
* tests/misc/basename.pl: Likewise.
* tests/misc/ls-misc.pl: Likewise.
* tests/misc/md5sum-bsd.sh: Likewise.
* tests/misc/shred-exact.sh: Likewise.
* tests/misc/sort.pl: Likewise.
* tests/misc/stdbuf.sh: Likewise.
* tests/misc/tac-continue.sh: Likewise.
* tests/rm/r-root.sh: Likewise.
* tests/tail-2/symlink.sh: Likewise.
2015-01-31 17:39:04 +01:00
Giuseppe Scrivano 8b2bf5295f sync: support syncing specified arguments
* m4/jm-macros.m4 (coreutils_MACROS): Check for syncfs().
* man/sync.x: Add references to syncfs, fsync and fdatasync.
* doc/coreutils.texi (sync invocation): Document the new feature.
* src/sync.c: Include "quote.h".
(AUTHORS): Include myself.
(MODE_FILE, MODE_DATA, MODE_FILE_SYSTEM, MODE_SYNC): New enum values.
(long_options): Define.
(sync_arg): New function.
(usage): Describe that arguments are now accepted.
(main): Add arguments parsing and add support for fsync(2),
fdatasync(2) and syncfs(2).
* tests/misc/sync.sh: New (and only) test for sync.
* tests/local.mk: Reference the new test.
* AUTHORS: Add myself to sync's authors.
* NEWS: Mention the new feature.
2015-01-28 10:50:45 +00:00
Bernhard Voelker e84046dbe6 tests: use compare-vs-/dev/null instead of 'test -s'
When some program produces unexpected output, that use of
compare-vs-/dev/null will ensure that the surprising output is
printed in the test's output.  With "test -s err" only, one
would have to instrument and rerun in order to see the offending
output.

* cfg.mk (sc_prohibit_and_fail_1): Exempt 'compare' from this check.
* tests/dd/misc.sh: Change "tests -s ... || fail=1" to
"compare /dev/null ... && fail=1".
* tests/misc/nice.sh: Likewise.
* tests/rm/read-only.sh: Likewise.
* tests/tail-2/inotify-race.sh: Likewise.
* tests/touch/no-dereference.sh: Likewise.

Suggested by Jim Meyering in
http://lists.gnu.org/archive/html/coreutils/2015-01/msg00042.html
2015-01-14 17:38:12 +01:00
Pádraig Brady 58cff8a009 tests: add extra protection against unexpected exits
Many tests use `program ... && fail=1` to ensure expected
error situations are indicated.  However that would mask
an unexpected exit (like a crash).  Therefore explicitly
check the expected exit code.
Note where error messages are also verified, the extra
protection is not added.

* tests/init.sh (returns_): A new helper function to
check the return code of a command, and used
throughout the tests.
* cfg.mk (sc_prohibit_and_fail_1): Add a syntax check
to avoid new instances of this issue.
2015-01-14 12:20:32 +00:00
Pádraig Brady eef0cd62dd maint: clean up some test issues identified with shellcheck
* tests/cp/cp-a-selinux.sh: Comment why unused variables are assigned.
Fix misspellings noticed while adjusting.
* tests/cp/fiemap-perf.sh: Fix quoting.
* tests/misc/shuf.sh: Avoid useless use of cat.
* tests/misc/printf-surprise.sh: Likewise.
2015-01-10 21:18:33 +00:00
Daiki Ueno fb13cdc727 maint: adjustments related to previous shuf crash fix
* tests/misc/shuf.sh: Improve the test so it detects
crashes in more cases.
* NEWS: Mention the previous fix.
2015-01-08 14:16:01 +00:00
Paul Eggert 63bc65fbe7 shuf: do not mishandle 'shuf -i0-0 1'
Problem reported by Daiki Ueno in: http://bugs.gnu.org/19520
* src/shuf.c (main): Avoid core dump if !input_range.
* tests/misc/shuf.sh: Test for this bug.
2015-01-05 19:40:21 -08:00
Pádraig Brady e0afeb0099 maint: update all copyright year number ranges
Run "make update-copyright" and then...

* tests/sample-test: Adjust to use the single most recent year.
* tests/du/bind-mount-dir-cycle-v2.sh: Fix case in copyright message,
so that year is updated automatically in future.
2015-01-01 04:52:17 +00:00
Pádraig Brady 3b7c9d1a31 tests: fix error message check on some systems
http://hydra.nixos.org/build/18129583 identified (on OS X)
an incorrect test assumption in the previous commit.

* gl/lib/xdectoint.c (__xnumtoint): Suppress the EINVAL
error message as it's redundant in this context.
* tests/misc/tail.pl: Suppress _optionally_ appended
strerror messages.
* tests/fmt/base.pl: Likewise.
* tests/pr/pr-tests.pl: Likewise.
* tests/split/l-chunk.sh: Likewise.
2014-12-25 02:30:05 +00:00
Pádraig Brady 6894816c65 diagnose too-large numbers better
Following on from commit v8.23-82-gaddae94, consistently diagnose
numbers that are too large, so as to distinguish from other errors,
and make the limits obvious.

* gl/modules/xdectoint: A new module implementing xdecto[iu]max(),
which handles the common case of parsing a bounded integer and
exiting with a diagnostic on error.
* gl/lib/xdectoimax.c: The signed variant.
* gl/lib/xdectoint.c: The parameterized implementation.
* gl/lib/xdectoint.h: The interface.
* gl/lib/xdectoumax.c: The unsigned variant.
* bootstrap.conf: Reference the new module.
* cfg.mk (exclude_file_name_regexp--sc_require_config_h_first):
Exclude the parameterized templates.
* src/csplit.c: Output EOVERFLOW or ERANGE errors if appropriate.
* src/fmt.c: Likewise.
* src/fold.c: Likewise.
* src/head.c: Likewise.
* src/ls.c: Likewise.
* src/nl.c: Likewise.
* src/nproc.c: Likewise.
* src/shred.c: Likewise.
* src/shuf.c: Likewise.
* src/stdbuf.c: Likewise.
* src/stty.c: Likewise.
* src/tail.c: Likewise.
* src/truncate.c: Likewise.
* src/split.c: Likewise.
* src/pr.c: Likewise.
* tests/pr/pr-tests.pl: Adjust to avoid matching errno diagnostic.
* tests/fmt/base.pl: Likewise.
* tests/split/l-chunk.sh: Likewise.
* tests/misc/shred-negative.sh: Likewise.
* tests/misc/tail.pl: Likewise.  Also remove the redundant
existing ERR_SUBST from test err-6.
* tests/ls/hex-option.sh: Check HEX/OCT options.
* tests/misc/shred-size.sh: Likewise.
* tests/misc/stty-row-col.sh: Likewise.
2014-12-19 11:56:20 +00:00
Pádraig Brady f57bfbb237 build: use the system printf routines in most cases
glibc <= 2.5 would crash when passed invalid long double values,
therefore internal gnulib routines were used, essentially only by od,
to output such invalid values.  Later glibc versions don't crash,
as per https://sourceware.org/bugzilla/show_bug.cgi?id=4586
and subsequently od was adjusted to use the system printf routines
through the use of the ftoastr module with commit v8.7-22-ga71c22f.
Consequently our testing of this feature was moot, and use of
the gnulib printf replacement for printf(1), od(1) and error(3) etc.
was redundant.

* configure.ac (gl_printf_safe): Unset so that we don't check that
"nan" is output for these long double values.
* tests/misc/od-float.sh: Adjust all existing checks to fail if od
exits with failure status (like crashing for example).  Add a new case
for one of the problematic invalid long double values for x86_64.
We only check that od exits successfully at present, which may change
if https://sourceware.org/bugzilla/show_bug.cgi?id=17661 is resolved.
2014-12-10 17:45:50 +00:00
Pádraig Brady 0e942fdba6 tests: chcon: avoid false failure with newer selinux
file_t is now mapped to unlabeled_t as per:
http://danwalsh.livejournal.com/68189.html

Therefore use the latter to ensure we match correctly.
This is needed on >= Fedora 21 for example,
while it also works on earlier releases.
2014-11-21 15:30:13 +00:00
Pádraig Brady 840da8a7c6 maint: tests: fix comments about retry_delay_
* tests/tail-2/F-vs-missing.sh: Comment with the correct total delay.
* tests/tail-2/F-vs-rename.shi: Likewise.
* tests/tail-2/flush-initial.sh: Likewise.
* tests/tail-2/inotify-hash-abuse.sh: Likewise.
* tests/tail-2/pipe-f2.sh: Likewise.
* tests/misc/chroot-fail.sh: Initialize can_chroot_root in all cases.
2014-10-30 00:06:11 +00:00
Bernhard Voelker f1466d180b tests: avoid false failure when comparing /proc files
At least the MHz number in /proc/cpuinfo may change, thus leading to
a false positive failure when comparing the expected against the
actual output file.  Use an invariant file instead: /proc/version.

* tests/misc/head-c.sh: s/cpuinfo/version/
2014-10-17 08:02:57 +02:00
Pádraig Brady d520929586 chroot: call chroot() unconditionally to handle bind mounted "/"
* src/chroot.c (is_root): Adjust to compare canonicalized paths
rather than inodes, to handle (return false in) the case where
we have a tree that is constructed by first bind mounting "/"
(thus having the same inode).
(main): Unconditionally call chroot() because it's safer
and of minimal performance benefit to avoid in this case.
This will cause inconsistency with some platforms
not allowing `chroot / true` for non root users.
* tests/misc/chroot-fail.sh: Adjust appropriately.
* NEWS: Mention the bug fixes.
Fixes http://bugs.gnu.org/18736
2014-10-16 00:45:32 +01:00
Pádraig Brady 560dad02c3 maint: avoid syntax-check failures in previous patch
* tests/misc/od-j.sh: Non standard comparison order.
* tests/split/b-chunk.sh: Confusing input file name.
* tests/tail-2/tail-c.sh: Redundant require ulimit.
2014-10-08 01:14:57 +01:00
Paul Eggert 2662702b9e wc: don't miscount /sys and similar file systems
Fix similar problems in head, od, split, tac, and tail.
Reported by George Shuklin in: http://bugs.gnu.org/18621
* NEWS: Document this.
* src/head.c (elseek): Move up.
(elide_tail_bytes_pipe, elide_tail_lines_pipe): New arg
CURRENT_POS.  All uses changed.
(elide_tail_bytes_file, elide_tail_lines_file):
New arg ST and remove arg SIZE.  All uses changed.
* src/head.c (elide_tail_bytes_file):
* src/od.c (skip): Avoid optimization for /sys files, where
st_size is bogus and st_size == st_blksize.
Don't report error at EOF when not optimizing.
* src/head.c, src/od.c, src/tail.c: Include "stat-size.h".
* src/split.c (input_file_size): New function.
(bytes_split, lines_chunk_split, bytes_chunk_extract): New arg
INITIAL_READ.  All uses changed. Use it to double-check st_size.
* src/tac.c (tac_seekable): New arg FILE_POS.  All uses changed.
(copy_to_temp): Return size of temp file.  All uses changed.
* src/tac.c (tac_seekable):
* src/tail.c (tail_bytes):
* src/wc.c (wc):
Don't trust st_size; double-check by reading.
* src/wc.c (wc): New arg CURRENT_POS.  All uses changed.
* tests/local.mk (all_tests): Add tests/misc/wc-proc.sh,
tests/misc/od-j.sh, tests/tail-2/tail-c.sh.
* tests/misc/head-c.sh:
* tests/misc/tac-2-nonseekable.sh:
* tests/split/b-chunk.sh:
Add tests for problems with /proc and /sys files.
* tests/misc/od-j.sh, tests/misc/wc-proc.sh, tests/tail-2/tail-c.sh:
New files.
2014-10-07 16:47:37 -07:00
Paul Eggert 4d8c4dfc21 test: check for Fedora 20 sort key bug
Problem reported by Göran Uddeborg in: http://bugs.gnu.org/18540
* tests/misc/sort.pl: New test 23.
2014-09-24 11:55:57 -07:00