Commit Graph
28202 Commits
Author SHA1 Message Date
Pádraig Brady eff511bd3c maint: clarify df's use of device IDs from /proc/self/mountinfo
* src/df.c (filter_mount_list): Clarify why we still stat even
though devices IDs may already be available.  Note using
/proc/self/mountinfo is still an advantage to get filtered items
with accurate device patchs in chroots and with bind mounts.
I.E. on older setups with static /etc/mtab, df will now
bypass that to get the more accuracte and dynamic info.
2015-06-28 03:56:41 +01:00
Pádraig Brady e0c5ca485c chroot: quote argument in error diagnostic
* src/chroot.c (main): Quote the passed argument,
to avoid confusing error messages.
2015-06-26 19:35:46 +01:00
Pádraig Brady f3a0e9ebb2 maint: clarify integer operations in recent commit
* src/factor.c (print_uintmaxes): Comment that the
value of n_out doesn't matter on error, and add an
explicit cast to avoid any future warnings.
Suggested by Jim Meyering RE commit v8.23-229-g4d2d6c5
2015-06-25 18:59:39 +01:00
Pádraig Brady cacd9bf9c2 tests: avoid false failure when running as root
* tests/misc/sync.sh: Ensure dir is unreadable before
including the permission check.
2015-06-25 14:10:47 +01:00
Pádraig Brady 1d82751e2c tests: avoid false failures with LD_PRELOAD=libasan.so.2
The LD_PRELOAD checks by -fsanitize=address are overly strict:
https://groups.google.com/forum/#!topic/address-sanitizer/jEvOJgkDqQk
A workaround is to first export LD_PRELOAD=libasan.so.2
The tests below are adjusted so that workaround is not discarded.

* tests/cp/no-ctx.sh: Append to $LD_PRELOAD.
* tests/df/no-mtab-status.sh: Likewise.
* tests/df/skip-duplicates.sh: Likewise.
* tests/ls/getxattr-speedup.sh: Likewise.
* tests/rm/r-root.sh: Likewise.
* tests/cp/nfs-removal-race.sh: Likewise.  Also check that
LD_PRELOAD is effective to aid future maintainability
and avoid false failure if libasan.so.2 is not preloaded.
2015-06-25 13:41:24 +01:00
Pádraig Brady 381b797784 build: update to latest gnulib
Mainly for -fsanitize=address and -fsanitize=undefined fixes
2015-06-25 13:21:08 +01:00
Pádraig Brady ad95353bf0 build: allow build to complete with -fsanitize=address
* src/make-prime-list.c (main): When building with
the above option, avoid this build stopping error:
"LeakSanitizer: detected memory leaks"
2015-06-25 02:46:07 +01:00
Pádraig Brady 77547ba7b2 maint: avoid undefined behavior in qsort call
GCC 5.1.1 -fsanitize=undefined with glibc 2.21 is returning:
"runtime error: null pointer passed as argument 1,
 which is declared to never be null"
* src/ptx.c (sort_found_occurs): Avoid the call with no entries.
2015-06-24 19:15:31 +01:00
Pádraig Brady 4d2d6c5b7c factor: avoid interspersed lines for parallel runs
* src/factor.c (n_out): A new global variable to track
how much data has been written to stdout.
(print_factors_single): Use n_out to determine whether
to flush the current (and previous) lines.
* tests/misc/factor-parallel.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
2015-06-24 17:04:01 +01:00
Pádraig Brady 32c97093db seq: handle exponents more consistently
src/seq.c (scan_arg): Set precision and width _after_ exponentiation.
For example, this will make '1.1e1 12' and '11 1.2e1' equivalent.
One can still set the precision by specifying extra precision on
the start value, or more naturally with a precision on a step value.
* tests/misc/seq-precision.sh: Add new cases.
2015-06-24 17:04:01 +01:00
Pádraig Brady 60c8e31d92 seq: use consistent output format with hex integers
* src/seq.c (scan_arg): Set precision to 0 for hex constants
(while avoiding hex floats).  This will use then use the
fast path for these arguments.  Note we also set the precision
of inf to 0 here, which ensures we use consistent precision
on output where possible.
* tests/misc/seq-precision.sh: Add corresponding test cases.
2015-06-24 17:03:59 +01:00
Pádraig Brady fc6825b231 seq: support inf last item more generally/efficiently
* src/seq.c (main): Call seq_fast for infinite last value.
This implicitly avoids format conversion on the
999999 -> 1000000 transition.
* src/seq.c (seq_fast): Generalize the buffer handling,
and adjust to handle the "inf" last value specifics.
* tests/misc/seq-precision.sh: A new test.
* tests/local.mk: Reference the new test.
2015-06-24 17:03:35 +01:00
Pádraig Brady 53883af063 doc: list numfmt in the main menu with "Numeric operations"
* doc/coreutils.texi (main menu): Add numfmt.
2015-06-22 17:56:48 +01:00
Pádraig Brady 4f824b3f36 numfmt: don't hardcode floating point limits
* src/numfmt.c (MAX_UNSCALED_DIGITS): Set this to LDBL_DIG
rather than hardcoding at 18 for better portability.
* tests/misc/numfmt.pl: Restrict limit tests to supported platforms.
2015-06-22 17:22:04 +01:00
Pádraig Brady bc9d16b79f numfmt: handle leading zeros correctly
* src/numfmt.c (simple_strtod_int): Don't count leading zeros
as significant digits.  Also have leading zeros as optional
for floating point numbers.
* tests/misc/numfmt.pl: Add test cases.
* NEWS: Mention the fix.
2015-06-22 02:21:02 +01:00
Pádraig Brady a3c3e1e9e6 numfmt: avoid integer overflow when rounding
Due to existing limits this is usually triggered
with an increased precision.  We also add further
restrictions to the output of increased precision numbers.

* src/numfmt.c (simple_round): Avoid intmax_t overflow.
(simple_strtod_int): Count digits consistently
for precision loss and overflow detection.
(prepare_padded_number): Include the precision
when excluding numbers to output, since the precision
determines the ultimate values used in the rounding scheme
in double_to_human().
* tests/misc/numfmt.pl: Add previously failing test cases.
* NEWS: Mention the fix.
2015-06-22 02:02:05 +01:00
Pádraig Brady 0a279f6190 numfmt: support user specified output precision
* src/numfmt.c (usage): Update the --format description
to indicate precision is allowed.
(parse_format_string): Parse a precision specification
like the standard printf does.
(double_to_human): Honor the precision in --to mode.
* tests/misc/numfmt.pl: New tests.
* doc/coreutils.texi (numfmt invocation): Mention the new feature.
* NEWS: Likewise.
2015-06-21 04:16:29 +01:00
Dylan Cali 71063bc858 numfmt: implement support for field ranges
* src/numfmt.c: Replace field handling code with logic that understands
field range specifiers.  Instead of processing a single field and
printing line prefix/suffix around it, process each field in the line
checking whether it has been included for conversion.  If so convert and
print, otherwise just print the unaltered field.
(extract_fields): Removed.
(skip_fields): Removed.
(process_line): Gutted and heavily reworked.
(process_suffixed_number): FIELD is now passed as an arg instead of
using a global.
(parse_field_arg): New function that parses field range specifiers.
(next_field): New function that returns pointers to the next field in
a line.
(process_field): New function that wraps the field conversion logic
(include_field): New function that checks whether a field should be
converted
(compare_field): New function used for field value comparisons in a
gl_list.
(free_field): New function used for freeing field values in a gl_list.
Global variable FIELD removed.
New global variable all_fields indicates whether all fields should be
processed.
New global variable all_fields_after stores the first field of a N-
style range.
New global variable all_fields_before stores the last field of a -M
style range.
New global variable field_list stores explicitly specified fields to
process (N N,M or N-M style specifiers).
(usage): Document newly supported field range specifiers.
* bootstrap.conf: Include xlist and linked-list modules.  numfmt now
uses the gl_linked_list implementation to store the field ranges.
* tests/misc/numfmt.pl: Add tests for 'cut style' field ranges.
Adjust existing tests as partial output can occur before an error
Remove test for the 'invalid' field -5.. this is now a valid range.
* gnulib: update to avoid compiler warnings in linked-list.
* NEWS: Mention the new feature.
2015-06-19 19:59:21 +01:00
Pádraig Brady 5863426dcf doc: use correct units in df | numfmt example
* src/numfmt.c (usage): Don't scale output from df
so that numfmt outputs the correct values.
2015-06-19 14:50:36 +01:00
Pádraig Brady 6fadd46acd numfmt: handle suffixes consistently with --{from,to}-unit
* src/numfmt.c (unit_to_umax): Support SI (power of 10) suffixes
with the --from-unit and --to-unit options.  Treat suffixes like
is done with --from=auto, which for example will change the meaning
of --to-unit=G to that of --to-unit=Gi.  The suffix support was
previously undocumented and it's better to avoid the traditional
coreutils suffix handling in numfmt by default.
* doc/coreutils.texi: Document the new behavior.  Also fix a typo
mentioning {from,to}=units=.
* tests/misc/numfmt.pl: Adjust accordingly.
* NEWS: Mention the change in behavior.
2015-06-19 14:50:34 +01:00
Pádraig Brady a262318f83 maint: remove stale online manual items at release
* gnulib: Update to get the new gnu-web-doc-update with --mirror option.
* README-release: Use the --mirror option in the instructions.
Also clarify and update various release steps.
2015-06-17 14:50:27 +01:00
Pádraig Brady aee9826278 tail: display consistent diagnostics upon file replacement
* src/tail.c (recheck): Display diagnostices for replaced files
even with reused inodes which is a common case.
* tests/tail-2/F-vs-missing.sh: Use correct diagnostic in comment.
* tests/tail-2/F-vs-rename.sh: Likewise.
2015-06-09 11:46:29 +01:00
Pádraig Brady 6878ae412d tail: display file headers correctly with inotify
* src/tail.c (tail_forever_inotify): Use the fspec pointer to
distinguish previously output files, rather than a descriptor
from the inotify event.  That event descriptor was that of
the parent directory when files were created or renamed etc.
(check_fspec): Adjust for the new comparison.  Also show the
header when the file is truncated, since we show data
in this case also.
* tests/tail-2/F-headers.sh: A new test case.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
2015-06-09 11:14:57 +01:00
Pádraig Brady 8f2231aa24 maint: sync .gitignore items with gnulib entries
* .gitignore: Add entries for potentially generated headers.
Also remove a couple of items already present in lib/.gitignore.
* cfg.mk (sc_gitignore_missing): A new syntax check rule to
identify missing .gitignore entries.
(sc_gitignore_redundant): A new syntax check rule to
identify redundant .gitignore entries.
Reported by Tomas Nordin.
2015-06-07 17:24:21 +01:00
Pádraig Brady 3150ec900c tests: fix false failure in recent test adjustment
* configure.ac: Comment on why we link rather than run the test,
and remove the moot __ELF__ check since we never ran it anyway,
and the new CFLAGS and LDFLAGS are a more direct test of support.
* tests/misc/wc-parallel.sh: Fix a syntax error in the previous change.
* tests/misc/md5sum-parallel.sh: Use better error checking, consistent
with that used in wc-parallel.sh.
2015-06-06 14:27:54 +01:00
Paul Eggert 079652ee6b build: port to AIX
Problems reported by Michael Felt, and and part of this fix taken
from code suggested by Pádraig Brady in:
http://bugs.gnu.org/20733#112
* configure.ac (stdbuf_supported): Check for warnings, and
for -fPIC and -shared, for AIX.
* src/stat.c (STRUCT_STATVFS): Define to struct statvfs64 if
STATFS is statvfs64.
* src/sync.c (sync_arg) [_AIX]: Open in write mode,
since AIX fsync doesn't work on read-only file descriptors.
* tests/misc/wc-parallel.sh: Skip test if xargs -P does not work.
2015-06-06 00:46:13 -07:00
Paul Eggert 95413f7dff build: update gnulib submodule to latest 2015-06-06 00:46:13 -07:00
Paul Eggert 5998435209 build: update gnulib submodule to latest 2015-06-05 07:55:03 -07:00
Paul Eggert 7eaf8c8eec build: port single_binary_prog to POSIX shell
Problem reported privately by Michael Felt.
* Makefile.am (install-exec-hook):
* src/local.mk (src/coreutils_symlinks, src/coreutils_shebangs)
(clean-local):
Port to POSIX shell, which doesn't allow 'for i in ; do ...'.
2015-06-04 12:16:04 -07:00
Pádraig Brady da355c9860 build: add a dependency on the gnulib tempname module
* bootstrap.conf: Add "tempname" which is needed by mktemp(1).
The explicit dependency supports running gnulib-tool with
the --conditional-dependencies option, used to minimize built
modules.  Note on a Fedora 22 system, that results in avoiding
redundant builds of: areadlinkat.o asnprintf.o fd-hook.o
fseterr.o printf-args.o printf-parse.o sockets.o vasnprintf.o.
However --conditional-dependencies is not enabled, since it
currently precludes the inclusion of gnulib tests.
2015-06-03 04:41:58 +01:00
Pádraig Brady 44f08f240e build: update to latest gnulib
Mainly with build fixes for Mac OS X.
2015-06-02 01:25:35 +01:00
Pádraig Brady d796a29f2f tests: fix race in tail test without inotify
* tests/tail-2/wait.sh: Without inotify, skip a portion of the test
that checks that -F never outputs from a tailed descriptor
after the followed name is recreated, because tail_forever()
doesn't guarantee that.
Noticed at http://hydra.nixos.org/build/22766288
2015-06-01 17:02:02 +01:00
Pádraig Brady b09f34b80f tests: fix false failure on loaded systems
* tests/misc/uniq-perf.sh: Use our standard 10s timeout,
which is sufficient to trigger the failure and also
avoids a false failure on slow/loaded systems.
Noticed at http://hydra.nixos.org/build/22766288
2015-06-01 15:38:23 +01:00
Pádraig Brady b9842a6153 maint: avoid new coverity warnings
* src/sync.c (sync_arg): Initialise variable to avoid
unitialized access if assert is disabled.
* src/head.c (elide_tail_bytes_file): Support this function
with ---presume-input-pipe and larger files,
which regressed with commit v8.23-47-g2662702.
(elide_tail_lines_file): Likewise.
* src/dd.c (dd_copy): Explicitly don't try to ftruncate()
upon failure to lseek() (the existing check against
st_size was already protecting that).
* src/factor.c (factor_using_squfof): Assert (only when
linting due to performance) to avoid the implication of
divide by zero.
* src/od.c (read_block): Remove dead code.
* src/tac.c (tac_seekable): Likewise.
* src/ls.c (gobble_file): Likewise.
2015-05-30 00:41:09 +01:00
Pádraig Brady 17621b594a build: remove workarounds for unsupported gettext versions
Now that we depend on gettext >= 0.19.2 remove the workaround
for issues in autopoint 0.18.3.  Note the scheme currently used in
newer gettext (autopoint) to avoid these issues requires
autoconf >= 2.69, therefore we update this requirement also.

Note the gettext version dependence from gnulib comes from
gnulib using gettext macros, and coreutils indirectly depends on
the gettext module due to:
http://git.sv.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=41dca647
bootstrap will then update m4/po.m4 and thus require a
supportng gettext version.

* bootstrap: Remove moot warning (resyncing with gnulib).
* configure.ac (AC_PREREQ): Change to 2.69 (now 3 years old).
2015-05-28 02:47:15 +01:00
Pádraig Brady 1aa71d9633 tests: ln/hard-to-sym FAIL rather than ERROR when ln fails
* tests/ln/hard-to-sym.sh: Only call framework_failure_ when ln
returns success.
2015-05-28 02:05:43 +01:00
Pádraig Brady 67db18ce0f copy: prefer our hardlink to symlink emulation on OS X 10.10
* src/copy.c (CAN_HARDLINK_SYMLINKS): Don't enable use of linkat()
on Darwin 14, as the gnulib fallback emulation there doesn't
preserve ownership and timestamps etc.  This fixes a test failure
in tests/cp/link-symlink.sh
* tests/cp/link-deref.sh: Adjust accordingly.
2015-05-28 01:42:15 +01:00
Pádraig Brady a41172a5c4 maint: remove sys/types.h include order check
* src/system.h: This was inadvertently ineffective due to
a typo in commit v8.9-10-ge1aaf89 (Jan 2011), but has
not caused any issues, so remove.
2015-05-26 02:16:48 +01:00
Pádraig Brady 4ce7488e8d mkdir: fix -pZ with existing parent directories
When the parent directory exists and has a different
default context to the final directory, the context
was incorrectly left as that of the parent directory.

* src/mkdir.c (process_dir): Because defaultcon() is called for
existing ancestors (as it must be to avoid races), then we must
unconditionally call restorecon() on the last component due to
the already documented caveat with make_dir_parents().
Alternatively you could temp disable o->set_security_context
around make_dir_parents(), but that would be subject to races.
* tests (tests/mkdir/restorecon.sh): Add a TODO for improvement.
Reference mknod and mkfifo with print_ver_.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/20616
2015-05-23 02:04:24 +01:00
Assaf Gordon b5f332cfe0 tests: df-output: accept multiple spaces in header
* tests/df/df-output.sh: Allow for multiple spaces in the header line
of 'df', resulting from alignment with disk sizes >= 10TB.
2015-05-22 20:45:23 +01:00
Pádraig Brady edc5bc5020 tests: fix non POSIX constructs causing failures with dash
* tests/cp/no-ctx.sh: Scope of `var=val func` is inconsistent
across shells, so avoid that construct with functions.
* tests/df/no-mtab-status.sh: Likewise.
* tests/tail-2/inotify-race.sh: `read` needs an argument.
* tests/tail-2/inotify-race2.sh: Likewise.
2015-05-19 15:00:57 +01:00
Pádraig Brady 719c15a6f2 build: fix 'dist' and 'syntax-check' targets in VPATH build
* cfg.mk: Various syntax-check adjustments so that it's
not assumed the $builddir is the base distribution directory.
* Makefile.am: Likewise for the 'dist' target.
2015-05-17 16:00:48 +01:00
Pádraig Brady 524a86e4f5 build: avoid issues with case insensitive file systems
* cfg.mk (sc_case_insensitive_file_names): A new syntax-check rule.
* tests/tail-2/descriptor-vs-rename.sh: Rename from
tests/tail-2/f-vs-rename.sh
* tests/local.mk: Reference the renamed test.
Reported by Jim Meyering.
2015-05-17 15:50:27 +01:00
Bernhard Voelker 99dfd5a661 tests: fix check for local file system in inotify-rotate-resources.sh
* tests/tail-2/inotify-rotate-resources.sh: s/(is_local_dir)/\1_/
2015-05-14 21:31:10 +02:00
Pádraig Brady 7f0f58fdd6 tests: fix async allocation race on BTRFS
* tests/dd/sparse.sh: Sync files before checking allocations,
which may be done asynchronously on NFS and BTRFS at least.
Also mark this test as very expensive on remote file systems.
* tests/du/2g.sh: Likewise, also use fallocate if available
to efficiently allocate the large file, otherwise skip
on remote file systems.
* tests/tail-2/inotify-rotate-resources.sh: Use the more
standard is_local_dir_() to check remoteness.
* tests/cp/fiemap-empty.sh: Comment on the sync issue
for this currerntly unused test.
Fixes http://bugs.gnu.org/20570
2015-05-14 14:08:02 +01:00
Pádraig Brady 45b8fe430d timeout: with --foreground don't send SIGCONT
* src/timeout.c (cleanup): Don't send SIGCONT to the monitored program
when --foreground is specified, as it's generally not needed for
foreground programs, and can cause intermittent signal delivery
issues with monitors like GDB for example.
* doc/coreutils.texi (timeout invocation): Mention that SIGCONT
is not sent with --foreground.
* NEWS: Mention the behavior change.
2015-05-13 13:48:56 +01:00
Pádraig Brady 703747f892 split: auto set suffix len for --numeric-suffixes=<N --number=N
Supporting `split --numeric-suffixes=1 -n100` for example.

* doc/coreutils.texi (split invocation): Mention the two
use cases for the FROM parameter, and the consequences on
the suffix length determination.
* src/split.c (set_suffix_length): Use the --numeric-suffixes
FROM parameter in the suffix width calculation, when it's
less than the number of files specified in --number.
* tests/split/suffix-auto-length.sh: Add test cases.
Fixes http://bugs.gnu.org/20511
2015-05-13 12:43:40 +01:00
Assaf Gordon 79111d1553 doc: clarify the operation of wc -L
* src/wc.c (usage): State that it calculates display width.
* doc/coreutils.texi (wc invocation): Detail the distinct
items used to determine the display width.
2015-05-13 02:54:19 +01:00
Pádraig Brady b28ff6a3c6 tail: consistently output all data for truncated files
Generally if logs are truncated, they're truncated to 0 length,
so output all existing data when our heuristic determines truncation.
Note with inotify, truncate() and write() are often determined
independently and so all data would be written if that was the case.

* src/tail.c (check_fspec): Reset file offset to 0 upon truncation.
(tail_forever): Likewise.
(recheck): Add a FIXME for the related issue where tail may lose
data due to tail discounting older log files too early.
* tests/tail-2/truncate.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the fix.
2015-05-11 23:16:14 +01:00
Pádraig Brady cf06a872e7 tail: fix inotify startup races
The previous fixes to races in the various tail tests,
identified actual races in the tail inotify implementation.
With --follow=descriptor, if the tailed file was replaced before
the inotify watch was added, then any subsequent changes were ignored.
Similarly in --follow=name mode, all changes to a new name were
effectively ignored if that name was created after the original open()
but before the inotify_add_watch().

* src/tail.c (tail_forever_inotify): Fix 3 cases.
1. With -f, don't stop tailing when file removed before watch.
2. With -f, watch right file when file replaced before watch.
3. With -F, inspect correct file when replaced before watch.
Existing tests identify these when tail compiled with TAIL_TEST_SLEEP.
* tests/tail-2/inotify-rotate-resources.sh:
This test also identifies the issue with --follow=name
when TAIL_TEST_SLEEP is used.  Adjust so the test is immune
to such races, and also fail quicker on remote file systems.
* tests/tail-2/inotify-race2.sh: A new test using GDB,
based on inotify-race.sh, which tests the -F race
without needed recompilation with sleeps.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug.
2015-05-11 23:03:32 +01:00