Commit Graph
28234 Commits
Author SHA1 Message Date
Pádraig Brady a8cc9ce3f2 base64: use stricter validation on wrap column
* src/base64.c (main): Use the higher level xnumtoumax()
rather than xstrtoumax(), which is simpler and improves
validation of input.  Also pass the _empty_ rather than NULL
string as the suffixes parameter so that invalid trailing
characters are not allowed.  For example -w08 is now
flagged as an error, rather than being interpreted as 0.
A subsequent commit will further improve verification
of numbers with leading zeros by dropping backwards compatibility
wrt auto parsing oct and hex numbers.
* tests/misc/base64.pl: Add tests for invalid wrap values.
2015-09-03 00:34:12 +01:00
Pádraig Brady 89c517d9e2 base32: A new program similar to base64
Suggested in https://bugzilla.redhat.com/1250113

* AUTHORS: Add base32.
* THANKS.in: Add suggester.
* README: Reference the new program.
* NEWS: Mention the new program.
* src/.gitignore: Ignore the new binary.
* bootstrap.conf: Reference the gnulib base32 module.
* build-aux/gen-lists-of-programs.sh: Add base32.
* man/base32.x: A new template.
* man/.gitignore: Ignore the new man page.
* man/local.mk: Reference the new man page.
* doc/coreutils.texi (base32 invocation): Document the new command.
* src/local.mk: Adjust to build base32 based on base64.c.
* src/base64.c: Parameterize to use the correct headers,
functions and buffer sizes, depending on which binary
is being built.
* tests/misc/base64.pl: Adjust to test both base32 and base64.
* tests/misc/tty-eof.pl: Add base32 as a program that
accepts input on stdin without any options specified.
* scripts/git-hooks/commit-msg: Add base32 to the template.
2015-09-03 00:33:19 +01:00
Pádraig Brady affc8e8087 build: update gnulib submodule to latest
This includes a tweak to support building
the gnulib base32 module with -Wsuggest-attribute=const
2015-09-01 17:42:33 +01:00
Pádraig Brady a83f703a9b doc: clarify in --help/man where short options take no param
* src/shred.c (usage): For -u, separate the decscription
of the short and long option, to clarify that the short option
takes no parameter.
* src/split.c (usage): Likewise for -d.
* src/tee.c (usage): Likewise for -p.
* src/uniq.c (usage): Likewise for -D.

Suggested by Stephane Chazelas
2015-08-31 18:10:41 +01:00
Bernhard Voelker fd5f2b1569 tests: avoid FP of ls/stat-free-color.sh with newer glibc
Since glibc-2.22, specifically commit [0], the opendir() implementation
implicitly makes an additional stat call thus leading to a FP.
Seen on openSUSE:Tumbleweed since snapshot 20150821.

[0]
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=46f894d8c60a

* tests/ls/stat-free-color.sh: Change the test to verify that ls(1)
needs the same number of stat-like calls for a single, empty directory
argument as for one with a few directory entries (sub-directory,
regular file, symlink, etc.).
2015-08-30 22:49:35 +02:00
Pádraig Brady da5505dd7f doc: clarify where ambiguous if short options take no param
* doc/coreutils.texi (split invocation): Clarify that -d takes no param.
(uniq invocation): Likewise for -D.
(shred invocation): Likewise for -u.
(tee invocation): Likewise for -p.
2015-08-27 16:12:58 +01:00
Paul Eggert 97678abb6b ls: allow -w18446744073709551616
Problem reported by Beco in: http://bugs.gnu.org/21325
* src/ls.c (set_line_length): New function.
(decode_switches): Use it to decode COLUMNS and -w.
2015-08-22 18:31:13 -07:00
Bernhard Voelker 4ec9ccad79 maint: add syntax check to ensure larger man/*.x files have a Copyright
* cfg.mk (sc_man_check_x_copyright): Add rule to ensure that non-trivial
.x files in the 'man/' subdirectory, i.e., files exceeding a line count
of 20 or a byte count of 1000, contain a proper Copyright notice.
2015-08-18 20:27:48 +02:00
Paul Eggert 037e3b9847 build: fprintftime/nstrftime API changes
* bootstrap.conf (gnulib_modules): Add time_rz,
since the main source code now uses timezone_t.
* src/date.c (batch_convert, main, show_date):
* src/ls.c (align_nstrftime, long_time_expected_width)
(print_long_format):
* src/stat.c (human_time):
Use timezone_t rather than boolean to specify which time zone
is wanted.
* src/ls.c (localtz): New static var.
(main): Initialize it.
2015-07-23 18:52:15 -07:00
Paul Eggert 7131a33952 build: update gnulib submodule to latest 2015-07-23 18:52:15 -07:00
Pádraig Brady 6d67649d86 doc: discourage use of uname -i and -p options
* src/uname.c (usage): State that the non POSIX -i and -p options
are non-portable.
* doc/coreutils.texi (uname invocation): Mention the discrepancies
even across GNU/Linux distros, and that the results should
be used as informational only, rather than impacting any
logic decisions.
Fixes http://bugs.gnu.org/13001
2015-07-22 01:55:14 +01:00
Pádraig Brady 0b0d38b762 doc: improve man page for realpath -m
* src/realpath.c (usage): Mention 'directory' in the --help
output, so that ENOTDIR errors may be more easily investigated,
by inspecting the man page.
Reported at http://pad.lv/1474519
2015-07-15 14:37:05 +01:00
Peter Bray a652e7b63e tests: avoid test warning with perl >= 5.22
* doc/local.mk (sc-lower-case-var): Escape a literal
left curly bracket, needed with perl >= 5.22
Fixes http://bugs.gnu.org/21060
2015-07-15 11:39:49 +01:00
Peter Bray 9d2597161d sync: fix build with separate $(LIB_FDATASYNC)
* src/local.mk: Link with $(LIB_FDATASYNC) needed
on some Solaris 10 systems for example.
Fixes http://bugs.gnu.org/21059
2015-07-15 08:40:51 +01:00
Pádraig Brady 1a78f6a064 dircolors: add tmux entries
* src/dircolors.hin: tmux entries were added to ncurses in:
http://ncurses.scripts.mit.edu/?p=ncurses.git;a=blobdiff;\
f=misc/terminfo.src;h=ce9bdc3b;hp=7e576ef1;hb=be512fa0;hpb=ee1bcda2
2015-07-09 17:52:51 +01:00
Pádraig Brady 5e5d454037 shred: fix pattern selection for certain iteration counts
This was detected in about 25% of runs with gcc -fsanitize=address

  ERROR: AddressSanitizer: global-buffer-overflow on address ...
  READ of size 4 at 0x000000416628 thread T0
    #0 0x40479f in genpattern src/shred.c:782
    #1 0x4050d9 in do_wipefd src/shred.c:921
    #2 0x406203 in wipefile src/shred.c:1175
    #3 0x406b84 in main src/shred.c:1316
    #4 0x7f3454a1ef9f in __libc_start_main (/lib64/libc.so.6+0x1ff9f)
    #5 0x4025d8 (/tmp/coreutils-8.23/src/shred+0x4025d8)
  0x000000416628 is located 56 bytes to the left of
  global variable '*.LC49' from 'src/shred.c' (0x416660) of size 17
  0x000000416628 is located 12 bytes to the right of
  global variable 'patterns' from 'src/shred.c' (0x416540) of size 220
  SUMMARY: AddressSanitizer: global-buffer-overflow src/shred.c:782

* src/shred.c (gen_patterns): Restrict pattern selection
to the K available, which regressed due to v5.92-1462-g65533e1.
* tests/misc/shred-passes.sh: Add a deterministic test case.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/20998
2015-07-07 03:26:58 +01:00
Pádraig Brady c5ff0d989f maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2015-07-03 22:23:56 +01:00
Pádraig Brady f2ef324811 version 8.24
* NEWS: Record release date.
v8.24
2015-07-03 21:58:50 +01:00
Pádraig Brady 9a81dd803c maint: fix distclean-check failure due to THANKS
* Makefile.am: Remove our dependence on src/sort which
induces awkward dependencies for `make dist` since
THANKS will be rebuilt once src/sort is newer.
Instead we remove the problematic -f option to sort
which actually doesn't change the order given
our current input.
2015-07-03 21:08:26 +01:00
Pádraig Brady db3f9596ff maint: avoid false syntax-check failure in distcheck
* cfk.mk (sc_tests_executable): Restrict the check to git files,
so we don't flag any gnulib files added to test/ during
`make distcheck`.
2015-07-03 17:36:26 +01:00
Pádraig Brady 11091122c5 tests: update gnulib submodule and tests/init.sh to latest
* gnulib: Update to latest, with fixes to tests edge cases.
* tests/init.sh: Update from gnulib.
2015-07-03 15:50:56 +01:00
Pádraig Brady 1d4289b3c1 tests: avoid side effects of $SHELL environment variable
Since non interactive shells don't generally set $SHELL,
its value is propagated through the tests and may cause issues;
for example if $SHELL implicitly adjusts $PATH when run.
Instead we set $SHELL to that determined by the posix-shell module,
and use that consistently for all test sub scripts,
including those created thorugh the `split --filter` command.

* tests/local.mk: Explicitly set $SHELL to $(PREFERABLY_POSIX_SHELL)
which defaults to $CONFIG_SHELL and thus usually /bin/sh.
* tests/envvar-check: Remove bash environment variables with
side effects, in case /bin/bash was selected for $SHELL.
* tests/misc/help-version.sh: Remove redundant initialization of $SHELL.
* tests/install/strip-program.sh: Use $SHELL for sub script.
* tests/misc/sort-compress-hang.sh: Likewise.
* tests/misc/sort-compress-proc.sh: Likewise.
* tests/misc/sort-compress.sh: Likewise.
* tests/misc/timeout-group.sh: Likewise.
* tests/rm/fail-eperm.xpl: Remove redundant elision of bash env vars.
* tests/misc/pwd-long.sh: Likewise.
2015-07-03 13:02:21 +01:00
Pádraig Brady 970b891702 tests: avoid false failures on OpenBSD 5.7
* tests/du/threshold.sh: Homogenize getopt error messages.
* tests/misc/numfmt.pl: Likewise.
* tests/mv/i-3.sh: Skip on *BSD not just FreeBSD.
2015-07-02 19:15:08 +01:00
Pádraig Brady a645ce6c04 wc: fix reading of /proc files on aarch64
tests/misc/wc-proc.sh fails when the page size is 64K

* src/wc.c (wc): The lseek adjustment should be based on st_blksize,
rather than on the internal buffer size.  This is significant on
aarch64 where st_blksize in /proc is the 64K (the page size) and
thus larger than the internal buffer.
* src/split.c (main): Even though the similar processing is done
on the internal buffer size, that's based on st_blksize and
so fine in this regard.  Add an assert to enforce this.
Avoid this path for the undocumented ---io-blksize option.
2015-07-02 03:53:59 +01:00
Pádraig Brady 8bf2af685c build: update gnulib submodule to latest
Mainly with build fixes for FreeBSD and OS X.
2015-07-01 16:09:39 +01:00
Pádraig Brady 81c7fc4081 numfmt: increase precision on 32 bit FreeBSD
* m4/jm-macros.m4 (HAVE_FPSETPREC): Define if needed.
* src/numfmt.c (main): Call fpsetprec() if needed.
Fixes large-15 and large-16 test failures on 32 bit FreeBSD.
2015-07-01 12:29:59 +01:00
Pádraig Brady 491f54fb5a tests: avoid false failure on FreeBSD systems
* tests/misc/stty.sh: FreeBSD returns ENOTTY for
the TIOCEXT ioctl, so just avoid this option for now.
2015-07-01 12:27:29 +01:00
Pádraig Brady f2db30e676 factor: ensure atomic output through pipes
The new tests/misc/factor-parallel.sh test was
seen to fail on FreeBSD (derived) systems, which was
due to split(1) --filter reading partial lines
through pipes, as factor(1) was writing a little
over PIPE_BUF each time.

* src/factor.c (lbuf): A new structure to internally buffer lines.
(lbuf_alloc): A new function to allocate enough at program start.
(lbuf_putint): A new function to buffer a uintmax_t.
(lbuf_flush): A new function to write directly to standard output.
(lbuf_putc): A new function to buffer a character and if enough
lines are buffered, then output complete lines <= PIPE_BUF,
and continue to buffer the rest.
(main): Call the internal buffer allocator, and register
the final flush from the internal buffer at program exit.
2015-07-01 12:27:29 +01:00
Pádraig Brady e1358a7dd9 tests: fix false failure on slower systems
* tests/dd/stats.sh: Wait 20s for dd to write 250MB through a fifo,
rather than 10s for 500MB.  The failure was seen often on
a lightly loaded SPARC-Enterprise-T5220 running Solaris 10.
2015-07-01 12:27:29 +01:00
Jim Meyering f303fb6bbb maint: stdbuf.c: avoid the OS X putenv function
* bootstrap.conf (gnulib_modules): Add setenv, to make this
module dependency explicit; setenv is also used by split.
* src/stdbuf.c (set_LD_PRELOAD) [__APPLE__]: Use the OS X setenv
function, rather than putenv, per that documentation:
https://developer.apple.com/\
library/mac/documentation/Darwin/Reference/ManPages/man3/putenv.3.html
2015-06-29 14:18:59 -07:00
Jim Meyering 46416bb7c3 build: numfmt.c: avoid a shadowing warning
* src/numfmt.c (parse_field_arg): Rename parameter s/optarg/arg/,
to avoid shadowing getopt's global variable.
Otherwise, building on OS X, with --enable-gcc-warnings, I saw this:

  In file included from src/numfmt.c:19:0:
  src/numfmt.c: In function 'parse_field_arg':
  ./lib/config.h:3109:25: error: declaration of 'rpl_optarg' shadows\
    a global declaration [-Werror=shadow]
2015-06-29 11:11:49 -07:00
Assaf Gordon badecb3f6c numfmt: fix printf argument order
* src/numfmt.c (double_to_human): Fix the argument order
passed to snprintf, which happened to work on amd64 with
its separate va_arg storage area for floats¹,
but would fail tests for example on i686.

¹ https://blog.nelhage.com/2010/10/amd64-and-va_arg/
2015-06-29 12:59:19 +01:00
Pádraig Brady eff511bd3c maint: clarify df's use of device IDs from /proc/self/mountinfo
* src/df.c (filter_mount_list): Clarify why we still stat even
though devices IDs may already be available.  Note using
/proc/self/mountinfo is still an advantage to get filtered items
with accurate device patchs in chroots and with bind mounts.
I.E. on older setups with static /etc/mtab, df will now
bypass that to get the more accuracte and dynamic info.
2015-06-28 03:56:41 +01:00
Pádraig Brady e0c5ca485c chroot: quote argument in error diagnostic
* src/chroot.c (main): Quote the passed argument,
to avoid confusing error messages.
2015-06-26 19:35:46 +01:00
Pádraig Brady f3a0e9ebb2 maint: clarify integer operations in recent commit
* src/factor.c (print_uintmaxes): Comment that the
value of n_out doesn't matter on error, and add an
explicit cast to avoid any future warnings.
Suggested by Jim Meyering RE commit v8.23-229-g4d2d6c5
2015-06-25 18:59:39 +01:00
Pádraig Brady cacd9bf9c2 tests: avoid false failure when running as root
* tests/misc/sync.sh: Ensure dir is unreadable before
including the permission check.
2015-06-25 14:10:47 +01:00
Pádraig Brady 1d82751e2c tests: avoid false failures with LD_PRELOAD=libasan.so.2
The LD_PRELOAD checks by -fsanitize=address are overly strict:
https://groups.google.com/forum/#!topic/address-sanitizer/jEvOJgkDqQk
A workaround is to first export LD_PRELOAD=libasan.so.2
The tests below are adjusted so that workaround is not discarded.

* tests/cp/no-ctx.sh: Append to $LD_PRELOAD.
* tests/df/no-mtab-status.sh: Likewise.
* tests/df/skip-duplicates.sh: Likewise.
* tests/ls/getxattr-speedup.sh: Likewise.
* tests/rm/r-root.sh: Likewise.
* tests/cp/nfs-removal-race.sh: Likewise.  Also check that
LD_PRELOAD is effective to aid future maintainability
and avoid false failure if libasan.so.2 is not preloaded.
2015-06-25 13:41:24 +01:00
Pádraig Brady 381b797784 build: update to latest gnulib
Mainly for -fsanitize=address and -fsanitize=undefined fixes
2015-06-25 13:21:08 +01:00
Pádraig Brady ad95353bf0 build: allow build to complete with -fsanitize=address
* src/make-prime-list.c (main): When building with
the above option, avoid this build stopping error:
"LeakSanitizer: detected memory leaks"
2015-06-25 02:46:07 +01:00
Pádraig Brady 77547ba7b2 maint: avoid undefined behavior in qsort call
GCC 5.1.1 -fsanitize=undefined with glibc 2.21 is returning:
"runtime error: null pointer passed as argument 1,
 which is declared to never be null"
* src/ptx.c (sort_found_occurs): Avoid the call with no entries.
2015-06-24 19:15:31 +01:00
Pádraig Brady 4d2d6c5b7c factor: avoid interspersed lines for parallel runs
* src/factor.c (n_out): A new global variable to track
how much data has been written to stdout.
(print_factors_single): Use n_out to determine whether
to flush the current (and previous) lines.
* tests/misc/factor-parallel.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
2015-06-24 17:04:01 +01:00
Pádraig Brady 32c97093db seq: handle exponents more consistently
src/seq.c (scan_arg): Set precision and width _after_ exponentiation.
For example, this will make '1.1e1 12' and '11 1.2e1' equivalent.
One can still set the precision by specifying extra precision on
the start value, or more naturally with a precision on a step value.
* tests/misc/seq-precision.sh: Add new cases.
2015-06-24 17:04:01 +01:00
Pádraig Brady 60c8e31d92 seq: use consistent output format with hex integers
* src/seq.c (scan_arg): Set precision to 0 for hex constants
(while avoiding hex floats).  This will use then use the
fast path for these arguments.  Note we also set the precision
of inf to 0 here, which ensures we use consistent precision
on output where possible.
* tests/misc/seq-precision.sh: Add corresponding test cases.
2015-06-24 17:03:59 +01:00
Pádraig Brady fc6825b231 seq: support inf last item more generally/efficiently
* src/seq.c (main): Call seq_fast for infinite last value.
This implicitly avoids format conversion on the
999999 -> 1000000 transition.
* src/seq.c (seq_fast): Generalize the buffer handling,
and adjust to handle the "inf" last value specifics.
* tests/misc/seq-precision.sh: A new test.
* tests/local.mk: Reference the new test.
2015-06-24 17:03:35 +01:00
Pádraig Brady 53883af063 doc: list numfmt in the main menu with "Numeric operations"
* doc/coreutils.texi (main menu): Add numfmt.
2015-06-22 17:56:48 +01:00
Pádraig Brady 4f824b3f36 numfmt: don't hardcode floating point limits
* src/numfmt.c (MAX_UNSCALED_DIGITS): Set this to LDBL_DIG
rather than hardcoding at 18 for better portability.
* tests/misc/numfmt.pl: Restrict limit tests to supported platforms.
2015-06-22 17:22:04 +01:00
Pádraig Brady bc9d16b79f numfmt: handle leading zeros correctly
* src/numfmt.c (simple_strtod_int): Don't count leading zeros
as significant digits.  Also have leading zeros as optional
for floating point numbers.
* tests/misc/numfmt.pl: Add test cases.
* NEWS: Mention the fix.
2015-06-22 02:21:02 +01:00
Pádraig Brady a3c3e1e9e6 numfmt: avoid integer overflow when rounding
Due to existing limits this is usually triggered
with an increased precision.  We also add further
restrictions to the output of increased precision numbers.

* src/numfmt.c (simple_round): Avoid intmax_t overflow.
(simple_strtod_int): Count digits consistently
for precision loss and overflow detection.
(prepare_padded_number): Include the precision
when excluding numbers to output, since the precision
determines the ultimate values used in the rounding scheme
in double_to_human().
* tests/misc/numfmt.pl: Add previously failing test cases.
* NEWS: Mention the fix.
2015-06-22 02:02:05 +01:00
Pádraig Brady 0a279f6190 numfmt: support user specified output precision
* src/numfmt.c (usage): Update the --format description
to indicate precision is allowed.
(parse_format_string): Parse a precision specification
like the standard printf does.
(double_to_human): Honor the precision in --to mode.
* tests/misc/numfmt.pl: New tests.
* doc/coreutils.texi (numfmt invocation): Mention the new feature.
* NEWS: Likewise.
2015-06-21 04:16:29 +01:00
Dylan Cali 71063bc858 numfmt: implement support for field ranges
* src/numfmt.c: Replace field handling code with logic that understands
field range specifiers.  Instead of processing a single field and
printing line prefix/suffix around it, process each field in the line
checking whether it has been included for conversion.  If so convert and
print, otherwise just print the unaltered field.
(extract_fields): Removed.
(skip_fields): Removed.
(process_line): Gutted and heavily reworked.
(process_suffixed_number): FIELD is now passed as an arg instead of
using a global.
(parse_field_arg): New function that parses field range specifiers.
(next_field): New function that returns pointers to the next field in
a line.
(process_field): New function that wraps the field conversion logic
(include_field): New function that checks whether a field should be
converted
(compare_field): New function used for field value comparisons in a
gl_list.
(free_field): New function used for freeing field values in a gl_list.
Global variable FIELD removed.
New global variable all_fields indicates whether all fields should be
processed.
New global variable all_fields_after stores the first field of a N-
style range.
New global variable all_fields_before stores the last field of a -M
style range.
New global variable field_list stores explicitly specified fields to
process (N N,M or N-M style specifiers).
(usage): Document newly supported field range specifiers.
* bootstrap.conf: Include xlist and linked-list modules.  numfmt now
uses the gl_linked_list implementation to store the field ranges.
* tests/misc/numfmt.pl: Add tests for 'cut style' field ranges.
Adjust existing tests as partial output can occur before an error
Remove test for the 'invalid' field -5.. this is now a valid range.
* gnulib: update to avoid compiler warnings in linked-list.
* NEWS: Mention the new feature.
2015-06-19 19:59:21 +01:00