Commit Graph
28731 Commits
Author SHA1 Message Date
Assaf Gordon 5ab71e4d3f stat: output default formats for --terse in usage
Suggested by L A Walsh in https://bugs.gnu.org/28763 .

* src/stat.c (fmt_terse_fs): Define format for --terse -f here.
(fmt_terse_regular): Define format for --terse here.
(fmt_terse_selinux): Likewise for when SELinux is enabled.
(default_format): Use the above constants.
(usage): Output the formats for the terse modes.
2017-10-31 16:31:16 -06:00
Pádraig Brady b04ce61958 df: fix hang with fifo argument
* src/df.c (main): stat() before open(), and avoid
the optional open when given a fifo argument.
* tests/df/unreadable.sh: Add a test case.
* NEWS: Mention the fix.
Fixes https://bugs.gnu.org/29038
2017-10-30 09:47:43 -07:00
Jim Meyering f89c20bdf7 build: ls.c: apply _GL_ATTRIBUTE_PURE to more functions
* src/ls.c (DEFINE_SORT_FUNCTIONS): Apply _GL_ATTRIBUTE_PURE
to each strcmp-derived function definition, since GCC8 with
-Wsuggest-attribute=pure now warns it is needed.
2017-10-27 21:18:46 -07:00
Vincent Lefevre 61a8b5cb56 doc: reference statfs(2) in the stat(1) man page
* man/stat.x (SEE ALSO): Mention statfs(2) in addition to stat(2).
Note statfs() is generally used rather than statvfs(),
so we'll defer that reference to the SEE ALSO section of statfs(2).
Fixes https://bugs.gnu.org/28989
2017-10-25 21:36:19 -07:00
Pádraig Brady e818f19cf8 tests: avoid false failure when O_DIRECT isn't supported
* tests/dd/nocache_eof.sh: Only run the O_DIRECT tests
when 512 byte alignment is supported.  Otherwise with older
XFS on systems with > 1MiB pages, or on file systems not
supporting O_DIRECT, there would have been false failures.
* tests/dd/direct.sh: Clarify the skip message.
2017-10-24 20:08:05 -07:00
Pádraig Brady de15a497d1 dd: fix nocache regions passed to posix_fadvise()
Previously with oflag=direct the call to invalidate_cache()
was not passed to the kernel, as it was less than a page size,
and a subsequent call was not made to invalidate the pending space.
Similarly with oflag=nocache the pending space at EOF was
not invalidated.  Even though these amount to only a single page
in the page cache it can be significant.  For example on
XFS before kernel patch v4.9-rc1-4-g0ee7a3f, O_DIRECT files
would have been read inefficiently if any pages were cached,
even if they were already synced to storage.

* src/dd.c (i_nocache_eof, o_nocache_eof): New bools used
to control when we want invalidate_cache(,0) to clear to EOF.
(cache_round): Use IO_BUFSIZE (currently 132KiB) to minimize
calls to the relatively expensive advise function, rather
than page_size.  This also makes it clear that while the
kernel function operates on pages, this size is chosen for
performance reasons.
(invalidate_cache): Refactor to share more code between
input and output paths. Use i_nocache_eof and o_nocache_eof
rather than proxying off max_records.  Ensure we
invalidate full pages when clearing to EOF as the kernel
will ignore any non complete pages.  Fix the offset used
for the output path.
(dd_copy): Invalidate the cache of the input after the
offset is updated, for consistency and so we don't try to
invalidate before the start of the file.  When we read
EOF on input, set flags so that we invalidate to EOF.
(main): Invalidate to EOF in more cases, by depending
on the i_nocache_eof and o_nocache_eof flags.
* doc/coreutils.texi (dd invocation): Clarify the alignment
and persisted caveats on the example applying "nocache"
to part of a file.
* tests/dd/nocache_eof.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Issue reported by Eric Bergen.
2017-10-24 19:38:12 -07:00
Michael Stone 9fa178fccd doc: mention QUOTING_STYLE env var in ls man page
* src/ls.c (usage): Mention QUOTING_STYLE with the --quoting-style
option, and indicate it has lower precedence than that option.
2017-10-24 10:29:35 -07:00
Pádraig Brady 84f7231dfa maint: apply suggested cleanup to recent stty.c change
This should have been part of commit v8.28-17-gf926f7c
* src/stty.c (check_argument): Align line continuation chars,
and ensure the function macro is immune to usage with if/else.
Suggested by Jim Meyering and Paul Eggert.
2017-10-23 23:31:40 -07:00
Pádraig Brady cc19f63be3 b2sum: fix crash with --check and truncated input
* src/md5sum.c (split_3): Ensure we don't walk off
the end of the string.
* tests/misc/b2sum.sh: Add test cases.
Fixes https://bugs.gnu.org/28860
2017-10-23 23:17:17 -07:00
Pádraig Brady f926f7ce0e stty: fix processing of options when -F is specified
This was a latent issue that became significant with
the addition of the -F option in FILEUTILS-3_16n-56-ge46a424

* src/stty.c (apply_settings): Refactor argument checking
to a function macro.  Augment the argument check to ignore
NULLed out arguments (already processed -F).
* NEWS: Mention the fix.
* tests/misc/stty-invalid.sh: Add a test case.
Fixes https://bugs.gnu.org/28859
2017-10-23 23:15:25 -07:00
Pádraig Brady cbf35912da timeout: fix a small race that would ignore command exit
This fixes a regression from commit v8.26-39-g2f69dba

* src/timeout.c (block_cleanup_and_chld): Rename from block_cleanup
to indicate we also block SIGCHLD to avoid the race where SIGCHLD
fires between waitpid() polling and sigsuspend() waiting for a signal.
* NEWS: Mention the fix.
2017-10-23 23:10:42 -07:00
Thomas Jarosch cccd01b4da timeout: fix regression when invoked with blocked SIGCHLD
We inherit the signal mask from our parent process,
therefore ensure SIGCHLD is not blocked.

If SIGCHLD is blocked, sigsuspend() won't be interrupted
when the child process exits and we hang until the timeout (SIGALRM).

This fixes a regression from commit v8.26-39-g2f69dba

* src/timeout.c (install_sigchld): Ensure SIGCHLD is unblocked.
* NEWS: Mention the issue.
2017-10-23 23:09:02 -07:00
Pádraig Brady 856b828320 build: reinstate distribution of man pages
man pages change little between systems,
so falling back to distributed pages make sense
when cross compiling or lacking perl.

* man/local.mk: Add all man pages to EXTRA_DIST
so that they're distributed in the generated tarball.
Use the dummy-man page generator if cross compiling.
Set TZ to avoid a distcheck failure where man pages
used a diffent month than those rebuilt (with a .timestamp).
* man/dummy-man: Only fall back to generating a stub
if copying an existing man page fails.
* man/help2man: Sync portable TZ=UTC0 specification
from upstream help2man.
* NEWS: Mention the build-related change.
Fixes https://bugs.gnu.org/28574
2017-10-01 17:32:48 -07:00
Pádraig Brady 6774b991db maint: remove a duplicate entry from THANKS
* .mailmap: Prefer Colin Watson's last used email address.
2017-10-01 17:32:30 -07:00
Paul Eggert 4b8c92b940 copy: revert recent patch for vulnerable dirs
I plan to propose a better patch to catch vulnerable parent
directories.
* NEWS, doc/coreutils.texi (Target directory): Document this.
* src/cp.c, src/install.c, src/ln.c, src/mv.c:
Do not include targetdir.h.
(target_directory_operand): Remove test for vulnerable parents.
* src/cp.c (stat_target_operand): Remove.  All uses removed.
* src/local.mk (noinst_HEADERS): Remove src/targetdir.h.
(src_ginstall_SOURCES, src_cp_SOURCES, src_ln_SOURCES)
(src_mv_SOURCES): Remove src/targetdir.c.
* src/targetdir.c, src/targetdir.h: Remove.
* tests/mv/vulnerable-target.sh: Remove.
* tests/local.mk (all_root_tests): Remove it.
2017-09-24 23:10:56 -07:00
Pádraig Brady 63d2f05f52 tests: fix test hang on case insenitive file systems
* tests/split/filter.sh: Due to an invalid 'FILE = zero.in'
construct trying to initialize a FILE variable, it would
instead try to run the FILE command which is present on
macOS 10.13 with APFS.
We also remove a redundant duplicate test clause introduced
during a rebase, and simplify the piped timeout command,
to avoid requiring a subshell and associated quoting.
* THANKS.in: Add the reporter Jack Howarth.
Fixes https://bugs.gnu.org/28506
2017-09-24 12:37:39 -07:00
Pádraig Brady 3a3f5f359e tests: avoid a false failure in expr test with UTF8
* tests/misc/expr.pl: Skip the quote varying tests in
the multi-byte locales as these tests aren't that interesting
in those locales.  Also ERR_SUBST is already defined for
some tests so awkward to redefine to munge UTF8 quotes to ASCII.
2017-09-20 22:21:31 -07:00
Assaf Gordon 454d7f31c5 expr: add detailed syntax error messages
Show offending argument instead of a generic 'syntax error' message.
Suggested by Bernhard Voelker in https://bugs.gnu.org/28461#13 .

* src/expr.c (syntax_error): Remove.
(required_more_args): New function.
(eval7, main): Replace syntax_error call with detailed die message.
* tests/misc/expr.pl: Add tests for new messages.
2017-09-19 23:51:54 -06:00
Pádraig Brady a608f1e781 maint: fix new syntax-check failures from HTTPS adjustments
* cfg.mk [old_NEWS_hash]: update with `make update-NEWS-hash`.
[sc_long_lines]: Avoid flagging (long) URLs in NEWS.
* src/sort.c: Tweak to a shorter line.
* src/tail.c: Likewise.
Introduced in v8.28-4-gbe87d61
2017-09-19 22:32:00 -07:00
Pádraig Brady 0be99382d9 maint: fix new syntax check failures from copy restrictions
* doc/coreutils.texi: Remove doubled word.
* src/targetdir.c: Explicitly mark exported function.
* tests/local.mk: This is not a root only test.
* tests/mv/vulnerable-target.sh: Use returns_.
Introduced in v8.28-3-g44ccd1c
2017-09-19 22:21:28 -07:00
Pádraig Brady c34f8d5c78 shred: reinstate --remove file name length obfuscation
This was unintentionally removed in v8.27-60-g2ae1460
* src/shred.c (wipename): Interate through all name lengths.
* tests/misc/shred-remove.sh: Add test cases.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/28507
2017-09-19 21:24:38 -07:00
Paul Eggert 4cb3f4faa4 maint: copy bootstrap from Gnulib 2017-09-19 01:25:13 -07:00
Paul Eggert be87d61299 all: prefer HTTPS in URLs 2017-09-19 01:22:54 -07:00
Paul Eggert 44ccd1c465 copy: check for vulnerable target dirs
* NEWS, doc/coreutils.texi (Target directory): Document this.
* src/cp.c, src/install.c, src/ln.c, src/mv.c: Include targetdir.h.
(target_directory_operand): Use the new targetdir_operand_type
function to check for vulnerable target directories.
* src/cp.c (stat_target_operand): New function.
(target_directory_operand, do_copy): Use it.
* src/local.mk (noinst_HEADERS): Add src/targetdir.h.
(src_ginstall_SOURCES, src_cp_SOURCES, src_ln_SOURCES)
(src_mv_SOURCES): Add src/targetdir.c.
* src/targetdir.c, src/targetdir.h: New files.
* tests/mv/vulnerable-target.sh: New test.
* tests/local.mk (all_root_tests): Add it.
2017-09-19 00:14:30 -07:00
Bernhard Voelker 97c5045435 ptx: avoid infloop due to zero-length matches with -S regex
* src/ptx.c (find_occurs_in_text): Die with an appropriate error
diagnostic when the given regular expression returns a match of
length 0.
* tests/misc/ptx.pl (S-infloop): Add a test.
* NEWS (Bug fixes): Mention the fix.

Fixes https://bugs.gnu.org/28417 which was detected using
Symbolic Execution techniques developed in the course of the
SYMBIOSYS research project at COMSYS, RWTH Aachen University.
2017-09-14 07:50:27 +02:00
Pádraig Brady 5d9d07bbb1 maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2017-09-01 19:25:50 -07:00
Pádraig Brady e9d231112a version 8.28
* NEWS: Record release date.
v8.28
2017-09-01 18:53:58 -07:00
Pádraig Brady 38c2b405dd tests: fix false failure in recent ls --hyperlink test
* tests/ls/hyperlink.sh: If the hostname or any part of
the absolute path would be changed due to URL encoding,
the test would fail.  Therefore simplify to remove
these components of the URL from consideration.
2017-09-01 00:03:08 -07:00
Pádraig Brady c6f9c75c16 maint: avoid a syntax-check failure
* .gitignore: Remove lines indicated by sc_gitignore_redundant
in a freshly checked out repo.
2017-08-31 21:29:39 -07:00
Pádraig Brady 2ba96977c9 tests: exclude the expensive gnulib fts-tests
* gnulib: The only change in this gnulib update
is the tagging of the fts-tests module as longrunning,
which gnulib-tool currently implicitly excludes.
This test was seen to take about 20s and 285MB.
Reported by Assaf Gordon on space restricted VMs.
2017-08-30 22:49:24 -07:00
Pádraig Brady 173bee1bdf tty: don't distinguish input errors
* src/tty.c (main): Don't distinguish ENOTTY from other errors,
because isatty() doesn't portably distinguish errors.
Solaris returns ENOENT for all input errors for example.
Musl also returns ENOENT, and ENODEV may be returned as disscussed at:
http://openwall.com/lists/musl/2017/04/06/6
* tests/misc/tty.sh: Adjust accordingly.
2017-08-30 20:46:49 -07:00
Pádraig Brady 5ee9c8f7a6 tests: avoid printf '0*d' construct unsupported by ash
* tests/ln/sf-1.sh: Generate specific length with space padding
which is supported.
Reported by Assaf Gordon on Alpine Linux.
2017-08-30 20:39:54 -07:00
Pádraig Brady 1e8c458210 tests: skip tests upon failure to set SELinux context
On some setups the root:object_r:tmp_t context is invalid.
This does indicate a limitation in the test framework,
but for now we'll relax this to skipping the tests.
The tests still run on a Fedora 25 system for example.

* tests/cp/cp-a-selinux.sh: Upon chcon error, skip rather than ERROR.
* tests/install/install-Z-selinux.sh: Likewise.
* tests/misc/chcon.sh: Likewise.
* tests/misc/runcon-no-reorder.sh: Likewise.
* tests/misc/selinux.sh: Likewise.
* tests/mkdir/restorecon.sh: Likewise.
2017-08-30 17:18:17 -07:00
Kamil Dudka 87a95504bb expr: fix a recently introduced memory leak
* src/expr.c (eval6): Free memory allocated by mbs_logical_substr().

Introduced in v8.27-47-ga9f2be5.  Detected by Coverity Analysis:

Error: RESOURCE_LEAK:
src/expr.c:851: leaked_storage: Variable "s" going out of scope
leaks the storage it points to.
849|             char *s = mbs_logical_substr (l->u.s, pos, len);
850|             v = str_value (s);
851|->         }
852|         freev (l);
853|         freev (i1);
2017-08-30 12:04:29 -07:00
Pádraig Brady db8d1bb92c build: fix build of renameat2 on Alpine Linux
* gnulib: The only change included in this update
it the added check for the presence of <linux/fs.h>
which is not present on Alpine Linux by default.
2017-08-30 01:33:06 -07:00
Pádraig Brady d952949829 tty: fix exit code with EINVAL
* src/tty.c (main): All systems mention that isatty()
man return EINVAL as well as (the POSIX compliant) ENOTTY.
Also Centos 6 was seen to return EINVAL from ttyname().
* tests/misc/tty.sh: Fix a test issue where we assume
standard input is always a valid tty.
Reported by Assaf Gordon on OpenSolaris 5.10 and 5.11,
and Centos 6.5
2017-08-30 01:29:53 -07:00
Pádraig Brady f5d7c0842e runcon: revert "disable use of the TIOCSTI ioctl"
This reverts commit v8.27-97-g8cb06d4 because
the setsid() fallback was not implemented correctly
and disabling the ioctl was not a complete solution
to the security issue of the child being passed
the tty of the parent.

Given runcon is not really a sandbox command,
the advice is to use `runcon ... setsid ...`
to avoid this particular issue.
2017-08-30 00:28:20 -07:00
Pádraig Brady f169345506 stat: fix determination of max name length on BSD systems
We only use one of statfs or statvfs for `stat -f`
and on the BSDs we use statfs which doesn't have the
f_namelen member.  However on OpenBSD and later FreeBSD
systems statfs does provide f_namemax, so use that.

* NEWS: Mention the improvement for OpenBSD and FreeBSD.
* m4/stat-prog.m4: Check for f_namemax in the statfs struct.
* src/stat.c: Return '?' rather than '*' when we can't
determine the max length of the file system.
* tests/ln/sf-1.sh: This test was failing on all BSDs
due to '*' being returned for the max length which
caused the test to attempt to create 1Mi+1 names.
The test now uses a short name when we can't determine
the max name length to use.

Reported by Assaf Gordon on various BSD based systems.
2017-08-29 23:42:54 -07:00
Pádraig Brady 3ebdc3e1af stat,tail: support "AAFS" AppArmor file system
* src/stat.c (human_fstype): This file system is used
to manage AppArmor policy in the Linux kernel.
2017-08-29 01:04:32 -07:00
Pádraig Brady ba76db563f all: update gnulib submodule to latest
* bootstrap: Sync timestamp update.
2017-08-29 00:38:20 -07:00
Pádraig Brady 8cb06d4b44 runcon: disable use of the TIOCSTI ioctl
Similar to the issue with SELinux sandbox (CVE-2016-7545),
children of runcon can inject arbitrary input to the terminal
that would be run at the originating terminal privileges.

The new libseccomp dependency is widely available and used
on modern SELinux systems, but is not available by default
on older systems like RHEL6 etc.

* m4/jm-macros.m4: Check for libseccomp and
warn if unavailable on selinux supporting systems.
* src/local.mk: Link runcon with -lseccomp.
* src/runcon.c (disable_tty_inject): A new function to
disable use of the TIOCSTI using libseccomp, or with setsid()
where libseccomp is unavailable.
* tests/misc/runcon-no-inject.sh: A new test that uses
python to make the TIOCSTI call, and ensure that doesn't succeed.
* tests/local.mk: Reference the new test
* NEWS: Mention the fix.
Addresses http://bugs.gnu.org/24541
2017-08-29 00:38:19 -07:00
Pádraig Brady 799bac0d06 ls: support --hyperlink to output file:// URIs
Terminals such as iTerm2 and VTE based terminals
(as of version 0.49.1), support hyperlinks when
passed terminals codes as described at:
https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda

* src/ls.c (gobble_file): Allocate an absolute file name to output.
(quote_name): Output the absolute name with the appropriate codes.
(file_escape): A new function to encode file names as per rfc8089.
(main): Handle the new option and call the file_escape_init() helper.
Disable --dired when --hyperlink is specified.
(print_dir): Get the absolute file name here too, so that the
directory name can be linkified.
* NEWS: Mention the new feature.
* tests/ls/hyperlink.sh: Add a new test.
* tests/local.mk: Reference the new test.
* doc/coreutils.texi (ls invocation): Describe --hyperlink.
2017-08-29 00:38:19 -07:00
Pádraig Brady 4e112e90b1 doc: remove older ChangeLog items
This saves about 0.5MB uncompressed from the tarball.

* Makefile.am: Following on from v8.26-34-g2c64bc8
update the oldest documented version to 8.18 which
is now about 5 years old.  Also remove older ChangeLogs
that were previously thought to be for changes not
in the git history, but are adequately recorded upon review.
* build-aux/ChangeLog-2007: Remove file.
* lib/ChangeLog-2007: Likewise.
* m4/ChangeLog-2007: Likewise.
2017-08-29 00:38:19 -07:00
Colin Watson 57dea5ed07 env: add --chdir option
This is useful when chaining with other commands that run commands in a
different context, while avoiding using the shell to cd, and thus
having to consider shell quoting the chained command.

* NEWS (New features): Document the new option.
* doc/coreutils.texi (env invocation): Likewise.
* src/env.c (usage): Likewise.
(main): Implement the new option.
* tests/misc/env.sh: Test the new option.
2017-08-29 00:38:19 -07:00
Pádraig Brady 2686052667 tests: don't fail tests when failing to write files
* tests/sample-test: Use framework_error_ rather than fail=1
* tests/chown/deref.sh: Likewise.
* tests/chown/preserve-root.sh: Likewise.
* tests/cp/src-base-dot.sh: Likewise.
* tests/dd/unblock-sync.sh: Likewise.
* tests/du/2g.sh: Likewise.
* tests/du/inacc-dest.sh: Likewise.
* tests/du/one-file-system.sh: Likewise.
* tests/fmt/goal-option.sh: Likewise.
* tests/ln/hard-backup.sh: Likewise.
* tests/ls/color-dtype-dir.sh: Likewise.
* tests/ls/m-option.sh: Likewise.
* tests/ls/stat-dtype.sh: Likewise.
* tests/ls/time-style-diag.sh: Likewise.
* tests/ls/x-option.sh: Likewise.
* tests/misc/chcon.sh: Likewise.
* tests/misc/nohup.sh: Likewise.
* tests/misc/od-N.sh: Likewise.
* tests/misc/sort-compress.sh: Likewise.
* tests/misc/tac-continue.sh: Likewise.
* tests/misc/time-style.sh: Likewise.
* tests/mv/backup-dir.sh: Likewise.
* tests/mv/dir2dir.sh: Likewise.
* tests/rm/dir-no-w.sh: Likewise.
* tests/rm/dir-nonrecur.sh: Likewise.
* tests/rm/inaccessible.sh: Likewise.
* tests/rm/interactive-always.sh: Likewise.
* tests/rm/interactive-once.sh: Likewise.
* tests/rm/rm3.sh: Likewise.
* tests/rm/v-slash.sh: Likewise.
* tests/touch/relative.sh: Likewise.
2017-08-29 00:37:59 -07:00
Josef CejkaandBernhard Voelker a19ff5d817 df: avoid stat() for dummy file systems with -l
When systemd is configured to automount a remote file system - see
'man systemd.automount(5)', then the mount point is initially
mounted by systemd with the file system type "autofs".
When the resource is used later on, then the wanted file system is
mounted over that mount point on demand.
'df -l' triggered systemd to mount the file system because it called
stat() on the mount point.
Instead of single-casing "autofs" targets, we can avoid stat()ing
all dummy file systems (which includes "autofs"), because those are
skipped later on in get_dev() anyway.

*src/df.c (filter_mount_list): Also skip dummy file systems unless
the -a option or a specific target are given.
* NEWS: Mention the fix.

Co-authored-by: Bernhard Voelker <mail@bernhard-voelker.de>

Fixes http://bugzilla.suse.com/show_bug.cgi?id=1043059
2017-08-28 23:22:28 -07:00
Assaf Gordon 7449f0d0ab doc: add 'realpath usage examples' section
* doc/coreutils.texi (Realpath usage examples): New section.
2017-08-28 23:22:28 -07:00
Assaf Gordon ac48f7135e doc: fix realpath index entry
The 'readlink' node has '@findex realpath' in it. This results in
    info doc/coreutils.info realpath
incorrectly jumping to the 'readlink' node (instead of the 'realpath'
node). Change it to @cindex instead.

* doc/coreutils.texi (readlink): Change '@findex realpath' to @cindex.
2017-08-28 23:22:28 -07:00
Assaf Gordon 65b7bf790c realpath: improve usage description for --relative-{to,base}
* src/realpath.c (usage): Explicitly say 'DIR' instead of 'FILE' for
--relative-{to,base} parameters, to avoid giving the impression
that regular files can be used as relative base.
* doc/coreutils.texi (realpath): Same.
2017-08-28 23:22:28 -07:00
Pádraig Brady ac2eebc224 ls: consistently quote symlink targets
* src/ls.c (gobble_file): Disable the optimization to avoid quoting
if the symlink target itself needs quoting.  This was introduced
with the quoting alignment adjustments in v8.25-106-g01971c0
* tests/ls/symlink-quote.sh: Add a test.
* tests/local.mk: Reference the test.
* NEWS: Mention the fix.
2017-08-24 19:26:32 -07:00