Commit Graph
147 Commits
Author SHA1 Message Date
Pádraig Brady 3067a9293a maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2022-01-02 16:15:55 +00:00
Pádraig Brady e8b56ebd53 chmod: fix exit status when ignoring symlinks
* src/chmod.c: Reorder enum so CH_NOT_APPLIED
can be treated as a non error.
* tests/chmod/ignore-symlink.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/50784
2021-09-24 23:42:20 +01:00
Pádraig Brady 772173a832 cksum: support digest detection for tagged format
Support `cksum --check FILE` without having to specify a digest
algorithm, allowing for more generic file check instructions.
This also supports mixed digest checksum files, supporting
more robust multi digest checks.

* src/digest.c (algorithm_from_tag): A new function to
identify the digest algorithm from a tagged format line.
(split3): Set the algorithm depending on tag, and update
the expected digest length accordingly.
* tests/misc/cksum-c.sh: Add a new test.
* tests/local.mk: Reference the new test.
* tests/misc/md5sum.pl: Adjust to more generic error.
* tests/misc/sha1sum.pl: Likewise.
* doc/coreutils.texi (md5sum invocation): Mention the new -c feature.
* NEWS: Mention the new feature.
2021-09-15 20:44:18 +01:00
Pádraig Brady 8362f2552a digest: add support for sm3
Add message digest sm3, which uses the OSCCA SM3 secure
hash (OSCCA GM/T 0004-2012 SM3) generic hash transformation.

* bootstrap.conf: Add the sm3 module.
* doc/coreutils.texi: Mention the cksum -a option.
* src/digest.c: Provide support for --algorithm='sm3'.
* tests/misc/sm3sum.pl: Add a new test (from Tianjia Zhang)
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.

Tested-by: Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
2021-09-15 20:44:18 +01:00
Pádraig Brady ad6c8e1181 cksum: add --algorithm option to select digest mode
* src/digest.c: Organize HASH_ALGO_CKSUM to be table driven,
and amalgamate all digest algorithms.
(main): Parse all options if HASH_ALGO_CKSUM, and disallow
--tag, --zero, and --check with the traditional bsd, sysv, and crc
checksums for now.
* src/local.mk: Reorganize to include all digest modules in cksum.
* tests/misc/cksum-a.sh: Add a new test.
* tests/misc/b2sum.sh: Update to default to checking with cksum,
as b2sum's implementation diverges a bit from the others.
* tests/local.mk: Reference the new test.
* doc/coreutils.texi (cksum invocation): Adjust the summary to
identify the new mode, and document the new --algorithm option.
* man/cksum.x: Adjust description to be more general.
* man/*sum.x: Add [See Also] section referencing cksum(1).
* NEWS: Mention the new feature.
2021-09-15 20:44:16 +01:00
Paul Eggert c75779cac3 ls: rename --null to --zero (Bug#49716)
* NEWS, doc/coreutils.texi (General output formatting):
* src/ls.c (usage):
Document this.
* src/ls.c (ZERO_OPTION): Rename from NULL_OPTION.
All uses changed.
(long_options): Rename --null to --zero.
(dired_dump_obstack, main, print_dir): Use '\n' instead of
eolbyte where eolbyte must equal '\n'.
(decode_switches): Decode --zero instead of --null.
--zero also implies -1, -N, --color=none, --show-control-chars.
Use easier-to-decipher code to set ‘format’ and ‘dired’.
Reject attempts to combine --dired and --zero.
* tests/local.mk: Adjust to test script renaming.
* tests/ls/zero-option.sh: Rename from tests/ls/null-option.sh,
and test --zero instead of --null.
2021-07-27 18:08:39 -07:00
Paul Eggert 7d44ac224c ls: add --null option (Bug#49716)
* NEWS, doc/coreutils.texi (General output formatting):
* src/ls.c (usage): Document this.
* src/ls.c (NULL_OPTION): New constant.
(long_options): Add --null.
(eolbyte): New static var.
(dired_dump_obstack, main, print_dir, print_current_files)
(print_many_per_line, print_horizontal, print_with_separator):
Output eolbyte instead of '\n'.
(decode_switches): Decode --null.
* tests/ls/null-option.sh: New file.
* tests/local.mk (all_tests): Add it.
2021-07-26 00:59:37 -07:00
Pádraig Brady 4eb2e9c11a tests: ensure we test SEEK_DATA where used
fiemap is no longer the default copy implementation,
so check for SEEK_DATA support instead as that's preferred.
This will ensure better test coverage on systems without fiemap.

* init.cfg: Replace fiemap_capable_ with seek_data_capable_.
This is best supported with python 3 so prefer that.
* tests/seek-data-capable: A new test script checking for
SEEK_DATA support on the passed file name,
called from seek_data_capable_.
* tests/fiemap-capable: Remove no longer used probing script.
* tests/cp/fiemap-perf.sh: Renamed to tests/cp/sparse-perf.sh
* tests/cp/fiemap-2.sh: Renamed to tests/cp/sparse-2.sh
* tests/cp/fiemap-extents.sh: Renamed to tests/cp/sparse-extents.sh
* tests/cp/sparse-fiemap.sh: Renamed to tests/cp/sparse-extents-2.sh
* tests/cp/fiemap-FMR.sh: Renamed to tests/cp/copy-FMR.sh
* tests/local.mk: Reference the renamed tests.
2021-05-12 22:14:11 +01:00
Carl Edquist 751ad58e00 ls: add --sort=width option to sort by file name width
This helps identify the outliers for long filenames, and also produces
a more compact display of columns when listing a directory with many
entries of various widths.

* src/ls.c (sort_type, sort_types, sort_width): New sort_width sort
type.
(sort_args): Add "width" sort arg.
(cmp_width, fileinfo_width): New sort function and helper for file name
width.
(quote_name_width): Add function prototype declaration.
(usage): Document --sort=width option.
* doc/coreutils.texi: Document --sort=width option.
* tests/ls/sort-width-option.sh: New test for --sort=width option.
* tests/local.mk: Reference new test.
* NEWS: Mention the new feature.
2021-04-11 14:58:38 +01:00
Pádraig Brady 9de1d153f8 rmdir: diagnose non following of symlinks with trailing slash
GNU/Linux is unusual here in that rmdir("symlink/") returns ENOTDIR,
whereas Solaris and FreeBSD at least, will follow the symlink
and remove the target directory.  We don't make the behavior
on Linux kernels consistent, but at least clarify
the confusing error message.

* src/rmdir (main): Output a specific error message for the above case.
(remove_parents): In the error message, don't assume intermediate paths
are directories, as they could be symlinks.
* tests/rmdir/symlink-errors.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the improvement.
2021-02-19 11:35:30 +00:00
Pádraig Brady 2de30c7350 cat: extend --show-ends to show \r\n as ^M$
- \r\n is common a line end combination
  - catting such a file without options causes it to display normally
  - overwriting the first char with $, loses info

* src/cat.c (cat): Convert \r preceeding a \n to ^M.
* tests/misc/cat-E.sh: New test.
* tests/local.mk: Reference new test.
* tests/misc/cat-proc.sh: Fix typo.
* doc/coreutils.texi (cat invocation): Mention the new behavior.
* NEWS: Mention the improvement.
2021-02-10 14:17:26 +00:00
Pádraig Brady 32ff1d1313 maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2021-01-01 16:36:09 +00:00
Pádraig Brady ffb7ebfa25 tests: add a test for cksum
* tests/misc/cksum.sh: Test basic operation.
* tests/local.mk: Reference the new test.
2020-12-28 16:21:38 +00:00
Nishant Nayan 6bf108358a rm: do not skip files upon failure to remove an empty dir
When removing a directory fails for some reason, and that directory
is empty, the rm_fts code gets the return value of the excise call
confused with the return value of its earlier call to prompt,
causing fts_skip_tree to be called again and the next file
that rm would otherwise have deleted to survive.

* src/remove.c (rm_fts): Ensure we only skip a single fts entry,
when processing empty dirs.  I.e. only skip the entry
having successfully removed it.
* tests/rm/empty-immutable-skip.sh: New root-only test.
* tests/local.mk: Add it.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/44883
2020-11-26 16:51:38 +00:00
Pádraig Brady 6fc695cb4a ls: fix crash printing SELinux context for unstatable files
This crash was identified by Cyber Independent Testing Lab:
https://cyber-itl.org/2020/10/28/citl-7000-defects.html
and was introduced with commit v6.9.90-11-g4245876e2

* src/ls.c (gobble_file): Ensure scontext is initialized
in the case where files are not statable.
* tests/ls/selinux-segfault.sh: Renamed from proc-selinux-segfault.sh,
and added test case for broken symlinks.
* tests/local.mk: Adjust for the renamed test.
* NEWS: Mention the bug fix.
2020-11-11 17:28:54 +00:00
Jason Kim 9c8a385aa4 ls: allow --classify to be ignored for non tty output
Have the `ls` `--classify` option take an optional argument for when to
classify ("always", "auto", "never"), just like the optional argument
for `--color`.  When the optional argument is not specified, default to
"always" for backwards compatibility.

* src/ls.c (usage): Update help text.
(decode_switches): Support an optional argument for --classify.
* tests/ls/classify.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
2020-04-27 13:48:32 +01:00
Pádraig Brady f2034aa9bb tests: avoid a false failure on OpenIndiana 11
* tests/misc/timeout-parameters.sh: Split the large timeout
handling to ...
* tests/misc/timeout-large-parameters.sh: ... here, so that
the 3 second delay is contained in its own test, and if
the test is skipped due invalid handling within timeout(1),
it will be more apparent.
Also adjust the check so we skip whenever the kernel timer
fires immediately, to handle the buggy OpenIndiana 11 kernel also.
Reported by Bruno Haible.
2020-03-01 23:36:05 +00:00
Colin Watson 05a99f7d7f ls: issue error message on removed directory
If the current directory has been removed, then "ls" confusingly
produced no output and no error message, indistinguishable from
running on an empty directory.

* src/ls.c (print_dir): Report ENOENT on GNU/Linux if readdir
finds no directory entries at all, not even "." or "..",
and a recheck with the getdents syscall returns ENOENT.
We recheck with getdents() as POSIX states that
"The directory entries for dot and dot-dot are optional".
* tests/ls/removed-directory.sh: New file.
* tests/local.mk (all_tests): Add new test.
* NEWS: Mention the change in behavior.
Reported by Owen Thomas.
2020-02-27 14:11:37 +00:00
Pádraig Brady 8e81d44b52 uniq: avoid strcoll() to improve performance and consistency
strcoll() is only significant to uniq(1) if it returns 0,
and it generally only does so with buggy locales or mismatched
locales and data.  Some systems may have strcoll()
return 0 for equivalent normalized unicode forms,
but for consistency across platforms strcoll() is avoided.
The various cases are defined in the new test.
This is consistent with newer POSIX standards as discussed at:
https://www.austingroupbugs.net/view.php?id=963

* src/uniq.c: s/xstrcoll/memcmp/.
* tests/local.mk: Reference the new test.
* tests/misc/uniq-collate.sh: Add a new test.
* NEWS: Mention the change in behavior.
Fixes https://bugs.gnu.org/38627
2020-02-24 10:12:42 +00:00
Pádraig Brady 2cecc3cc99 ls: support --time=creation to show/sort birth time
* src/ls.c (usage): Reorganize help for --time,
and add description for --time=birth.
(do_statx): Store btime in mtime if available.
(get_stat_btime): A new function to read the creation time
from the appropriate stat structure member.
(cmp_btime): A new function to compare birth time.
(print_long_format): Output '?' when birth time unavailable.
* doc/coreutils.texi: Document --time={birth,creation}.
* tests/local.mk: Reference the new test.
* tests/ls/birthtime.sh: Add a new test.
* NEWS: Mention the new feature.
2020-02-03 19:37:56 +00:00
Pádraig Brady aaba82431c maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2020-01-01 14:16:56 +00:00
Assaf Gordon 07f811a3c0 seq: fix superfluous output line
Under certain circumstances seq prints an extra line when the output
format has custom format with characters following the printed numbers:

    $ seq -f "%g " 1000000 1000000
     1e+06
     1e+06

This is due to the "print_extra_number" logic using strings to determine
whether a 'extra number' is needed, but only one string was trimmed
when using a custom printf format.

Prompted by https://lists.gnu.org/r/coreutils/2019-08/msg00001.html

* NEWS: Mention fix.
* src/seq.c (print_numbers): Trim the 'x0_str' string before comparing
it to the previous 'x_str' string.
* tests/misc/seq-extra-number.sh: Add this scenario.
* tests/local.mk (all_tests): Add new test.
2019-08-02 18:29:25 -06:00
Assaf Gordon 95adadd9a4 env: new options --{default,ignore,block}-signal[=SIG]
New options to set signal handlers for the command being executed.
--block-signal suggested by Paul Eggert in http://bugs.gnu.org/34488#71
--default-signal is useful to overcome the POSIX limitation that shell
must not override inherited signal state, e.g. the second 'trap' here is
a no-op:

   trap '' PIPE && sh -c 'trap - PIPE ; seq inf | head -n1'

Instead use:

   trap '' PIPE && sh -c 'env --default-signal=PIPE seq inf | head -n1'

Similarly, the following will prevent CTRL-C from terminating the
program:

   env --ignore-signal=INT seq inf > /dev/null

See https://bugs.gnu.org/34488#8

* NEWS: Mention new options.
* doc/coreutils.texi (env invocation): Document new options.
* man/env.x: Add example of --default-signal=SIG usage.
(SEE ALSO): Mention sigprocmask.
* src/env.c (signals): New global variable.
(longopts): Add new options.
(usage): Print new options.
(parse_signal_params): Parse comma-separated list of signals, store in
signals variable.
(reset_signal_handlers): Set each signal to SIG_DFL/SIG_IGN.
(parse_block_signal_params): Parse command-line options.
(set_signal_proc_mask): Call sigprocmask to block/unblock signals.
(main): Process new options.
* src/local.mk (src_env_SOURCES): Add operand2sig.c.
* tests/misc/env-signal-handler.sh: New test.
* tests/local.mk (all_tests): Add new test.
2019-03-04 00:37:07 -08:00
Pádraig Brady a5202bd585 wc: treat non breaking space as a word separator
* src/wc.c (iswnbspace): A new function to match
characters in this class.
(isnbspace): Likewise for single byte charsets.
(main): Initialize posixly_correct from the environment,
to allow disabling honoring NBSP in non C locales.
(wc): Call is[w]nbspace() along with is[w]space.
* bootstrap.conf: Ensure btowc is available.
* tests/misc/wc-nbsp.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the change in behavior.
2019-02-25 22:49:44 -08:00
Bernhard Voelker 44af84263e all: detect --help and --version more consistently
For select programs which accept only --help and --version options
(in addition to non-option arguments), process these options before
any other options.

Before:

  $ dd bs=1 --help
  dd: unrecognized option '--help'
  Try 'dd --help' for more information.

  $ yes me --help
  me --help
  me --help
  ...

After:
Any occurrence of '--help' in the arguments (prior to '--') will
show the help screen.

Discussed in https://bugs.gnu.org/33468 .

* NEWS: Mention change.
* src/cksum.c, src/dd.c, src/hostid.c, src/hostname.c, src/link.c,
src/logname.c, src/nohup.c, src/sleep.c, src/tsort.c, src/unlink.c,
src/uptime.c, src/users.c, src/whoami.c, src/yes.c (main): Replace
parse_long_options() + getopt_long() calls with
parse_gnu_standard_options_only(); Remove <getopt.h> inclusion;
Remove empty 'struct long_options' variable;
* tests/misc/help-version-getopt.sh: Add test.
* tests/local.mk (all_tests): Reference it.
2019-02-24 19:32:29 +01:00
Pádraig Brady 03fbb5ecec seq: output decimal points consistently with invalid locales
* src/seq.c (print_numbers): Only reset the locale if it
was successfully set originally.
* tests/misc/seq-locale.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the fix.
2019-02-03 17:18:08 -08:00
Pádraig Brady 39ca7731e2 ls: with --group-directories-first, also group symlinked dirs
* src/ls.c (is_linked_directory): A new function to
also consider symlinked directories.
(main): Rename check_symlink_color to check_symlink_mode,
and enable that with --group-directories-first.
(DIRFIRST_CHECK): Adjust to use is_linked_directory,
rather than just is_directory.
(gobble_file): Simplify to always update f->linkmode
if the stat() succeeds.
* tests/ls/group-dirs.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the change in behavior.
Suggested by Amin Bandali in
https://lists.gnu.org/r/coreutils/2018-12/msg00017.html
2019-01-12 19:23:45 -08:00
Assaf Gordon 8dfcf38af1 maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2019-01-01 16:51:48 -07:00
Assaf Gordon 2f438fa9f5 basenc: A new program complementary to base64/base32
Encodes/decodes data in various common formats:
base64,base64url,base32,base32,base16,base2,z85.

Discussed here:
https://lists.gnu.org/r/coreutils/2018-11/msg00014.html
https://lists.gnu.org/r/coreutils/2018-12/msg00019.html

* AUTHORS: Add basenc.
* README: Reference the new program.
* NEWS: Mention the new program.
* build-aux/gen-lists-of-programs.sh: Add basenc.
* doc/coreutils.texi: (basenc invocation): Document the new command.
* man/.gitignore: Ignore the generated man page.
* man/basenc.x: A new template, with few examples.
* man/local.mk: Reference the new man page.
* scripts/git-hooks/commit-msg: Allow basenc as program prefix.
* src/.gitignore: Ignore the new binary.
* src/basenc.c:
(usage): Mention new options.
(main): Handle new options.
(isbase*, base*_length, base*_encode, base*_decode_ctx): Implement new
encoding/decoding formats.
* src/local.mk: Add new program.
* tests/local.mk: Add new test.
* tests/misc/basenc.pl: New tests.
* tests/misc/help-version.sh (basenc_setup): use '--version' for default
invocation (basenc errors with no parameters).
2018-12-27 23:46:02 -07:00
Pádraig Brady 8d328706c6 echo: always process escapes when POSIXLY_CORRECT is set
* src/echo.c (main): Always enable backslash processing if
POSIXLY_CORRECT is set.
* tests/misc/echo.sh: Add (the first) test for the echo command.
* tests/local.mk: Reference the new test.
* tests/misc/printf.sh: Update a stale comment.
* doc/coreutils.texi (echo invocation). Mention that POSIXLY_CORRECT
now always enables backslash processing.
* NEWS: Mention the change in behavior.
Fixes https://bugs.gnu.org/32703
Issue identified by Eric Blake.
2018-10-27 03:05:24 -07:00
Bernhard Voelker 1a674a036b test: add -N unary operator
Bash knows 'test -N FILE'.  Add it to GNU 'test' as well.

* src/test.c (unary_operator): Add a case for 'N'.
(usage): Document it.
* doc/coreutils.texi (node File characteristic tests): Likewise.
* NEWS (New features): Likewise.
* tests/misc/test-N.sh: Add a test.
* tests/local.mk (all_tests): Reference it.
2018-10-26 11:05:25 +02:00
Pádraig Brady d542945c3a tests: fix false failures when perl not available
* tests/local.mk: Reference the stub that skips perl tests,
with the correct path.
2018-06-27 02:38:59 -07:00
Bernhard Voelker 5fb8eb091f tests: initialize fail=0 to avoid "unary operator expected" errors
With an uninitialized variable 'fail', the unquoted use like
  test $fail = 1
lead to the shell error
  "unary operator expected".

The uninitialized 'fail' variable was a side effect of
  https://git.sv.gnu.org/cgit/gnulib.git/commit/?id=e91c0d4f9
which was pulled into coreutils-v8.26 with
  https://git.sv.gnu.org/cgit/coreutils.git/commit/?id=ef9650170
Coreutils test code relied and relies on 'fail' to be initialized,
so initialize that variable here.

* tests/local.mk (TESTS_ENVIRONMENT): Initialize fail=0.
2018-06-23 22:37:42 +02:00
Assaf Gordon 668306ed86 env: add -S/--split-string option
Adopted from FreeBSD's env(1), useful for specifing multiple
parameters on a shebang (#!) script line, e.g:

   #!/usr/bin/env -S perl -w -T

Discussed in https://lists.gnu.org/r/coreutils/2018-04/msg00011.html

* src/env.c (valid_escape_sequence,escape_char,scan_varname,
  extract_varname,validate_split_str,build_argv,
  parse_split_string): New functions.
  (main): Process new option and call parse_split_string.
  (usage): Mention new option.
* tests/misc/env-S.pl: Test new option from the command line.
* tests/misc/env-S-script.sh: Test new option from shebang scripts.
* tests/local.mk (all_tests): Add new tests.
* man/env.x (OPTIONS): Show a brief example of -S usage and point to
the full documentation for more information.
* doc/coreutils.texi (env invocation): Detail usage of -S/--split-string
option.
* NEWS: Mention new option.
2018-06-20 21:15:31 -07:00
Kaxandra Labat 24053fbd8f ls: ignore case when coloring file extensions
* src/ls.c (get_color_indicator): s/STREQ_LEN/c_strncasecmp/
* src/dircolors.hin: Remove a now redundant entry.
* tests/ls/color-ext.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the change in behavior.
2018-06-20 21:02:30 -07:00
Pádraig Brady d3daa95096 cp: fix symlink checks when overwriting files
Ensure this _does_ recreate the symlink
  Given "path1" and "path2" are on different devices.
  $ touch "path1/file"
  $ cd path2/; ln -s path1/file
  $ cp -dsf path1/file .

Ensure this does _not_ overwrite file
  $ touch file
  $ ln -s file l1
  $ cp -sf l1 file

* src/copy.c (same_file_ok): Remove device ids from consideration,
instead deferring to future EXDEV with --link or allowing
the first case above to work.
Also ensure that we do not exist this function too early,
when the destination file is not a symlink, which protects
against the second case.
* tests/cp/cross-dev-symlink.sh: Add a test for the first case.
* tests/cp/same-file.sh: Add a test for the second case above.
* NEWS: Mention the bug fixes.
* THANKS.in: Mention the reporters who also analyzed the code.
Fixes https://bugs.gnu.org/31364
2018-05-15 09:55:19 -07:00
Paul Eggert 8d6acfd853 ls: -A now overrides -a
Problem reported by Karl Berry (Bug#30963).
* NEWS: Mention this.
* src/ls.c (decode_switches): Implement this.
* tests/ls/a-option.sh: New file.
* tests/local.mk (all_tests): Add it.
2018-03-27 15:05:22 -07:00
Pádraig Brady ece7157933 maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2018-01-01 15:57:22 +00:00
Bernhard VoelkerandPádraig Brady 052cce9e3e tests: verify usage vs. getopt
Verify that all options mentioned in usage are actually recognized
by the program.

* tests/misc/usage_vs_getopt.sh: Add test.
* tests/local.mk (all_tests): Reference it.

Co-authored-by: Pádraig Brady <P@draigBrady.com>
2017-11-29 13:29:00 +01:00
Pádraig Brady 31dd7a0de2 tail: seek to the end of block devices
* src/tail.c (tail_bytes): Try lseek(..., SEEK_END) when
we can't determine the file size.
* tests/tail-2/end-of-device.sh: Add a new root only test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the improvement.
Paul Eggert suggested using lseek() (rather than ioctl(BLKGETSIZE64)).
Fixes https://bugs.gnu.org/29259
2017-11-18 22:26:02 -08:00
Pádraig Brady de15a497d1 dd: fix nocache regions passed to posix_fadvise()
Previously with oflag=direct the call to invalidate_cache()
was not passed to the kernel, as it was less than a page size,
and a subsequent call was not made to invalidate the pending space.
Similarly with oflag=nocache the pending space at EOF was
not invalidated.  Even though these amount to only a single page
in the page cache it can be significant.  For example on
XFS before kernel patch v4.9-rc1-4-g0ee7a3f, O_DIRECT files
would have been read inefficiently if any pages were cached,
even if they were already synced to storage.

* src/dd.c (i_nocache_eof, o_nocache_eof): New bools used
to control when we want invalidate_cache(,0) to clear to EOF.
(cache_round): Use IO_BUFSIZE (currently 132KiB) to minimize
calls to the relatively expensive advise function, rather
than page_size.  This also makes it clear that while the
kernel function operates on pages, this size is chosen for
performance reasons.
(invalidate_cache): Refactor to share more code between
input and output paths. Use i_nocache_eof and o_nocache_eof
rather than proxying off max_records.  Ensure we
invalidate full pages when clearing to EOF as the kernel
will ignore any non complete pages.  Fix the offset used
for the output path.
(dd_copy): Invalidate the cache of the input after the
offset is updated, for consistency and so we don't try to
invalidate before the start of the file.  When we read
EOF on input, set flags so that we invalidate to EOF.
(main): Invalidate to EOF in more cases, by depending
on the i_nocache_eof and o_nocache_eof flags.
* doc/coreutils.texi (dd invocation): Clarify the alignment
and persisted caveats on the example applying "nocache"
to part of a file.
* tests/dd/nocache_eof.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Issue reported by Eric Bergen.
2017-10-24 19:38:12 -07:00
Paul Eggert 4b8c92b940 copy: revert recent patch for vulnerable dirs
I plan to propose a better patch to catch vulnerable parent
directories.
* NEWS, doc/coreutils.texi (Target directory): Document this.
* src/cp.c, src/install.c, src/ln.c, src/mv.c:
Do not include targetdir.h.
(target_directory_operand): Remove test for vulnerable parents.
* src/cp.c (stat_target_operand): Remove.  All uses removed.
* src/local.mk (noinst_HEADERS): Remove src/targetdir.h.
(src_ginstall_SOURCES, src_cp_SOURCES, src_ln_SOURCES)
(src_mv_SOURCES): Remove src/targetdir.c.
* src/targetdir.c, src/targetdir.h: Remove.
* tests/mv/vulnerable-target.sh: Remove.
* tests/local.mk (all_root_tests): Remove it.
2017-09-24 23:10:56 -07:00
Pádraig Brady 0be99382d9 maint: fix new syntax check failures from copy restrictions
* doc/coreutils.texi: Remove doubled word.
* src/targetdir.c: Explicitly mark exported function.
* tests/local.mk: This is not a root only test.
* tests/mv/vulnerable-target.sh: Use returns_.
Introduced in v8.28-3-g44ccd1c
2017-09-19 22:21:28 -07:00
Paul Eggert be87d61299 all: prefer HTTPS in URLs 2017-09-19 01:22:54 -07:00
Paul Eggert 44ccd1c465 copy: check for vulnerable target dirs
* NEWS, doc/coreutils.texi (Target directory): Document this.
* src/cp.c, src/install.c, src/ln.c, src/mv.c: Include targetdir.h.
(target_directory_operand): Use the new targetdir_operand_type
function to check for vulnerable target directories.
* src/cp.c (stat_target_operand): New function.
(target_directory_operand, do_copy): Use it.
* src/local.mk (noinst_HEADERS): Add src/targetdir.h.
(src_ginstall_SOURCES, src_cp_SOURCES, src_ln_SOURCES)
(src_mv_SOURCES): Add src/targetdir.c.
* src/targetdir.c, src/targetdir.h: New files.
* tests/mv/vulnerable-target.sh: New test.
* tests/local.mk (all_root_tests): Add it.
2017-09-19 00:14:30 -07:00
Pádraig Brady f5d7c0842e runcon: revert "disable use of the TIOCSTI ioctl"
This reverts commit v8.27-97-g8cb06d4 because
the setsid() fallback was not implemented correctly
and disabling the ioctl was not a complete solution
to the security issue of the child being passed
the tty of the parent.

Given runcon is not really a sandbox command,
the advice is to use `runcon ... setsid ...`
to avoid this particular issue.
2017-08-30 00:28:20 -07:00
Pádraig Brady 8cb06d4b44 runcon: disable use of the TIOCSTI ioctl
Similar to the issue with SELinux sandbox (CVE-2016-7545),
children of runcon can inject arbitrary input to the terminal
that would be run at the originating terminal privileges.

The new libseccomp dependency is widely available and used
on modern SELinux systems, but is not available by default
on older systems like RHEL6 etc.

* m4/jm-macros.m4: Check for libseccomp and
warn if unavailable on selinux supporting systems.
* src/local.mk: Link runcon with -lseccomp.
* src/runcon.c (disable_tty_inject): A new function to
disable use of the TIOCSTI using libseccomp, or with setsid()
where libseccomp is unavailable.
* tests/misc/runcon-no-inject.sh: A new test that uses
python to make the TIOCSTI call, and ensure that doesn't succeed.
* tests/local.mk: Reference the new test
* NEWS: Mention the fix.
Addresses http://bugs.gnu.org/24541
2017-08-29 00:38:19 -07:00
Pádraig Brady 799bac0d06 ls: support --hyperlink to output file:// URIs
Terminals such as iTerm2 and VTE based terminals
(as of version 0.49.1), support hyperlinks when
passed terminals codes as described at:
https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda

* src/ls.c (gobble_file): Allocate an absolute file name to output.
(quote_name): Output the absolute name with the appropriate codes.
(file_escape): A new function to encode file names as per rfc8089.
(main): Handle the new option and call the file_escape_init() helper.
Disable --dired when --hyperlink is specified.
(print_dir): Get the absolute file name here too, so that the
directory name can be linkified.
* NEWS: Mention the new feature.
* tests/ls/hyperlink.sh: Add a new test.
* tests/local.mk: Reference the new test.
* doc/coreutils.texi (ls invocation): Describe --hyperlink.
2017-08-29 00:38:19 -07:00
Pádraig Brady ac2eebc224 ls: consistently quote symlink targets
* src/ls.c (gobble_file): Disable the optimization to avoid quoting
if the symlink target itself needs quoting.  This was introduced
with the quoting alignment adjustments in v8.25-106-g01971c0
* tests/ls/symlink-quote.sh: Add a test.
* tests/local.mk: Reference the test.
* NEWS: Mention the fix.
2017-08-24 19:26:32 -07:00
Jim Meyering 83ed661947 groups: do not exit early
Most programs take care to operate on all command-line-specified
operands before exiting.  That is an important feature that allows
to identify all problems with the first run.  However, groups would
exit upon the first problematic user name.
Bug introduced via commit v6.10-56-g167b8025ac.
* src/groups.c (main): Do not exit immediately upon error.
* tests/misc/groups-process-all.sh: New file. Test for this.
* tests/local.mk (all_tests): Add it.
* NEWS (Bug fixes): Mention this.
2017-07-10 10:17:57 +02:00