Commit Graph
8161 Commits
Author SHA1 Message Date
Kristoffer Brånemyr a7533917e0 cksum: use more efficient slice by 8 algorithm
A 100MB file improves from 2.50s to 1.80s on a Sparc T5220
A 100MB file improves from 0.54s to 0.13s on an i3-2310M

* bootstrap.conf: Explicitly depend on byteswap,
since now used directly by coreutils.
* src/cksum.c (cksum): Process in multiples of 8 bytes.
(main): Adjust for generation of expanded crctab.
* src/cksum.h: Split now larger crctab to separate header.
* src/local.mk: Reference the new header.
* NEWS: Mention the improvement.
2020-12-28 16:21:38 +00:00
Paul Eggert fb64712c4d build: update gnulib submodule to latest
* src/make-prime-list.c (free): Undef, since Gnulib's free-posix
module now defines this to rpl_free on some platforms.
2020-12-24 22:03:13 -08:00
Pádraig Brady e5cb4aaa0b doc: add seq inf and sleep inf examples to texinfo
* doc/coreutils.texi (seq invocation): Mention "inf" is supported,
and describe that it's handled specially to generate infinite
whole integer sequences.  Also mention that such infinite generation
is supported for integer steps up to 200.
(sleep invocation): Give `sleep inf` as an example to sleep forever.
* src/seq.c: Add a comment on SEQ_FAST_STEP_LIMIT, to say it's
reflected in the texinfo description.
2020-12-18 14:54:57 +00:00
Paul Eggert 3e61d5dd31 doc: document mkdir -m -p better
Chris Colohan wrote that the man page did not do enough to dispel
a common misunderstanding that “contributed to one of the scariest
outages Google has ever seen” (Bug#45258).
* doc/coreutils.texi (mkdir invocation):
* src/mkdir.c (usage): Document -m vs -p better.
2020-12-15 11:58:48 -08:00
KOBAYASHI Takashi 567fc2c2ad nl: fix --section-delimiter handling of single characters
* src/nl.c (main): Enforce the POSIX specified
behavior of assuming ':' is specified after a single
character argument to -d.
* tests/misc/nl.sh: Add a test case.
* NEWS: Mention the bug fix.
2020-12-15 14:11:55 +00:00
Pádraig Brady 1935528ce9 doc: mention the GNU extensions to nl --section-delimiter
* doc/coreutils.texi (nl invocation): Mention the GNU extensions
of allowing arbitrary length and empty delimiter strings.
* src/nl.c (usage): Likewise.
* tests/misc/nl.sh: Add test cases for the GNU extensions.
2020-12-15 14:11:51 +00:00
Pádraig Brady b0d527fb40 maint: refactor nl section delimiter handling
* src/nl.c (main): Update the default delimiter characters
when passed two characters with --section-delimiter.
Avoid redundant copies for the body and footer delimiter strings,
and instead, just offset into the header string.
(check_section): Avoid redundant comparing of 2 bytes of memory
for an empty delimiter.
2020-12-15 13:17:39 +00:00
Arman Absalan 14562f8b20 chroot,comm,join: fix usage options style
* src/chroot.c (usage): Fix indentation of options.
* src/comm.c: Likewise.
* src/join.c: Likewise.
2020-12-08 18:08:17 +01:00
Pádraig Brady e361bff8a5 date: with --debug, show the output format
The format can be determined from --options or the locale,
so it's useful to output the format string being used.

* src/date.c (show_date): Show the output format
along with the date being shown.
* tests/misc/date-debug.sh: Adjust accordingly.
Addresses https://bugs.gnu.org/44960
2020-12-01 13:51:06 +00:00
Tim Gates 5ed804a9b8 maint: fix typo, characteres -> characters
* src/expr.c: Fix typo in comment.
2020-11-27 19:56:02 +00:00
Nishant Nayan 6bf108358a rm: do not skip files upon failure to remove an empty dir
When removing a directory fails for some reason, and that directory
is empty, the rm_fts code gets the return value of the excise call
confused with the return value of its earlier call to prompt,
causing fts_skip_tree to be called again and the next file
that rm would otherwise have deleted to survive.

* src/remove.c (rm_fts): Ensure we only skip a single fts entry,
when processing empty dirs.  I.e. only skip the entry
having successfully removed it.
* tests/rm/empty-immutable-skip.sh: New root-only test.
* tests/local.mk: Add it.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/44883
2020-11-26 16:51:38 +00:00
Pádraig Brady e53bb7ad05 maint: remove no longer needed se_const helper
This was needed before libselinux-2.3 (May 2014),
but modern releases have the correct const declarations.

* src/chcon.c: Remove se_const() wrapper.
* src/cp.c: Likewise.
* src/install.c: Likewise.
* src/mkdir.c: Likewise.
* src/mkfifo.c: Likewise.
* src/mknod.c: Likewise.
* src/system.h: Likewise.
* gnulib: update to pick up const correctness fixes in selinux stubs.
2020-11-23 22:25:55 +00:00
Paul Eggert d12f5da6d4 install: suppress "Operation not supported" false alarms
At least, I *think* they are false alarms.  An SELinux expert eye
would be welcome.
* src/install.c (setdefaultfilecon): If selabel_lookup fails
due to either ENOTSUP or ENODATA, don’t diagnose the issue.
Problem reported by Kamil Dudka in:
https://lists.gnu.org/r/coreutils/2020-11/msg00050.html
2020-11-23 01:48:45 -08:00
Paul Eggert b8a543fe02 maint: propagate errno better in selinux.c
* src/selinux.c: Don’t include die.h; no longer needed.
(computecon, defaultcon, restorecon): Propagate errno.
(defaultcon, restorecon): Don’t diagnose errors or exit, as that’s
the caller’s responsibility.
2020-11-23 01:00:23 -08:00
Pádraig Brady 7b341f084b maint: use absolute paths with selabel_lookup
* src/selinux.c: selabel_lookup requires absolute paths
(while only older matchpathcon before libselinux < 2.1.5 2011-0826 did).
* po/POTFILES.in: Readd src/selinux.c since we now have
a translatable error message.
2020-11-23 01:00:22 -08:00
Bernhard Voelker 344cf0ee5f maint: minor cleanup
The previous commit introduced a couple of syntax-check failures.

* .gitignore (/lib/se-label.h): Add entry to silence the
sc_gitignore_missing check.  Sort entries in C locale.
* po/POTFILES.in (src/selinux.c): Remove entry as this source doesn't
contain any translatable strings anymore; avoids a sc_po_check failure.
* src/mv.c: Replace tabs by spaces to avoid complaints by
sc_prohibit_tab_based_indentation.
2020-11-22 15:17:17 +01:00
Paul Eggert 3aaa42deaa maint: port from matchpathcon to selabel_lookup
Ubuntu 20.10 is using a newer version of libselinux that
complains that matchpathcon is obsolete.  Rewrite the code
that it uses the recommended selabel_lookup instead.
* m4/jm-macros.m4 (coreutils_MACROS): Do not check for
matchpathcon_init_prefix, as it is no longer used.
* src/copy.c (set_file_security_ctx): Omit process_local arg,
as it is equivalent to !x->set_security_context.  All callers changed.
* src/copy.h (struct cp_options): set_security_context is now of
type struct selabel_handle *, not bool.  All uses changed.
* src/cp.c, src/install.c, src/mkdir.c, src/mkfifo.c, src/mknod.c:
* src/mv.c: Include selinux/label.h.
(main): Use selabel_open for set_security context.
* src/install.c (matchpathcon_init_prefix): Remove; now unused.
(get_labeling_handle): New static function.
(setdefaultfilecon, main): Use it.
(setdefaultfilecon): Do something regardless of
ENABLE_MATCHPATHCON, which seems to be a revenant macro.
(setdefaultfilecon): Use selabel_lookup instead of the obsolescent
matchpathcon.  Report an error unless it fails due to ENOENT.
* src/local.mk (src_ginstall_CPPFLAGS): Remove.
* src/selinux.c: Include selinux/label.h
Do not include die.h, error.h, canonicalize.h.
(defaultcon, restorecon_private, restorecon):
New arg HANDLE.  All callers changed.
Use selabel_lookup rather than matchpathcon.
(restorecon_private, restorecon): Don’t lose track of errno.
* src/selinux.c, src/selinux.h:
(restorecon): Don’t call ‘error’; that’s the caller’s job.
Use HAVE_SELINUX_LABEL_H, not HAVE_SELINUX_SELINUX_H,
in case there is some weird system with the former but not the latter.
* src/selinux.h (struct selinux_handle): Add forward decl.
2020-11-21 18:41:25 -08:00
Paul Eggert 5c8e2716b3 build: port to Solaris 10
* src/local.mk (src_ln_LDADD, src_mktemp_LDADD, src_tac_LDADD):
Add $(LIB_CLOCK_GETTIME), since these use tempname which uses
clock_gettime if getrandom fails.  On platforms like Solaris 10,
clock_gettime is not in the standard C library.
2020-11-21 18:41:25 -08:00
Pádraig Brady 2bc66c5ea7 tr: fix crash validating -c with some case char classes
This crash was identified by Cyber Independent Testing Lab:
https://cyber-itl.org/2020/10/28/citl-7000-defects.html
and was introduced with commit v8.5-163-g3f48829c2

* src/tr.c (validate_case_classes): Don't apply these
extra case alignment checks in the --complement case,
which is even more restrictive as to the contents of SET2.
* tests/misc/tr-case-class.sh: Add a test case,
for a large SET1, which caused the length adjustment
in validate_case_classes to underflow and trigger the assert.
* NEWS: Mention the bug fix.
2020-11-14 16:51:49 +00:00
Pádraig Brady 6fc695cb4a ls: fix crash printing SELinux context for unstatable files
This crash was identified by Cyber Independent Testing Lab:
https://cyber-itl.org/2020/10/28/citl-7000-defects.html
and was introduced with commit v6.9.90-11-g4245876e2

* src/ls.c (gobble_file): Ensure scontext is initialized
in the case where files are not statable.
* tests/ls/selinux-segfault.sh: Renamed from proc-selinux-segfault.sh,
and added test case for broken symlinks.
* tests/local.mk: Adjust for the renamed test.
* NEWS: Mention the bug fix.
2020-11-11 17:28:54 +00:00
Pádraig Brady beafa5c0a3 timeout: support sub-second timeouts on macOS
* m4/jm-macros.m4: Check for setitimer.
* src/timeout.c: Use setitimer if timer_settime is not available.
* NEWS: Mention the improvement.
2020-11-07 21:35:01 +00:00
Pádraig Brady 165a80f6e6 maint: avoid strncat warning on GCC
GCC 10.1.1 without optimization gives:

  error: ‘strncat’ argument 2 declared attribute ‘nonstring’
    [-Werror=stringop-overflow=]
    strncat (comment, UT_ID (utmp_ent), utmpsize);

Note the strncat man page says that:
  "src does not need to be null-terminated
   if it contains n or more bytes."
And the POSIX spec says that the second (source) parameter
is an array not a string.
So I think it's incorrect for strncat to require src be a string type.
This constraint seems to be being added to the gcc builtin strncat,
as specifiying -fno-builtin also avoids the warning.
Note specifying any optimization level also avoids the warning.

* src/who.c (make_id_equals_comment): Avoid the issue by using
stpcpy + stzncpy, instead of strcpy + strncat.
This pattern is used elsewhere in who.c
2020-11-07 21:34:34 +00:00
Pádraig Brady ff80b6b0a0 stat,tail: sync file system constants from the linux kernel
* src/stat.c: Add magic constants for "devmem", and
"zonefs" file systems.
* NEWS: Mention the improvement.
2020-10-28 14:16:29 +00:00
Pádraig Brady 53324dd611 maint: cleanup operation of fs-magic-compare
* src/local.mk: Ensure we map 2 hex digits to 4,
so that we don't output already handled Z3FOLD file system (0x33).
Also hide the generation command for src/fs.h.
2020-10-28 14:16:12 +00:00
Pádraig Brady 51cfaef655 doc: make blank lines before --help consistent
* src/basenc.c (usage): Remove extraneous blank line,
to be consistent with other utilities that have options.
* src/realpath.c: Likewise.
* src/runcon.c: Likewise.
Addresses https://bugs.gnu.org/44248
2020-10-27 19:23:46 +00:00
Jim Meyering cc816fb978 maint: avoid new sort.c warning from upcoming GCC11
gcc version 11.0.0 20201025 (experimental) warns that
src/sort.c:1655:1: warning: function might be candidate for attribute \
  'pure' if it is known to return normally [-Wsuggest-attribute=pure]
* src/sort.c (limfield): Mark as pure.
2020-10-26 07:47:36 -07:00
Kamil Dudka 4278e6615f dd: drop old workaround for lseek() bug in Linux kernel
The workaround triggers warnings from newer kernel versions in case
a user does not have sufficient privileges for the MTIOCGET ioctl.

* src/dd.c (skip_via_lseek): Drop wrapper function no longer needed.
(skip): Use lseek() directly.
(advance_input_after_read_error): Likewise.

Reported-by: Nir Soffer at https://bugzilla.redhat.com/1876840
Fixes https://bugs.gnu.org/44235
2020-10-26 13:31:36 +00:00
KOBAYASHI Takashi 2c898597ea nl: support a negative --line-increment
* src/nl.c (main): Allow -i to accept down to INTMAX_MIN.
* tests/misc/nl.sh: Add test cases.
* NEWS: Mention the new feature.
2020-10-26 13:15:46 +00:00
Pádraig Brady fe41d6c651 nl: only fail if need to output overflowed numbers
Previously we would have failed immediately upon internal overflow,
which didn't output the full line being processed, and assumed
there would be another numbered line.

* src/nl.c (line_no_overflow): A new global to track overflow.
(print_lineno): Only fail if about to output an overflowed number.
(reset_lineno): A new function to refactor resetting of the number,
and which also clears line_no_overflow.
* tests/misc/nl.sh: Add a test case.
2020-10-25 16:57:19 +00:00
Paul Eggert 00ea4bacf6 build: update gnulib submodule to latest
* gl/lib/randperm.c, src/cp-hash.c, src/ls.c, src/sort.c, src/tail.c:
Change all instaces of hash_delete to hash_remove to accommodate
change to Gnulib API.
2020-10-19 10:58:53 -07:00
Grigorii Sokolik d9b33b1535 maint: remove already handled FIXME in tail.c
* src/tail.c: Remove FIXME to follow a file name in a recreated
directory. The comment was added in commit v8.5-191-g61b77891c
while the fix (albeit not using inotify) was added in
commit v8.27-21-gba5fe2d4b
2020-10-17 16:54:17 +01:00
Benno Schulenberg ebaffa5094 doc: fix punctuation in stat --help
* src/stat.c (usage): Replace a mistaken semicolon with a colon,
and replace mistaken backticks with single quotes.  Also reorder
some words, for clarity.
Fixes https://bugs.gnu.org/43707
2020-09-29 21:57:30 +01:00
Emanuele Giacomelli 7cf45f4f6a csplit: fix regex suppression with specific match count
* src/csplit.c (process_regexp): Process the line suppression
in all invocations so that the last match is suppressed.
Previously with a non infinite match count,
the last regex pattern was not suppressed.
* NEWS: Mention the bug fix.
* tests/misc/csplit-suppress-matched.pl: Add a test case.
Fixes https://bugs.gnu.org/42764
2020-08-08 21:50:13 +01:00
Paul Eggert fd0fdeff94 build: update gnulib submodule to latest
* src/local.mk (src_expr_LDADD, src_factor_LDADD):
Adjust to Gnulib renaming of LIB_GMP to LIBGMP.
2020-07-28 13:45:47 -07:00
Paul Eggert fa7a5074d7 date: clarify the Epoch
* src/date.c (usage): Mention the Epoch under %s for clarity,
and capitalize.
2020-07-24 13:33:40 -07:00
Paul Eggert 9462c1202d factor: port to --without-libgmp
* src/factor.c (mp_factor_using_division): Use mpz_fdiv_q_2exp
instead of its no-longer-documented mpz_div_2exp alias.
(print_factors): Use mpz_out_str instead of gmp_printf.
2020-07-18 18:46:35 -07:00
Paul Eggert 55d985064c factor: explain why non-GMP code (Bug#42269)
* doc/coreutils.texi (factor invocation):
* src/factor.c: Explain why the two-word algorithm is useful.
2020-07-08 18:59:41 -07:00
Paul Eggert 95955e6746 factor: treat ' +bignum' like non-bignum
* src/factor.c (strto2uintmax): Instead of here ...
(print_factors): ... skip spaces and '+' here, so that
bignums are treated like non-bignums.
* tests/misc/factor.pl (bug-gmp-plus_2_sup_128_plus_1): New test.
2020-07-08 08:11:57 -07:00
Paul Eggert 1304644488 maint: use Gnulib libgmp module
This lets use assume multiple-precision arithmetic on all
platforms, simplifying the code.
* bootstrap.conf (gnulib_modules): Add libgmp.
* configure.ac: Don’t call cu_GMP, as this is now done by Gnulib.
* m4/gmp.m4: Remove.
* src/expr.c, src/factor.c: Use gmp.h unconditionally.
* src/factor.c: Use the simpler ‘#ifndef mpz_inits’ to
determine whether there is an mpz_inits macro.
2020-07-07 14:48:21 -07:00
Paul Eggert 4b04a0c3b7 cp: use copy_file_range if available
* NEWS: Mention this.
* bootstrap.conf (gnulib_modules): Add copy-file-range.
* src/copy.c (sparse_copy): Try copy_file_range if not
looking for holes.
2020-06-25 18:53:43 -07:00
Paul Eggert a6eaee501f cp: use SEEK_DATA/SEEK_HOLE if available
If it works, prefer lseek with SEEK_DATA and SEEK_HOLE to FIEMAP,
as lseek is simpler and more portable (will be in next POSIX).
Problem reported in 2011 by Jeff Liu (Bug#8061).
* NEWS: Mention this.
* src/copy.c (lseek_copy) [SEEK_HOLE]: New function.
(enum scantype): New constants ERROR_SCANTYPE, LSEEK_SCANTYPE.
(union scan_inference): New type.
(infer_scantype): Last arg is now union scan_inference *,
not struct extent_scan *.  All callers changed.
Prefer SEEK_HOLE to FIEMAP if both work, since
SEEK_HOLE is simpler and more portable.
(copy_reg): Do the fdadvise after initial scan, in case the scan
fails.  Report an error if the initial scan fails.
(copy_reg) [SEEK_HOLE]: Use lseek_copy if scantype says so.
2020-06-25 18:53:43 -07:00
Paul Eggert 2fcd0f3328 cp: avoid copy_reg goto
* src/copy.c (copy_reg): Redo to avoid label and goto.
2020-06-25 18:53:43 -07:00
Paul Eggert 761ba28400 cp: refactor extent_copy
* src/copy.c (extent_copy): New arg SCAN, replacing
REQUIRE_NORMAL_COPY.  All callers changed.
(enum scantype): New type.
(infer_scantype): Rename from is_probably_sparse and return
the new type.  Add args FD and SCAN.  All callers changed.
2020-06-25 18:53:43 -07:00
Pádraig Brady 6de819a2e3 doc: fix punctuation in man pages
* man/chmod.x: Add missing punctuation.
* src/expand-common.c: Likewise.
* src/numfmt.c: Likewise.
* src/rm.c: Likewise.

Fixes https://bugs.gnu.org/41962
2020-06-21 22:43:20 +01:00
Bernhard Voelker f5644c986c stat,tail: add support for the VBOXSF file system
* src/stat.c (human_fstype): Add case for the 'vboxsf' file system type
which is used for VirtualBox Shared Folders mounted in VirtualBox guest
VMs.
* NEWS: Mention the Improvement.
Fixes https://bugs.gnu.org/41935
2020-06-20 16:07:19 +02:00
Paul Eggert 25725f9d41 cp: default to COW
Likewise for ‘install’.  Proposed in Bug#24400, and long past due.
* NEWS:
* doc/coreutils.texi (cp invocation):
* src/copy.h (enum Reflink_type): Document this.
* src/cp.c (cp_option_init):
* src/install.c (cp_option_init): Implement this.
2020-06-18 22:38:06 -07:00
Tobias Stoeckmann 0fe9fdece7 maint: avoid signed integer overflows
Since -LONG_MIN results in LONG_MIN again, the operation itself is
a signed integer overflow.

This can be observed with the following calls (best if compiled
with -ftrapv or -fsanitize=undefined):

  $ numfmt --padding=-9223372036854775808
  $ seq 1e-9223372036854775808

Technically, the change in seq "reduces" the precision, but a double
or long double that small would be represented as 0 anyway.

* src/numfmt.c: Explicitly disallow --padding=LONG_MIN.
* src/seq.c: Treat 1e$LONG_MIN as 1e-$LONG_MAX.
* tests/misc/numfmt.pl: Add a test case.
* tests/misc/seq-precision.sh: Likewise.

Fixes https://bugs.gnu.org/41850
2020-06-15 23:07:03 +01:00
Bernhard Voelker 189776ff3b doc: timeout: improve documentation of the exit status
* doc/coreutils.texi (timeout invocation): Document that the exit
status is 137 when the KILL signal is used, regardless of whether that
signal is sent to COMMAND or timeout.
* src/timeout.c (usage): Likewise. Also split out and expand
on the possible exit status values to a separate table.

Discussed at https://bugs.gnu.org/41634
2020-06-07 15:41:48 +02:00
Paul Eggert b4ab357e42 maint: use getrandom, not getentropy
This makes for one Gnulib module less, and at runtime there’s
typically just one getrandom syscall instead of several for large
nonces.
* gl/lib/randread.c: Include sys/random.h instead of sys/time.h
and unistd.h.
(get_nonce): Use getrandom, not getentropy.
* gl/modules/randread (Depends-on):
Depend on getrandom, not getentropy.
* src/shred.c (main):
* src/shuf.c (main):
* src/sort.c (random_md5_state_init):
Say "getrandom" rather than "getentropy" in (unlikely) diagnostic.
2020-06-01 15:54:50 -07:00
Paul Eggert b3ff4113fd maint: use getentropy and new tempname modules
Update gnulib submodule to latest and use its new features.
Gnulib’s new getentropy module means coreutils can now assume
getentropy instead of approximating it, badly in some cases.
Gnulib’s improvements to the tempname module mean coreutils no
longer needs to maintain private patches.
* bootstrap.conf (gnulib_modules): Remove gettimeofday.
* gl/lib/randread.c (NAME_OF_NONCE_DEVICE): Remove.
(get_nonce): Return success indicator.  Remove bytes_bound arg.
All callers changed.  Rewrite by using getentropy instead of
reading the nonce device and falling back on gettimeofday.
Fail if getentropy fails.
(randread_new): Return NULL (setting errno) if get_nonce fails.
All callers changed.
* gl/lib/tempname.c.diff, gl/lib/tempname.h.diff:
* gl/modules/tempname.diff: Remove.
* gl/modules/randread (Depends-on):
Depend on getentropy, not gettimeofday.
* src/ptx.c (swallow_file_in_memory):
* src/shuf.c (read_input):
Adjust to read_file changes in Gnulib.
* src/shred.c (main):
* src/shuf.c (main):
* src/sort.c (random_md5_state_init):
Diagnose the new form of randread_new failures: randread_new can
fail now when !random_source, meaning getentropy failed.
2020-05-31 22:39:48 -07:00