Commit Graph
6921 Commits
Author SHA1 Message Date
Jim Meyering 9bcd045f81 copy: always use FIEMAP_FLAG_SYNC, for now
* src/extent-scan.c (extent_need_sync): Always return true,
to make the sole caller always use FIEMAP_FLAG_SYNC.
This will doubtless have an undesirable performance impact,
but we'll mitigate that shortly, by using extent_copy only on
files with holes.
2011-04-20 17:11:15 +02:00
Ondřej Vašík 9f688b8363 dircolors: add .ear, .war, .sar, for Java jar-like archives
* src/dircolors.hin: Add .ear, .war, .sar, for Java jar-like archives
Suggested by Ville Skyttä in https://bugzilla.redhat.com/616497.
2011-04-17 11:21:13 +02:00
Jim Meyering 53ee267945 maint: remove unnecessary inclusion of <stdio.h>
* src/extent-scan.c: Don't include <stdio.h>.  It was not used.
2011-04-12 12:14:26 +02:00
Jim Meyering b80e2c7c25 maint: install: remove support for --preserve_context ("_", not "-")
* src/install.c: Its use has elicited a warning for two years.
Use --preserve-context instead.
* NEWS (changes in behavior): Mention this.
2011-04-11 14:32:56 +02:00
Jim Meyering 9288068ff8 maint: reorder install.c to eliminate declarations of static functions
* src/install.c: Remove static function declarations.
2011-04-11 14:32:56 +02:00
Jim Meyering e6f4c34167 maint: rename variables for clarity...
and to avoid a false-positive "TO to" in new doubled word check.
* src/install.c (change_timestamps): Rename parameters for
readability.  Make the comment match the code.
2011-04-11 14:32:55 +02:00
Jim Meyering b2dfc3b5e5 maint: remove doubled words in comments, e.g., s/to to/to/
* tests/ls/color-norm: s/to to/to/
* gl/lib/mbsalign.h (mbs_align_t): s/or or/or/
* src/extent-scan.c (extent_scan_read): s/the the/the/
* src/libstdbuf.c: s/the the/the/
* tests/misc/stdbuf: s/on on/on/
* ChangeLog-2005: s/for\n\tfor /for\n\t/
2011-04-11 14:32:55 +02:00
Pádraig Brady 80be785c95 maint: misc typo fixes
* src/fiemap.h: s/can not/cannot/
* NEWS: s/in/is/
* doc/coreutils.texi: Remove spurious "and".
s/effect/affect/
Use matched ``...'' quotes.
2011-04-11 11:13:14 +02:00
Pádraig Brady 18f5a8537c copy: handle mergeable extents across fiemap scans
* extent-scan.h (extent_scan_free): Init the pointer to NULL,
and reset the count to 0, so that we can realloc the buffer.
* src/extent-scan.c (extent_scan_init): Likewise.
(extent_scan_read): Loop over multiple fiemap scans, so we handle
mergeable extents that span across fiemap scan boundaries.  Once
we have enough unique extents, return so as to minimize memory use.
2011-04-06 23:15:19 +01:00
Pádraig Brady 8b2bedadb5 copy: fix an unlikely memory leak when a fiemap copy fails
* src/copy.c (extent_copy): Free the extents array when
sparse_copy() fails.
2011-04-06 23:08:54 +01:00
Jim Meyering 6227c8e8de tests: minor improvement of sc_tight_scope rule
* src/Makefile.am (sc_tight_scope): Adjust rule to use an eval-based
trap-setting for-loop rather than 4x hard-coded 128+N constants.
Also catch SIGQUIT (3).  Tweak comments.
2011-04-04 18:21:31 +02:00
Jim Meyering 03583f8d64 maint: prohibit direct use of strncmp: prefer STREQ_LEN, STRNCMP_LIT
* cfg.mk (sc_prohibit_strncmp): New rule, mostly from libvirt.
* src/system.h (STREQ_LEN, STRPREFIX, STRNCMP_LIT): Define.
* src/df.c (get_dev, get_point): Convert.
* src/extent-scan.c (extent_need_sync): Likewise.
* src/ls.c (is_colored, decode_switches): Likewise.
(parse_ls_color, (print_color_indicator): Likewise.
* src/md5sum.c (split_3): Likewise.
* src/split.c (main, emit_ancillary_info): Likewise.
* src/tr.c (look_up_char_class): Likewise.
* src/uname.c (main): Likewise.
* src/who.c (scan_entries): Likewise.
2011-04-03 21:55:27 +02:00
Jim Meyering 1c3654cb1f copy: require fiemap sync also for 2.6.38 kernels
* src/extent-scan.c (extent_need_sync): Require sync also for 2.6.38.
Without this, part of the cp/fiemap-empty test would fail both on
F15-to-be (2.6.38.1-6.fc15.x86_64) and rawhide.  For details, see
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22190
2011-04-03 12:16:27 +02:00
Jim Meyering 45b332f7ab cp: always detect copy-into-self: avoid infloop w/large PATH_MAX
When running the erroneous command, cp -rl A D D, and depending on the
structure of directories A and D and the file system type (because that
changes order of dir. entry traversal), cp would sometimes fail to
detect that D was being copied into D, and would create D/D/D/D/D/...
until it hit PATH_MAX or exhausted some resource.
I noticed this via the occasional failure of the cp/into-self test
when run using a ZFS file system.  It is occasional because the bug
is dependent on the order in which directory entries are traversed,
and that is apparently indeterminate with ZFS.
Technically, with the current recursive implementation, there is no
risk of an infinite loop, due to stack limitations, but with an
eventual fts-based implementation, it might have iterated until
disk space or inodes are exhausted.
* src/copy.c (copy_dir): Avoid copy-into-self interminable loop on
systems with large PATH_MAX.  On other systems, diagnose the copy-into-
self error consistently.  Handle the parameter,
first_dir_created_per_command_line_arg, correctly when there are two
or more sub-directories.
2011-04-02 16:14:47 +02:00
Pádraig Brady bfdb6b585b copy: process empty extents more efficiently
* src/copy.c (extent_copy): Treat an allocated but empty extent
much like a hole.  I.E. don't read data we know is going to be NUL.
Also we convert the empty extent to a hole only when SPARSE_ALWAYS
so that the source and dest have the same allocation.  This will
be improved soon, when we use fallocate() to do the allocation.
* tests/cp/fiemap-empty: A new test for efficiency and correctness
of copying empty extents.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the change in behavior.
2011-04-01 15:04:18 +01:00
Pádraig Brady b56b53bd70 copy: protect against overlapping extents
* src/extent-scan.c (extent_scan_read): Add a more stringent check
for OFF_T overflow, to ensure subsequent code is immune.
Detect overlapping extents and adjust, so as files always copied.
Detection using a single scan with fallback to a standard copy
was thought too expensive in memory or time.
* NEWS: Mention the fix
2011-04-01 14:42:46 +01:00
Pádraig Brady 594292a1d8 copy: link rather than copy symlinks, when --link used
This bug was introduced in commit ca9e212c, 2009-09-24,
"cp, mv: use linkat to guarantee semantics", which
inadvertently disabled the creation of hardlinks to symlinks.
However rather than implementing the intention of that commit
and relying on gnulib linkat emulation, we'll revert to the
previous emulation as that maintains ownership and timestamps.

* src/copy.c (copy_internal): Use our existing hardlink to
symlink emulation when link() might dereference the symlink.
Also ensure that we copy the timestamps of the original symlink
when we use the emulation.
* tests/cp/link-symlink: Add a test to ensure timestamps copied.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the fix.
Reported by Ruediger Meier
2011-04-01 14:17:27 +01:00
Pádraig Brady f69876e558 copy: with fiemap copy, only sync when needed
* src/extent-scan.h (struct extent_scan): Add the fm_flags member to
pass to the fiemap scan.
* src/extent-scan.c (extent_need_sync): A new function used to
detect Linux kernels before 2.6.38.
(extent_scan_init): Add FIEMAP_FLAG_SYNC when needed.
* tests/cp/sparse-fiemap: Adjust comment.
* NEWS: Mention the change in behavior.
Indirectly suggested by Mike Frysinger
2011-03-31 16:46:42 +01:00
Jim Meyering fde07ff436 maint: correct formatting style in a header
* src/find-mount-point.h: Move "*" to where it belongs.
Move "const", too.
* src/find-mount-point.c: Move "const" to conform.
* src/Makefile.am (sc_tight_scope): Allow `*'s before the function name.
Use perl's -l option and drop the \n after (and quotes around) $1.
2011-03-28 11:51:35 +02:00
Pádraig Brady 99679fff62 df: fix alignment of columns
* src/df.c (alloc_table_row): A new function to allocate storage
for a row of strings.
(print_table): A new function to interate over all stored strings in
the table, and apply alignment honoring the max width of each column.
(get_header): Renamed from print_header, and adjusted accordingly.
(get_dev): Renamed from show_dev.  Also we no longer wrap longer
device names over two lines, which can be an unexpected issue for
scripts parsing the output from df.
(get_disk): s/show_/get_/
(get_point): Likewise.
(get_entry): Likewise.
(get_all_entries): Likewise.
* NEWS: Mention the change.
2011-03-23 16:44:49 +00:00
David A. Wheeler 3f31ec950b test: accept "==" as a synonym for "="
Make GNU coreutils' test recognize "==" as a synonym for "=".
This is already the case in GNU coreutils' expr, bash, ksh,
busybox ash, FreeBSD-current /bin/sh and /bin/test, and
OpenBSD's /bin/sh.

Before, env test a '==' a would fail with this diagnostic:
"test: ==: binary operator expected".  Now, it succeeds.
* src/test.c: Accept "==" as a synonym for "=".
* doc/coreutils.texi (String tests): Document it.
Reported as http://debbugs.gnu.org/8263
Also see http://austingroupbugs.net/view.php?id=375
2011-03-22 07:12:01 +01:00
Jim Meyering 4f591fdd0b sort: avoid memory pressure of 130MB/thread when reading from pipe
* src/sort.c (INPUT_FILE_SIZE_GUESS): Decrease initial allocation
factor used to size buffer used when reading a non-regular file.
For motivation, see discussion here:
http://thread.gmane.org/gmane.comp.gnu.coreutils.general/878/focus=887
2011-03-16 16:30:02 +01:00
Jim Meyering e0245ae0ae sort: spawn fewer threads for small inputs
* src/sort.c (SUBTHREAD_LINES_HEURISTIC): Do not spawn a new thread
for every 4 lines.  Increase this from 4 to 128K.  128K lines seems
appropriate for a 5-year-old dual-core laptop, but it is too low for
some common combinations of short lines and/or newer systems.
* NEWS (Bug fixes): Mention it.
2011-03-13 18:13:17 +01:00
Pádraig Brady 3ed0384c41 copy: merge similar extents before processing
* src/extent-scan.c (extent_scan_read):  Merge adjacent extents
that vary only in size, so that we may process them more efficiently.
This will be especially useful when we introduce fallocate()
so that we don't reproduce fragmentation in the destination.
2011-03-11 00:50:10 +00:00
Paul Eggert 194c1e89a4 dd: avoid or diagnose some problems with short reads
* src/dd.c (warn_partial_read): New static var.
(iread): Diagnose partial reads if needed.
(iwrite): Don't diagnose them here; not needed any more.
(scanargs): Determine whether partial reads should be diagnosted.
2011-03-05 16:06:34 -08:00
Pádraig Brady 5f3115535b dd: add a flag to discard cached data
* src/dd.c (FFS_MASK): A new macro (Find First Set) refactored
from the following enum as it's now used twice.
(usage): Mention the new 'nocache' flag.
(cache_round): A new function to help ignore requests
to drop cache, that are less than page_size.
(invalidate_cache): A new function to call posix_fadvise()
with the appropriate offset and length.  Note we don't
use fdadvise() so we can detect errors when count=0.
(dd_copy): Call invalidate_cache() for the portions read.
(iwrite): Likewise for the portions written.
(main): Call invalidate_cache for page_size slop or
for full file when count=0.
* cfg.mk (sc_dd_O_FLAGS): Adjust to pass.
* doc/coreutils.texi (dd invocation): Describe the 'nocache' flag,
and give some examples of how it can be used.
* tests/dd/nocache: A new test.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the new feature.
2011-03-05 02:33:53 +00:00
Pádraig Brady 433a7c614f doc: clarify that dd bs= can immediately output short reads
* doc/coreutils.texi (dd invocation): Clarify that bs= can
cause parital reads to be immediately written to output.
* src/dd.c (usage): Hint that bs= can cause partial writes.
See http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8171
2011-03-05 02:21:45 +00:00
Pádraig Brady 7525860fb9 doc: group dd conv= options that are actually flags
* src/dd.c (usage): Move 'sync' up with other data transformation
options.  Having it alongside 'fsync' and 'fdatasync' is
particularly confusing.  Also the double line description of
the 'sync' option, serves as a visual break from the "flag"
type options that follow.
* doc/coreutils.texi (dd invocation):  Apply the same grouping
as above, by splitting the "conv=" table in two.
2011-03-05 02:01:16 +00:00
Pádraig Brady e1788d9ed9 dd: warn when we disable oflag=direct not at EOF
An alternative to this is to auto enable iflag=fullblock
when oflag=direct and bs= is specified.
It was thought better though, to warn about the specific issue,
and give full control of dd's options to the user.

* src/dd.c (iwrite): Warn, when we write after having
disabled O_DIRECT.
See https://bugzilla.redhat.com/show_bug.cgi?id=614605
2011-03-05 01:48:55 +00:00
Jim Meyering caaf2899f6 du: don't infloop for --files0-from=DIR
* src/du.c (main): Fail on AI_ERR_READ error, rather than merely
diagnosing and continuing.  Based on a patch by Stefan Vargyas.
Also move the handling of AI_ERR_EOF into the case stmt.
Do not report ferror/fclose(stdin) failure when we've
already diagnosed e.g., failure to read the DIR, above.
Bug introduced by 2008-11-24 commit 031e2fb5, "du: read and
process --files0-from= input a name at a time,".
* src/wc.c: Handle read failure as with du: do not exit
immediately, but rather go on to print any total and to clean-up.
As above, move the handling of AI_ERR_EOF into the case stmt.
* tests/du/files0-from-dir: New file, to test both du and wc.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it.
2011-03-03 09:05:44 +01:00
Jim Meyering 7cfd12c78e wc: avoid NULL dereference on out-of-memory error
* src/wc.c (main): Diagnose failed argv_iter_init_* failure,
rather than falling through and dereferencing NULL.
Bug introduced by 2008-11-25 commit c2e56e0d,
"wc: read and process --files0-from= input a name at a time,".
* NEWS (Bug fixes): Mention it.
2011-03-03 09:01:17 +01:00
Jim Meyering 2895f44e89 stdbuf: avoid even the appearance of a possible use-after-free
There was an execution path by which "libstdbuf" could be used after
being freed, but that would happen only if there were no libstdbuf.so
alongside the stdbuf program and there had been an installation error
leading to absence of the file, PKGLIBDIR/libstdbuf.so.
* src/stdbuf.c (set_LD_PRELOAD): Rearrange loop to make it perfectly
clear that there is no possibility of use-after-free.
Steve Grubb reported this possible use-after-free of "libstdbuf".
2011-02-18 23:31:29 +01:00
Paul Eggert 43066fe2ca uptime: omit unnecessary #if
* src/uptime.c (print_uptime): Omit unnecessary "#if defined
HAVE_GETLOADAVG || defined C_GETLOADAVG".  This #if is always
true, and removing it will help us simplify the gnulib getloadavg
module.
2011-02-14 21:05:55 -08:00
Pádraig Brady 275c831a3c copy: adjust fiemap handling of sparse files
Don't depend on heuristics to detect sparse files
if fiemap is available.  Also don't scan for new
holes unless --sparse=always has been specified.

* src/copy.c (extent_copy): Pass the user specified
sparse mode, and handle as described above.
2011-02-11 00:13:31 +00:00
Pádraig Brady 6f95025c9d copy: suppress redundant lseeks when using fiemap
* src/copy.c (extent_copy): Suppress redundant lseek()s in both
the source and dest files, when there is no hole between extents.
2011-02-11 00:13:31 +00:00
Jim Meyering 2e636af1ef cut: don't segfault for large unbounded range
* src/cut.c (set_fields): When computing the maximum range endpoint,
take into consideration the start of any unbounded range, like "999-".
* NEWS (Bug fixes): Mention it.
* tests/misc/cut (big-unbounded-b,c,f): Add tests.
Reported by Paul Marinescu in http://debbugs.gnu.org/7993
The bug was introduced on 2004-12-04 via commit 7380cf79.
2011-02-07 09:25:51 +01:00
Jim Meyering 9f61806875 copy: don't let a failed lseek go undiagnosed
Upon failed lseek, sparse_copy_finalize would mistakenly return true.
Admittedly, that is very unlikely, since that particular lseek
is attempted only if the preceding call to sparse_copy induced
a hole at EOF (via lseek on the destination FD).  However, now
that sparse_copy has an output parameter, N_READ, there is no
longer any reason to call lseek (fd, 0, SEEK_CUR), so...
* src/copy.c (sparse_copy_finalize): Remove the function.
(copy_reg): Call ftruncate with n_read, rather than
sparse_copy_finalize with its now-unnecessary lseek.
Lasse Collin spotted the bug in sparse_copy_finalize.
2011-02-05 22:40:57 +01:00
Jim Meyering d5fa424a9c cp: avoid spurious failure on any non-linux kernel
* src/extent-scan.c (extent_scan_read) [!linux]: Always set
scan->initial_scan_failed so caller knows not to report the failure.
2011-02-04 15:01:39 +01:00
Pádraig Brady d846bc8bab sort: fix --debug key highlighting when key start after key end
This case was overlooked in commit bdde34f9, 2010-08-05,
"sort: tune and refactor --debug code, and fix minor underlining bug"

* src/sort.c (debug_key):  Don't adjust the key end when
it's before the key start.
* tests/misc/sort-debug-keys: Add a test case.
2011-02-03 10:19:34 +00:00
Pádraig Brady 84e5274f7e cp: fix the buffer size used when writing zeros
* src/copy.c (write_zeros): This bug caused 4 or 8 bytes to
be written at a time which is very inefficient.  One could
trigger the issue with `cp --sparse=never sparse non-sparse`
on a file system that supports fiemap.
2011-01-31 23:21:04 +00:00
Jim Meyering f8f57dd76b cp: always initialize extent_copy's output parameter
* src/copy.c (extent_copy): Otherwise it would be used uninitialized.
2011-01-31 20:55:34 +01:00
Jim Meyering cc0645fc64 cp: fix copying a sparse file to a pipe
The recent FIEMAP-related changes made it so the unusual case of
copying a sparse file to a non-regular destination (e.g., a pipe)
would erroneously write one byte too many to that destination.
That happened because extent_copy assumed that it could use lseek
to obtain the number of bytes written to the output file descriptor.
That was valid only for regular files.
* src/copy.c (sparse_copy): Add a parameter, to be used by extent_copy,
but not by reg_copy.  Adjust callers.
(extent_copy): Maintain new local, dest_pos, using new arg, n_read.
Don't call lseek on dest_fd; use new var, dest_pos, instead.
(copy_reg): Add unused arg.
2011-01-31 18:26:06 +01:00
Jim Meyering aeed6c1d89 maint: remove unused inclusion of "fiemap.h"
* src/copy.c: Don't include "fiemap.h".
2011-01-31 15:17:12 +01:00
Jim Meyering 31dc21f693 maint: update copyright year lists in new files
* src/extent-scan.h: Update copyright year list.
* src/extent-scan.c: Likewise.
* tests/cp/sparse-fiemap: Likewise.
2011-01-31 13:42:49 +01:00
Jim Meyering fc3c8e3cdd copy, tee: assume EINTR is always defined: remove #ifdefs
Don't use "#ifdef EINTR".  dd.c has been doing that since 2004.
* src/copy.c (sparse_copy): Remove #ifdef...#endif around EINTR use.
* src/tee.c (tee_files): Remove #ifdef...#endif around EINTR use.
If we need it, add something like this in system.h:
/* When EINTR is not defined, define it to an improbable value
   so that each use does not have to be #ifdef'd.  */
 #ifndef EINTR
 # define EINTR 999988
 #endif
2011-01-30 20:44:12 +01:00
Jim Meyering 0f91b6174e copy: make extent_copy use sparse_copy, rather than its own code
* src/copy.c (extent_copy): Before this change, extent_copy would fail
to create holes, thus breaking --sparse=auto and --sparse=always.
I.e., copying a large enough file of all zeros, cp --sparse=always
should introduce a hole, but with extent_copy, it would not.
2011-01-30 20:44:12 +01:00
Jim Meyering 8da69cbf30 copy: remove obsolete comment
* src/copy.c (sparse_copy): Remove now-obsolete comment about
how we used to work around lack of ftruncate.  Combine nested
if conditions into one.
2011-01-30 20:44:12 +01:00
Jim Meyering 82c7f192bd copy: factor sparse-copying code into its own function, because
we're going to have to use it from within extent_copy, too.
* src/copy.c (sparse_copy): New function, factored out of...
(copy_reg): ...here.
Remove now-unused locals.
2011-01-30 20:44:12 +01:00
Jim Meyering f5743a6f20 fiemap copy: avoid leak-on-error
* src/copy.c (extent_copy): Don't leak an extent_scan buffer on
failed lseek, read, or write.
2011-01-30 20:44:12 +01:00
Jim Meyering 3953f7523e fiemap copy: avoid a performance hit due to very small buffer
* src/copy.c (extent_copy): Don't let what should have been a
temporary reduction of buf_size (to handle a short ext_len) become
permanent and thus impact the performance of all further iterations.
2011-01-30 20:44:12 +01:00