Commit Graph
378 Commits
Author SHA1 Message Date
Collin Funk 56531e52a3 dd: don't place a hole when seeking N bytes when N is less than obs
* src/dd.c (main): Also account for seeking bytes in a few conditions.
* tests/dd/bytes.sh: Add a test case.
* tests/dd/fail-ftruncate-fstat.sh (fail_ftruncate_fstat): New function.
Use it to test seeking on bytes, and make sure fstat and ftruncate are
called.
* NEWS: Mention the bug fix.

Discovered by Anthropic's OSS Scanner, as bug ANT-2026-NLK8TXH1.
2026-10-29 16:23:47 -07:00
Collin Funk 4be9843dc9 tests: dd: fix a test that would never finish on GNU/Hurd
On GNU/Hurd, this test would get stuck in an infinite loop like the
following:

    + kill -s INFO 16535
    + sleep .01
    + kill -s INFO 16535
    + sleep .01
    + kill -s INFO 16535
    + sleep .01
    [...]

The signals being sent that frequently seem to prevent 'dd' from
performing writes, since it outputs the following during that time:

    0+0 records in
    0+0 records out
    0 bytes copied, 7161.11 s, 0.0 kB/s
    0+0 records in
    0+0 records out
    0 bytes copied, 7161.88 s, 0.0 kB/s
    [...]

* tests/dd/stats.sh: Use a larger interval between signals on GNU/Hurd.
2026-09-10 22:22:37 -07:00
Pádraig Brady 1ea33dafe9 tests: dd: avoid false failure due to restricted memory
* tests/dd/no-allocate.sh: Expand memory restriction
(and allocated buffers) by 6MiB, to give more head-room
to the shell.
Reported and tested by Bruno Haible on OpenIndiana (from 2025).
2026-09-10 14:24:28 +01:00
Pádraig Brady 4fc469f850 tests: dd: avoid false failure with older strace
* tests/dd/fail-ftruncate-fstat.sh: Ensure strace intercepts
the call, which was not the case on strace 6.19 on Alpine 3.24.1.
2026-09-08 18:35:09 +01:00
oech3 e4db330242 tests: dd: support referencing of external test command
* tests/dd/stderr.sh: Use env instead of
$abs_top_builddir for external usage of tests.

Link: https://github.com/coreutils/coreutils/pull/340
2026-08-31 12:43:13 +01:00
oech3 cfd13b061e tests: dd: replace LD_PRELOAD of fail-ftruncate-fstat.sh by strace
* tests/dd/fail-ftruncate-fstat.sh: Replace LD_PRELOAD by strace
to support static executables.

Link: https://github.com/coreutils/coreutils/pull/330
2026-08-20 16:25:16 +01:00
Pádraig Brady 2653c5c5e2 tests: dd: ensure nocache flag does not fail with pipes
* tests/dd/nocache.sh: Whether fadvise() is called (uutils)
or avoided (GNU), ensure there is no failure when nocache
is only the ancillary operation being requested.

Link: https://github.com/coreutils/coreutils/pull/276
2026-07-13 17:55:26 +01:00
Collin Funk 57fe9ecd69 tests: dd: check that files are opened with O_TRUNC
* tests/dd/misc.sh: Add the test case.
2026-06-24 21:08:33 -07:00
oech3 e964f2f431 tests: dd: add test case for OOM with skip=1
* tests/dd/no-allocate.sh: Add test cases for OOM with skip=1.
https://github.com/coreutils/coreutils/pull/265
2026-05-20 13:09:28 +01:00
Collin Funk fbd3072360 tests: dd: avoid a false failure when ulimit fails
This test failure was seen on NetBSD 10.0 machine, where ulimit failed
with EINVAL.

* tests/dd/bytes.sh: Don't fail the test case if ulimit fails.
Reported by Bruno Haible.
2026-05-18 17:32:58 -07:00
Pádraig Brady 4bf753a801 dd: avoid recursive parsing of multipliers
* src/dd.c (parse_integer): Use iterative rather than recursive parsing,
to avoid potential stack overflow.
* tests/dd/bytes.sh: Add a test case.
https://github.com/coreutils/coreutils/issues/254
2026-04-23 13:58:46 +01:00
Pádraig Brady 0b7d8620f7 tests: fix false failure due to ulimit usage
* tests/dd/no-allocate.sh: Run getlimits_ to ensure
we have $SSIZE_MAX etc. available.  Also give some buffer
for the configured ulimit to leave space for returns_ to work.
2026-04-16 19:03:13 +01:00
oech3 3558839bdd tests: dd: ensure memory exhaustion is handled gracefully
* tests/dd/no-allocate.sh: Ensure we exit 1 upon mem allocation failure.
Also check other buffer size edge cases.
https://github.com/uutils/coreutils/issues/11436
https://github.com/uutils/coreutils/issues/11580
https://github.com/coreutils/coreutils/pull/235
2026-04-01 16:54:12 +01:00
Pádraig Brady 178c48154d tests: dd: avoid false failure with no controlling terminal
* tests/dd/misc.sh: test -w /dev/tty is not a strong enough check,
we need to actually open /dev/tty to ensure it's available.
It's not available under setsid for example.
2026-04-01 13:31:57 +01:00
oech3 ee5092971e tests: dd: check that erroneous seeks are not done in output
* tests/dd/misc.sh: Add test case for of=/dev/tty.
The same occurs for /dev/stdout, but that varies
in the test hardness so is best avoided.
https://github.com/coreutils/coreutils/pull/234
2026-03-31 12:09:26 +01:00
Pádraig Brady 58d88d2435 tests: dd: fix false failure on NetBSD 10
* tests/dd/partial-write.sh: Skip the test if
nothing written at all, as was seen on NetBSD 10.
Reported by Bruno Haible.
2026-03-16 22:34:58 +00:00
Pádraig Brady 711fa8f9a5 dd: always diagnose partial writes on write failure
* src/dd.c (dd_copy): Increment the partial write count upon failure.
* tests/dd/partial-write.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/80583
2026-03-12 21:19:33 +00:00
Pádraig Brady 6218f2b7a3 maint: tests: avoid EPERM specific handling
* tests/dd/fail-ftruncate-fstat.sh: Use $EPERM rather than
mapping specific strings to the Linux $EPERM.
* tests/mkdir/smack-no-root.sh: Use $EPERM rather than hardcoding.
2026-02-18 18:31:49 +00:00
Pádraig Brady 75c671415e tests: dd: avoid EPERM false failure on Solaris
* tests/dd/fail-ftruncate-fstat.sh: Also map the "Not owner"
error for EPERM seen on earlier Solaris 11 releases.
Reported by Bruno Haible.
2026-01-24 14:53:30 +00:00
Collin Funk a96abcbbeb tests: dd: avoid a false test failure on Solaris 11.4
* tests/dd/fail-ftruncate-fstat.sh: Replace the message used by Solaris
11.4 for strerror (EPERM) with the one we expect.
2026-01-19 18:26:45 -08:00
Collin Funk 73d3a49f45 maint: run 'make update-copyright' 2026-01-01 10:56:16 -08:00
Collin Funk b12593d124 dd: don't continue copying when ftruncate fails using seek= and of=
* src/dd.c (main): Reduce the scope of exit_status. Exit immediately if
ftruncate fails.
* tests/dd/fail-ftruncate-fstat.sh: New test.
* tests/local.mk (all_tests): Add the new test.
* NEWS: Mention the bug fix.
2025-12-25 15:16:57 -08:00
Pádraig Brady 2692d599ec tests: fix non-portable use of printf '\x..'
* tests/dd/conv-case.sh: Use octal instead.
* tests/ls/hyperlink.sh: Likewise.
* tests/sort/sort-locale.sh: Likewise.
2025-12-08 22:32:48 +00:00
Collin Funk 71514b4983 tests: dd: add tests for conv=lcase and conv=ucase
* tests/dd/conv-case.sh: New test.
* tests/local.mk (all_tests): Add it.
2025-11-14 10:59:55 -08:00
Pádraig Brady df30ce36e8 tests: dd: avoid false failure on Solaris 10
* tests/dd/no-allocate.sh: Give some extra space to the
determined vm limit.
2025-09-20 22:12:58 +01:00
Pádraig Brady 2b0887fdd5 tests: dd: ensure posix_fadvise errors are handled
* tests/dd/nocache_fail.sh: Add a test case for the recent fix.
2025-03-10 12:46:14 +00:00
Pádraig Brady 28b176085f maint: update all copyright year number ranges
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...

* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2025-01-01 09:33:08 +00:00
Paul Eggert cc58b65736 dd: port big-offset test to macOS 12.6
* tests/dd/skip-seek-past-file.sh: Do not assume that
seek to exactly OFF_T_MAX should fail; it works on macOS 12.6
and POSIX allows this.  Come to think of it, it should work
on Solaris too, if someone ever comes across a Solaris host
with a file system that allows such files.
2024-11-20 11:11:54 -08:00
Pádraig Brady c6be9649a7 tests: fix inconsistent use of getlimits_
* tests/dd/skip-seek-past-file.sh: s/eval $(getlimits)/getlimits_/
2024-11-19 20:03:01 +00:00
Paul Eggert 52e024b7b7 dd: skip some alloc tests on ZFS
* tests/dd/sparse.sh: Skip some tests on ZFS.
2024-05-16 23:30:55 -07:00
Pádraig Brady aa50ea55ea tests: make ulimit -v interact better with ASAN
ulimit -v is generally not supported with ASAN, giving errors like:
  "ReserveShadowMemoryRange failed while trying to map 0x... bytes.
   Perhaps you're using ulimit -v"

* tests/cp/link-heap.sh: Mention ASAN as a possible reason for skipping.
* tests/csplit/csplit-heap.sh: Likewise.
* tests/cut/cut-huge-range.sh: Likewise.
* tests/dd/no-allocate.sh: Likewise.
* tests/printf/printf-surprise.sh: Likewise.
* tests/rm/many-dir-entries-vs-OOM.sh: Likewise.
* tests/head/head-c.sh: Only skip the part of the test needing ulimit.
* tests/split/line-bytes.sh: Likewise.
2024-01-17 23:59:49 +00:00
Pádraig Brady a966dcdb69 maint: update all copyright year number ranges
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...

* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Manually update copyright year,
until we fully sync with gnulib at a later stage.
* tests/sample-test: Adjust to use the single most recent year.
2024-01-01 13:27:23 +00:00
Pádraig Brady e8e81fc44b dd: fix parsing of numbers with more than two multipliers
* src/dd.c (parse_integer): Use recursion to support more than two
multipliers.  Also protect suffix[-1] access to ensure we don't
inspect before the passed string.
* tests/dd/bytes.sh: Add test cases.
* doc/coreutils.texi (dd invocation): Note the support for specifying
many multipliers in a number.
* NEWS: Mention the bug fix.
Fixes https://bugs.debian.org/1037275
2023-06-11 23:10:34 +01:00
Pádraig Brady 01755d36e7 maint: update all copyright year number ranges
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...

* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Manually update copyright year,
until we fully sync with gnulib at a later stage.
* tests/sample-test: Adjust to use the single most recent year.
2023-01-01 14:50:15 +00:00
Paul Eggert 97e9778296 dd: counts ending in "B" now count bytes
This implements my suggestion in Bug#54112.
* src/dd.c (usage): Document the change.
(parse_integer, scanargs): Implement the change.
Omit some now-obsolete checks for invalid flags.
* tests/dd/bytes.sh: Test the new behavior, while retaining
checks for the now-obsolete usage.
* tests/dd/nocache_eof.sh: Avoid now-obsolete usage.
2022-02-22 18:56:53 -08:00
Paul Eggert 4439ef3ec4 dd: support iseek= and oseek=
Alias iseek=N to skip=N, oseek=N to seek=N (Bug#45648).
* src/dd.c (scanargs): Parse iseek= and oseek=.
* tests/dd/skip-seek.pl (sk-seek5): New test case.
2022-02-22 09:04:27 -08:00
Paul Eggert e94d95075d dd: improve integer overflow checking
* src/dd.c: Prefer signed to unsigned types where either will do,
as this helps improve checking with gcc -fsanitize=undefined.
Limit the signed types to their intended ranges.
(MAX_BLOCKSIZE): Don’t exceed IDX_MAX - slop either.
(input_offset_overflow): Remove; overflow now denoted by negative.
(parse_integer): Return INTMAX_MAX on overflow, instead of unspecified.
Do not falsely report overflow for ‘00x99999999999999999999999999999’.
* tests/dd/misc.sh: New test for 00xBIG.
* tests/dd/skip-seek-past-file.sh: Adjust to new diagnostic wording.
New test for BIGxBIG.
2022-01-27 13:04:14 -08:00
Pádraig Brady 3067a9293a maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2022-01-02 16:15:55 +00:00
Pádraig Brady 32ff1d1313 maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2021-01-01 16:36:09 +00:00
Paul Eggert af941bc523 tests: simplify since expr now works on bignums
* cfg.mk (sc_prohibit_expr_unsigned): Remove.
* tests/dd/skip-seek-past-dev.sh (DEV_OFLOW):
* tests/id/setgid.sh (gp1):
* tests/misc/cut-huge-range.sh (CUT_MAX):
* tests/misc/expr.pl:
* tests/misc/sort-discrim.sh:
Assume expr works on bignums.
* tests/misc/cut-huge-range.sh (subtract_one):
Remove; no longer needed.
2020-07-08 08:11:57 -07:00
Andreas Schwab 6a3d2883fe tests: avoid spurious testsuite failure
* tests/dd/stats.sh: Increase timeout.
Fixes https://bugs.gnu.org/42135
2020-06-30 13:44:34 +01:00
Pádraig Brady de73a867ca tests: avoid false failures on darwin 19.2.0
With these adjustments, all tests pass on macOS Catalina.

* tests/dd/sparse.sh: Adjust so that systems like apfs that
don't create holes < 16 MiB do not fail erroneously.
* tests/touch/trailing-slash.sh: Darwin was seen to dereference
symlinks to files when given a trailing slash, so avoid
that particular case.
2020-02-29 22:45:00 +00:00
Pádraig Brady aaba82431c maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2020-01-01 14:16:56 +00:00
Kevin Locke 3a87ad5124 doc: clarify dd sparse detection is by *output* block
The wording of the dd --help text suggests that output will be skipped
for sparse *input* blocks (i.e. that NUL-checking is done on input
blocks) while the code actually checks/skips all-NUL *output* blocks.[1]

* src/dd.c (usage): Update the --help text to clarify the above.
* tests/dd/sparse.sh: Ensure sparseness is controlled with obs.

[1]: https://superuser.com/a/1136358
2019-05-26 12:53:26 +01:00
Assaf Gordon 8dfcf38af1 maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2019-01-01 16:51:48 -07:00
Pádraig Brady ece7157933 maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2018-01-01 15:57:22 +00:00
Pádraig Brady a58ce6f023 tests: fix false failure in new dd/nocache_eof test
* test/dd/nocache_eof.sh: Also handle fadvise64_64 which is
used on 32 bit x86.  Note strace internally maps fadvise64_64
to {arm,xtensa}_fadvise64_64.
2017-12-10 17:13:18 -08:00
Pádraig Brady 5f02cddc18 dd: support iflag=direct with arbitrary sized files
* src/dd.c (iread): Handle read error with a non-aligned
file offset in the O_DIRECT case.  This is not an issue
on XFS at least, but on EXT4 the final read will return
EINVAL rather than the expected 0 to indicate EOF.
* tests/dd/direct.sh: Test the iflag=direct case also.
* NEWS: Mention the improvement.
2017-11-24 16:28:57 -08:00
Pádraig Brady e818f19cf8 tests: avoid false failure when O_DIRECT isn't supported
* tests/dd/nocache_eof.sh: Only run the O_DIRECT tests
when 512 byte alignment is supported.  Otherwise with older
XFS on systems with > 1MiB pages, or on file systems not
supporting O_DIRECT, there would have been false failures.
* tests/dd/direct.sh: Clarify the skip message.
2017-10-24 20:08:05 -07:00
Pádraig Brady de15a497d1 dd: fix nocache regions passed to posix_fadvise()
Previously with oflag=direct the call to invalidate_cache()
was not passed to the kernel, as it was less than a page size,
and a subsequent call was not made to invalidate the pending space.
Similarly with oflag=nocache the pending space at EOF was
not invalidated.  Even though these amount to only a single page
in the page cache it can be significant.  For example on
XFS before kernel patch v4.9-rc1-4-g0ee7a3f, O_DIRECT files
would have been read inefficiently if any pages were cached,
even if they were already synced to storage.

* src/dd.c (i_nocache_eof, o_nocache_eof): New bools used
to control when we want invalidate_cache(,0) to clear to EOF.
(cache_round): Use IO_BUFSIZE (currently 132KiB) to minimize
calls to the relatively expensive advise function, rather
than page_size.  This also makes it clear that while the
kernel function operates on pages, this size is chosen for
performance reasons.
(invalidate_cache): Refactor to share more code between
input and output paths. Use i_nocache_eof and o_nocache_eof
rather than proxying off max_records.  Ensure we
invalidate full pages when clearing to EOF as the kernel
will ignore any non complete pages.  Fix the offset used
for the output path.
(dd_copy): Invalidate the cache of the input after the
offset is updated, for consistency and so we don't try to
invalidate before the start of the file.  When we read
EOF on input, set flags so that we invalidate to EOF.
(main): Invalidate to EOF in more cases, by depending
on the i_nocache_eof and o_nocache_eof flags.
* doc/coreutils.texi (dd invocation): Clarify the alignment
and persisted caveats on the example applying "nocache"
to part of a file.
* tests/dd/nocache_eof.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Issue reported by Eric Bergen.
2017-10-24 19:38:12 -07:00