Commit Graph
393 Commits
Author SHA1 Message Date
Sylvestre Ledru 64ea3f0553 tests: sort: check that a -t-like operand after -- is a file
* tests/sort/sort.pl (dash-dash-t): New test.
https://github.com/uutils/coreutils/pull/15823

Link: https://github.com/coreutils/coreutils/pull/450
2026-11-09 00:24:53 +00:00
Collin Funk 79063df078 sort: don't share compressed temp files for the same input when merging
* src/sort.c (sortlines): Don't share compressed temporary files when
the same input is given multiple times.
* tests/sort/sort-compress-hang.sh: Add a test case that would
previously hang.
* NEWS: Mention the bug fix.

Discovered by Anthropic's OSS Scanner, as bug ANT-2026-MYSN7JAW.
2026-11-08 12:01:22 -08:00
Sylvestre Ledru 307bdd9441 tests: sort: ensure sort rejects -o combined with --debug
Reported at https://github.com/uutils/coreutils/issues/15998

* tests/sort/sort.pl: Add test case.

Link: https://github.com/coreutils/coreutils/pull/433
2026-11-07 12:35:10 +00:00
Pádraig Brady d8df89c29d tests: fix false failure with spaces in the path
* tests/sort/sort-compress.sh: Quote the found sort exe.
Reported by Bruno Haible.
2026-11-03 13:23:47 +00:00
Collin Funk f30c663c73 sort: use a default PATH when the environment variable is undefined
* gnulib: Update to get commit 19e602707d (findprog-in: Use a default
PATH like POSIX specifies., 2026-11-02).
* tests/sort/sort-compress.sh: Add a test case.
* NEWS: Mention the bug fix.

Discovered by Anthropic's OSS Scanner, as bug ANT-2026-1R7RAVAG.
2026-11-02 15:30:16 -08:00
Sylvestre Ledru fdca506d98 tests: sort: check -t given twice must be the same separator
* tests/sort/sort.pl (tab-dup1, tab-dup2, tab-incompat1)
(tab-incompat2, tab-incompat3): New tests.  Ensure differing -t
separators, including NUL, evoke "incompatible tabs", while
repeating the same one is accepted.
https://github.com/uutils/coreutils/pull/15707

Link: https://github.com/coreutils/coreutils/pull/399
2026-10-25 13:43:21 +00:00
Pádraig Brady ff1141fcec tests: make process cleanup more robust
* tests/cat/cat-buf.sh: Always call 'wait' in case the process already
exited before being killed.
* tests/cp/existing-perm-race.sh: Likewise.
* tests/cp/file-perm-race.sh: Likewise.
* tests/cp/parent-perm-race.sh: Likewise.
* tests/cp/sparse-to-pipe.sh: Likewise.
* tests/du/move-dir-while-traversing.sh: Likewise.
* tests/help/help-version.sh: Likewise.
* tests/misc/stdbuf.sh: Likewise.
* tests/mv/i-3.sh: Likewise.
* tests/printf/printf-surprise.sh: Likewise.
* tests/rm/isatty.sh: Likewise.
* tests/sort/sort-compress-proc.sh: Likewise.
* tests/sort/sort-spinlock-abuse.sh: Likewise.
* tests/tail/F-headers.sh: Likewise.
* tests/tail/F-vs-missing.sh: Likewise.
* tests/tail/F-vs-rename.sh: Likewise.
* tests/tail/append-only.sh: Likewise.
* tests/tail/assert-2.sh: Likewise.
* tests/tail/assert.sh: Likewise.
* tests/tail/debug.sh: Likewise.
* tests/tail/descriptor-vs-rename.sh: Likewise.
* tests/tail/follow-name.sh: Likewise.
* tests/tail/follow-stdin.sh: Likewise.
* tests/tail/follow-symlink.sh: Likewise.
* tests/tail/inotify-dir-recreate.sh: Likewise.
* tests/tail/inotify-hash-abuse.sh: Likewise.
* tests/tail/inotify-hash-abuse2.sh: Likewise.
* tests/tail/inotify-parent-hash.sh: Likewise.
* tests/tail/inotify-rotate.sh: Likewise.
* tests/tail/pid.sh: Likewise.
* tests/tail/pipe-f2.sh: Likewise.
* tests/tail/retry.sh: Likewise.
* tests/tail/symlink.sh: Likewise.
* tests/tail/tail-n0f.sh: Likewise.
* tests/tail/truncate.sh: Likewise.
* tests/tail/wait.sh: Likewise.
* tests/tee/tee.sh: Likewise.
* tests/timeout/timeout-group.sh: Likewise.
* tests/rm/dangling-symlink.sh: Likewise.
Also avoid redundant cleanup_ calls.
* tests/tail/flush-initial.sh: Likewise.
* tests/tail/inotify-rotate-resources.sh: Likewise.
2026-10-25 13:11:49 +00:00
Sylvestre Ledru eac02dd7a2 tests: sort: check long lines survive an external sort
* tests/sort/sort.pl (@long_lines): New variable.
(@Tests): Add a test case sorting lines much longer than the buffer
with a small -S, so they go through temporary files.
https://github.com/uutils/coreutils/pull/15645

Link: https://github.com/coreutils/coreutils/pull/388
2026-10-24 16:15:03 +01:00
Sylvestre Ledru 782219ffa2 tests: sort: check -b and -M skip only space and tab
* tests/sort/sort.pl (@Tests): Add test cases ensuring a leading \f
or \r is not skipped by -b or -M, while space and tab still are.
https://github.com/uutils/coreutils/pull/15647

Link: https://github.com/coreutils/coreutils/pull/388
2026-10-24 16:15:02 +01:00
Sylvestre Ledru c0684d5bbb tests: sort: check a whole-line key's own reverse flag
* tests/sort/sort.pl (@Tests): Add test cases for a single key
covering the whole line with its own 'r' option, and for such a key
not inheriting the global -r.
https://github.com/uutils/coreutils/pull/15626

Link: https://github.com/coreutils/coreutils/pull/388
2026-10-24 16:15:02 +01:00
Collin Funk 98c8b158b6 tests: make checks for $LOCALE_FR and $LOCALE_FR_UTF8 consistent
* tests/cp/non-utf8-name.sh: Check for $LOCALE_FR and $LOCALE_FR_UTF8
being empty or the string "none".
* tests/date/percent-percent.sh: Likewise.
* tests/du/block-size.sh (test_unit): Likewise.
* tests/ls/non-utf8-hidden.sh: Likewise.
* tests/mktemp/bad-unicode.sh: Likewise.
* tests/sort/sort-unique.sh: Likewise.
* tests/df/problematic-chars.sh:  Check for $LOCALE_FR_UTF8 being empty
or the string "none".
* tests/expand/mb.sh: Likewise.
* tests/fold/fold-characters.sh: Likewise.
* tests/fold/fold-nbsp.sh: Likewise.
* tests/fold/fold-spaces.sh: Likewise.
* tests/fold/fold-zero-width.sh: Likewise.
* tests/join/join-utf8.sh: Likewise.
* tests/misc/sleep.sh: Likewise.
* tests/nl/multibyte.sh: Likewise.
* tests/paste/multi-byte.sh: Likewise.
* tests/printf/printf-mb.sh: Likewise.
* tests/printf/printf-quote.sh: Likewise.
* tests/sort/sort-debug-keys.sh: Likewise.
* tests/unexpand/mb.sh: Likewise.
2026-10-04 15:31:59 -07:00
Pádraig Brady 83009c56fe tests: sort: avoid a false failure on very slow systems
* tests/sort/sort-NaN-infloop.sh: Up the timeout to about 240x
a "standard" system, as this was seen to timeout on a
mipsel VM using QEMU emulation.
Reported by Bruno Haible.
2026-09-10 16:12:19 +01:00
Collin Funk 9e4b42982a tests: sort: avoid a failure on GNU/Hurd
On GNU/Hurd, file descriptor limits set via 'ulimit -n' seem to be
ineffective when set below the default:

    $ (ulimit -n; ulimit -n 512; ulimit -n; \
         tee $(seq 1024) < /dev/null; echo $?)
    1024
    512
    tee: 1022: Too many open files
    tee: 1023: Too many open files
    tee: 1024: Too many open files
    1

* tests/sort/sort-merge-fdlimit.sh: Don't fail if using 'ulimit -n' to
decrease the file descriptor limit is ineffective.
2026-09-08 23:53:25 -07:00
Collin Funk 1ab303a901 tests: sort: check that non-blank characters aren't treated as blank
* tests/sort/sort.pl (@Tests): Add a few test cases.
2026-08-26 20:19:56 -07:00
Collin Funk 6ac1d0907a tests: sort: check LC_ALL and LC_COLLATE
* tests/sort/sort-locale.sh: Also check LC_COLLATE.

Link: https://github.com/coreutils/coreutils/pull/269
2026-08-20 17:37:01 +01:00
Sylvestre Ledru af1ba4244c tests: sort: check byte-wise ordering in the C.UTF-8 locale
* tests/sort/sort-locale.sh: C.UTF-8 has a UTF-8 codeset but the
collation rules of C, so add a case ensuring the ordering there stays
byte-wise rather than following the linguistic rules exercised for
en_US.iso8859-1 and fr_FR.UTF-8 above.

Link: https://github.com/coreutils/coreutils/pull/323
2026-07-27 13:52:51 +01:00
Collin Funk 05d237cc56 tests: sort: check that invalid --parallel arguments are diagnosed
* tests/sort/sort.pl (@Tests): Add a few test cases.
2026-06-21 19:11:51 -07:00
Sylvestre Ledru 5008d5d112 tests: sort: cover punctuation weight in locale collation
* tests/sort/sort-locale.sh: In a UTF-8 locale '-' carries a minimal
(shifted) weight, so 'file1' sorts before 'file-2' even though '-'
precedes '1' in the C/byte order.  This is the opposite of the C locale
result, so it also exercises the differs-from-C path.
Identified here: https://github.com/uutils/coreutils/pull/12677
Closes https://github.com/coreutils/coreutils/pull/283
2026-06-09 14:08:02 +01:00
Sylvestre Ledru 6f8600f5d1 tests: sort: check sort -m preserves long lines
* tests/sort/sort-merge.pl: Add 'long-lines' case ensuring that
merging input whose lines exceed the read buffer does not split
lines on buffer boundaries.
https://github.com/coreutils/coreutils/pull/270
2026-05-29 19:25:40 +01:00
Pádraig Brady dce828c3d6 tests: fix false failure with spaces in hierarchy
* tests/sort/sort-buffer-size.sh: Use `pwd` rather than $PWD,
and quote appropriately.
Reported by Bruno Haible.
2026-05-04 23:38:51 +01:00
Pádraig Brady 38cf97c1ff sort: use more dynamic memory allocation with pipes
The default memory allocation with pipes was too passive/static,
resulting in not allocating enough memory to enable threading.
By dynamically reallocating the buffer when reading from
unknown sized inputs we better use available memory and threads.

  $ time seq 10000000 -1 0 | sort-old >/dev/null
  real	0m16.523s
  user	0m16.900s
  sys	0m0.167s

  $ time seq 10000000 -1 0 | sort-old -S1G >/dev/null
  real	0m12.263s
  user	0m29.646s
  sys	0m0.527s

  $ time seq 10000000 -1 0 | sort-new >/dev/null
  real	0m12.994s
  user	0m31.266s
  sys	0m0.716s

It also avoids the overhead of writing to temp files
for modestly sized inputs. For example the following
input would induce interaction with temp storage:

  $ seq 125000 | wc -c
  763895

* src/sort.c (sort_buffer_size): Rename to ...
(sort_buffer_policy): ... here, and adjust to set
an initial size and limit, rather than just a size.
(fillbuf): Add a POLICY parameter, and use that
to call maybe_growbuf() as needed.
(maybe_growbuf): Return true if POLICY dictates we
should grow the buffer, and try_growbuf() was
able to reallocate the larger buffer.
* tests/sort/sort-buffer-size.sh: Add a new test.
* tests/local.mk: Reference new test.
* NEWS: Mention the improvement.
Related to https://bugs.gnu.org/10877
2026-04-29 13:16:21 +01:00
oech3 f638c29df6 tests: sort: ensure --batch-size=(rlimit+1) fails
* tests/sort/sort-merge-fdlimit.sh: Add test case.
https://github.com/coreutils/coreutils/pull/182
2026-02-12 17:27:49 +00:00
Collin Funk ec77664a31 tests: avoid failures if 'ulimit -n' cannot set file descriptor limits
This fixes test failures seen on Haiku.

* tests/ls/recursive.sh: Run 'ls' even if ulimit fails.
* tests/split/r-chunk.sh: Run 'split' even if ulimit fails.
* tests/sort/sort-merge-fdlimit.sh: Skip test if 'ulimit -n' cannot set
file descriptor limits.
Reported by Bruno Haible.
2026-01-24 11:20:56 -08:00
Collin Funk 73d3a49f45 maint: run 'make update-copyright' 2026-01-01 10:56:16 -08:00
Pádraig Brady 06ed0625bd tests: fix recent portability issue with printf
* tests/sort/sort-locale.sh: Avoid non portable printf \u....
* cfg.mk (sc_env_printf): Add a new syntax check to flag future cases.
2025-12-11 22:02:55 +00:00
Pádraig Brady 2692d599ec tests: fix non-portable use of printf '\x..'
* tests/dd/conv-case.sh: Use octal instead.
* tests/ls/hyperlink.sh: Likewise.
* tests/sort/sort-locale.sh: Likewise.
2025-12-08 22:32:48 +00:00
Pádraig Brady faa566372d test: re-enable tests that had a false requirement on coreutils' kill
Recently we've not built our kill command by default,
so reduce test dependence on that.

* tests/sort/sort-compress-proc.sh: `kill -l [exit_status]` is well
supported, and is a POSIX requirement.
* tests/timeout/timeout.sh: There were actually no kill invocations
in this test at all.
2025-11-29 12:17:27 +00:00
Pádraig Brady d6fc91f37a tests: add tests for locale ordering
* tests/sort/sort-locale.sh: Check sort,ls have the same (non C) order.
* tests/local.mk: Reference the new test.
2025-11-11 14:29:36 +00:00
Pádraig Brady b294aff3fe sort: fix silent exit upon SIGPIPE from --compress-program
* src/sort.c (main): Ignore SIGPIPE so we've more control over
how we handle for stdout and compression programs.
(sort_die): Handle EPIPE from stdout and mimic a standard SIGPIPE,
otherwise reverting to a standard exit(SORT_FAILURE);
* tests/sort/sort-compress-proc.sh: Add a test case.
* NEWS: Mention the bug fix.
2025-10-29 19:03:23 +00:00
Collin Funk 2b3eb795c8 sort: use the more efficient posix_spawn to invoke --compress-program
* NEWS: Mention the improvement. Mention that 'sort' will continue
without compressing temporary files if the program specified by
--compress-program cannot be executed.
* doc/coreutils.texi (sort invocation): Document the behavior when the
program specified by --compress-program cannot be executed.
* src/sort.c: Include spawn.h.
(MAX_FORK_TRIES_COMPRESS, MAX_FORK_TRIES_DECOMPRESS): Remove definition.
(MAX_TRIES_COMPRESS, MAX_TRIES_DECOMPRESS): New definitions based on
MAX_FORK_TRIES_COMPRESS and MAX_FORK_TRIES_DECOMPRESS.
(async_safe_die): Remove function.
(posix_spawn_file_actions_move_fd): New function.
(pipe_fork): Remove function.
(pipe_child): New function based on pipe_fork. Return an error number
instead of a pid. Use posix_spawnp instead of calling fork and expecting
the caller to exec.
(maybe_create_temp): Call pipe_child instead of pipe_fork. Print a
warning to standard error if --compress-program cannot be executed and
the error is different than the previous call. Remove code for the child
process.
(open_temp): Remove code for the child process. Improve error message.
* tests/sort/sort-compress.sh: Add a test case for when the program
specified by --compress-program does not exist.
2025-10-25 12:59:08 -07:00
Collin Funk c9a30d6781 maint: use consistent references to standard files in messages
* cfg.mk (sc_standard_outputs): Add a grep command for source files.
* src/du.c (main): Use standard input instead of stdin, standard output
instead of stdout, and standard error instead of stderr in messages.
* src/nohup.c (main): Likewise.
* src/sort.c (main): Likewise.
* src/split.c (main): Likewise.
* src/stdbuf.c (main): Likewise.
* src/wc.c (main): Likewise.
* tests/du/files0-from.pl (@Tests): Adjust test case to new messages.
* tests/sort/sort-files0-from.pl: Likewise.
* tests/wc/wc-files0-from.pl: Likewise.
2025-08-05 18:46:04 -07:00
Pádraig Brady 083f89528e tests: sort: add a test for equivalent floats
* tests/sort/sort-float.sh: Ensure equivalent floats
are treated as such.
2025-07-12 19:58:06 +01:00
Paul Eggert 3a9ec02138 tests: fix fraction comparison in sort-float
Problem reported by Cosima Neidahl <https://bugs.gnu.org/78985#13>.
* tests/sort/sort-float.sh: At top level, use C locale at first.
(dbl_minima_order): Assume C locale.
Use string comparison for the fractional parts.

2025-07-10  Paul Eggert  <eggert@cs.ucla.edu>

tests: fix integer overflow in sort-float
Problem reported by Cosima Neidahl <https://bugs.gnu.org/78985>.
* tests/sort/sort-float.sh (dbl_minima_order):
Use expr instead of test, to avoid problems with integers
too large for the shell.
2025-07-11 14:45:04 -07:00
Paul Eggert 8f9fc8f08c tests: fix integer overflow in sort-float
Problem reported by Cosima Neidahl <https://bugs.gnu.org/78985>.
* tests/sort/sort-float.sh (dbl_minima_order):
Use expr instead of test to compare fractions,
to avoid problems with integers too large for the shell.
2025-07-10 10:18:06 -07:00
Pádraig Brady 21a662b1ee tests: sort: expand multiple output testing
* tests/sort/sort.pl: Ensure duplicate
output strings are supported.
2025-05-28 21:11:34 +01:00
Pádraig Brady 8c9602e3a1 sort: fix buffer under-read (CWE-127)
* src/sort.c (begfield): Check pointer adjustment
to avoid Out-of-range pointer offset (CWE-823).
(limfield): Likewise.
* tests/sort/sort-field-limit.sh: Add a new test,
which triggers with ASAN or Valgrind.
* tests/local.mk: Reference the new test.
* NEWS: Mention bug fix introduced in v7.2 (2009).
Fixes https://bugs.gnu.org/78507
2025-05-20 19:36:00 +01:00
Pádraig Brady 9ed5770b5e tests: fix false failure with multi-byte thousands separators
* tests/sort/sort-h-thousands-sep.sh: sv_SE defaults to UTF-8
on macOS 18, so avoid the test for multi-byte separators.
Fixes https://bugs.gnu.org/77509
2025-04-04 12:00:24 +01:00
Pádraig Brady fbfd886e59 sort: drop "note" from a --debug message
* src/sort.c (key_warnings): Remove "note " from the start
of a usually informational message, as this simplifies translation.
* tests/sort/sort-debug-warn.sh": Adjust accordingly.
Fixes https://bugs.gnu.org/75763
2025-01-22 17:55:54 +00:00
Pádraig Brady 28b176085f maint: update all copyright year number ranges
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...

* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2025-01-01 09:33:08 +00:00
Pádraig Brady 79c628901d tests: simplify treatment of the $LOCALE_FR_UTF8 variable
* tests/df/problematic-chars.sh: Rely on gnulib setting
this to "none" where not usable.
* tests/misc/sleep.sh: Likewise.
* tests/printf/printf-mb.sh: Likewise.
* tests/printf/printf-quote.sh: Likewise.
* tests/sort/sort-debug-keys.sh: Likewise.
2024-02-27 14:27:04 +00:00
Pádraig Brady a966dcdb69 maint: update all copyright year number ranges
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...

* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Manually update copyright year,
until we fully sync with gnulib at a later stage.
* tests/sample-test: Adjust to use the single most recent year.
2024-01-01 13:27:23 +00:00
Pádraig Brady ebb985dc41 tests: run locale tests on more systems
* tests/misc/numfmt.pl: Determine the thousands grouping character
in use, rather than skipping locale tests when it's not a space.
For example fr_FR.UTF-8 uses "NARROW NO-BREAK SPACE" as the grouping
char on modern glibc systems at least.
* tests/sort/sort-h-thousands-sep.sh: Likewise.
2023-12-29 18:04:29 +00:00
Paul Eggert 2dddc87214 maint: spelling fixes, including author names
Most of this just affects commentary and documentations.  The only
significant behavior change is translating author names via
proper_name_lite rather than proper_name_utf8, or not translating
them at all.  proper_name_lite is good enough for coreutils and
avoids the bloat that had coreutils not using Gnulib proper_name.
* bootstrap.conf (gnulib_modules): Use propername-lite instead
of propername.
(XGETTEXT_OPTIONS): Look for proper_name_lite instead of for
proper_name_utf8.
* cfg.mk (local-checks-to-skip): Remove
sc_proper_name_utf8_requires_ICONV, since we no longer use
proper_name_utf8.
(old_NEWS_hash): Update.
(sc_check-I18N-AUTHORS): Remove; no longer needed.
2023-08-28 14:06:43 -07:00
Paul Eggert 9348edb6b6 sort: port sort-merge-fdlimit test to Solaris 10
* tests/sort/sort-merge-fdlimit.sh: Give 'sort' fd 6 too.
Needed for the same reason sort-continue.sh needed a ulimit -n boost.
2023-08-27 20:50:04 -07:00
Paul Eggert 5c9998fbab sort: port sort-continue test back to Solaris 10
* tests/sort/sort-continue.sh: Use ulimit -n 7 not -n 6.  On
Solaris 10 'sort' uses Gnulib mkostemp, which calls Gnulib
getrandom, which opens /dev/urandom to calculate the temp file's
name, which means 'sort' needs one more file descriptor to work.
2023-08-27 19:15:31 -07:00
Pádraig Brady fc5e68612f tests: fix false failure due to locale on alpine
* tests/sort/sort-debug-keys.sh: Decimal point was seen to be '.'
on fr_FR.UTF-8 on Alpine Linux 3.18, so add an extra guard
to ensure we've a ',' as the decimal point on this locale.
Fixes https://bugs.gnu.org/65310
2023-08-15 22:55:41 +01:00
Sylvestre Ledru 15925d0e5b tests: move tests to a directory per utility
* cfg.mk: Adjust syntax check exclusion paths.
* tests/local.mk: Adjust for renamed tests.
2023-06-19 13:12:37 +01:00
Jim Meyering 15b0d75930 tests: remove directory, tests/sort/
* configure.ac (AC_CONFIG_FILES): Remove tests/sort/Makefile.
* tests/Makefile.am (SUBDIRS): Remove sort.
* tests/misc/sort: New file, with tests from...
* tests/sort/Test.pm: ...here.  Remove file.
2008-05-09 08:11:12 +02:00
Andreas Schwab cb3147d298 sort: add --sort=... option.
* src/sort.c (SORT_OPTION): New enum.
(sort_args, sort_types): Define.
(usage, long_options, main): New option --sort.
* tests/sort/Test.pm: Test it.
* doc/coreutils.texi (sort invocation): Document --sort option.
* NEWS: Mention this.
2008-02-20 08:41:29 +01:00
Jim Meyering ae96ae296b Clarify a comment.
* tests/sort/Test.pm: Replace a vague ChangeLog reference with a URL.
2008-02-02 09:04:24 +01:00