As described in commit v9.10-283-g5cb0cca3b
avoid using ulimit with $SHELL -c, and explict
invocation of the shell like that can require lots of memory,
often due to loading the locale archive.
* tests/basenc/bounded-memory.sh: Avoid $SHELL -c with ulimit.
* tests/cut/bounded-memory.sh: Likewise.
* tests/expand/bounded-memory.sh Likewise.
* tests/pr/bounded-memory.sh Likewise.
* tests/unexpand/bounded-memory.sh Likewise.
* tests/fold/fold-zero-width.sh: Bring the ulimit bound
down to the standard 6M over the determined limit.
With field delimiter = line delimiter we need to know
if there is any more data to be read, as field delimiter
in the last byte of the file is treated differently.
So reiterate the loop to ensure enough read()s to make
the appropriate determination.
* src/cut.c (bytesearch_field_delim_ok): Expand the range
of bytes that can be simply searched for. 0xF5-0xFF can't
appear in valid UTF-8 characters, and so may be used as
delimiters in UTF-8 input, so it's worth optimizing for.
* tests/cut/cut.pl: Add a test case (mainly as documentation).
This is about 20x faster.
Note we only do the delimiter search once per chunk,
and it's usually quick as delimiters wouldn't be too far
into the a chunk if present, so we don't bother
to cache the found delimiter.
Support ignoring leading and trailing whitespace.
E.g. this matches awk's default field splitting mode.
* src/cut.c
* tests/cut/cut.pl: Add test cases.
To improve compatibility with toybox/busybox scripts.
* doc/coreutils.texi (cut invocation): Add -O description.
* src/cut.c: Support -O as well as --output-delimiter
* tests/cut/cut.pl: Adjust one case to use -O.
Both the i18n patch and FreeBSD/macOS support this option.
They do differ in behavior somewhat as the i18n patch
may output more bytes than requested.
$ printf '\xc3\xa9b\n' | i18n-cut -n -b1
é
There is also a bug in the i18n patch with multi-byte
at the start of a line:
$ printf '\xc3\xa9b\n' | i18n-cut -n -b1-2
éb
We follow the FreeBSD behavior since it seems more
useful to have -b be a hard limit, rather than a soft limit.
This also reduces the possibility of duplicate character output
with separate cut invocations with non overlapping byte ranges.
* src/cut.c (cut_bytes_no_split): A new function
similar to cut_characters, to handle multi-byte characters
with byte limit semantics.
* tests/cut/cut.pl: Add test cases.
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...
* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
ulimit -v is generally not supported with ASAN, giving errors like:
"ReserveShadowMemoryRange failed while trying to map 0x... bytes.
Perhaps you're using ulimit -v"
* tests/cp/link-heap.sh: Mention ASAN as a possible reason for skipping.
* tests/csplit/csplit-heap.sh: Likewise.
* tests/cut/cut-huge-range.sh: Likewise.
* tests/dd/no-allocate.sh: Likewise.
* tests/printf/printf-surprise.sh: Likewise.
* tests/rm/many-dir-entries-vs-OOM.sh: Likewise.
* tests/head/head-c.sh: Only skip the part of the test needing ulimit.
* tests/split/line-bytes.sh: Likewise.
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...
* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Manually update copyright year,
until we fully sync with gnulib at a later stage.
* tests/sample-test: Adjust to use the single most recent year.
Also, diagnose the '-' in "cut -f -" as an invalid range, rather
than interpreting it as the unlimited range, "1-".
* NEWS: Mention these changes.
* src/cut.c (set_fields): Don't interpret an accumulator "value"
of 0 as an unspecified range endpoint.
Give better diagnostics.
Adjust a comment so that it is true also for 64-bit size_t.
* tests/cut/Test.pm: Add tests for the above.
* Transform all Makefile.am files so that when running "make check",
CU_TEST_NAME is set to the name of the test. This is so that when I
run valgrind-enabled (--log-file-qualifier=CU_TEST_NAME) "make check"
on the entire package it is more convenient to map a leak or error
found in a valgrind log file back to the offending test.
Use this command:
(echo tests/Makefile.am.in; find tests -name Makefile.am) \
|xargs perl -pi -e '/^(\s*)PATH=...VG_PATH_PREFIX/ and ' \
-e 'print $1,q|CU_TEST_NAME=`basename $(abs_srcdir)`,$$tst |,"\\\n"'
Remove from CVS, since ./bootstrap generates them automatically.
* .cvsignore: Add INSTALL, Makefile.in, *.cache, *.lineno, *.log.
Remove more-specific entries. This catches files like configure.lineno.
* man/.cvsignore: Add Makefile.in.
* src/.cvsignore: Add Makefile.in.
Remove .version, dir.c, install, mvdir, stamp-v, vdir.c, version.c.
For .cvsignore file under the tests directory:
Add Makefile.in. Sort entries if necessary. Remove *.I, *.E,
*.X, *.O, *-tests, build-script, mk-script if they're never
created in this directory.
* build-aux/.cvsignore: New file.
* doc/.cvsignore: Add Makefile.in, coreutils.html, coreutils.pdf,
coreutils.ps, coreutils.tps. Remove coreutils.cm (dunno what it
is, but the makefile doesn't mention it). Remove coreutils.info
as it is subsumed by coreutils.info*.
* lib/.cvsignore: Add Makefile.in, getdate.tab.h.
Remove stat.c, sysexit.h.
* m4/.cvsignore: Remove Makefile, dev-ino.m4, glibc2.m4, intdiv0.m4,
inttypes-h.m4, inttypes-pri.m4, jm-glibc-io.m4, lcmessage.m4,
lock.m4, printf-posix.m4, same-inode.m4, size_max.m4,
uintmax_t.m4, ulonglong.m4, visibility.m4, xsize.m4. Many of
these are put in automatically by ../bootstrap, but that's a
bootstrap bug that I plan to fix shortly.
* po/.cvsignore: More ../bootstrap-related fixes, plus remove old
cruft. Add *.po, LINGUAS, Makevars. Remove *.cat, *.msg,
cat-id-dbl.c, messages.mo, stamp-cat-id.
backed out due to updates provoked by the copyright changes.
* tests/Makefile.am.in (PATH): Prepend $(VG_PATH_PREFIX), so that
it propagates to the derived Makefile.am files.
($(srcdir)/Makefile.am): Mark generated .am files as read-only,
so we don't mistakenly edit them again.
* tests/cut/Makefile.am: Regenerate.
* tests/head/Makefile.am: Likewise.
* tests/join/Makefile.am: Likewise.
* tests/pr/Makefile.am: Likewise.
* tests/sort/Makefile.am: Likewise.
* tests/tac/Makefile.am: Likewise.
* tests/tail/Makefile.am: Likewise.
* tests/test/Makefile.am: Likewise.
* tests/tr/Makefile.am: Likewise.
* tests/uniq/Makefile.am: Likewise.
* tests/wc/Makefile.am: Likewise.