Commit Graph
9507 Commits
Author SHA1 Message Date
Pádraig Brady 25f0702eaa maint: cut: various code cleanups and comments
* src/cut.c: Document some functions, and remove extraneous
 abstractions.
2026-04-05 13:15:56 +01:00
Pádraig Brady 5d339d583d cut: support no delimiter match fast path with -s
* src/cut.c (cut_fields_bytesearch): Just skip the data with -s.
2026-04-05 13:15:56 +01:00
Pádraig Brady 032ecdee9b maint: cut: cleanup context management for byte search
* src/cut.c: Hoist at_eof into context so we're not
querying it multiple times.  Also add a helper
to explicitly init bytesearch_context.
2026-04-05 13:15:56 +01:00
Pádraig Brady 24571c41f3 cut: optimize UTF-8 input with 0xF5-0xFF delimiters
* src/cut.c (bytesearch_field_delim_ok): Expand the range
of bytes that can be simply searched for. 0xF5-0xFF can't
appear in valid UTF-8 characters, and so may be used as
delimiters in UTF-8 input, so it's worth optimizing for.
* tests/cut/cut.pl: Add a test case (mainly as documentation).
2026-04-05 13:15:56 +01:00
Pádraig Brady 7ceb4c348b maint: cut: simplify mbbuf_fill
We can only byte search with uni-byte or utf-8.
utf-8 implicitly can't false match a delimiter at buffer boundary.
So don't worry about finding the exact utf8 boundary at end of buffer,
rather just ensuring the buffer always starts with a valid character
(by ensuring MCEL_LEN_MAX-1 moved to start of buffer on each refill).
2026-04-05 13:15:56 +01:00
Pádraig Brady 5dd1a0bc77 maint: refactor is_utf8_charset helper to system.h
* src/cut.c: Move from here.
* src/numfmt.c: Likeise.
* src/system.h: To here.
2026-04-05 13:15:56 +01:00
Pádraig Brady 36751ab993 cut: optimize per character memcpy
$ time src/cut-before -f10 -w ll.in >/dev/null
real	0m4.309s

$ time src/cut-after -f10 -w ll.in >/dev/null
real	0m3.136s

* src/cut.c (cut_bytes): Add a new helper that avoids
the memcpy call in the common case of adding characters to a buffer.
2026-04-05 13:15:56 +01:00
Pádraig Brady 4ae2275952 cut: refactor skip_line_remainder logic
Simplify and optimize field exhaustion logic:

$ time LC_ALL=C src/cut-before -f1 -dc as.in >/dev/null
real	0m0.057s

$ time LC_ALL=C src/cut-after -f1 -dc as.in >/dev/null
real	0m0.023s

* src/cut.c (cut_fields_bytesearch): Refactor.
2026-04-05 13:15:56 +01:00
Pádraig Brady 30c19a1279 maint: cut simplify cut_fields_bytesearch
refactor line delimiter output,
and resetting of parse record state.
2026-04-05 13:15:56 +01:00
Pádraig Brady 1da3ccced2 cut: enable fast path for all delimiter lengths
1. Removed !have_pending_line from the fast path condition.
This is safe because:
  - field_1_n_bytes == 0 already ensures we haven't started
    buffering field 1 content
  - The fast path correctly continues any pending partial line
    by writing raw bytes including the completing \n
2. Added have_pending_line = false after the fast path write,
since all lines up to the last \n are now complete.

$ time src/cut.before -f1 -dç sl.in >/dev/null
real	0m0.081s
$ time src/cut.after -f1 -dç sl.in >/dev/null
real	0m0.012s

$ time src/cut.before -f10 -dç sl.in >/dev/null
real	0m0.081s
$ time src/cut.after -f10 -dç sl.in >/dev/null
real	0m0.012s
2026-04-05 13:15:56 +01:00
Pádraig Brady 7a124eb585 cut: optimize -f with -d longer than lines
$ time src/cut.before -f1 -dç sl.in >/dev/null
real	0m0.157s

$ time src/cut.after -f1 -dç sl.in >/dev/null
real	0m0.084s
2026-04-05 13:15:56 +01:00
Pádraig Brady e73dbeb5ab cut: optimize -b for short lines
For a 40% performance increase it's worth reinstating the simple
original cut_bytes() which avoids data copying and function calls.
Once a longer line is encountered we defer to the buffered variant.

$ time src/cut.before -b2 sl.in  >/dev/null
real	0m0.101s

$ time src/cut.after -b2 sl.in  >/dev/null
real	0m0.060s
2026-04-05 13:15:56 +01:00
Pádraig Brady f5cab8e30c cut: optimize per character field scanning
Use specialized loops rather than branching per character,
giving a 28% increase.

$ time src/cut -f1 -w ll.in >/dev/null
real	0m7.199s

$ time src/cut -f1 -w ll.in >/dev/null
real	0m5.204s
2026-04-05 13:15:56 +01:00
Pádraig Brady a78ef2b65c cut: prefer c_isblank() to c32issep()
12% perf increase with:

$ time src/cut -f2 -w ll.in >/dev/null
real	0m6.469s

$ time src/cut -f2 -w ll.in >/dev/null
real	0m5.689s
2026-04-05 13:15:56 +01:00
Pádraig Brady 9817599526 cut: avoid fwrite calls for smaller amounts of data
This is quite significant:

yes abcdfeg | head -n1MB > big-file

$ time src/cut-before -b1,3 big-file >/dev/null
real	0m0.050s

$ time src/cut-after -b1,3 big-file >/dev/null
real	0m0.029s
2026-04-05 13:15:56 +01:00
Pádraig Brady a16d56d60c cut: optimize -b by avoiding per byte iteration
Always memchr(line_delim) which is fast and allows:

- skipping whole segments when the next selected byte is beyond them
- skipping unselected prefixes in bulk
- writing contiguous selected spans in bulk

This wins for lines >= 4 characters,
but is slower lines <= 3 characters, especially if selecting bytes 1-3.
That is unusual though.
2026-04-05 13:15:56 +01:00
Pádraig Brady ea6a7ba547 cut: optimize when no delimiter in input
This is about 20x faster.
Note we only do the delimiter search once per chunk,
and it's usually quick as delimiters wouldn't be too far
into the a chunk if present, so we don't bother
to cache the found delimiter.
2026-04-05 13:15:56 +01:00
Pádraig Brady 700ffc51a1 cut: optimize -w for uni-byte case
* src/cut.c: Limit search to SPACE and TAB
2026-04-05 13:15:56 +01:00
Pádraig Brady c1d7b492c6 doc: cut: document the -w option
* src/cut.c (usage): Mention blank characters are used to separate.
* doc/coreutils.texi (cut invocation): Likewise.  Also describe
the 'trimmed' argument and the relation to -F.
2026-04-05 13:15:56 +01:00
Pádraig Brady cf25ef286a cut: refactor find_bytesearch_field_terminator to be stateful
Allows better/simpler avoidance of repeated line/delim scans

TODO: speed up our really slow cut_fields_mb_any.
Compare for example:
  time src/cut -w  -f1 ll.in >/dev/null #14s
  time src/cut -d, -f1 ll.in >/dev/null #.1s
Could adjust so that LC_ALL=C does memchr2(space,tab) ?
2026-04-05 13:15:56 +01:00
Pádraig Brady 0adb7c6edd cut: avoid repeated searchs for line_delim in the multi-byte delim case
TODO: Refactor all this into find_bytesearch_field_terminator.
Also handle in the delim_length==1 case.
2026-04-05 13:15:56 +01:00
Pádraig Brady d9825aa9b1 cut: refactor all byte search to find_bytesearch_field_terminator
TODO: Perhaps also add search only fields mode
to avoid rescans of very long lines
2026-04-05 13:15:56 +01:00
Pádraig Brady 6250b59ef9 cut: optimize -f when finished processing fields for a line
TODO: simplify and compare perf
2026-04-05 13:15:56 +01:00
Pádraig Brady 352a396a16 cut: optimize -f for fhe common case of single byte delimiters
* TODO: perf comparison
2026-04-05 13:15:56 +01:00
Pádraig Brady 2a6b36ff5b cut: optimize -d '?' in UTF-8 case
ensure all ascii delims are processed with byte search in UTF-8
2026-04-05 13:15:56 +01:00
Pádraig Brady f6b3055f74 cut: merge cut_fields and cut_fields_bytesearch
TODO: See why this is much slower:
time LC_ALL=C.UTF-8 src/cut -f1 -dc as.in > /dev/null
2026-04-05 13:15:56 +01:00
Pádraig Brady b3ef6231bd cut: refactor -f to byte search and character processing
Not sure about this at all.
Only worthwhile if can also remove cut_fields_line_delim.

Refactored src/cut.c so the old UTF-8 byte-search path is now the
general byte-search field engine for all safe byte-search cases:
ordinary single-byte delimiters and valid UTF-8 delimiters. The old
cut_fields path did not go away completely; it is now
cut_fields_line_delim and is used only when the field delimiter equals
the record delimiter, because -d $'\n' and -z -d '' have different
semantics from normal line-based field splitting.

As part of that, I also folded the duplicated “start selected field”
logic into a shared helper, and renamed the byte-search helpers to match
their broader use. The current dispatcher in src/cut.c is now:
whitespace parser, then line- delimiter field mode, then byte-search
field mode, then the decoded multibyte parser.
2026-04-05 13:15:56 +01:00
Pádraig Brady 16b1ff40ae cut: fix 25% perf regression mentioned in previous change
* src/cut.c: Prefer strstr() over memmem() as the former is
optimized per platforma on GLIBC, where as the latter is only
optimized for S390 as of glibc-2.42

TODO: look at merging cut_fields and cut_fields_mb_utf8
as they're both byte search
2026-04-05 13:15:56 +01:00
Pádraig Brady 26028fb2c6 cut: use bounded memory in utf8 mode when possible
TODO: See why a bit slower than old code

$ time src/cut.old -f1 -dç mb.in >/dev/null
real	0m0.136s
user	0m0.096s
sys	0m0.039s

$ time src/cut.new -f1 -dç mb.in >/dev/null
real	0m0.170s
user	0m0.139s
sys	0m0.030s
2026-04-05 13:15:56 +01:00
Pádraig Brady 801686242e cut: faster utf8 processing
TODO: improve to use bounded memory where possible
2026-04-05 13:15:56 +01:00
Pádraig Brady a14ac29629 cut: support -F as an alias for -f -w -O ' '
To improve compatibility with toybox/busybox scripts.
2026-04-05 13:15:56 +01:00
Pádraig Brady 74f6692aa0 maint: cut: refactor buffered and ordinary field scanning
* src/cut.c: Merge scan_mb_field and read_mb_field_to_buffer
2026-04-05 13:15:56 +01:00
Pádraig Brady eb1f057746 cut: support --whitespace-delimited=trimmed
Support ignoring leading and trailing whitespace.
E.g. this matches awk's default field splitting mode.

* src/cut.c
* tests/cut/cut.pl: Add test cases.
2026-04-05 13:15:56 +01:00
Pádraig Brady 77ccacb9a7 cut: support -O as an alias for --output-delimiter
To improve compatibility with toybox/busybox scripts.

* doc/coreutils.texi (cut invocation): Add -O description.
* src/cut.c: Support -O as well as --output-delimiter
* tests/cut/cut.pl: Adjust one case to use -O.
2026-04-05 13:15:56 +01:00
Pádraig Brady f644b4ca53 cut: refactor multi-byte updates
* src/cut.c: 160 fewer lines

Helpers extracted (replacing repeated inline patterns):
- write_line_delim(), write_pending_line_delim(), reset_item_line()
  - line boundary code used by cut_bytes{,no_split}, cut_characters
- write_selected_item()
  - output-delimiter + write logic used by all three byte/char functions
- reset_field_line()
  - field line reset used by cut_fields_mb_any

Field functions unified via cut_fields_mb_any(stream, whitespace_mode):
- struct mbfield_parser encapsulates the whitespace vs.
  fixed-delimiter state (saved char, mode flag)
- mbfield_get_char() - dispatches to saved-char or direct read
- mbfield_terminator()
  - returns FIELD_{DATA,DELIMETER,LINE_DELIMITER} based on mode
- read_mb_field_to_buffer()
  - replaces the two duplicated first-field buffering loops
- scan_mb_field(mbbuf, parser, pending, write_field)
  - replaces the four duplicated field scan loops
  (print+skip × two modes) with a single function and a write_field bool
- cut_fields_mb and cut_fields_ws are now trivial wrappers
2026-04-05 13:15:56 +01:00
Pádraig Brady 57110d8bae cut: implement -n to avoid outputting partial characters
Both the i18n patch and FreeBSD/macOS support this option.
They do differ in behavior somewhat as the i18n patch
may output more bytes than requested.

  $ printf '\xc3\xa9b\n' | i18n-cut -n -b1
  é

There is also a bug in the i18n patch with multi-byte
at the start of a line:

  $ printf '\xc3\xa9b\n' | i18n-cut -n -b1-2
  éb

We follow the FreeBSD behavior since it seems more
useful to have -b be a hard limit, rather than a soft limit.
This also reduces the possibility of duplicate character output
with separate cut invocations with non overlapping byte ranges.

* src/cut.c (cut_bytes_no_split): A new function
similar to cut_characters, to handle multi-byte characters
with byte limit semantics.
* tests/cut/cut.pl: Add test cases.
2026-04-05 13:15:56 +01:00
Pádraig Brady 74047ec55e cut: adjust error message to be less specific
* src/cut.c (main): Cater for both misplaced -w and -d.
2026-04-05 13:15:56 +01:00
Pádraig Brady 19aa72b4ea cut: implement -w,--whitespace-delimited
* src/cut.c (cut_fields_ws): A new function handling both
uni-byte and multi-byte cases.
* tests/cut/cut.pl: Add a test cases.
2026-04-05 13:15:56 +01:00
Pádraig Brady 32f1de5b4f cut: support single byte -d with GB18030 input
* src/cut.c
* tests/cut/mb-non-utf8.sh
* tests/local.mk
2026-04-05 13:15:55 +01:00
Pádraig Brady 94ddf45a60 cut: support single byte -d that may be part of multi-byte
Note this is a slight divergence from the i18n patch
as that switched to uni-byte for any single byte delimiter
that is not valid multi-byte.

That results in possibly splitting in the middle of
a valid multi-byte character.

Instead we only split on a single byte when they're
not part of a multi-byte character.

* src/cut.c
2026-04-05 13:15:55 +01:00
Pádraig Brady a021b0b698 cut: support multi-byte field delimiters
* src/cut.c
* tests/cut/cut.pl
2026-04-05 13:15:55 +01:00
Pádraig Brady 97703386e6 cut: support multi-byte input with -c
* src/cut.c
* tests/cut/cut.pl
2026-04-05 13:15:55 +01:00
Pádraig Brady fb78200249 maint: cut: refactor output calls
* src/cut.c (cut_fields): Refactor calls to fwrite() and putchar()
2026-04-05 13:15:55 +01:00
Pádraig Brady a04a1054f8 doc: cut: reorder --complement alphabetically in help
* src/cut.c (usage): Move placement of --comlement description.
* doc/coreutils.texi (cut invocation): Likewise.
2026-04-05 13:15:55 +01:00
Pádraig Brady 1173ebb7c8 doc: cut: clarify description of -b and -c
* src/cut.c (usage): State the arguments are positions,
in case users may think they were values.
2026-04-05 13:15:55 +01:00
Collin Funk 38fc6bde64 basename: avoid duplicate strlen calls on the suffix
$ ltrace -c ./src/basename-prev -s a $(seq 100000) > /dev/null
    % time     seconds  usecs/call     calls      function
    ------ ----------- ----------- --------- --------------------
     50.00   30.030316          75    400000 strlen
    [...]
    $ ltrace -c ./src/basename -s a $(seq 100000) > /dev/null
    % time     seconds  usecs/call     calls      function
    ------ ----------- ----------- --------- --------------------
     42.88   22.413953          74    300001 strlen
    [...]

* src/basename.c (remove_suffix, perform_basename): Add a length
argument for the suffix and use it instead of strlen.
(main): Calculate the suffix length. Refactor code to avoid calling
perform_basename in multiple places.
2026-04-04 14:55:56 -07:00
Paul Eggert b64f9bfe4f date: simplify -u by not calling putenv
* src/date.c (TZSET): Remove; no longer needed.
(main): Simplify -u’s implementation by passing "UTC0" to tzalloc,
rather than by setting TZ in the environment and then calling getenv.
The old way of doing things dates back to before we had tzalloc.
* configure.ac (LOCALTIME_CACHE): Remove; no longer needed.
2026-04-02 18:54:35 -07:00
Collin Funk 4cd0644472 date: avoid calling putenv multiple times unnecessarily
Adding environment variables can become quite expensive in some
admittedly unlikely situations.

    $ for i in $(seq 10000); do export A$i=A$i; done
    $ time ./src/date-prev -u $(yes -- -u | head -n 100000)
    Sun Mar 29 01:59:49 AM UTC 2026

    real	0m3.753s
    user	0m3.684s
    sys	0m0.050s
    $ time ./src/date -u $(yes -- -u | head -n 100000)
    Sun Mar 29 02:00:00 AM UTC 2026

    real	0m0.061s
    user	0m0.022s
    sys	0m0.045s

* src/date.c (main): Only add TZ=UTC0 to the environment once.
2026-03-28 18:57:49 -07:00
Collin Funk 753fa26918 maint: remove unnecessary return statements
* src/env.c (initialize_signals): Remove return at the end of the
function.
* src/who.c (print_runlevel): Likewise.
2026-03-28 12:48:38 -07:00
Collin Funk eec7b2a131 who: avoid locking standard output for each user with the -q option
* src/who (list_entries_who): Prefer putchar and fputs to printf.
Simplify separator tracking.
2026-03-28 12:45:14 -07:00