Files
coreutils/tests/cut
Pádraig Brady f1101ce852 cut: -c: optimize ASCII and UTF-8 processing
These are the most common input encodings so worth optimizing for.

Performance was tested with GCC 16 on an i7-5600U with
CFLAGS='-march=native -O3 -flto' with the following setup:

$ yes | head -n10M > sl.in
$ yes $(yes eeeaae | head -n10K | paste -s -d' ') | head -n10K > llw.in
$ yes $(yes eeeaae | head -n9 | paste -s -d' ') | head -n1M > asw.in
$ yes $(yes éééááé | head -n9 | paste -s -d$'\xe2\x80\x83') |
   head -n1M > mbw.in

for type in sl llw asw mbw; do
  cat $type.in >/dev/null;
  for imp in '-before' '-after'; do
    echo ============ "${imp:-base}" $type ==============;
    for d in -c; do
      fields='-f1 -f10 -f100'
      test "$d" = "-c" && { fields='-c1 -c10 -c100000'; d=''; }
      for f in $fields; do
        for loc in C.UTF-8; do
          # Skip -b for UTF-8 as no different
          test "$loc" = C.UTF-8 && echo "$f" | grep -q -- -b \
           && continue
          # Skip multi-byte delimiter for C and not allowed
          test "$loc" = C && test $(echo -n "$d" | wc -c) -ge 4 \
           && continue
          bin=src/cut${imp}
          LC_ALL=$loc $bin $f $d /dev/null 2>/dev/null &&
          hyperfine --warmup=2 -m2 -M6 \
           "LC_ALL=$loc $bin $f $d $type.in >/dev/null" ||
          printf 'Benchmark 1: %s\n  unsupported\n\n' \
           "LC_ALL=$loc $bin $f $d $type.in >/dev/null"
        done;
      done;
    done;
  done;
done

After a little post-processing of the results, we get:

-- src/cut-before

| command         |       sl |      llw |      asw |      mbw |
| --------------- | -------- | -------- | -------- | -------- |
| UTF8 -c1        |  70.7 ms | 103.5 ms |  26.9 ms |  48.6 ms |
| UTF8 -c10       |  55.1 ms | 103.9 ms |  48.6 ms | 127.3 ms |
| UTF8 -c100000   |  54.7 ms |  1.585 s | 156.9 ms | 555.9 ms |

-- src/cut-after

| command         |       sl |      llw |      asw |      mbw |
| --------------- | -------- | -------- | -------- | -------- |
| UTF8 -c1        |  90.1 ms | 103.6 ms |  28.0 ms |  50.9 ms |
| UTF8 -c10       |  39.7 ms | 103.2 ms |  40.0 ms | 127.5 ms |
| UTF8 -c100000   |  39.0 ms | 104.4 ms |  18.5 ms |  30.3 ms |

* bootstrap.conf: Explicitly depend on unistr/u8-check.
* src/cut.c (cut_characters_mode): Quickly skip over unselected
ASCII/UTF-8 data.
* tests/cut/cut.pl: Add test cases.
2026-09-29 13:34:04 +01:00
..