Commit Graph
1686 Commits
Author SHA1 Message Date
Pádraig Brady 64a5d1092c build: update gnulib to latest
* NEWS: Mention the fix to nproc honoring the affinity mask
on systems with more than 1024 processors.
2024-11-11 14:54:03 +00:00
Pádraig Brady a94929715c cksum: add support for --algorithm=crc32b
$ echo -n '123456789' | cksum --raw -a crc32b | basenc --base16
  CBF43926

* bootstrap.conf: Explicitly depend on the crc module.
* doc/coreutils.texi (cksum): Add "crc32b" as an argument to -a.
* src/cksum.c (crc32b_sum_stream): A new function similar to
crc_sum_stream, but which does not include the length in
the CRC calculation.
* src/cksum.h: Add crc32b_sum_stream prototype.
* src/digest.c: Add "crc32b" as an argument to -a.
* tests/cksum/cksum.sh: Refactor to test both crc and crc32b.
* tests/cksum/cksum-a.sh: Add "crc32b" case.
* tests/cksum/cksum-base64.pl: Likewise.
* tests/misc/read-errors.sh: Likewise.
* NEWS: Mention the new feature.
2024-11-02 19:57:35 +00:00
Collin Funk ee231019e1 test: add string operators added by POSIX 2024
* src/test.c (binop): Recognize the ">" and "<" operators.
(three_arguments): Likewise.
(binary_operator): Implement the "<" and ">" operators.
(usage): Add operators to --help output.
* tests/test/test.pl (@Tests): Add functionality tests.
* doc/coreutils.texi (test invocation, String tests): Document new
operators.
* NEWS: Mention the new feature.
2024-11-02 14:03:49 +00:00
Pádraig Brady f48123ef27 doc: mention new timeout short options in NEWS
* NEWS: Mention the timeout -f and -p options.
2024-11-01 15:58:42 +00:00
Paul Eggert 2bd3db74d5 factor: improve fd buffering
* src/factor.c (struct lbuf_, lbuf, lbuf_alloc): Remove.
All uses removed.
(FACTOR_PIPE_BUF): Now a constant instead of a macro.
Increase to PIPE_BUF if available.
(lbuf_buf, lbuffered): New static vars, replacing lbuf.
All uses changed.
(lbuf_flush): Avoid unlikely recursion on write failure.
(lbuf_putc): Now simply adds a byte to the buffer.
(lbuf_putnl): Do the work of the old lbuf_putc ('\n').
Use changed.  Use memrchr to find the newline.
(lbuf_putint): Widths are now int, not size_t.
2024-09-27 17:42:59 -07:00
Pádraig Brady 5cecd703e5 printf: add indexed argument support
* src/printf.c (print_formatted): Add support for %i$ indexed args.
* tests/printf/printf-indexed.sh: Add a new file of test cases.
* tests/local.mk: Reference the new test file.
* doc/coreutils.texi (printf invocation): Mention how mixed
processing of indexed and sequential references are supported,
unlike the printf(2) library function.
* NEWS: Mention the new (POSIX:2024) feature.
Addresses https://bugs.gnu.org/73068
2024-09-13 17:29:02 +01:00
Pádraig Brady 403cb48413 install: dereference source symlinks when comparing
* NEWS: Mention the change in behavior.
* src/install.c (need_copy): s/lstat/stat/ for the source.
* tests/install/install-C.sh: Add test cases
(and improve existing test case which wan't valid
due to the existing non standard modes on test files).
Addresses https://bugs.gnu.org/72707
2024-08-19 14:47:07 +01:00
Pádraig Brady 84f8202287 ls: add support for explicit file name sorting
Support overriding previous sorting options
with an explicit --sort=name option.

* doc/coreutils.texi (ls invocation): Document the new option.
* src/ls.c (usage): Likewise.
(sort_args): Add the "name" entry, and sort to be consistent
with the ordering presented in --help.
* tests/ls/ls-time.sh: Add test cases.
* NEWS: Mention the new feature.
Suggested by: Tzvetelin Katchov
2024-08-11 20:40:15 +01:00
Paul Eggert cb11d2ab5e tail: support counts > 2**64
* src/tail.c (tail_lines): If skipping all input, use lseek if
possible.
(parse_options): Allow counts to exceed 2**64.
(main): Don’t subtract 1 from UINTMAX_MAX, since it stands
for infinity in this context.
(main): Also don’t read anything when given infinite elisions.
* tests/tail/tail.pl: Adjust to match new behavior.  Rename err-5
test to big-c and expect the invocation to succeed, since ‘tail
-c99999999999999999999’ now succeeds instead of (unnecessarily)
failing.
2024-08-10 19:30:01 -07:00
Paul Eggert ad094f20ed nproc: support --ignore counts > 2**64
* src/nproc.c (main): Allow --ignore arg to exceed 2**64.
2024-08-10 19:30:01 -07:00
Paul Eggert 67eff6359d nl: support -l counts > 2**64
* src/nl.c (main): Allow -l arg to exceed 2**64.
2024-08-10 19:30:01 -07:00
Paul Eggert 22ac54858c head: support counts > 2**64
* src/head.c (head): Optimize for -n-HUGE, where HUGE exceeds
2**64 - 2.
(string_to_integer): Return UINTMAX_MAX for too-large numbers,
instead of failing.
(main): Omit no-lnger-necessary test for byte count overflow.
2024-08-10 19:30:01 -07:00
Paul Eggert bfbb3ec7f7 shuf: fix randomness bug
Problem reported by Daniel Carpenter <https://bugs.gnu.org/72445>.
* gl/lib/randread.c (randread_new): Fill the ISAAC buffer
instead of storing at most BYTES_BOUND bytes into it.
2024-08-03 23:22:53 -07:00
Pádraig Brady 3ce31e6f19 stat,tail: sync with latest Linux file systems
* src/stat.c (human_fstype): Rename "FUSEBLK" to "FUSE" to sync with
kernel adjustments.  Add "bcachefs", and "pidfs".  Both are local,
with the latter being similar to "proc" which is also local.
* NEWS: Mention the change in behavior, and the improvement.
2024-05-28 13:38:34 +01:00
Pádraig Brady ccf47cad93 wc: increase I/O size from 16 KiB to 256KiB
Similarly to commit v9.4-143-gfcfba90d0,
and enabled for AVX by commit v9.5-25-g0e4450103.

This was seen to improve AVX performance by about 10%
on an AMD 7800X3D (Ryzen 7 (2023)) CPU,
while having neutral AVX performance,
on an Intel i7-5600U (Broadwell-U (2015)) CPU.
With avx not enabled, this gives about a 3% performance boost,
on an Intel i7-5600U.

* src/wc.c: Use the centrally configured optimum buffer size.
* src/wc_avx2.c: Likewise.
* NEWS: Mention the change in performance.
2024-05-09 15:03:56 +01:00
Pádraig Brady de49e993ea cp: actually support --update=none-fail
* src/cp.c: Add the entries for the --update=none-fail option.
* tests/mv/update.sh: Add a test case.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/70727
2024-05-03 10:18:50 +01:00
Paul Eggert fb543b6b82 tail: avoid infloop with -c on /dev/zero
Problem reported by Ionut Nicula in:
https://bugs.gnu.org/70477
* src/tail.c (tail_bytes): Do not loop forever on commands
like 'tail -c 4096 /dev/zero'.
* tests/tail/tail-c.sh: Test this fix.
2024-04-19 21:45:43 -07:00
Paul Eggert 9ab1f9db94 ls: -f now means -a -U
Problem reported by Toby Kelsey <https://bugs.gnu.org/70418>.
* src/ls.c (decode_switches): -f now simply means -a -U.
(usage): Adjust to match.
2024-04-17 14:42:57 -07:00
Pádraig Brady be3a216ab4 doc: NEWS: mention sort being more efficient with /proc files
* NEWS: Mention the improvement in commit v9.5-6-g8ff390328
2024-04-07 13:51:31 +01:00
Pádraig Brady 8f3989d586 build: update gnulib submodule to latest
* gnulib: Update to latest.
* NEWS: Document 2 bugs fixed.
2024-04-04 11:20:52 +01:00
Pádraig Brady d2df669c9c maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2024-03-28 15:13:45 +00:00
Pádraig Brady cc8689a07c version 9.5
* NEWS: Record release date.
2024-03-28 14:47:20 +00:00
Pádraig Brady 30039cb23a cp: with --no-preserve=mode ensure set-group-ID bits maintained on dirs
This issue was introduced in commit v8.19-145-g24ebca6

* src/copy.c (copy_internal): On systems that don't support ACLs,
the fallback default chmod done on directories should maintain
the set-group-ID, as that's generally auto-set by the system.
* NEWS: Mention the fix.
Reported by Bruno Haible on Alpine (with tests/cp/preserve-mode.sh)
2024-03-25 22:27:15 +00:00
Pádraig Brady 193449b173 env: add -a,--argv0 to set the first argument passed to exec
Using the shell's exec -a feature can be awkward
so add support for setting overriding argv[0].
This gives env full control over the arguments it passes.

* src/env.c: Accept -a,--argv0 and set argv[0] appropriately.
* tests/env/env.sh: Add test cases.
* doc/coreutils.texi (env invocation): Describe -a,--argv0.
* NEWS: Mention the new feature.
2024-03-22 13:12:52 +00:00
Paul Eggert 5d3fd24f42 mv: new option --exchange
* src/copy.h (struct cp_options): New member 'exchange'.
* src/copy.c (copy_internal): Support the new member.
* src/mv.c (EXCHANGE_OPTION): New constant.
(long_options): Add --exchange.
(usage): Document --exchange.
(main): Support --exchange.
* tests/mv/mv-exchange.sh: New test case.
* tests/local.mk (all_tests): Add it.
2024-03-20 14:40:29 -07:00
Pádraig Brady 425b8a2f53 chmod: fix TOCTOU security issue with symlink replacement
This is an issue with -[H]R mode, where an attacker
may replace a traversed file with a symlink
between where we stat() the file and chmod() the file.

* src/chmod.c (process_file): Remove the first !S_ISLNK guard
as that's now just an optimization, and instead consistently
apply fchmodat() to files/symlinks.  Ensure AT_SYMLINK_NOFOLLOW
is set when traversing in default (-H) mode.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/11108
2024-03-19 23:56:45 +00:00
Pádraig Brady 07a69fc3ba chmod: add support for -h, -H,-L,-P, --dereference options
There have been various requests to add -h to avoid following symlinks
for security reasons.  This wasn't provided previously as chmod(1)
already ignored symlinks unless specified on the command line.
Note chmod defaults to -H mode rather than the chown default of -P,
as usually chown can work directly on symlinks and so defaults
to not traversing those specified on the command line.
Note FreeBSD chmod does default to -P mode, but we retain the -H mode
default also for compatibility with existing chmod behavior.

Adding -HLP will allow chmod to disable traversing CLI symlinks to dirs.
Adding -h will allow to disable following CLI symlinks to files/dirs,
  also operating on all symlinks on systems that support that.
Adding --dereference will be significant with -H (the default).  I.e.
  symlinks to dirs not recursed, but symlinks are dereferenced.
Adding these options will also be consistent with chown(1), chgrp(1),
and chmod(1) on other systems.

Note since chmod(1) currently ignores symlinks by default,
and -h is primarily a mechanism to avoid following symlinks, rather than
for operating on the symlink itself, we make -h try to chmod a symlink,
but ignore ENOTSUP.  In that way we're consistent with chown(1)
where it also ignores ENOTSUP for symlinks, and we don't fail when
trying to be extra secure with command line params.

* doc/coreutils.texi (chmod invocation): Reference the -H,-L,-P
descriptions, and adjust the corresponding macros to say
the default is -H or -P as appropriate.
Add --dereference and -h,--no-dereference descriptions.
* man/chmod.x: Adjust discussion of symlink handling.
* src/chmod.c (main): Accept new options and set
fts flags appropriately.
(process_file): Process / dereference symlinks as necessary.
* src/system.h (emit_symlink_recurse_options): A new function
refactored from chown.c and chmod.c usage().
* tests/chmod/symlinks.sh: New test for the new options.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
2024-03-19 23:43:53 +00:00
Pádraig Brady bf73a7602a pinky: disable location canonicalization by default
Behave like who(1) in requiring --lookup to enable this
often slow feature.  pinky(1) is supposed to be lightweight after all.

* doc/coreutils.texi (who invocation): Adjust the description to no
longer reference dialup, and be more general about the still significant
delays.
(pinky invocation): Reference the same --lookup description.
* src/pinky.c (main): Accept --lookup to enable DNS lookups.
* NEWS: Mention the change in behavior.
Fixes https://bugs.debian.org/628815
2024-03-19 23:43:53 +00:00
Grisha Levit b8d1b00e21 env,kill,timeout: support unnamed signals
Some signals with values less that the max signal number for the system
do not have defined names.  For example, currently on amd64 Linux,
signals 32 and 33 do not have defined names, and Android has a wider
gap of undefined names where it reserves some realtime signals.

Previously the signal listing in env ended up reusing the name
of the last printed valid signal (the repeated HUP below):

    $ env --list-signal-handling true
    HUP        ( 1): IGNORE
    HUP        (32): BLOCK
    HUP        (38): IGNORE

..and the corresponding signal numbers were rejected as operands for the
env, kill, and timeout commands.

This patch removes the requirement that sig2str returns 0 for a signal
number associated with an operand.  This allows unnamed signals to be in
the sets `env' attempts to manipulate when a --*-signal option is used
with no argument, and kill(1) and timeout(1) to send such unnamed
signals.

* src/operand2sig.c (operand2sig): Drop signame argument, accept all
signal numbers <= SIGNUM_BOUND.  All callers updated.
* src/env.c (parse_signal_action_params, reset_signal_handlers)
(parse_block_signal_params, set_signal_proc_mask)
(list_signal_handling): Accept all signal numbers <= SIGNUM_BOUND,
use SIG%d for printing if necessary.
* src/kill.c (list_signals, main): Likewise.
(send_signals): Check errno from kill(3) for bad signo.
* src/timeout.c (main): Update operand2sig call.
* tests/misc/kill.sh: Test listing all signal numbers.
* NEWS: Mention the improvement.
2024-03-13 16:15:35 +00:00
Pádraig Brady ab4ffc8503 timeout: fix narrow race in failing to kill processes
* src/timeout.c (main): Block cleanup signals earlier so that cleanup()
is not runnable until monitored_pid is in a deterministic state.
This ensures we always send a termination signal to the child
once it's forked.
* NEWS: Mention the bug fix.
Reported at https://github.com/coreutils/coreutils/issues/82
2024-03-12 14:57:47 +00:00
Pádraig Brady c1cf5148a1 timeout: fix race where we might kill arbitrary processes
* src/timeout.c (cleanup): Handle the case where monitored_pid
might be -1, which could happen if a signal was received
immediately after a failed fork() call.  In that case it would
send the termination signal to all processes that the timeout
process has permission to send signals too.
* NEWS: Mention the bug fix.
2024-03-11 15:12:36 +00:00
Pádraig Brady 9d8890d872 mv: revert add --swap (-x) option
Since this functionality is recently available
in the exch(1) utility from util-linux,
it was thought best not to complicate mv with it.

This reverts commit 6cd2d5e533
2024-03-05 21:34:00 +00:00
Pádraig Brady c877c7c1ee doc: fix spellings in NEWS
* NEWS: s/user/uses/; s/exisiting/existing/
2024-03-03 12:27:07 +00:00
Petr Malat 6cd2d5e533 mv: add --swap (-x) option to atomically swap 2 paths
renameat2() syscall allows atomically swapping 2 paths on one
file system. Expose this ability to the user with --swap.

* doc/coreutils.texi: Describe mv --swap option.
* src/mv.c (main): Support --swap.
* tests/mv/mv-swap.sh: Add test for mv -x.
* tests/local.mk: Reference new test.
* NEWS: Mention the new option.
2024-03-03 12:18:45 +00:00
Pádraig Brady fcfba90d0d cat,cp,mv,dd,install,split: set the default IO size to 256KiB
* src/ioblksize.h: Add updated test results and
increase value from 128KiB to 256KiB, which was last
updated 10 years ago.
* NEWS: Mention the improvement.
2024-02-28 22:26:52 +00:00
lvgenggeng e397ba1a31 mktemp: fix template diagnostic with --suffix
* src/mktemp.c (main): When --suffix is specified, TEMPLATE
points to the meraged buffer DEST_NAME. As X's in the suffix are
not significant to the generated random characters, the diagnostic
for too few X's should only refer to the template portion.
* tests/misc/mktemp.pl: Adjust accordingly.
* NEWS: Mention the bug fix.
2024-02-27 14:59:33 +00:00
Pádraig Brady 49912bac28 cp,mv: add --update=none-fail to fail if existing files
* src/cp.c (main): Add support for --update=none-fail to provide the
functionality of diagnosing files in the destination,
and exiting with failure status.
(usage): Mark -n as deprecated.
* src/mv.c: Likewise.
* src/copy.h: Add UPDATE_NONE_FAIL definition.
* src/system.h (emit_update_parameters_note): Add --update=none-fail
description.
* doc/coreutils.texi (cp invocation): Likewise.
Also mention why -n is deprecated.
* tests/mv/update.sh: Add a test case, including precedence
with -n and other --update options.
* tests/cp/cp-i.sh: Verify that --backup and --update=none{,-fail}
are mutually exclusive.
* tests/mv/mv-n.sh: Likewise.
* NEWS: Mention the new feature.
Addresses https://bugs.gnu.org/62572
2024-02-26 17:33:02 +00:00
Pádraig Brady 9907b6f5dc cp,mv: reinstate that -n exits with success if files skipped
* src/cp.c (main): Adjust so that -n will exit success if skipped files.
* src/mv.c (main): Likewise.
* doc/coreutils.texi (cp invocation): Adjust the description of -n.
* src/system.h (emit_update_parameters_note): Adjust --update=none
comparison.
* tests/cp/cp-i.sh: Adjust -n exit status checks.
* tests/mv/mv-n.sh: Likewise.
* NEWS: Mention the change in behavior.
Fixes https://bugs.gnu.org/62572
2024-02-26 17:33:02 +00:00
Paul Eggert d9dc02b09a doc: mention -lcrypto change in NEWS 2024-02-25 21:24:04 -08:00
Daan De Meyer a357cfb021 cp: add --keep-directory-symlink option
When recursively copying files into OS trees, it often happens that
some subdirectory of the source directory is a symlink in the target
directory. Currently, cp will fail in that scenario with the error:

"cannot overwrite non-directory %s with directory %s"

However, we'd like cp in this scenario to follow the destination
directory symlink and copy the files into the symlinked directory
instead. Let's support this by adding a new option
--keep-directory-symlink that makes cp follow destination directory
symlinks.

We name the option --keep-directory-symlink to keep consistent with
tar which has the same option with the same effect.

* doc/coreutils.texi (cp invocation): Describe the new option.
* src/copy.h: Add the new setting.
* src/copy.h: Adjust to follow symlinks if setting enabled.
* src/cp.c (usage): Describe the new option.
(main): Accept the new option.
* tests/cp/keep-directory-symlink.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
2024-02-22 14:37:47 +00:00
Pádraig Brady 76604db7d2 od: support half precision floating point
Rely on compiler support for _Float16 and __bf16
to support -fH and -fB formats respectively.
I.e. IEEE 16 bit, and brain 16 bit floats respectively.
Modern GCC and LLVM compilers support both types.

clang-sect=half-precision-floating-point
https://gcc.gnu.org/onlinedocs/gcc/Half-Precision.html
https://clang.llvm.org/docs/LanguageExtensions.html#$clang-sect
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0192r4.html
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p1467r9.html

This was tested on:
gcc 13, clang 17 x86 (Both types supported)
gcc 7 aarch64 (Only -fH supported)
gcc 13 ppc(be) (Neither supported. Both will be with GCC 14)

* src/od.c: Support -tf2 or -tfH to print IEEE 16 bit floating point,
or -tfB to print Brain 16 bit floating point.
* configure.ac: Check for _Float16 and __bf16 types.
* doc/coreutils.texi (od invocation): Mention the new -f types.
* tests/od/od-float.sh: Add test cases.
* NEWS: Mention the new feature.
Addresses https://bugs.gnu.org/68871
2024-02-05 13:30:45 +00:00
Pádraig Brady c01ee18934 doc: split -C: test and document a heap overflow
This was introduced in coreutils 9.2 through commit v9.1-184-g40bf1591b,
and was fixed in coreutils 9.5 through commit v9.4-111-gc4c5ed8f4.
This issue has been assigned CVE-2024-0684.

* NEWS: Mention the bug fix.
* tests/split/line-bytes.sh: Add a test case.
Reported by Valentin Metz.
2024-01-18 16:52:36 +00:00
Pádraig Brady a966dcdb69 maint: update all copyright year number ranges
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...

* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Manually update copyright year,
until we fully sync with gnulib at a later stage.
* tests/sample-test: Adjust to use the single most recent year.
2024-01-01 13:27:23 +00:00
Pádraig Brady 9e67ccd745 sort: fix thousands grouping handling on single byte locales
* gl/lib/strnumcmp-in.h (numcompare): After commit v9.0-8-g6cafb122f,
we need to treat characters as signed to avoid invalid comparisons
between negative integers and unsigned characters.
* NEWS: Mention the bug fix.
2023-12-29 18:04:36 +00:00
Christian Göttsche 98d463ef5d copy,install: avoid unnecessary security context translations
Do not perform SELinux context translation for operations not involving
user input or output.  Context translation converts MCS/MLS labels into
human readable form, which is useful for user facing applications like
ls(1) or the --context=CTX argument of cp(1).

* src/copy.c (set_process_security_ctx): Use raw selinux variants.
* src/install.c (need_copy): Likewise.
(setdefaultfilecon): Likewise.
* src/selinux.c (computecon): Likewise.
(defaultcon): Likewise.
* tests/cp/no-ctx.sh: Add raw variants to preload lib.
* NEWS: Mention the improvement.
2023-12-19 17:34:33 +00:00
Pádraig Brady da091b3ab3 chgrp: add --from parameter similar to chown
* doc/coreutils.texi (chown invocation): Convert --from option
description to a macro and call from ...
(chgrp description): ... here.
* src/chown-core.h (emit_from_option_description): A new function
refactored from ...
* src/chown.c (usage): ... here, and called from ...
* src/chgrp.c (usage): ... here.
(main): Accept the --from option as chown(1) does.
* po/POTFILES.in: Add chown-core.h as now translated.
* tests/chown/basic.sh: Decouple the root user from id 0.
* tests/chgrp/from.sh: A new test largely based on chown/basic.sh.
* tests/local.mk: Reference the new test.
* NEWS: Mention the new feature.
Suggested by Ed Neville.
2023-12-15 13:19:27 +00:00
dann frazier 73d119f4f8 tail: fix tailing sysfs files where PAGE_SIZE > BUFSIZ
* src/tail.c (file_lines): Ensure we use a buffer size >= PAGE_SIZE when
searching backwards to avoid seeking within a file,
which on sysfs files is accepted but also returns no data.
* tests/tail/tail-sysfs.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/67490
2023-12-01 23:01:32 +00:00
Pádraig Brady 615167cc4d numfmt: support lowercase 'k' for Kilo and Kibi
For consistency with the "SI" standard, and with other coreutils
which output a lowercase 'k' in "SI" mode.

* src/numfmt.c (suffix_power): Treat 'k' like 'K' on input.
(double_to_human): Output lowercase 'k' in SI mode.
(usage): Adjust accordingly.
* doc/coreutils.texi: Mention 'k' accepted, and printed in SI mode.
* tests/misc/numfmt.pl: Adjust accordingly.
* NEWS: Mention the change in behavior.
Fixes https://bugs.gnu.org/47103
2023-11-27 19:41:49 +00:00
Paul Eggert 11b01fc21f join,uniq: support multi-byte separators
* NEWS: Mention this.
* bootstrap.conf (gnulib_modules): Remove cu-ctype, as this module
is now more trouble than it’s worth.  All uses removed.
Add skipchars.
* gl/lib/cu-ctype.c, gl/lib/cu-ctype.h, gl/modules/cu-ctype:
Remove.
* gl/lib/skipchars.c, gl/lib/skipchars.h, gl/modules/skipchars:
* tests/misc/join-utf8.sh:
New files.
* src/join.c: Include skipchars.h and mcel.h instead of cu-ctype.h.
(tab): Now mcel_t, not int.  All uses changed.
(output_separator, output_seplen): New static vars.
(eq_tab, newline_or_blank, comma_or_blank): New functions.
(xfields, prfields, prjoin, add_field_list, main):
Support multi-byte characters.
* src/numfmt.c: Include ctype.h, skipchars.h.
Do not include cu-ctype.h.
(newline_or_blank): New function.
(next_field): Support multi-byte characters.
* src/sort.c: Include ctype.h instead of cu-ctype.h.
(inittables): Open-code field_sep since it no longer exists.
‘sort’ is not multi-byte safe yet, but when it is this code
will need revamping anyway.
* src/uniq.c: Include mcel.h and skipchars.h instead of cu-ctype.h.
(newline_or_blank): New function.
(find_field): Support multi-byte characters.
* tests/local.mk (all_tests): Add tests/misc/join-utf8.sh
2023-10-30 00:58:04 -07:00
Pádraig Brady 8c735f6585 base32,base64: disallow non-canonical encodings
This will make decoding more resilient to corruption
whether due to transmission errors or nefarious adjustment.
See https://eprint.iacr.org/2022/361.pdf

* gnulib: Update to commit 3f463202bd enforcing canonical encoding.
* tests/basenc/base64.pl: Add test cases, and adjust existing cases.
* NEWS: Mention the change in behavior.
2023-10-28 13:13:34 +01:00