Commit Graph
21 Commits
Author SHA1 Message Date
Sylvestre Ledru ee4ee881be tests: basenc: ensure --base58 decodes input of more than 1024 digits
Reported at https://github.com/uutils/coreutils/issues/15776

* tests/basenc/large-input.sh: Add a base58 round trip.

Link: https://github.com/coreutils/coreutils/pull/440
2026-11-07 12:35:10 +00:00
Sylvestre Ledru 529612a8ae tests: basenc: ensure --base64url decodes input without '=' padding
Reported at https://github.com/uutils/coreutils/issues/15664

* tests/basenc/basenc.pl: Add test cases.

Link: https://github.com/coreutils/coreutils/pull/409
2026-11-07 12:35:10 +00:00
Pádraig Brady 4c4f2f55c3 basenc: fix incorrect base32hex input validation
* src/basenc.c (base32hex_decode_ctx_wrapper): Ensure we don't
accept base32 only characters for base32hex.
* tests/basenc/basenc.pl: Ensure negative base32hex tests have the
appropriate number of characters to test character validation
rather than just input size validation.
* NEWS: Mention the bug fix.

Discovered by Anthropic's OSS Scanner, as bug ANT-2026-EP1XO2G4.
2026-10-22 13:54:39 +01:00
Pádraig Brady d58ea74d01 tests: avoid $SHELL -c "ulimit ..." pattern
As described in commit v9.10-283-g5cb0cca3b
avoid using ulimit with $SHELL -c, and explict
invocation of the shell like that can require lots of memory,
often due to loading the locale archive.
* tests/basenc/bounded-memory.sh: Avoid $SHELL -c with ulimit.
* tests/cut/bounded-memory.sh: Likewise.
* tests/expand/bounded-memory.sh Likewise.
* tests/pr/bounded-memory.sh Likewise.
* tests/unexpand/bounded-memory.sh Likewise.
* tests/fold/fold-zero-width.sh: Bring the ulimit bound
down to the standard 6M over the determined limit.
2026-04-15 15:08:27 +01:00
Collin Funk 73d3a49f45 maint: run 'make update-copyright' 2026-01-01 10:56:16 -08:00
Pádraig Brady b4e02e0ef4 tests: fix false failure in recent memory limit test
* tests/basenc/bounded-memory.sh: Ensure we skip the test
upon failure to determine the memory lower bound.
Reported by Bruno Haible.
2025-10-13 13:22:38 +01:00
Pádraig Brady dd3eab13ee tests: basenc: use less redundant naming
Rename to less redundant names, now that we use
a separate test directory per util.

* tests/basenc/basenc-bounded-memory.sh -> .../bounded-memory.sh
* tests/basenc/basenc-large.sh -> .../large-input.sh
* tests/local.mk: Reference new names.
2025-10-11 11:28:22 +01:00
Pádraig Brady 86234ebffa tests: fix memory limit determination in new test
* tests/basenc/basenc-bounded-memory.sh: The passed command
needs to succeed for memory limit determination to work.
2025-10-11 11:27:47 +01:00
Collin Funk f88f0e5d44 tests: basenc: add a test for bounded memory operation
* tests/basenc/basenc-bounded-memory.sh: New file.
* tests/local.mk (all_tests): Add the test.
2025-10-11 01:33:15 -07:00
Pádraig Brady 2ef53e5b04 basenc: --base58: fix buffer overflow with input > 15MB
base58_length() operated naively on an int
which resulted in an overflow to a negative number
for any input > 2^31-1/138, i.e. 15,561,475 bytes.

* src/basenc.c (base_length): Change input and output
parameter types from int to idx_t since this needs to
cater for the full input size in the base58 case.
(base58_length): Likewise.  Also reorder the calculation
to be less exact, but doing the division first
to minimize the chance of overflow (which now on 64 bit
would only happen for inputs > around 6 Exa bytes).
* tests/basenc/basenc-large.sh: Add a new test,
that triggers with valgrind or ASAN.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
2025-09-23 20:35:34 +01:00
Pádraig Brady 4141ae3e49 maint: basenc: refactor all encodings to use finalize
Finalize was required for base58, but it's a more general mechanism
which simplifies the logic for all encodings

* src/basenc.c (do_decode): Always call base_decode_ctx_finalize(),
rather than the awkward double loop at end of buffer.
* tests/basenc/basenc.pl: Add basenc finalization tests.
2025-09-10 23:24:46 +01:00
Pádraig Brady d9b68362ed basenc: ensure partial padding with newlines induces an error
* src/basenc.c (has_padding): A more robust helper to
identify padding in the presence of trailing newlines.
(do_decode): Use has_padding() rather than just looking
at the last character.
* tests/basenc/base64.pl: Fully test commit v9.4-53-g378dc38f4
by ensuring partially padded data is diagnosed.
baddecode9 is the case fixed in this commit.
* NEWS: Mention the bug fix.
2025-09-08 19:39:47 +01:00
Pádraig Brady 02c3d43e3a basenc: add base58 support
A 58 character encoding that:
 - avoids visually ambiguous 0OIl characters
 - uses only alphanumeric characters
Described at:
 - https://tools.ietf.org/html/draft-msporny-base58-03

This implementation uses GMP (or gnulib's gmp fallback).
Performance is good in comparison to other implementations.
For example when using libgmp on an i7-5600U system,
encoding is 530 times faster, and decoding 830 times faster
than the implementation using arbitrary precision ints in cpython 3.13.

Memory use is proportional to the size of input.

Encoding benchmarks:

  $ time yes | head -c65535 | src/basenc --base58 -w0 >file.enc
  real    0m0.018s

  ./configure --quiet --without-libgmp && make -j $(nproc)
  $ time yes | head -c65535 | src/basenc --base58 -w0 >file.enc
  real    0m3.431s

  # dnf install python3-base58
  $ time yes | head -c65535 | base58 >file.enc  # cpython 3.13
  real    0m9.700s

Decoding benchmarks:

  $ time src/basenc --base58 -d <file.enc >/dev/null
  real    0m0.010s

  $ ./configure --without-libgmp && make  # gnulib gmp
  $ time src/basenc --base58 -d <file.enc >/dev/null
  real    0m0.145s

  $ time base58 -d <file.enc >/dev/null  # cpython 3.13
  real    0m8.302s

* src/basenc.c (base_decode_ctx_finalize, base_encode_ctx_init,
base_encode_ctx, base_encode_ctx_finalize): New functions to
provide more general processing functionality.
(base58_{de,en}code_ctx{_init,,_finalize}): New functions to
accumulate all input before calling ...
(base58_{de,en}code): ... the GMP based encoding/decoding routines.
(do_encode, do_decode): Call the ctx variants if enabled.
* doc/coreutils.texi (basenc invocation): Describe the new option,
and indicate the main use case being interactive user use.
* src/local.mk: Link basenc with GMP.
* tests/basenc/basenc.pl: Add test cases.
* NEWS: Mention the new feature.
2025-08-09 22:10:27 +01:00
Pádraig Brady c480428e37 basenc: fix stripping of '=' chars in some encodings
* src/basenc.c (do_decode): With -i ensure we strip '=' chars
if there is no padding for the chosen encoding.
* tests/basenc/basenc.pl: Add a test case.
* NEWS: Mention the bug fix.
2025-08-09 09:59:00 +01:00
Pádraig Brady 28b176085f maint: update all copyright year number ranges
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...

* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2025-01-01 09:33:08 +00:00
Daniel Hofstetter e331dcbc81 maint: tests: update deprecated perl backreference syntax
* tests/basenc/basenc.pl: perl warns that $1 is better than \1,
so update to the preferred form.
2024-12-16 18:14:36 +00:00
Pádraig Brady a966dcdb69 maint: update all copyright year number ranges
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...

* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Manually update copyright year,
until we fully sync with gnulib at a later stage.
* tests/sample-test: Adjust to use the single most recent year.
2024-01-01 13:27:23 +00:00
Pádraig Brady 8c735f6585 base32,base64: disallow non-canonical encodings
This will make decoding more resilient to corruption
whether due to transmission errors or nefarious adjustment.
See https://eprint.iacr.org/2022/361.pdf

* gnulib: Update to commit 3f463202bd enforcing canonical encoding.
* tests/basenc/base64.pl: Add test cases, and adjust existing cases.
* NEWS: Mention the change in behavior.
2023-10-28 13:13:34 +01:00
Pádraig Brady 5f538c27a1 basenc: --base16: also allow lower case with --ignore-garbage
* src/basenc.c (isbase16): Also return true for lower case.
* tests/basenc/basenc.pl: Add a test case.
Reported by Paul Eggert.
2023-10-25 14:04:00 +01:00
Pádraig Brady d733f2ec26 basenc: --base16: support lower case hex digits
* src/basenc.c (base16_decode_ctx): Convert to uppercase
before converting from hex.
* tests/basenc/basenc.pl: Add a test case.
* NEWS: Mention the change in behavior.
Addresses https://bugs.gnu.org/66698
2023-10-23 14:04:38 +01:00
Pádraig Brady caa716803a tests: move all basenc tests to their own directory
* tests/misc/base64.pl: Move to tests/basenc/base64.pl
* tests/misc/basenc.pl: Move to tests/basenc/basenc.pl
* tests/local.mk: Adjust accordingly
2023-10-06 18:22:35 +01:00