Commit Graph
40 Commits
Author SHA1 Message Date
Pádraig Brady 606f54d157 tests: numfmt: fix false failures on some systems
* tests/misc/numfmt.pl: Verify that printf field width specs
count characters and not bytes before enabling locale tests.
This was seen on FreeBSD 14.0 and Solaris 11 OpenIndiana.
Reported by Bruno Haible
2024-03-26 20:09:35 +00:00
Pádraig Brady c0f02a8f5b tests: avoid false failure with partial locale info
* tests/misc/numfmt.pl: Some systems with the fr_FR.UTF-8
locale installed, do not have a thousands grouping character defined.
In this case we skip the locale tests which depend on a non empty
grouping character.
2024-03-25 18:40:53 +00:00
Pádraig Brady a966dcdb69 maint: update all copyright year number ranges
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...

* gnulib: Update included in this commit as copyright years
are the only change from the previous gnulib commit.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Manually update copyright year,
until we fully sync with gnulib at a later stage.
* tests/sample-test: Adjust to use the single most recent year.
2024-01-01 13:27:23 +00:00
Pádraig Brady 76ec516242 tests: numfmt: fix test related to lower case 'k' SI unit
* tests/misc/numfmt.pl: Following on from v9.4-86-g615167cc4,
adjust this test accordingly.  This test was being skipped
on some systems, and so only noticed now.
Reported by Jim Meyering.
2023-12-29 18:04:36 +00:00
Pádraig Brady ebb985dc41 tests: run locale tests on more systems
* tests/misc/numfmt.pl: Determine the thousands grouping character
in use, rather than skipping locale tests when it's not a space.
For example fr_FR.UTF-8 uses "NARROW NO-BREAK SPACE" as the grouping
char on modern glibc systems at least.
* tests/sort/sort-h-thousands-sep.sh: Likewise.
2023-12-29 18:04:29 +00:00
Pádraig Brady 615167cc4d numfmt: support lowercase 'k' for Kilo and Kibi
For consistency with the "SI" standard, and with other coreutils
which output a lowercase 'k' in "SI" mode.

* src/numfmt.c (suffix_power): Treat 'k' like 'K' on input.
(double_to_human): Output lowercase 'k' in SI mode.
(usage): Adjust accordingly.
* doc/coreutils.texi: Mention 'k' accepted, and printed in SI mode.
* tests/misc/numfmt.pl: Adjust accordingly.
* NEWS: Mention the change in behavior.
Fixes https://bugs.gnu.org/47103
2023-11-27 19:41:49 +00:00
Pádraig Brady 850a056441 maint: avoid syntax check failure
* tests/misc/numfmt.pl: Keep lines <= 80 chars.
2023-08-31 21:59:02 +01:00
Paul Eggert cbb6dfec55 maint: remove need for mbsalign
This simplifies memory allocation a bit, and removes an arbitrary
limitation from numfmt, which formerly limited cell output to 127
bytes.
* bootstrap.conf (gnulib_modules): Remove mbsalign, strncat.
Add strnlen (the code already used strnlen directly, and we were
saved only because Gnulib used the module indirectly)
* gl/lib/mbsalign.c, gl/lib/mbsalign.h, gl/modules/mbsalign:
* gl/modules/mbsalign-tests, gl/tests/test-mbsalign.c: Remove.
* src/df.c, src/ls.c: Do not include mbsalign.h.
(MBSWIDTH_FLAGS): New constant, now used for all
mbswidth calls.  All callers changed to check for -1 return.
* src/df.c (struct field_data_t): ‘width’ is now int not size_t,
since mbswidth can’t do widths greater than INT_MAX anyway.
Replace ‘align’ with ‘align_right’.  All uses changed.
(print_table): Redo to avoid the need for ambsalign.
(get_header, get_dev): mbswidth returns int, not size_t.
* src/ls.c (MAX_MON_WIDTH): Remove; no longer used.
(abmon_init): Use strnlen to cheaply discard too-long month names.
Align by hand instead of using mbsalign.
* src/numfmt.c: Include stdckdint.h, mbswidth.h.
Do not include mbsalign.h.
(padding_buffer_size): Now idx_t.  All uses changed.
(padding_width): Now intmax_t, since it’s no longer an object
size.  Its sign now records alignment.  All uses changed.
(zero_padding_width): Now int, since it’s given to sprintf.
All uses changed.
(padding_alignment): Remove; it’s now taken from padding_width’s sign.
(double_to_human): Return string length.  BUF_SIZE arg is now idx_t.
Include suffix in output.  All callers changed.  Simplify by not
calling strncat or stpcpy.  Calculate fmt size bound more carefully.
(setup_padding_buffer): Remove.  All uses removed.
(parse_format_string): Use intmax_t, not long, for pad.
On overflow, set widths to large values that cause later code
to do the right thing, rather than separately checking for
overflow here.
(prepare_padded_number): Return bool, not int 0/1.  New arg
PADDING.  All uses changed.  Do not limit padded output to 127
bytes; instead, use xpalloc to expand the output buffer.
(print_padded_number): New arg PADDING.  All uses changed.
(process_suffixed_number): Simplify.
(main): Take extremum if xstrtoimax overflows, as this does
the right thing.
* tests/misc/numfmt.pl: New test suf-20 to test for truncation bug.
Remove tests pad-3.2, fmt-err-7, as they’re no longer invalid but
are quite expensive.
2023-08-29 23:57:40 -07:00
Pádraig Brady f4567ed953 all: further adjustments for new Ronna, Quetta SI prefixes
* src/dd.c (parse_integer): Support Q,R suffixes.
* src/od.c (main): Likewise.
* src/split.c (main): Likewise.
* src/stdbuf.c (parse_size): Likewise.
* src/truncate.c (main): Likewise.
* src/sort.c (specify_size_size): Likewise.
Also line length syntax check fix.
* tests/misc/numfmt.pl: Adust top end large number checks
to the new largest values.
* doc/coreutils.texi (numfmt invocation): Add a numfmt example.
* NEWS: Tweak to aid searchability.
2023-01-06 14:26:40 +00:00
Paul Eggert e1ff4ee0f6 numfmt: add support for new SI prefixes
* src/dd, src/head.c, src/od.c, src/sort.c, src/stdbuf.c, src/tail.c:
(usage):
* src/system.h (emit_size_note):
Mention new SI prefixes.
* src/du.c (main):
* src/head.c (head_file):
* src/numfmt.c (suffix_power, suffix_power_char, prepare_padded_number):
* src/shred.c (main):
* src/sort.c (unit_order):
* src/tail.c (parse_options):
Support new SI prefixes.
* src/numfmt.c (MAX_ACCEPTABLE_DIGITS): Increase to 33.
(zero_and_valid_suffixes, valid_suffixes): New constants,
with new SI prefixes.
(valid_suffix, unit_to_umax): Use them.
(prepare_padded_number): Diagnose "999Q" instead of "999Y".
* tests/misc/numfmt.pl, tests/misc/sort.pl:
Adjust tests to match new max.
2023-01-05 12:47:41 -08:00
Pádraig Brady 01755d36e7 maint: update all copyright year number ranges
Update to latest gnulib with new copyright year.
Run "make update-copyright" and then...

* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Manually update copyright year,
until we fully sync with gnulib at a later stage.
* tests/sample-test: Adjust to use the single most recent year.
2023-01-01 14:50:15 +00:00
Pádraig Brady 3067a9293a maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2022-01-02 16:15:55 +00:00
Pádraig Brady 32ff1d1313 maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2021-01-01 16:36:09 +00:00
Tobias Stoeckmann 0fe9fdece7 maint: avoid signed integer overflows
Since -LONG_MIN results in LONG_MIN again, the operation itself is
a signed integer overflow.

This can be observed with the following calls (best if compiled
with -ftrapv or -fsanitize=undefined):

  $ numfmt --padding=-9223372036854775808
  $ seq 1e-9223372036854775808

Technically, the change in seq "reduces" the precision, but a double
or long double that small would be represented as 0 anyway.

* src/numfmt.c: Explicitly disallow --padding=LONG_MIN.
* src/seq.c: Treat 1e$LONG_MIN as 1e-$LONG_MAX.
* tests/misc/numfmt.pl: Add a test case.
* tests/misc/seq-precision.sh: Likewise.

Fixes https://bugs.gnu.org/41850
2020-06-15 23:07:03 +01:00
Pádraig Brady aaba82431c maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2020-01-01 14:16:56 +00:00
Assaf Gordon 8dfcf38af1 maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2019-01-01 16:51:48 -07:00
Tobias Stoeckmann d1a754c827 cut: improve large file support on 32 bit
Increase max range from SIZE_MAX to UINTMAX_MAX, which will
allow cut to support line lengths up to the max file size
on all systems. The inherent SIZE_MAX limitation in cut was
removed with the enhancements in https://bugs.gnu.org/13127.
Also numfmt gets similarly increased --field ranges due to
shared code.

* src/cut.c: s/size_t/uintmax_t/.
* src/numfmt.c: Likewise.
* src/set-fields.c: Likewise.
* src/set-fields.h: Likewise.
* tests/misc/cut-huge-range.sh: Adjust accordingly.
* tests/misc/numfmt.pl: Likewise.
* NEWS: Mention the improvement.
2018-03-28 00:00:09 -07:00
Pádraig Brady ece7157933 maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2018-01-01 15:57:22 +00:00
Paul Eggert be87d61299 all: prefer HTTPS in URLs 2017-09-19 01:22:54 -07:00
Pádraig Brady e7a2580b96 maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2017-01-01 22:38:02 +00:00
Assaf Gordon 672663e1b0 numfmt: add the -z,--zero-terminated option
* doc/coreutils.texi (numfmt invocation): Reference the description.
* src/numfmt.c: Parameterize '\n' references.
* tests/misc/numfmt.pl: Add tests for character and field processing.
* NEWS: Mention the new feature.
2016-01-13 11:11:36 +00:00
Pádraig Brady b16e999f55 maint: update all copyright year number ranges
Run "make update-copyright" and then...

* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
2016-01-01 14:10:41 +00:00
Assaf Gordon ba24c42e13 numfmt: use new set-fields module to parse --field
numfmt --field=LIST can accept the same options as cut.

* bootstrap.conf: remove xlist, linked-list
* src/local.mk: link numfmt with set-fields
* src/numfmt.c: use set-fields.c instead of custom field parsing code.
  (include_field): adapt to new code.
* tests/misc/numfmt.pl: add new tests, adapt current tests to new
  error message wording from set-fields.c
2015-09-12 02:27:32 +00:00
Pádraig Brady 970b891702 tests: avoid false failures on OpenBSD 5.7
* tests/du/threshold.sh: Homogenize getopt error messages.
* tests/misc/numfmt.pl: Likewise.
* tests/mv/i-3.sh: Skip on *BSD not just FreeBSD.
2015-07-02 19:15:08 +01:00
Pádraig Brady 4f824b3f36 numfmt: don't hardcode floating point limits
* src/numfmt.c (MAX_UNSCALED_DIGITS): Set this to LDBL_DIG
rather than hardcoding at 18 for better portability.
* tests/misc/numfmt.pl: Restrict limit tests to supported platforms.
2015-06-22 17:22:04 +01:00
Pádraig Brady bc9d16b79f numfmt: handle leading zeros correctly
* src/numfmt.c (simple_strtod_int): Don't count leading zeros
as significant digits.  Also have leading zeros as optional
for floating point numbers.
* tests/misc/numfmt.pl: Add test cases.
* NEWS: Mention the fix.
2015-06-22 02:21:02 +01:00
Pádraig Brady a3c3e1e9e6 numfmt: avoid integer overflow when rounding
Due to existing limits this is usually triggered
with an increased precision.  We also add further
restrictions to the output of increased precision numbers.

* src/numfmt.c (simple_round): Avoid intmax_t overflow.
(simple_strtod_int): Count digits consistently
for precision loss and overflow detection.
(prepare_padded_number): Include the precision
when excluding numbers to output, since the precision
determines the ultimate values used in the rounding scheme
in double_to_human().
* tests/misc/numfmt.pl: Add previously failing test cases.
* NEWS: Mention the fix.
2015-06-22 02:02:05 +01:00
Pádraig Brady 0a279f6190 numfmt: support user specified output precision
* src/numfmt.c (usage): Update the --format description
to indicate precision is allowed.
(parse_format_string): Parse a precision specification
like the standard printf does.
(double_to_human): Honor the precision in --to mode.
* tests/misc/numfmt.pl: New tests.
* doc/coreutils.texi (numfmt invocation): Mention the new feature.
* NEWS: Likewise.
2015-06-21 04:16:29 +01:00
Dylan Cali 71063bc858 numfmt: implement support for field ranges
* src/numfmt.c: Replace field handling code with logic that understands
field range specifiers.  Instead of processing a single field and
printing line prefix/suffix around it, process each field in the line
checking whether it has been included for conversion.  If so convert and
print, otherwise just print the unaltered field.
(extract_fields): Removed.
(skip_fields): Removed.
(process_line): Gutted and heavily reworked.
(process_suffixed_number): FIELD is now passed as an arg instead of
using a global.
(parse_field_arg): New function that parses field range specifiers.
(next_field): New function that returns pointers to the next field in
a line.
(process_field): New function that wraps the field conversion logic
(include_field): New function that checks whether a field should be
converted
(compare_field): New function used for field value comparisons in a
gl_list.
(free_field): New function used for freeing field values in a gl_list.
Global variable FIELD removed.
New global variable all_fields indicates whether all fields should be
processed.
New global variable all_fields_after stores the first field of a N-
style range.
New global variable all_fields_before stores the last field of a -M
style range.
New global variable field_list stores explicitly specified fields to
process (N N,M or N-M style specifiers).
(usage): Document newly supported field range specifiers.
* bootstrap.conf: Include xlist and linked-list modules.  numfmt now
uses the gl_linked_list implementation to store the field ranges.
* tests/misc/numfmt.pl: Add tests for 'cut style' field ranges.
Adjust existing tests as partial output can occur before an error
Remove test for the 'invalid' field -5.. this is now a valid range.
* gnulib: update to avoid compiler warnings in linked-list.
* NEWS: Mention the new feature.
2015-06-19 19:59:21 +01:00
Pádraig Brady 6fadd46acd numfmt: handle suffixes consistently with --{from,to}-unit
* src/numfmt.c (unit_to_umax): Support SI (power of 10) suffixes
with the --from-unit and --to-unit options.  Treat suffixes like
is done with --from=auto, which for example will change the meaning
of --to-unit=G to that of --to-unit=Gi.  The suffix support was
previously undocumented and it's better to avoid the traditional
coreutils suffix handling in numfmt by default.
* doc/coreutils.texi: Document the new behavior.  Also fix a typo
mentioning {from,to}=units=.
* tests/misc/numfmt.pl: Adjust accordingly.
* NEWS: Mention the change in behavior.
2015-06-19 14:50:34 +01:00
Pádraig Brady e0afeb0099 maint: update all copyright year number ranges
Run "make update-copyright" and then...

* tests/sample-test: Adjust to use the single most recent year.
* tests/du/bind-mount-dir-cycle-v2.sh: Fix case in copyright message,
so that year is updated automatically in future.
2015-01-01 04:52:17 +00:00
Pádraig Brady 10a9652435 numfmt: support zero padding using --format="%010f"
* src/numfmt.c (setup_padding_buffer): Simplify the code by not
explicitly dealing with heap exhaustion.
(parse_format_string): Likewise.  Handle multiple grouping
modifiers as does the standard printf.  Handle the new leading
zero --format modifier.
(double_to_human): Use more defensive coding against overwriting
stack buffers.  Honor the leading zeros width.
(usage): Mention the leading zero --format modifier.
(main): Allow --padding in combo with a --format (width),
as the number of leading zeros are useful independent of
the main field width.
* doc/coreutils.texi (numfmt invocation): Likewise.
* tests/misc/numfmt.pl: Add new test cases.
* NEWS: Mention the improvement.
2014-05-02 01:59:31 +01:00
Pádraig Brady d695bc8bc6 maint: various cleanups
* tests/misc/numfmt.pl: Fix comment misspelling.
* src/cut.c: Likewise.
* src/tsort.c (detect_loop): Replace an fprintf() with error().
2014-04-05 01:59:39 +01:00
Bernhard Voelker 275c078fb4 maint: update all copyright year number ranges
Run "make update-copyright", but then also run this,
  perl -pi -e 's/2\d\d\d-//' tests/sample-test
to make that one script use the single most recent year number.
2014-01-02 22:19:59 +01:00
Bernhard Voelker dffaa8db2d maint: update copyright year number ranges of numfmt sources
Run "make update-copyright".

* src/numfmt.c: Update copyright year number range.  This file has
obviously been added to coreutils after and without the annual update.
* tests/misc/numfmt.pl: Likewise.
2013-08-02 08:31:36 +02:00
Bernhard Voelker 0c452fe97a maint: make some shell and perl scripts executable in 'tests/'
Some newer test scripts - partially ones from me - are not executable.
It does not seem to be a problem, but for consistency and to avoid
future problems on unusual platforms or shells change the permissions
by adding the executable bit.

* cfg.mk (sc_tests_executable): Add new syntax-check rule to ensure
that all test scripts are executable.
* tests/df/df-output.sh: Change file mode from 644 to 755.
* tests/du/threshold.sh: Likewise.
* tests/factor/run.sh: Likewise.
* tests/init.sh: Likewise.
* tests/misc/csplit-suppress-matched.pl: Likewise.
* tests/misc/numfmt.pl: Likewise.
* tests/tail-2/retry.sh: Likewise.
2013-07-22 01:07:52 +02:00
Stefano Lattarini ba60445f73 tests: numfmt: use the printf program, not the shell builtin
* tests/misc/numfmt.pl: Avoid a spurious failure when
/bin/sh is dash (as can happen on Debian systems).
2013-05-27 00:24:00 +01:00
Pádraig Brady 3309e880fb maint: consolidate developer debug messages
Both factor and numfmt recently introduced debug messages
for developers, enabled by --verbose and ---devdebug respectively.
There were a few issues though:
 1. They used different mechanisms to enable these messages.
 2. factor used --verbose which might be needed for something else
 3. They used different methods to output the messages,
    and numfmt used error() which added an unwanted newline
 4. numfmt marked all these messages for translation and factor
    marked a couple.  We really don't need these translated.
So we fix the above issues here while renaming the enabling
option for both commands to ---debug (still undocumented).

* src/factor.c (verbose): Rename to dev_debug and change from int to
bool as it's just a toggle flag.
(long_options): Rename --verbose to ---debug.
* src/system.h (devmsg): A new inline function to output a message
if enabled by a global dev_debug variable in the compilation unit.
* src/numfmt.c: Use devmsg() rather than error().
Also remove the translation tags from these messages.
Also change debug flag to bool from int.
* tests/misc/numfmt.pl: Adjust for the ---devdebug to ---debug change.
* cfg.mk (sc_marked_devdiagnostics): Add a syntax check to ensure
translations are not added to devmsg calls.

Reported by Göran Uddeborg in http://bugs.gnu.org/13665
2013-02-10 20:30:53 +00:00
Pádraig Brady 54deb54db9 tests: skip numfmt grouping tests on some systems
* tests/misc/numfmt.pl: When the system locale grouping doesn't
match our expected format for grouping 1234 in the fr_FR locale,
reset the locale to 'C' so as to skip all locale tests.
2013-02-10 05:04:09 +00:00
Assaf Gordon 8a303a8963 numfmt: a new command to format numbers
* AUTHORS: Add my name.
* NEWS: Mention the new program.
* README: Reference the new program.
* src/numfmt.c: New file.
* src/.gitignore: Ignore the new binary.
* build-aux/gen-lists-of-programs.sh: Update.
* scripts/git-hooks/commit-msg: Allow numfmt: commit prefix.
* po/POTFILES.in: Add new c file.
* tests/misc/numfmt.pl: A new test file giving >93% coverage.
* tests/local.mk: Reference the new test.
* man/.gitignore: Ignore the new man page.
* man/local.mk: Reference the new man page.
* man/numfmt.x: A new template.
* doc/coreutils.texi: Document the new command.
2013-02-04 23:40:32 +00:00