Commit Graph
699 Commits
Author SHA1 Message Date
Paul Eggert ccef237b83 maint: sync bootstrap
* bootstrap: Sync from gnulib.
2023-01-30 09:58:54 -08:00
Paul Eggert 4b580108ee maint: sync README-hacking
* README-hacking: Sync from coreutils.
2023-01-30 09:58:54 -08:00
Paul Eggert 5c359da036 sed: fix int width in decl
* sed/sed.h (lcmd_out_line_len): Declare intmax_t, not idx_t.
Problem reported by Bruno Haible in:
https://lists.gnu.org/r/sed-devel/2023-01/msg00001.html
2023-01-30 09:58:54 -08:00
Paul Eggert 8c802c5078 maint: quote 'like this' not `like this' 2023-01-30 09:58:54 -08:00
Jim Meyering fa9e4e4fc0 tests: reflect recent diagnostic change
v4.9-14-g3d75b4a changed the diagnostic, but did not update
the occurrences of the diagnostic in expected test outputs.
* testsuite/compile-errors.sh: Accommodate changed diagnostic.
* testsuite/mb-y-translate.sh: Likewise.
* testsuite/normalize-text.sh: Likewise.
Reported by Bruno Haible in https://bugs.gnu.org/61159
2023-01-29 17:36:34 -08:00
Bruno Haible c2683c7ba2 build: accommodate ACL change in updated gnulib
* sed/local.mk (sed_sed_LDADD): Add $(QCOPY_ACL_LIB).
Reported in https://lists.gnu.org/r/sed-devel/2023-01/msg00001.html
2023-01-29 17:36:34 -08:00
Jim Meyering 4d231bb07d build: update gnulib to latest
* lib/.gitignore, m4/.gitignore: Also reflect regenerated .gitignore files.
2023-01-29 17:18:21 -08:00
Jim Meyering 3d75b4a278 maint: reword to avoid a too-long-line warning
* sed/compile.c (compile_program): Improve diagnostic wording
that also happens to shorten the line so it's length is < 80.
2023-01-01 17:38:09 -08:00
Jim Meyering f6df64598f maint: expand a few TABs to placate make syntax-check
* sed/execute.c (alloc): Indent with spaces, not TABs.
2023-01-01 17:30:41 -08:00
Jim Meyering 805a40757e build: update gnulib to latest 2023-01-01 17:20:41 -08:00
Jim Meyering d81710cc8d maint: update copyright dates 2023-01-01 17:19:34 -08:00
Paul Eggert 602c0cb92c maint: add test cases for integer overflow
* testsuite/execute-tests.sh: Add some tests with
enormous integers.
2022-12-19 13:47:19 -08:00
Paul Eggert a9b1b679b6 sed: improve integer overflow checking
Fix some some longstanding but unlikely integer overflows.
Internally, 'sed' now more often prefers signed integer arithmetic,
which can be checked automatically via 'gcc -fsanitize=undefined'.
* basicdefs.h (countT): Remove.  All uses replaced
with a more-specific signed type, e.g., idx_t.
Similarly, change uses of types like size_t to
signed types like idx_t when appropriate.
(REALLOC): Remove; no longer used.  We now use xpalloc
because that detects integer overflow in size calculations.
Also, we no longer use XCALLOC since the code never relies
on the storage being zero, and leaving it uninitialized is
more likely to catch errors when debugging implementations
are used.  We use XNMALLOC instead, or xpalloc.
* bootstrap.conf (gnulib_modules): Add stdckdint, strtoimax.
* lib/.gitignore, m4/.gitignore: Update for new Gnulib modules.
* sed/compile.c: Include stdckdint.h.
(VECTOR_ALLOC_INCREMENT): Remove; no longer used.
(in_integer): Return maximal value if integer overflow.
All callers changed to expect this.
(next_cmd_entry): Use xpalloc instead of reallocating by hand,
which might suffer integer overflow.
(normalize_text): Don’t rely on system-defined conversion
of out-of-range size_t to int.
(next_cmd_entry): Arg is now pointer, not pointer-to-pointer.
All uses changed.
* sed/debug.c (debug_print_function): Don’t attempt to
fwrite a null pointer with a zero size.
* sed/execute.c: Include <stdckdint.h>, "minmax.h".
(resize_line): LEN arg is now increment, not total length,
to avoid overflow when calculating total length.
All uses changed.  Do not assume lb->alloc * 2 cannot overflow.
(resize_line, line_copy): Use xpalloc instead of doing realloc by
hand, which might suffer integer overflow.
(str_append_modified): Do not add n to to->length until
after it's known this cannot overflow.
(read_file_line): Don’t assume ssize_t fits in long.
(get_backup_file_name): Don’t assume string length fits in int.
Do not assume PTR-1+1 works; behavior is undefined if PTR
is at buffer start.  Check for integer overflow in buffer
size calculation.
(read_pattern_space): Check for line number overflow.
(match_address_p): Check for address overflow.
(debug_print_line): Omit unnecessary test for in->active being null.
(execute_program): Check for Q overflow.
* sed/regexp.c: Include <stdckdint.h>.
(match_regex): Don’t assume TYPE_MAXIMUM (regoff_t) == INT_MAX.
* sed/sed.c: Include inttypes.h, for strtoimax.
(main): Use strtoimax, not atoi.
* sed/utils.c (init_buffer): Use xmalloc and xpalloc
instead of guessing sizes ourselves, and unnecessarily
initializing.
(resize_buffer): Remove; all callers changed to use xpalloc.
(free_buffer): Don’t call free (NULL), since we already
test whether the pointer is null.
2022-12-19 12:35:32 -08:00
Paul Eggert fc0655d782 sed: simplify by aborting rather than asserting
* sed/debug.c: Do not include assert.h.
(debug_print_function): Use plain abort () rather than assert (0).
This is simpler and just as useful.
2022-12-19 12:33:57 -08:00
Paul Eggert 3ac95905f4 sed: fix unlikely mkostemp-related overflow
* bootstrap.conf (gnulib_modules): Add mempcpy.
* sed/utils.c (ck_mkstemp): Use mempcpy instead of sprintf,
which doesn’t work with strings longer than INT_MAX.
2022-12-19 12:33:57 -08:00
Paul Eggert 03488eeea9 sed: do octal escapes ourselves
* sed/execute.c (do_list): Simplify by doing octal escapes
by hand (!) instead of having sprintf do it.  This makes it
easier for compilers and humans see that buffer overflow
is impossible.  And it’s quite a bit faster if the input
consists mostly of odd control bytes.
2022-12-19 12:33:57 -08:00
Paul Eggert 2aa368b970 sed: simplify how diagnostics are translated
* basicdefs.h, sed/regexp.c (N_): Remove; no longer used.
* bootstrap.conf (XGETTEXT_OPTIONS):
Treat bad_prog as a function that translates its first arg.
Omit unnecessary flags.
* sed/compile.c, sed/regexp.c (errors): Remove this array of
concatenated strings, and associated macros.  All users of these
macros now simply call bad_prog ("diagnostic") instead of bad_prog
(_(DIAGNOSTIC_MACRO)).  This is easier to read and maintain, and
made it easy to catch two diagnostics that were never properly
translated before: "cannot specify modifiers on empty regexp",
"missing filename in r/R/w/W commands".
* sed/compile.c (bad_command): Remove; no longer used.
(vbad_prog): New function, with most of the old bad_prog contents.
(bad_prog): Now variadic, like printf.  Always translate WHY.
All callers changed.
(bad_prog_notranslate): New function.
2022-12-19 12:33:57 -08:00
Paul Eggert 3bb0a12761 maint: fix list of libraries to link to
* sed/local.mk (sed_sed_LDADD): Append all the libraries
that ’bootstrap’ says should be included.
2022-12-18 14:40:04 -08:00
Paul Eggert 8f600f2df2 sed: fix symlink bufsize readlink check
Problem reported by Hauke Mehrtens.
* sed/utils.c (follow_symlink): Fix typo when checking size of
second and later symlink, when that symlink is so large that it
does not fit into the buffer.  Although the bug is not a buffer
overflow, it does cause sed to mishandle the symlink.
* testsuite/follow-symlinks.sh: Test for the bug.
2022-12-17 20:57:23 -08:00
Paul Eggert b82f32d708 maint: update .gitignore 2022-12-17 20:57:23 -08:00
Jim Meyering 836f68dfb8 maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2022-11-06 13:51:47 -08:00
Jim Meyering 7e2e575a36 version 4.9
* NEWS: Record release date.
v4.9
2022-11-06 13:49:47 -08:00
Jim Meyering 3fe543665c build: update gnulib to latest, and testsuite/init.sh 2022-11-06 11:38:14 -08:00
Jim Meyering 8efd09941e tests: do not rely on unportable "grep -q"
* testsuite/inplace-selinux.sh: Avoid unportable "grep -q".
Instead, just redirect to /dev/null.
2022-11-06 10:57:11 -08:00
Jim Meyering 59c6f17a71 test: add a test case and mention the change in NEWS
* testsuite/misc.pl: Add a test to exercise the preceding change.
* NEWS (Changes in behavior): Mention it.
2022-10-23 23:10:26 -07:00
Oğuz f5abeba48e sed: handle the unspecified "n as delimiter alias" case more sensibly
Print the less-surprising variant in a corner case of POSIX-unspecified
behavior.  Before, this would print "n".  Now, it prints "X":
  printf n | sed 'sn\nnXn'; echo
* sed/compile.c (match_slash): Remove special handling of 'n'.
Reported in https://bugs.gnu.org/40242
2022-10-23 10:52:23 -07:00
Jim Meyering c9a1669bcb build: update gnulib to latest, for Solaris static_assert fix 2022-10-23 08:42:07 -07:00
Jim Meyering 777817bb9a build: fix am_DISTCHECK_CONFIGURE_FLAGS typo
* Makefile.am (AM_DISTCHECK_CONFIGURE_FLAGS): Correct typo
in recent change: s/am_/AM_/
2022-10-16 22:41:06 -07:00
Jim Meyering da2316c9d4 build: update gnulib to latest, for getdelim vs macOS workaround 2022-10-16 22:19:37 -07:00
Jim Meyering a4210b6e5b maint: avoid syntax-check failure
* Makefile.am (am_DISTCHECK_CONFIGURE_FLAGS): Change
DISTCHECK_CONFIGURE_FLAGS to
am_DISTCHECK_CONFIGURE_FLAGS.
2022-10-16 22:18:39 -07:00
Jim Meyering 4f4a002ffd maint: tell git to ignore new generated files in lib/
* lib/.gitignore: Add /assert.h and /stdckdint.h
2022-10-16 22:16:10 -07:00
Jim Meyering d1d7eeafe4 build: work when perl is installed but not in /usr/bin/perl
Also support building on systems without perl and when cross-compiling.
* configure.ac (AC_ARG_ENABLE [bold-man-page-references]): New
configure-time option, using code copied from coreutils.
(BUILD_MAN_PAGE, BUILD_DUMMY_MAN_PAGE): Remove definitions.
(HAVE_PERL): Remove bogus conditional.
* doc/dummy-man: New file, from coreutils, with slight adapation.
* build-aux/help2man: Update from coreutils.
* doc/local.mk (EXTRA_DIST): Distribute dummy-man.
(run_help2man): New variable.
(doc/sed.1): Use it rather than simply invoking help2man.
Simplify to use only one copy of this rule, now that run_help2man
works both with and without perl and when cross-compiling.
Move the --name=... string argument into...
* doc/sed.x: ... here. Also make the "Synopsis" appear in generated file.
In https://lists.gnu.org/r/sed-devel/2022-10/msg00005.html,
Bruno Haible reported build failure with perl in a different
location and (in another report) another failure with no perl at all.
2022-10-16 22:13:11 -07:00
Jim Meyering 0fe201e7f4 build: fix THANKS generation
* .mailmap: Building THANKS would fail due to thanks-gen finding
two different email addresses for Mike Frysinger.  Map the older
one to the newer one.
2022-10-16 20:12:20 -07:00
Jim Meyering 23eaab0669 build: update gnulib to latest 2022-10-16 10:37:36 -07:00
Marvin Schmidt bd37192878 tests: remove stray export-ending semicolon in test setup
* testsuite/local.mk (TESTS_ENVIRONMENT): Remove the erroneous ";"
that effectively elided all following var=val pairs from the test
environment. Report and patch from http://bugs.gnu.org/57831
Bug introduced via v4.2.2-100-gb250bd4.
2022-09-15 12:51:36 -07:00
Paul Eggert c07b5066b9 maint: pacify ‘make syntax-check’
* THANKS.in: Put J.T. before Jakub again, as apparently
that’s how it’s done in Fedora 36 which is bleeding edge.
* cfg.mk (sc_THANKS_in_sorted): Update for Fedora 36 sort.
(sc_gitignore_missing): The lib files to ignore are now
in lib/.gitignore, not .gitignore.
2022-07-03 21:11:22 -05:00
Paul Eggert 40dba28572 sed: fix infloop with symlink cycles
* bootstrap.conf (gnulib_modules):
Add eloop-threshold, idx, minmax, readlink.
* configure.ac: Do not check for lstat; no longer needed.
(ENABLE_FOLLOW_SYMLINKS): Remove; all uses removed.
(TEST_SYMLINKS): Depend only on readlink.
* sed/utils.c: Include eloop-threshold.h, idx.h, minmax.h.
(SSIZE_IDX_MAX): New macro.
(follow_symlink): Rewrite to not loop when given a symlink cycle.
Do not use lstat, since readlink suffices.  Use just one memory
buffer, not two; this simplifies memory management.
* testsuite/follow-symlinks.sh: Adjust diagnostics to
to match revised behavior.  Test for symlink loops.
2022-07-03 18:17:41 -05:00
Paul Eggert bfdc4d6ee4 build: update gnulib submodule to latest 2022-07-02 17:54:48 -05:00
Jim Meyering 207f759098 maint: exempt doc/fdl.texi from long-line check
* cfg.mk (FILTER_LONG_LINES): Also exempt doc/fdl.texi.
2022-07-02 15:45:53 -07:00
Paul Eggert 1bc68cb4d5 sed: fix temp file cleanup
Without this fix, the code would sometimes use FP after calling
fclose (FP), which has undefined behavior in C.
Problem found with --enable-gcc-warnings and GCC 12.
* sed/execute.c (open_next_file): Do not register here,
as it’s too late and this can cause the file to not
be cleaned up.
* sed/sed.c (G_file_to_unlink, register_cleanup_file, cancel_cleanup):
Move from here to utils.c.
(cleanup): Call remove_cleanup_file instead of doing it by hand.
* sed/utils.c (struct open_file): Remove member temp
(which was always false) and fclose_failed (which was
not enough to prevent calling fclose with a bad pointer).
All uses changed.
(register_open_file): Do not access p->fp after it’s fclosed,
as that has undefined behavior in C.
Use xmalloc instead of xcalloc, since we initialize all members.
(G_file_to_unlink, register_cleanup_file, cancel_cleanup):
Move from utils.c to here.
(remove_cleanup_file): New function.
(ck_mkstemp): Fix a screwup when mkostemp succeeded but
set_binary_mode or fdopen failed: we might misuse a null pointer,
or forget to clean up the newly-created temp file.
(ck_getdelim): Rename local to avoid confusion with global.
(mark_as_fclose_failed): Remove.  All uses removed.
(ck_fclose): Remove entry from open_files before attempting
to fclose it, so that panicking doesn’t try to fclose it again.
(do_ck_fclose): New arg NAME so that there’s no need to
call mark_as_fclose_failed, which inspected FP after fclosing
it, which is undefined behavior.
(ck_rename): Omit arg UNLINK_IF_FAIL.  All callers changed.
The cleanup handler removes this file now, as needed.
2022-07-02 17:06:39 -05:00
Paul Eggert a2c48eceac .gitconfig: add gt-bug* for some test cases 2022-07-02 17:06:39 -05:00
Paul Eggert de2194891a Remove unused BOOST.tests, PCRE.tests, SPENCER.tests 2022-07-02 17:06:39 -05:00
Paul Eggert e69bb8436f Remove unused files testcases.h, ptestcases.h 2022-07-02 17:06:39 -05:00
Paul Eggert d120d7f586 maint: stop using fdl module
This is as per current Gnulib recommendations.
2022-07-02 17:06:39 -05:00
Paul Eggert 5adab913a8 Update COPYING, bootstrap from Gnulib 2022-07-02 17:06:39 -05:00
Paul Eggert cc351c8251 Update, simplify, and sort .gitignore files 2022-07-02 17:06:39 -05:00
Paul Eggert 49c90357b9 Add NEWS item for dfa fix in Gnulib 2022-07-02 17:06:39 -05:00
Chris Marusich b8f98f7dc3 tests: export CONFIG_HEADER to test scripts
Before this change, the testsuite/inplace-selinux test fails in some
cases when SELinux is available.  The reason it fails is because the
require_selinux_ function in init.cfg expects the CONFIG_HEADER
environment variable to be defined, but in fact it is not defined.
Even though we do invoke AM_CONFIG_HEADER in configure.ac, and even
though the CONFIG_HEADER variable does get set in the resulting
Makefile, nothing actually exports the CONFIG_HEADER variable, so the
test is doomed to fail whenever it is not skipped.  Fix this by
exporting the variable.

* testsuite/local.mk (TESTS_ENVIRONMENT): Add CONFIG_HEADER to the
list of variables to export to the environment of the test scripts.

Reported by Vineet Jain <vkjain@gmail.com> in
https://bugs.gnu.org/36150 .  The fix was suggested by Timothy Sample
<samplet@ngyro.com> in the same bug report.
2022-02-13 11:08:03 -08:00
Jim Meyering 0c8d88bc61 maint: make update-copyright 2022-01-01 15:00:18 -08:00
Jim Meyering 13c54aac5b build: update gnulib to latest; also bootstrap and init.sh 2022-01-01 14:58:10 -08:00