Commit Graph
810 Commits
Author SHA1 Message Date
Paul Eggert 228652f200 sed: pacify ODS 12.6
* sed/compile.c (bad_prog, bad_prog_notranslate)
(mark_subst_opts): Omit some unreachable code (including an
unreachable ‘unreachable ()’!) to pacify Oracle Developer Studio 12.6.
2026-08-17 01:17:23 -07:00
Paul Eggert 598c3d0b95 sed: omit some multithreading cruft
sed is single-threaded, so does not need multithreading support.
* bootstrap.conf (avoided_gnulib_modules): Avoid lock, not lock-tests.
(bootstrap_post_import_hook): Copy lib/glthread/lock.h by hand,
as we now avoid ‘lock’.
* configure.ac (GNULIB_EXCLUDE_SINGLE_THREAD)
(GNULIB_LOCALENAME_SINGLE_THREAD)
(GNULIB_MBRTOWC_SINGLE_THREAD, GNULIB_REGEX_SINGLE_THREAD)
(GNULIB_WCHAR_SINGLE_LOCALE, GNULIB_MBRTOC32_REGULAR):
New macros.
* lib/.gitignore, m4/.gitignore: Adjust accordingly.

sed: also avoid ‘lock’ module

* bootstrap.conf
2026-08-17 01:17:23 -07:00
Paul Eggert 6d22645e41 sed: basicdefs.h cleanup
Also, remove EMX support, as EMX is no longer used.
* basicdefs.h: Don’t include stdbool.h;
the Gnulib bool module means it’s not needed any more.
(PATH_MAX): Remove; no longer used.
(initialize_main): Remove.  No longer needed now that we
no longer support EMX.  All uses removed.
* sed/execute.c (open_next_file):
* sed/utils.c (ck_fdopen): Remove EMX support.
2026-08-17 01:17:23 -07:00
Paul Eggert f137d0a95f sed: MB_CUR_MAX→MB_LEN_MAX for speed
* sed/execute.c (str_append_modified): Use MB_LEN_MAX
not MB_CUR_MAX, as it’s a bit faster and there’s little
point to space optimization here.
2026-08-17 01:17:23 -07:00
Paul Eggert 0222c34e3a sed: don’t cache MB_CUR_MAX
These days MB_CUR_MAX is so fast on its own, and we so rarely use it,
that there’s little point to our caching it.
* sed/mbcs.c: Remove.
All uses of mb_cur_max replaced by MB_CUR_MAX.
All uses of initialize_mbcs removed.
* sed/local.mk (sed_sed_SOURCES): Remove sed/mbcs.c.
2026-08-17 01:17:23 -07:00
Paul Eggert 2b155a6c8b sed: support multibyte delimiters
* basicdefs.h: Do not include wchar.h, wctype.h.
(ISASCII, ISBLANK, ISSPACE): Remove.  All uses replaced
by c32isblank, c32isspace.
* bootstrap.conf (gnulib_modules): Add c32isblank, c32isprint,
c32isspace, c32rtomb, c32toower, c32toupper, mcel.  Remove btowc,
localcharset, mbrlen, mbrtowc, mbsinit, wcrtomb, wctob, wctype-h.
* configure.ac: Do not check for headers locale.h, errno.h,
wchar.h, wctype.h, mcheck.h.
Do not check for isascii.  Remove ac_cv_func_wcscoll hack.
(XFAIL_TESTS): Remove check for whether to run UTF-8 Cyrillic
tests, as I hope the other changes in this patch should fix
problems in that area.
* lib/.gitignore, m4/.gitignore: Adjust
* sed/compile.c (struct prog_info.prev): New member.
Adjust all users of this struct to update and use it when needed.
(INCHAR_EOF): New constant.
(inchar): Return a char32_t value for each wide character, a
negative byte value for encoding errors, and INCHAR_EOF for end of
input.  All callers changed.
(savchar): Now takes no arguments.  All callers changed.
(in_nonblank): Use char32_t tests rather than ctype.h tests.
(add_prev_to_buffer): New function.
(add_then_next, match_slash, read_text):
Use add_prev_to_buffer instead of add1_buffer when the idea is to
copy a character from input to output, as the character might be
multibyte and add1_buffer does only single-byte.
(match_slash): Support multibyte delimiters.  Test for matchers
individually rather than via strchr, to make it clearer that we
are not testing for '\0'.
* sed/compile.c (add_then_next, snarf_char_class):
* sed/debug.c (debug_print_mb):
* sed/execute.c (line_init, line_reset, line_copy, line_append)
(line_exchange):
Remove now-unnecessary arg.  All callers changed.
* sed/compile.c (compile_program, normalize_text):
* sed/debug.c (debug_print_regex_pattern):
* sed/execute.c (str_append_modified, do_list, translate_mb)
(debug_print_line):
Use mcel_scan/c32rtomb1 not mbrtowc1/mbrlen1/wcrtomb,
as mcel_scan should work better on MS-Windows.
* sed/debug.c (debug_print_char): Replace args c and clen
with an mcel_t arg g.  All callers changed.
(debug_print_mb): Adjust to new meaning of int “characters”.
* sed/execute.c (struct line.mbstate): Remove.  All uses removed.
(str_append): Omit no-longer-needed code put in just to update a
no-longer-needed mbstate_t.
(str_append_modified): Don’t worry about c32rtomb1 returning
(size_t) -1, as it now returns idx_t and anyway that cannot happen.
* sed/mbcs.c: Don’t include string.h, localcharset.h.
(is_mb_char, is_utf8): Remove.  All uses removed.
* sed/sed.h: Include mcel.h.
(mbrtowc1, mbrlen1): Remove.  All callers changed to use mcel_scan.
(c32rtomb1): New function.
(IS_MB_CHAR): Remove; no longer used.
* sed/utils.c (add1_buffer): Now returns void, since no caller
used the return value.  2nd arg is now char, not int.
Remove special case for EOF.  All callers changed.
* testsuite/mb-bad-delim.sh, testsuite/mb-y-translate.sh:
Multibyte delimiters are no longer prohibited, so adjust test
cases accordingly.  Put newlines at end of text files,
as required by POSIX.
2026-08-17 01:17:23 -07:00
Paul Eggert 11f413d9c0 doc: say a:i::r:?
Document that there is no command that is like r but
acts before the current line (Bug#36201).
2026-08-17 01:17:23 -07:00
Paul Eggert c3744164d4 sed: streamline utils.c a bit
* sed/utils.c (quotef): Now static.
(ck_fdopen): Define only if needed.
(add_buffer): Remove; unused.
2026-08-17 01:17:23 -07:00
Paul Eggert 49336f7e4b sed: fix EOF value assumption
* sed/sed.c (main): Don’t assume EOF == -1; it’s not required
by either POSIX or C.
2026-08-17 01:17:23 -07:00
Paul Eggert 156efe87ae sed: prefer plain char *
Instead of converting pointers between char * and unsigned char *,
convert their members between char and unsigned char.
This is a little cleaner as pointer casts are too powerful in C.
* sed/compile.c (struct prog_info, compile_program):
* sed/execute.c (execute_program):
* sed/sed.h (struct sed_cmd):
Use plain char * rather than unsigned char *.
All uses changed.
2026-08-17 01:17:23 -07:00
Paul Eggert da5cecf553 sed: 'l' and printable multibyte characters
Fix a longstanding bug where the 'l' command mistakenly
escapes bytes in a multibyte character.
* basicdefs.h (ISPRINT): Remove.  All uses removed.
* bootstrap.conf (gnulib_modules): Add wctype-h, which is now needed
directly instead of indirectly, as we now use iswprint.
* sed/debug.c (debug_print_char):
* sed/execute.c (do_list):
Use iswprint instead of ISPRINT, so that things work correctly
with multibyte locales.
* sed/debug.c (debug_print_char):
New args P and CLEN.  All uses changed.
* testsuite/cmd-l.sh: Test for the bug.
2026-08-17 01:17:23 -07:00
Paul Eggert 6c92794c2b sed: remove ISALNUM etc
* basicdefs.h (ISALNUM, ISALPHA, ISCNTRL, ISLOWER, ISPUNCT)
(ISUPPER, ISXDIGIT): Remove; unused.
2026-08-17 01:17:23 -07:00
Paul Eggert fe0a844c0a sed: ISDIGIT → c_isdigit
ISDIGIT is locale-dependent, and in theory could misfire if
any locale has a single-byte digit that is not '0'..'9'.
Fix this by using c_isdigit, which should be a bit faster anyway.
* basicdefs.h (ISDIGIT): Remove.  All uses replaced by c_isdigit.
* bootstrap.conf (gnulib_modules): Add c-ctype.
* lib/.gitignore: Update.
* sed/compile.c: Include c-ctype.h.
2026-08-17 01:17:23 -07:00
Paul Eggert d43a870aa2 sed: simplify reading commands from file
This refactoring should help make future changes easier.
* bootstrap.conf (gnulib_modules): Add read-file.
* configure.ac (HAVE_FOPEN_RT): Remove; no longer used.
* lib/.gitignore: Adjust for read-file files.
* sed/compile.c: Include errno.h, read-file.h.
(struct prog_info.file): Remove.  All uses removed.
(inchar, savchar, compile_program): Just read from the string.
Assume ! (NULL < NULL) && (NULL - NULL) == 0,
as per Gnulib guidelines.  This should speed things up here.
(compile_file): Read the file into a string, and process the string.
2026-08-17 01:17:23 -07:00
Paul Eggert 4e2995cd01 sed: simplify multibyte substitution
When handling the y command, or the case-changing replacement,
stay a bit more in the wchar_t world, to simplify allocation
and help make future changes simpler.
* basicdefs.h (STREQ_LEN): Remove; no longer used.
* cfg.mk (sc_prohibit_strncmp): Don’t suggest STREQ_LEN.
* sed/compile.c (compile_program):
* sed/execute.c (str_append_modified, translate_mb):
Instead of preserving the bytes of a multibyte character, just
save its wchar_t counterpart, or negative of its single-byte
encoding error.
* sed/debug.c (debug_print_mb): New static function.
(debug_print_translation): Use it to adjust to new data structure.
* sed/execute.c (translate_mb): Args are now NPAIRS and PAIR, not
TRANS.  Caller changed.
* sed/sed.h (struct sed_cmd.translate.mb): Now a struct with
an npairs and pair members, instead of a single char **.
All uses changed.
2026-08-17 01:17:23 -07:00
Paul Eggert bd3b429b80 sed: shrink struct sed_cmd translate slots
* sed/sed.h (struct sed_cmd): Rename translate to translate.sb,
and translatemb to translate.mb, as they are never both used.
All uses changed.
2026-08-17 01:17:23 -07:00
Paul Eggert ff5bec34b1 sed: remove MEMDUP, OB_MALLOC
* basicdefs.h (MEMDUP, OB_MALLOC):
Remove.  All uses of these macros replaced by function calls,
which are simpler nowadays.
2026-08-17 01:17:23 -07:00
Paul Eggert f0e2d07924 sed: remove YMAP_LENGTH
* sed/compile.c (YMAP_LENGTH): Remove.
All uses replaced by UCHAR_MAX + 1.
2026-08-17 01:17:23 -07:00
Paul Eggert 0e9c603e82 sed: prefer mbrlen to mbrtowc+NULL
* bootstrap.conf (gnulib_modules): Add mbrlen.
* lib/.gitignore, m4/.gitignore: Adjust
* sed/mbcs.c (is_mb_char), sed/sed.h (mbrlen1):
Prefer mbrlen to mbrtowc with NULL, as it can be a bit faster.
2026-08-17 01:17:23 -07:00
Paul Eggert 2d2a87d3b6 sed: more consistency about encoding errors
* sed/compile.c (compile_program, normalize_text):
Prefer separate locals to reusing locals, to make things a bit clearer.
* sed/execute.c (str_append_modified):
Treat invalid sequences consistently with elsewhere.
* sed/sed.c (SED_INLINE): New macro.
* sed/sed.h: Use _GL_INLINE_HEADER_BEGIN, _GL_INLINE_HEADER_END.
(SED_INLINE): Define if not already defined.
(MBRLEN): Remove this macro, replacing all uses with ...
(mbrlen1): ... this new function.  All callers changed.
This simplifies them, and fixes a few inconsistencies.
2026-08-17 01:17:23 -07:00
Paul Eggert 818d3a5675 sed: don’t squash size_t to int
* sed/mbcs.c (is_mb_char): Don’t rely on implementation-defined
behavior when converting values greater than INT_MAX to int.
2026-08-17 01:17:23 -07:00
Paul Eggert de39cf77c0 sed: fix panic printf format
* sed/utils.c (ck_fwrite): %jd → %td for idx_t.
This matters for i686.
2026-08-17 01:17:23 -07:00
Paul Eggert c1fc247803 sed: prefer mbszero to clear mbstate_t
* bootstrap.conf (gnulib_modules): Add mbszero.
* sed/compile.c (match_slash, compile_program, normalize_text):
* sed/execute.c (str_append, str_append_modified, line_init)
(line_reset, translate_mb):
* sed/mbcs.c (is_mb_char):
Use mbszero to clear mbstate_t, as it’s a bit faster.
2026-08-17 01:17:23 -07:00
Paul Eggert 61cb0a5547 sed: prefer streq to strcmp
* basicdefs.h (STREQ, STRPREFIX): Remove.  All uses removed.
* bootstrap.conf (gnulib_modules): Add streq.
* cfg.mk (local-checks-to-skip): Remove sc_prohibit_strcmp.
* sed/compile.c (get_openfile, check_final_program):
* sed/execute.c (closedown):
* sed/mbcs.c (initialize_mbcs):
Prefer streq to strcmp.
2026-08-17 01:17:23 -07:00
Paul Eggert b05def1241 sed: prefer assignment to memcpy/memset
* sed/compile.c (next_cmd_entry, match_slash, setup_label)
(compile_program, check_final_program):
* sed/execute.c (str_append, str_append_modified, line_init)
(line_reset, line_copy, line_append, line_exchange, open_next_file):
* sed/mbcs.c (is_mb_char):
Prefer assignment to memcpy/memset when that is easy.
* sed/execute.c (SIZEOF_LINE): Remove.
2026-08-17 01:17:23 -07:00
Paul Eggert ff5921491a build: update gnulib submodule to latest
In addition to the usual ‘git submodule foreach git pull origin master’,
this also makes the following changes to adjust to current Gnulib:
* bootstrap: Sync from gnulib.
* lib/.gitignore, m4/.gitignore: Update to match current Gnulib.
* sed/utils.c: Include min-eloop-threshold.h, not eloop-threshold.h.
(follow_symlink): Use MIN_ELOOP_THRESHOLD, not __eloop_threshold ().
2026-08-11 08:55:15 -07:00
Paul Eggert 31df9f71cb sed: minor int width fixes
* sed/compile.c (finish_program):
* sed/execute.c (reset_addresses):
Use idx_t, not int, for vars that might exceed INT_MAX in theory.
* sed/debug.c (block_level, debug_print_command):
Use off_t, not int, likewise.
2026-05-25 10:30:53 -07:00
Paul Eggert 163b6e8819 maint: one more .gitignore update
* m4/.gitignore: Remove wchar_t.m4.
Not sure why the previous iteration missed this.
2026-05-25 10:30:53 -07:00
Paul Eggert 82c7f52c04 maint: update .gitignore files for current Gnulib
Also, sort entries, and remove duplicates (e.g., already
specified in parent .gitignore).
2026-05-25 10:02:46 -07:00
Paul Eggert 4c806767f7 build: update gnulib submodule to latest 2026-05-25 10:02:46 -07:00
Paul Eggert 5b94ed12b9 maint: pacify gcc -Wuseless-cast
* basicdefs.h (OB_MALLOC): Omit useless cast.
* sed/execute.c (do_subst): Use compound literal instead of cast,
since the cast is sometimes useless, sometimes not.
Compound literals are safer anyway.
2026-05-25 10:02:46 -07:00
Paul Eggert 3ffece093f sed: escape unusual chars in file name diagnostics
Problem reported by Michał Majchrowicz and Marcin Wyczechowski.
* NEWS, THANKS.in: Mention this.
* bootstrap.conf (gnulib_modules): Add quotearg.
* sed/compile.c (vbad_prog):
* sed/execute.c (open_next_file):
Quote file names in diagnostics.
* sed/utils.c: Include quotearg.h.
(quotef, quotef_n): New functions, inspired by coreutils macros
of the same name.
* sed/utils.c (ck_fopen, ck_fdopen, ck_mkstemp, ck_fwrite)
(ck_fread, ck_getdelim, ck_fflush, follow_symlink, ck_rename):
2026-05-25 10:02:46 -07:00
Bruno Haible 560dc3864c maint: add gperf as a build prerequisite
* bootstrap.conf (buildreq): Add gperf.
2026-05-18 20:58:50 -07:00
Jim Meyering 57040a1f89 maint: simplify a test script
* testsuite/posix-backslash-warn.sh: Use simple echo, printf or
/dev/null rather than a trivial multi-line here-script.
2026-05-15 13:30:54 -07:00
Collin Funk 367d7e98f1 sed: don't warn about escaped special characters with --posix
* NEWS: Mention the bug fix.
* sed/compile.c: Don't warn about special characters.
* testsuite/posix-backslash-warn.sh: Add a few test cases.
* THANKS.in: Update.
Reported by Karl Ove Hufthammer in https://bugs.gnu.org/81037
2026-05-15 13:29:25 -07:00
Jim Meyering f8c5af67f3 build: avoid new failure when combining --enable-gcc-warnings and GCC16
* configure.ac: Add gl_WARN_ADD([-Wno-keyword-macro]), to suppress many
new warnings like this from GCC16:
  ./config.h:2418:9: warning: keyword 'restrict' defined as macro\
    [-Wkeyword-macro]
   2418 | #define restrict __restrict__
2026-05-06 17:59:30 -07:00
Jim Meyering dda43e9c75 gnulib: update to latest
* lib/.gitignore: Autoupdate to reflect additions.
* m4/.gitignore: Likewise.
2026-05-06 17:43:15 -07:00
Jim Meyering 4dcb36bc70 build: include stdint.h, for use of intmax_t
* sed/sed.h: Include stdint.h, to cover use of intmax_t.
Suggested by Bruno Haible in
https://lists.gnu.org/r/sed-devel/2026-05/msg00000.html
2026-05-06 17:29:14 -07:00
Bruno Haible 4ee49f606d tests: really skip read-error-stale-errno on Cygwin
* testsuite/read-error-stale-errno.sh: Skip when setting O_NONBLOCK on
an fd in a child process has no effect on the fd in the parent process.
Reported in https://lists.gnu.org/r/sed-devel/2026-04/msg00016.html
2026-04-23 11:51:17 -07:00
Jim Meyering 63b9c25c50 maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2026-04-21 18:42:52 -07:00
Jim Meyering 89b7a2224d version 4.10
* NEWS: Record release date.
v4.10
2026-04-21 18:32:56 -07:00
Jim Meyering f04e0da045 tests: skip C-locale sub-tests when "C" is multibyte
* testsuite/mb-bad-delim.sh: Don't run the sub-test that uses an
incomplete multibyte byte as a single-byte delimiter in C locale.
* testsuite/mb-y-translate.sh: Don't run the two sub-tests that
expect a y/// length-mismatch error in C locale.
Android's bionic libc uses UTF-8 even in the C locale, so these
sub-tests would always fail.
Reported by Bruno Haible in
https://lists.gnu.org/r/sed-devel/2026-04/msg00019.html
2026-04-21 08:30:31 -07:00
Jim Meyering 13604c8f68 tests: tolerate SELinux setfscreatecon warning on Android/Termux
* init.cfg (remove_selinux_warning_): New function.
* testsuite/in-place-hyphen.sh: Use it before comparing stderr.
* testsuite/in-place-suffix-backup.sh: Likewise.
* testsuite/panic-tests.sh: Likewise.
* testsuite/temp-file-cleanup.sh: Likewise.
Reported by Bruno Haible in
https://lists.gnu.org/r/sed-devel/2026-04/msg00019.html
2026-04-21 08:11:48 -07:00
Jim Meyering a8d3d77433 sed: fix two newline-lacking diagnostics
* sed/execute.c (open_next_file): Terminate each of two fprintf format
strings with "\n".
2026-04-21 07:55:02 -07:00
Brun Haible f491a1cc14 doc: fix typo in NEWS
* NEWS: s/getoph.h/getopt.h/
2026-04-21 06:20:45 -07:00
Jim Meyering 682d04b7d7 tests: skip read-error-stale-errno on Cygwin
* testsuite/read-error-stale-errno.sh: Skip when a FIFO with
O_NONBLOCK returns EOF instead of EAGAIN while a writer is still
attached.  Cygwin gets this wrong.
Reported by Bruno Haible in
https://lists.gnu.org/r/sed-devel/2026-04/msg00011.html
2026-04-20 17:26:50 -07:00
Jim Meyering 6ec335494f sed: s///g now works for input lines longer than 2GB
* sed/regexp.c (match_regex): Change type of 'ret' from int
to regoff_t, to avoid truncation results larger than INT_MAX.
* testsuite/regex-max-int.sh: Update this "very expensive"
test to expect correct output (bbbbb) rather than exit-code-4 panic.
* NEWS (Bug fixes): Mention this.
The following is related but mostly incidental (--debug only).
I tested via --debug, (where it generate 20GB of output!), but couldn't
easily add a test case without adding a further option to suppress
debug printing of each escaped input line.
* sed/execute.c (do_subst): Don't cast regoff_t values to (int) for
debug printing via %d. Instead, cast to (ptrdiff_t) and print via %td.
2026-04-20 16:59:50 -07:00
Jim Meyering 6b9b43c55c sed: -i --follow-symlinks: fix TOCTOU race (CVE-2026-5958)
When using -i with --follow-symlinks, sed resolved the symlink via
readlink() and then opened the original symlink path in a separate
syscall.  An attacker who swapped the symlink between those two
operations could cause sed to read from an attacker-controlled file
while writing the result to the originally resolved target, enabling
arbitrary file overwrite.  Fix by opening the already-resolved path
rather than re-traversing the symlink.
Reported by Michał Majchrowicz and Marcin Wyczechowski (AFINE Team).
* sed/execute.c (open_next_file): Use input->in_file_name (the
resolved path) rather than "name" (the original symlink) in the
ck_fopen call.
* NEWS: Mention this.
2026-04-20 07:47:24 -07:00
Jim Meyering fe548a4297 build: update gnulib to latest
* gnulib: Update to latest.
* lib/.gitignore: Add stdcountof.in.h.
2026-04-19 20:55:40 -07:00
Jim Meyering b99731ec2c tests: add a test for the just-fixed bug
* testsuite/read-error-stale-errno.sh: New file.
* testsuite/local.mk (T): Add it.
2026-04-19 20:17:47 -07:00