* sed/compile.c (bad_prog, bad_prog_notranslate)
(mark_subst_opts): Omit some unreachable code (including an
unreachable ‘unreachable ()’!) to pacify Oracle Developer Studio 12.6.
sed is single-threaded, so does not need multithreading support.
* bootstrap.conf (avoided_gnulib_modules): Avoid lock, not lock-tests.
(bootstrap_post_import_hook): Copy lib/glthread/lock.h by hand,
as we now avoid ‘lock’.
* configure.ac (GNULIB_EXCLUDE_SINGLE_THREAD)
(GNULIB_LOCALENAME_SINGLE_THREAD)
(GNULIB_MBRTOWC_SINGLE_THREAD, GNULIB_REGEX_SINGLE_THREAD)
(GNULIB_WCHAR_SINGLE_LOCALE, GNULIB_MBRTOC32_REGULAR):
New macros.
* lib/.gitignore, m4/.gitignore: Adjust accordingly.
sed: also avoid ‘lock’ module
* bootstrap.conf
Also, remove EMX support, as EMX is no longer used.
* basicdefs.h: Don’t include stdbool.h;
the Gnulib bool module means it’s not needed any more.
(PATH_MAX): Remove; no longer used.
(initialize_main): Remove. No longer needed now that we
no longer support EMX. All uses removed.
* sed/execute.c (open_next_file):
* sed/utils.c (ck_fdopen): Remove EMX support.
These days MB_CUR_MAX is so fast on its own, and we so rarely use it,
that there’s little point to our caching it.
* sed/mbcs.c: Remove.
All uses of mb_cur_max replaced by MB_CUR_MAX.
All uses of initialize_mbcs removed.
* sed/local.mk (sed_sed_SOURCES): Remove sed/mbcs.c.
* basicdefs.h: Do not include wchar.h, wctype.h.
(ISASCII, ISBLANK, ISSPACE): Remove. All uses replaced
by c32isblank, c32isspace.
* bootstrap.conf (gnulib_modules): Add c32isblank, c32isprint,
c32isspace, c32rtomb, c32toower, c32toupper, mcel. Remove btowc,
localcharset, mbrlen, mbrtowc, mbsinit, wcrtomb, wctob, wctype-h.
* configure.ac: Do not check for headers locale.h, errno.h,
wchar.h, wctype.h, mcheck.h.
Do not check for isascii. Remove ac_cv_func_wcscoll hack.
(XFAIL_TESTS): Remove check for whether to run UTF-8 Cyrillic
tests, as I hope the other changes in this patch should fix
problems in that area.
* lib/.gitignore, m4/.gitignore: Adjust
* sed/compile.c (struct prog_info.prev): New member.
Adjust all users of this struct to update and use it when needed.
(INCHAR_EOF): New constant.
(inchar): Return a char32_t value for each wide character, a
negative byte value for encoding errors, and INCHAR_EOF for end of
input. All callers changed.
(savchar): Now takes no arguments. All callers changed.
(in_nonblank): Use char32_t tests rather than ctype.h tests.
(add_prev_to_buffer): New function.
(add_then_next, match_slash, read_text):
Use add_prev_to_buffer instead of add1_buffer when the idea is to
copy a character from input to output, as the character might be
multibyte and add1_buffer does only single-byte.
(match_slash): Support multibyte delimiters. Test for matchers
individually rather than via strchr, to make it clearer that we
are not testing for '\0'.
* sed/compile.c (add_then_next, snarf_char_class):
* sed/debug.c (debug_print_mb):
* sed/execute.c (line_init, line_reset, line_copy, line_append)
(line_exchange):
Remove now-unnecessary arg. All callers changed.
* sed/compile.c (compile_program, normalize_text):
* sed/debug.c (debug_print_regex_pattern):
* sed/execute.c (str_append_modified, do_list, translate_mb)
(debug_print_line):
Use mcel_scan/c32rtomb1 not mbrtowc1/mbrlen1/wcrtomb,
as mcel_scan should work better on MS-Windows.
* sed/debug.c (debug_print_char): Replace args c and clen
with an mcel_t arg g. All callers changed.
(debug_print_mb): Adjust to new meaning of int “characters”.
* sed/execute.c (struct line.mbstate): Remove. All uses removed.
(str_append): Omit no-longer-needed code put in just to update a
no-longer-needed mbstate_t.
(str_append_modified): Don’t worry about c32rtomb1 returning
(size_t) -1, as it now returns idx_t and anyway that cannot happen.
* sed/mbcs.c: Don’t include string.h, localcharset.h.
(is_mb_char, is_utf8): Remove. All uses removed.
* sed/sed.h: Include mcel.h.
(mbrtowc1, mbrlen1): Remove. All callers changed to use mcel_scan.
(c32rtomb1): New function.
(IS_MB_CHAR): Remove; no longer used.
* sed/utils.c (add1_buffer): Now returns void, since no caller
used the return value. 2nd arg is now char, not int.
Remove special case for EOF. All callers changed.
* testsuite/mb-bad-delim.sh, testsuite/mb-y-translate.sh:
Multibyte delimiters are no longer prohibited, so adjust test
cases accordingly. Put newlines at end of text files,
as required by POSIX.
Instead of converting pointers between char * and unsigned char *,
convert their members between char and unsigned char.
This is a little cleaner as pointer casts are too powerful in C.
* sed/compile.c (struct prog_info, compile_program):
* sed/execute.c (execute_program):
* sed/sed.h (struct sed_cmd):
Use plain char * rather than unsigned char *.
All uses changed.
Fix a longstanding bug where the 'l' command mistakenly
escapes bytes in a multibyte character.
* basicdefs.h (ISPRINT): Remove. All uses removed.
* bootstrap.conf (gnulib_modules): Add wctype-h, which is now needed
directly instead of indirectly, as we now use iswprint.
* sed/debug.c (debug_print_char):
* sed/execute.c (do_list):
Use iswprint instead of ISPRINT, so that things work correctly
with multibyte locales.
* sed/debug.c (debug_print_char):
New args P and CLEN. All uses changed.
* testsuite/cmd-l.sh: Test for the bug.
ISDIGIT is locale-dependent, and in theory could misfire if
any locale has a single-byte digit that is not '0'..'9'.
Fix this by using c_isdigit, which should be a bit faster anyway.
* basicdefs.h (ISDIGIT): Remove. All uses replaced by c_isdigit.
* bootstrap.conf (gnulib_modules): Add c-ctype.
* lib/.gitignore: Update.
* sed/compile.c: Include c-ctype.h.
This refactoring should help make future changes easier.
* bootstrap.conf (gnulib_modules): Add read-file.
* configure.ac (HAVE_FOPEN_RT): Remove; no longer used.
* lib/.gitignore: Adjust for read-file files.
* sed/compile.c: Include errno.h, read-file.h.
(struct prog_info.file): Remove. All uses removed.
(inchar, savchar, compile_program): Just read from the string.
Assume ! (NULL < NULL) && (NULL - NULL) == 0,
as per Gnulib guidelines. This should speed things up here.
(compile_file): Read the file into a string, and process the string.
When handling the y command, or the case-changing replacement,
stay a bit more in the wchar_t world, to simplify allocation
and help make future changes simpler.
* basicdefs.h (STREQ_LEN): Remove; no longer used.
* cfg.mk (sc_prohibit_strncmp): Don’t suggest STREQ_LEN.
* sed/compile.c (compile_program):
* sed/execute.c (str_append_modified, translate_mb):
Instead of preserving the bytes of a multibyte character, just
save its wchar_t counterpart, or negative of its single-byte
encoding error.
* sed/debug.c (debug_print_mb): New static function.
(debug_print_translation): Use it to adjust to new data structure.
* sed/execute.c (translate_mb): Args are now NPAIRS and PAIR, not
TRANS. Caller changed.
* sed/sed.h (struct sed_cmd.translate.mb): Now a struct with
an npairs and pair members, instead of a single char **.
All uses changed.
* bootstrap.conf (gnulib_modules): Add mbrlen.
* lib/.gitignore, m4/.gitignore: Adjust
* sed/mbcs.c (is_mb_char), sed/sed.h (mbrlen1):
Prefer mbrlen to mbrtowc with NULL, as it can be a bit faster.
* sed/compile.c (compile_program, normalize_text):
Prefer separate locals to reusing locals, to make things a bit clearer.
* sed/execute.c (str_append_modified):
Treat invalid sequences consistently with elsewhere.
* sed/sed.c (SED_INLINE): New macro.
* sed/sed.h: Use _GL_INLINE_HEADER_BEGIN, _GL_INLINE_HEADER_END.
(SED_INLINE): Define if not already defined.
(MBRLEN): Remove this macro, replacing all uses with ...
(mbrlen1): ... this new function. All callers changed.
This simplifies them, and fixes a few inconsistencies.
In addition to the usual ‘git submodule foreach git pull origin master’,
this also makes the following changes to adjust to current Gnulib:
* bootstrap: Sync from gnulib.
* lib/.gitignore, m4/.gitignore: Update to match current Gnulib.
* sed/utils.c: Include min-eloop-threshold.h, not eloop-threshold.h.
(follow_symlink): Use MIN_ELOOP_THRESHOLD, not __eloop_threshold ().
* sed/compile.c (finish_program):
* sed/execute.c (reset_addresses):
Use idx_t, not int, for vars that might exceed INT_MAX in theory.
* sed/debug.c (block_level, debug_print_command):
Use off_t, not int, likewise.
* basicdefs.h (OB_MALLOC): Omit useless cast.
* sed/execute.c (do_subst): Use compound literal instead of cast,
since the cast is sometimes useless, sometimes not.
Compound literals are safer anyway.
Problem reported by Michał Majchrowicz and Marcin Wyczechowski.
* NEWS, THANKS.in: Mention this.
* bootstrap.conf (gnulib_modules): Add quotearg.
* sed/compile.c (vbad_prog):
* sed/execute.c (open_next_file):
Quote file names in diagnostics.
* sed/utils.c: Include quotearg.h.
(quotef, quotef_n): New functions, inspired by coreutils macros
of the same name.
* sed/utils.c (ck_fopen, ck_fdopen, ck_mkstemp, ck_fwrite)
(ck_fread, ck_getdelim, ck_fflush, follow_symlink, ck_rename):
* NEWS: Mention the bug fix.
* sed/compile.c: Don't warn about special characters.
* testsuite/posix-backslash-warn.sh: Add a few test cases.
* THANKS.in: Update.
Reported by Karl Ove Hufthammer in https://bugs.gnu.org/81037
* configure.ac: Add gl_WARN_ADD([-Wno-keyword-macro]), to suppress many
new warnings like this from GCC16:
./config.h:2418:9: warning: keyword 'restrict' defined as macro\
[-Wkeyword-macro]
2418 | #define restrict __restrict__
* testsuite/mb-bad-delim.sh: Don't run the sub-test that uses an
incomplete multibyte byte as a single-byte delimiter in C locale.
* testsuite/mb-y-translate.sh: Don't run the two sub-tests that
expect a y/// length-mismatch error in C locale.
Android's bionic libc uses UTF-8 even in the C locale, so these
sub-tests would always fail.
Reported by Bruno Haible in
https://lists.gnu.org/r/sed-devel/2026-04/msg00019.html
* init.cfg (remove_selinux_warning_): New function.
* testsuite/in-place-hyphen.sh: Use it before comparing stderr.
* testsuite/in-place-suffix-backup.sh: Likewise.
* testsuite/panic-tests.sh: Likewise.
* testsuite/temp-file-cleanup.sh: Likewise.
Reported by Bruno Haible in
https://lists.gnu.org/r/sed-devel/2026-04/msg00019.html
* testsuite/read-error-stale-errno.sh: Skip when a FIFO with
O_NONBLOCK returns EOF instead of EAGAIN while a writer is still
attached. Cygwin gets this wrong.
Reported by Bruno Haible in
https://lists.gnu.org/r/sed-devel/2026-04/msg00011.html
* sed/regexp.c (match_regex): Change type of 'ret' from int
to regoff_t, to avoid truncation results larger than INT_MAX.
* testsuite/regex-max-int.sh: Update this "very expensive"
test to expect correct output (bbbbb) rather than exit-code-4 panic.
* NEWS (Bug fixes): Mention this.
The following is related but mostly incidental (--debug only).
I tested via --debug, (where it generate 20GB of output!), but couldn't
easily add a test case without adding a further option to suppress
debug printing of each escaped input line.
* sed/execute.c (do_subst): Don't cast regoff_t values to (int) for
debug printing via %d. Instead, cast to (ptrdiff_t) and print via %td.
When using -i with --follow-symlinks, sed resolved the symlink via
readlink() and then opened the original symlink path in a separate
syscall. An attacker who swapped the symlink between those two
operations could cause sed to read from an attacker-controlled file
while writing the result to the originally resolved target, enabling
arbitrary file overwrite. Fix by opening the already-resolved path
rather than re-traversing the symlink.
Reported by Michał Majchrowicz and Marcin Wyczechowski (AFINE Team).
* sed/execute.c (open_next_file): Use input->in_file_name (the
resolved path) rather than "name" (the original symlink) in the
ck_fopen call.
* NEWS: Mention this.