Commit Graph
649 Commits
Author SHA1 Message Date
Jim Meyering cb2bd20f0b maint: syntax-check: disable the indent check
* cfg.mk (local-checks-to-skip): Add sc_indent.
Otherwise, "make syntax-check" would fail for me on a
fedora 25 system.
2021-12-24 11:07:09 -08:00
Jim Meyering a465ca5f22 build: non-recursive Automake in a less hacky way
* bootstrap.conf (gnulib_modules): Remove
non-recursive-gnulib-prefix-hack.
(gnulib_tool_option_extras): Add --automake-subdir.
(bootstrap_post_import_hook): No need to massage lib/gnulib.mk.
* configure.ac [AM_INIT_AUTOMAKE]: Move subdir-objects onto the same line
as the function name, so gnulib-tool sees it.
2021-12-17 22:32:50 -08:00
Jim Meyering e84505f123 doc: fix more typos
* doc/sed.texi: Fix typos.
2021-10-02 07:58:40 -07:00
Antonio Diaz Diaz 6da11f2e5c doc: fix a typo
* doc/sed.texi (sed regular expressions):
s/considrations/considerations/
This addresses https://bugs.gnu.org/50943
2021-10-02 07:58:20 -07:00
Jim Meyering 3c1b6314a7 maint: avoid new syntax-check failure
* doc/sed.texi: Split lines longer than 80.
2021-09-05 06:17:07 -07:00
Jim Meyering fe2ecce4bf build: update gnulib to latest 2021-09-05 06:07:54 -07:00
Jim Meyering b217be9142 build: update gnulib to latest 2021-09-05 06:07:54 -07:00
Jim Meyering 4c90fc246a doc: tighten/clarify wording e.g., re "g" modifier
* doc/sed.texi: When declaring that an example replaces
all occurrences, use "g" and make the description more
precise.  Prompted by http://bugs.gnu.org/50361
Stop using "the word" when describing a match, to avoid the
implication of a connection with "word"-delimited (\b) matching.
Prefer to match "on lines ...", not "in lines".
2021-09-05 12:04:21 +02:00
Assaf Gordon 07c9c74c6b sed: allow '0rFILE' (insert FILE before the first line)
The 'r' command can be used with address zero, effectively prepending
a file to the beginning of the input file, e.g.:
    sed '0rA.TXT' B.TXT > C.TXT
is equivalent to:
    cat A.TXT B.TXT > C.TXT

With "sed -i", this allows safe in-place prepending of files.
A typical example would be adding a license header to multiple source
files:
    sed -i '0rLICENSE' *.c *.h
    find -iname '*.cpp' | xargs sed -i '0rLICENSE'

A current cumbersome alternative is:
    sed -i -e 'x;${p;x};1rA.TXT' -e '1d'  B.TXT

* NEWS: Mention new feature.
* sed/sed.h (struct readcmd): New struct. (struct sed_cmd): Use new
struct instead of a char* for the filename.
* sed/compile.c (compile_program): Expand conditional detecting invalid
usage of "0" address to allow "0r"; Adjust '0r' to '1r' with prepending
(instead of appending).
* sed/execute.c (execute_program): 'r' command: support prepending.
* sed/debug.c (debug_print_function): Use the new 'struct readcmd'.
* testsuite/cmd-0r.sh: New test.
* testsuite/local.mk (TESTS): Add new test.
* doc/sed.texi (Zero Address): New section. (Adding a header to multiple
files): New example section.
2021-08-16 12:34:20 -06:00
Assaf Gordon aa15df834a sed: extract print_file function
* sed/execute.c (dump_append_queue): Extract code to print a file to ...
(print_file): new function.
2021-08-11 12:31:15 -06:00
Renaud Pacalet 268c0083de doc: fix wording in 'line length adj.' example
Reported in https://bugs.gnu.org/49680 .

* doc/sed.texi: Fix wording, typo.
2021-08-04 15:02:56 -06:00
Jim Meyering 6568d364f4 sed: avoid potential double-fclose
Upon a failed temp file fclose, do_ck_fclose would call panic,
which would then attempt to fclose and unlink that same pointer.
Caught by gcc's new -Wanalyzer-double-fclose.
* sed/utils.c (struct open_file) [fclose_failed]: New member.
(panic): Don't double-close.
(register_open_file): Clear new member.
(mark_as_fclose_failed): Use new member to avoid double fclose.
(do_ck_fclose): Call mark_as_fclose_failed upon fclose failure.
2021-08-01 20:30:51 -07:00
Jim Meyering f3239cca6e maint: update obsolete constructs in configure.ac
* configure.ac: Avoid autoconf warnings:
Switch from obsolete AM_CONFIG_HEADER to AC_CONFIG_HEADERS, and quote.
Use AC_PROG_CC_STDC, not AC_PROG_CC.
Convert from obsolete AC_TRY_RUN to AC_RUN_IFELSE.
2021-08-01 20:30:51 -07:00
Jim Meyering 9cfcd06173 maint: avoid new warning about deprecated security_context_t
* sed/execute.c (open_next_file): Use char * in place of
deprecated security_context_t.
2021-08-01 20:30:42 -07:00
Jim Meyering 3f469c20da build: update gnulib to latest
* gnulib: update to latest
* .gitignore files: Reflect gnulib changes.
* bootstrap: update from gnulib
2021-07-10 14:46:53 -07:00
Assaf Gordon b352b74006 build: restore build rules for manual page
Reported by Kian Kasad in
https://lists.gnu.org/r/sed-devel/2020-12/msg00000.html ,
(continued in https://lists.gnu.org/r/sed-devel/2021-01/msg00000.html ).

* configure.ac: Enable buidling the manual page regardless of whether
building from git or from a tarball.
2021-01-05 17:02:41 -07:00
Assaf Gordon fb0c57d55e sed: silence -Wformat-nonoliteral warning
GCC compilation fails with:
  sed/compile.c:206:3: error: format not a string literal, argument types
                       not checked [-Werror=format-nonliteral]
    sprintf (unknown_cmd, msg, ch);
             ^~~~~~~
While there is suppression of "-Wformat-nonliteral" in configure.ac,
it later '-Wformat=2' which turns "-Wformat-nonliteral" on.

* sed/compile.c (bad_command): Add explicit "#pragma GCC ... ignore".
The format string is fixed and known ahead of time.
2021-01-05 17:02:41 -07:00
Assaf Gordon 71de42fd34 maint: update all copyright year number ranges
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* bootstrap.conf (gnulib_modules): Remove getopt module, depracated in
https://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=0abc38bd2a1398f0
* po/POTFILES.in: Remove getopt.c
* sed/utils.h (panic): Rename _GL_ATTRIBUTE_FORMAT_PRINTF to
_GL_ATTRIBUTE_FORMAT_PRINTF_STANDARD following gnulib's change:
https://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=387d654cabd7bc15
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
2021-01-05 17:02:10 -07:00
Jim Meyering 36e24f199f build: update to newer help2man
* build-aux/help2man: Update from coreutils.
This updates from 1.28 to 1.47.3
* cfg.mk (exclude_file_name_regexp--sc_long_lines):
Exempt help2man from the 80-column-max limit.
This addresses https://bugs.gnu.org/30471
2020-03-23 22:35:39 -07:00
Tobias Stoeckmann acabfdb582 sed: handle very long input lines with R (tiny change)
It is possible to trigger an out of bounds memory access when
using the sed command R with an input file containing very long
lines.

The problem is that the line length of parsed file is returned as
a size_t by ck_getdelim, but temporarily stored in an int and
then converted back into a size_t. On systems like amd64, on which
this problem can be triggered, size_t and int have different sizes.

If the input line is longer than 2 GB (which is parseable on amd64
or other 64 bit systems), this means that the temporarily stored
int turns negative. Converting the negative int back into a size_t
will lead to an excessively large size_t, as the conversion leads to
a lot of leading 1 bits.

Eventually ck_fwrite is called with this huge size_t which in turn
will lead to an out of bounds access on amd64 systems -- after all
the parsed text was just a bit above 2 GB, not near SIZE_MAX.

You can trigger this issue with GNU sed on OpenBSD like this:

$ dd if=/dev/zero bs=1M count=2049 | tr '\0' e > long.txt
$ sed Rlong.txt /etc/fstab
Segmentation fault (core dumped)

I was unable to trigger the bug on a Linux system with glibc due to
a bug in glibc's fwrite implementation -- it leads to a short write
and sed treats that correctly as an error.

* sed/execute.c (execute_program) [case 'R']: Declare result
to be of type size_t, not int.
* NEWS (Bug fixes): Mention it.
This addresses https://bugs.gnu.org/39166
2020-01-18 08:25:10 -08:00
Tobias Stoeckmann 61b5e58f18 sed: handle very long execution lines (tiny change)
If sed is called with an excessively long execution line, then it is
prone to an out of bounds memory access.

The problem is that the length of the execution line, which is a
size_t, is temporarily stored in an int. This means that on systems
which have a 64 bit size_t and a 32 bit int (e.g. linux amd64) an
execution line which exceeds 2 GB will overflow int. If it is just
slightly larger than 2 GB, the negative int value is used as an
array index to finish the execution line string with '\0' which
therefore triggers the out of bounds access.

This problem is probably never triggered in reality, but can be
provoked like this (given that 'e' support is compiled in):

$ dd if=/dev/zero bs=1M count=2049 | tr '\0' e > e-command.txt
$ sed -f e-command.txt /etc/fstab
Segmentation fault (core dumped)

Also adjust another int/size_t conversion, even though it is a
purely cosmetic change, because it can never be larger than 4096.

* sed/execute.c (execute_program) [case 'e']: Declare cmd_length
to be of type size_t, not int.  Likewise for "n" just below.
* NEWS (Bug fixes): Mention it.
This addresses https://bugs.gnu.org/39165
2020-01-18 08:25:10 -08:00
Jim Meyering 8827762c4a maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2020-01-14 20:10:42 -08:00
Jim Meyering 6205998563 version 4.8
* NEWS: Record release date.
v4.8
2020-01-14 20:08:55 -08:00
Jim Meyering ca9be2751e doc: mention changes inherited through gnulib
* NEWS (Improvements): Mention gnulib.
2020-01-14 20:08:45 -08:00
Jim Meyering 5d1590baaf build: update gnulib to latest (for updated perl.m4) 2020-01-11 21:09:23 -08:00
Jim Meyering 8f2f1b7182 tests: remove valgrind-added diagnostics from stderr
* testsuite/bug32271-2.sh: Filter out lines starting with "==".
This avoids unwarranted failures on e.g., windows systems that
would emit this to stderr:
+==16281== error calling PR_SET_PTRACER, vgdb might block
* testsuite/newline-dfa-bug.sh: Likewise.
Reported by Vagelis Prokopiou on Windows subsystem for Linux
(Debian 10) in https://lists.gnu.org/r/sed-devel/2020-01/msg00018.html
and by Bruno Haible for mingw in
https://lists.gnu.org/r/sed-devel/2020-01/msg00005.html
2020-01-11 20:49:51 -08:00
Jim Meyering def38eb00e sed: avoid MSVC build failure
* sed/sed.c (usage): Remove unnecessary forward declaration.
Reported by Bruno Haible in
https://lists.gnu.org/r/sed-devel/2020-01/msg00007.html
2020-01-11 19:24:08 -08:00
Jim Meyering 5766a98647 tests: avoid hpux11 test failure
* testsuite/title-case.sh: Correct printf usage not to trigger
unspecified behavior for '\(\)'. Instead, just use sed with its -r
option so those parentheses don't need any backslash at all.
Reported by Bruno Haible in
https://lists.gnu.org/r/sed-devel/2020-01/msg00008.html
2020-01-11 16:18:50 -08:00
Jim Meyering 598c3a0dd9 tests: handle missing perl
Before this change, the two perl-requiring tests would fail
on systems without perl. Now, they are skipped, as intended.
* bootstrap.conf (gnulib_modules): Add perl.
* testsuite/no-perl: New file. From coreutils.
* testsuite/local.mk (EXTRA_DIST): Add it.
(TESTSUITE_PERL): Include testsuite/ prefix.
2020-01-09 22:56:39 -08:00
Jim Meyering 6c175df230 maint: once again distribute .tar.gz files
* configure.ac: Reenable distribution of gzip-compressed
tarballs, on request from Jan Nieuwenhuizen in
https://lists.gnu.org/r/sed-devel/2020-01/msg00013.html
* NEWS (Release): Mention this.
2020-01-06 16:55:02 -08:00
Jim Meyering 3c1e83363b maint: space-before-open-paren: avoid syntax-check error
* sed/sed.c (selinux_support): Insert space before open-paren.
2020-01-05 13:47:31 -08:00
Jim Meyering 174718e3c1 build: update gnulib to latest (for thread-related test fixes)
* .gitignore: Add /lib/inttypes.h.
2020-01-05 13:44:17 -08:00
Jim Meyering 623f5cd2f9 maint: update all copyright year number ranges
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
2020-01-01 11:38:46 -08:00
Jim Meyering f8ab99468d maint: remove explicit mention of gnulib's threadlib module
* bootstrap.conf (gnulib_modules): Remove explicit mention of threadlib.
It was added back in 2012 to avoid a bootstrap failure; that explicit
mention has long been unnecessary.
2020-01-01 11:37:23 -08:00
Assaf Gordon f69b085d3e sed: set correct umask on temporary files
"sed -i" now creates temporary files with correct umask (limited to u=rwx).
Previously sed would incorrectly set umask, and combined with mkostemp
creating file with mode 0600, the result would be a file with
permission mode 0.

Reported by Dr N.W. Filardo <nwf20@cam.ac.uk>:
https://lists.gnu.org/r/sed-devel/2019-08/msg00000.html
  "The net effect is that this patch does not do what it says on the tin:
   it does not improve the security story at all.  Things continue to
   function because the subsequent operations are via f*() APIs, which
   take the open file handle, and in particular fchmod() will put the
   bits back to something sensible.

   However, when running atop, for example, fuse-style filesystems which do
   not keep open descriptors to underlying files, this is catastrophic:
   the underlying file will have I_SRWXU of zero, and so the filesystem
   server will be unable to open the file for the fchmod() and that's
   the end of that."

"fuse-overlayfs" is an example of a filesystem with such issues.
This change was made in commit 5156c19b23
and was released in sed 4.2.1.

* NEWS: Mention change.
* sed/utils.c (ck_mkstemp): Set correct umask.
2019-09-05 13:10:29 -06:00
Assaf Gordon a9cb52bcf3 tests: inplace-selinux.sh: skip if SELinux is not supported
Avoid false-positive failure on system with SELinux enabled when SED
is built without SELinux suppport.

Reported by Jeffrey Walton <noloader@gmail.com> in
https://bugs.gnu.org/35997 .

* testsuite/inplace-selinux.sh: Call 'sed --version' to check if sed
was built with SELinux support before continuing with the test.
2019-06-02 01:14:00 -06:00
Assaf Gordon 07ef8a3138 sed: report SELinux status in --version output
Report whether sed was built with SELinux support (determined at compile
time), and whether it is enabled on this system (determined at runtime).

Relates to https://bugs.gnu.org/35997 .

* sed/sed.c (selinux_support): New function, print SELinux status to
stdout. (main): Call selinux_support on --version.
2019-06-02 01:11:19 -06:00
Jim Meyering 2a7125a8b3 build: ensure no VLA is used
Cause developer builds to fail for any use of a VLA.
VLAs (variable length arrays) limit portability.
* configure.ac (nw): Remove -Wvla from the list of disabled warnings,
thus enabling the warning when configured with --enable-gcc-warnings.
(GNULIB_NO_VLA) Define, disabling use of VLAs in gnulib.  This commit
is functionally equivalent to coreutils' v8.30-44-gd26dece5d.
2019-01-20 22:22:45 -08:00
Jim Meyering c728a3fcb1 build: update gnulib to latest 2019-01-20 22:22:45 -08:00
Jim Meyering f850369d8c build: update gnulib to latest 2019-01-01 19:09:57 -08:00
Assaf Gordon 07df2c2e61 maint: update copyright dates for 2019
* all files: Run "make update-copyright".
2019-01-01 14:46:27 -07:00
Jim Meyering 8c60cebe0b maint: use https: in gnu mirror URL prefix, not http
This appears in the generated release announcement message.
* cfg.mk (url_dir_list): Use https: prefix, not http:.
2018-12-20 22:21:40 -08:00
Jim Meyering c0a4f0643f maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2018-12-20 22:04:03 -08:00
Jim Meyering 249cbda41c version 4.7
* NEWS: Record release date.
v4.7
2018-12-20 22:02:51 -08:00
Jim Meyering 799ce52011 sed: fix \b DFA-bug in C locale
Under some conditions, \b would mistakenly fail to match. E.g.,
this would mistakenly print "123-x" instead of "123":
  echo 123-x|LC_ALL=C sed 's/.\bx//'
* NEWS (Bug fixes): Mention it
* gnulib: Update to latest, for DFA regression fix.
* testsuite/word-delim.sh: New file, to test for the dfa.c regression.
* testsuite/local.mk (T): Add it.
Reported by Jan Palus in
https://lists.gnu.org/r/sed-devel/2018-12/msg00022.html
2018-12-20 21:23:11 -08:00
Jim Meyering 04eef69746 maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2018-12-19 23:25:32 -08:00
Jim Meyering 4383aeba93 version 4.6
* NEWS: Record release date.
v4.6
2018-12-19 23:23:46 -08:00
Jim Meyering 99f0df6c3d build: update gnulib to latest for linux/sparc c-stack fix 2018-12-17 06:57:28 -08:00
Jim Meyering 75b06ca114 build: update gnulib to latest for config.sub fix, etc.
* gnulib: Update for build- and test-related fixes,
including the config.sub fix for Solaris 10:
https://lists.gnu.org/r/grep-devel/2018-12/msg00013.html
* bootstrap: Update to latest.
2018-12-16 11:31:41 -08:00
Assaf Gordon 5155e56775 tests: fix obinary test error due to 'wc' spaces
On BSD systems "wc -c" prefixes the count with spaces,
leading to test error.
Discussed in https://lists.gnu.org/r/sed-devel/2018-12/msg00014.html .

* testsuite/obinary.sh: Remove leading spaces from 'wc -c' output.
2018-12-13 15:59:44 -07:00