Commit Graph
618 Commits
Author SHA1 Message Date
Jim Meyering 174718e3c1 build: update gnulib to latest (for thread-related test fixes)
* .gitignore: Add /lib/inttypes.h.
2020-01-05 13:44:17 -08:00
Jim Meyering 623f5cd2f9 maint: update all copyright year number ranges
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
2020-01-01 11:38:46 -08:00
Jim Meyering f8ab99468d maint: remove explicit mention of gnulib's threadlib module
* bootstrap.conf (gnulib_modules): Remove explicit mention of threadlib.
It was added back in 2012 to avoid a bootstrap failure; that explicit
mention has long been unnecessary.
2020-01-01 11:37:23 -08:00
Assaf Gordon f69b085d3e sed: set correct umask on temporary files
"sed -i" now creates temporary files with correct umask (limited to u=rwx).
Previously sed would incorrectly set umask, and combined with mkostemp
creating file with mode 0600, the result would be a file with
permission mode 0.

Reported by Dr N.W. Filardo <nwf20@cam.ac.uk>:
https://lists.gnu.org/r/sed-devel/2019-08/msg00000.html
  "The net effect is that this patch does not do what it says on the tin:
   it does not improve the security story at all.  Things continue to
   function because the subsequent operations are via f*() APIs, which
   take the open file handle, and in particular fchmod() will put the
   bits back to something sensible.

   However, when running atop, for example, fuse-style filesystems which do
   not keep open descriptors to underlying files, this is catastrophic:
   the underlying file will have I_SRWXU of zero, and so the filesystem
   server will be unable to open the file for the fchmod() and that's
   the end of that."

"fuse-overlayfs" is an example of a filesystem with such issues.
This change was made in commit 5156c19b23
and was released in sed 4.2.1.

* NEWS: Mention change.
* sed/utils.c (ck_mkstemp): Set correct umask.
2019-09-05 13:10:29 -06:00
Assaf Gordon a9cb52bcf3 tests: inplace-selinux.sh: skip if SELinux is not supported
Avoid false-positive failure on system with SELinux enabled when SED
is built without SELinux suppport.

Reported by Jeffrey Walton <noloader@gmail.com> in
https://bugs.gnu.org/35997 .

* testsuite/inplace-selinux.sh: Call 'sed --version' to check if sed
was built with SELinux support before continuing with the test.
2019-06-02 01:14:00 -06:00
Assaf Gordon 07ef8a3138 sed: report SELinux status in --version output
Report whether sed was built with SELinux support (determined at compile
time), and whether it is enabled on this system (determined at runtime).

Relates to https://bugs.gnu.org/35997 .

* sed/sed.c (selinux_support): New function, print SELinux status to
stdout. (main): Call selinux_support on --version.
2019-06-02 01:11:19 -06:00
Jim Meyering 2a7125a8b3 build: ensure no VLA is used
Cause developer builds to fail for any use of a VLA.
VLAs (variable length arrays) limit portability.
* configure.ac (nw): Remove -Wvla from the list of disabled warnings,
thus enabling the warning when configured with --enable-gcc-warnings.
(GNULIB_NO_VLA) Define, disabling use of VLAs in gnulib.  This commit
is functionally equivalent to coreutils' v8.30-44-gd26dece5d.
2019-01-20 22:22:45 -08:00
Jim Meyering c728a3fcb1 build: update gnulib to latest 2019-01-20 22:22:45 -08:00
Jim Meyering f850369d8c build: update gnulib to latest 2019-01-01 19:09:57 -08:00
Assaf Gordon 07df2c2e61 maint: update copyright dates for 2019
* all files: Run "make update-copyright".
2019-01-01 14:46:27 -07:00
Jim Meyering 8c60cebe0b maint: use https: in gnu mirror URL prefix, not http
This appears in the generated release announcement message.
* cfg.mk (url_dir_list): Use https: prefix, not http:.
2018-12-20 22:21:40 -08:00
Jim Meyering c0a4f0643f maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2018-12-20 22:04:03 -08:00
Jim Meyering 249cbda41c version 4.7
* NEWS: Record release date.
v4.7
2018-12-20 22:02:51 -08:00
Jim Meyering 799ce52011 sed: fix \b DFA-bug in C locale
Under some conditions, \b would mistakenly fail to match. E.g.,
this would mistakenly print "123-x" instead of "123":
  echo 123-x|LC_ALL=C sed 's/.\bx//'
* NEWS (Bug fixes): Mention it
* gnulib: Update to latest, for DFA regression fix.
* testsuite/word-delim.sh: New file, to test for the dfa.c regression.
* testsuite/local.mk (T): Add it.
Reported by Jan Palus in
https://lists.gnu.org/r/sed-devel/2018-12/msg00022.html
2018-12-20 21:23:11 -08:00
Jim Meyering 04eef69746 maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2018-12-19 23:25:32 -08:00
Jim Meyering 4383aeba93 version 4.6
* NEWS: Record release date.
v4.6
2018-12-19 23:23:46 -08:00
Jim Meyering 99f0df6c3d build: update gnulib to latest for linux/sparc c-stack fix 2018-12-17 06:57:28 -08:00
Jim Meyering 75b06ca114 build: update gnulib to latest for config.sub fix, etc.
* gnulib: Update for build- and test-related fixes,
including the config.sub fix for Solaris 10:
https://lists.gnu.org/r/grep-devel/2018-12/msg00013.html
* bootstrap: Update to latest.
2018-12-16 11:31:41 -08:00
Assaf Gordon 5155e56775 tests: fix obinary test error due to 'wc' spaces
On BSD systems "wc -c" prefixes the count with spaces,
leading to test error.
Discussed in https://lists.gnu.org/r/sed-devel/2018-12/msg00014.html .

* testsuite/obinary.sh: Remove leading spaces from 'wc -c' output.
2018-12-13 15:59:44 -07:00
Jim Meyering 58eb65b263 build: update gnulib to latest 2018-12-12 22:11:29 -08:00
Assaf Gordon 6187fc0e3a tests: add text/binary mode tests
This test is skipped on most platforms (which do not support O_BINARY
and do not differentiate between TEXT and BINARY files).

On cygwin, the test is skipped because default cygwin installations
do not use O_TEXT/O_BINARY modes (unless file systems are mounted with
special options).

In practice this test will only run on native Windows (e.g. when compiled
with mingw or msvc). Note that almost every other test will fail with
such a binary (due to excesssive "\r" in the output). Thus it is
recommended to explictly run just this test alone:

   make check SUBDIRS=. TESTS=testsuite/obinary.sh

The script tries to cover all cases mentioned in https://bugs.gnu.org/25459 .
See also https://lists.gnu.org/r/sed-devel/2018-10/msg00001.html .

* testsuite/obinary.sh: New test.
* testsuite/local.mk (TESTS): Add new test.
2018-12-10 00:15:34 -07:00
Assaf Gordon 46b9427132 sed: fix -b/--binary mode under windows/mingw
Discussed in https://lists.gnu.org/r/sed-devel/2018-10/msg00001.html .

* NEWS: Mention change.
* bootstrap.conf: Add gnulib's binary-io module.
* sed/sed.c (main): Set stdin/stdout to binary mode if needed.
* sed/utils.c (ck_mkstemp): Explicitly set binary mode on file
descriptor. It seems that on (non-cygwin) Windows the fdopen(3) call
ignores the 'b' in the 'mode' argument - and the file was always opened
in O_TEXT mode. Thus "--binary" was not working with "--inplace".
2018-12-05 21:07:57 -07:00
Assaf Gordon 159675c030 maint: update .gitignore
Following gnulib recent update.
* lib/.gitignore, m4/.gitignore: Update file list.
2018-12-05 21:07:57 -07:00
Assaf Gordon 51eb713854 build: update autoconf version requirements
Require version 2.64 to bootstrap. This is already required in practice
by bootstrap.conf, but was not updated in configure.ac.
After recent gnulib update, gnulib-tool would complain:

  $ ./gnulib/gnulib-tool
  ./gnulib/gnulib-tool: *** minimum supported autoconf version is 2.63.\
      Try adding AC_PREREQ([2.63]) to your configure.ac.
  ./gnulib/gnulib-tool: *** Stop.

Hence this update.

* configure.ac: Require autoconf version 2.64 or later.
2018-12-05 21:07:50 -07:00
Assaf Gordon b307652592 gnulib: update to latest (for dfa improvements) 2018-12-05 14:00:50 -07:00
Jannick dae3abf2af sed: fix memory leakage under lint
The NULL-initialized char string in_place_extension is free'ed everytime
it is redefined (using xstrdup) and at program exit with any return code.

See: https://lists.gnu.org/r/sed-devel/2018-11/msg00005.html

* sed/sed.c (main, cleanup): Free 'in_place_extension' if running with
lint.
2018-11-22 13:21:38 -07:00
Assaf Gordon 76ca1d2c53 build: rename stdin local variable
Under windows/msys2, defining 'bool stdin' variable results in
compilation errors:

  ../sed/execute.c:1252:8: error: expected declaration specifiers or '...'
  before numeric constant
   bool stdin = (input->fp && fileno (input->fp) == 0);
        ^~~~~

Reported by Jannick <thirdedition@gmx.net> in
https://lists.gnu.org/archive/html/sed-devel/2018-10/msg00019.html .

* sed/execute.c (debug_print_input): Rename 'stdin' to 'is_stdin'.
2018-11-01 10:06:47 -06:00
Jim Meyering 4708cf368a maint: fix a syntax-check nit
* sed/compile.c (MISSING_FILENAME): Insert a space before "(", and
in a following comment.
2018-10-28 04:21:20 -07:00
Jim Meyering 1c0866b1d3 tests: avoid unnecessary dependency on Data::Dump
* testsuite/debug.pl: Don't use Data::Dump.  Unnecessary.
That package, perl-Data-Dump, appear not to be default-installed
on at least a Fedora 29 beta system.
2018-10-27 15:53:03 -07:00
Jim Meyering 2bfa7984c8 sed: avoid UMR in --debug code path
* sed/debug.c (debug_print_function) [b, t, T]: For a b, t or T
command with no LABEL, do not access uninitialized memory.
I.e., print the label name only when there is one.
2018-10-27 15:47:41 -07:00
Assaf Gordon 3ac640ce3b sed: add --debug feature
$ seq 3 | sed --debug -e 's/./--&--/ ; 2d'
    SED PROGRAM:
      s/./--&--/
      2 d
    INPUT:   'STDIN' line 1
    PATTERN: 1
    COMMAND: s/./--&--/
    MATCHED REGEX REGISTERS
      regex[0] = 0-1 '1'
    PATTERN: --1--
    COMMAND: 2 d
    END-OF-CYCLE:
    --1--
    INPUT:   'STDIN' line 2
    PATTERN: 2
    COMMAND: s/./--&--/
    MATCHED REGEX REGISTERS
      regex[0] = 0-1 '2'
    PATTERN: --2--
    COMMAND: 2 d
    END-OF-CYCLE:
    INPUT:   'STDIN' line 3
    PATTERN: 3
    COMMAND: s/./--&--/
    MATCHED REGEX REGISTERS
      regex[0] = 0-1 '3'
    PATTERN: --3--
    COMMAND: 2 d
    END-OF-CYCLE:
    --3--

Discussed in https://lists.gnu.org/r/sed-devel/2018-07/msg00006.html
and https://lists.gnu.org/r/sed-devel/2018-10/msg00007.html .

* NEWS: Mention new option.
* doc/sed.texi (Program options): Mention new option.
* sed/debug.c: New unit with debug printing functions.
* sed/sed.h (debug_print_command, debug_print_char, debug_print_program,
debug): Declare functions and global variable.
(struct sed_cmd): Add label_name member variable.
* sed/compile.c (compile_program): Save the label's name.
(cleanup_program_filenames): extracted function to free filenames.
(check_final_program) Don't delete the filenames, instead move it to ...
(finish_program) ... here.
* sed/execute.c (debug_print_end_of_cycle, debug_print_input,
debug_print_line): New debug functions (cannot be defined in debug.c as
execute's structures are private).
(execute_program, process_files): Call debug functions.
* sed/sed.c: (DEBUG_OPTION): New option for getoptlong.
(debug): New global variable.
(usage): Mention new option.
(main): Process new option and call debug functions if needed.
* testsuite/debug.pl: New tests.
* testsuite/local.mk (T): Add new tests.
2018-10-24 14:00:23 -06:00
Assaf Gordon f649e21458 sed: change internal storage for 'R' command
For the 'R' command, keep a pointer to 'struct output' instead of just
to FILE*. No change in functionality. This will help future debug code
to know the filename of the associated file.

As a side effect, fix invalid code in W/w execution:
The previous code checked for 'cur_cmd->x.fp' (which was related to R
command, not W/w). Since 'x' is a union, 'x.fp' was not relevant to
the 'x.outf' (which is the struct associted with W/w commands).

* sed/sed.h (struct sed_cmd): Replace 'FILE*' with 'struct *output' for
R command.
* sed/compile.c (compile_program): Adjust as needed.
* sed/execute.c (execute_program): Adjust as needed.
2018-10-19 10:37:32 -06:00
Assaf Gordon 38e9a954ff sed: reject r/R/w/W commands without a filename
w/W (and s///w) commands Without a filename would print a confusing error
message:
  $ sed w
  sed: couldn't open file : No such file or directory
While r/R commands with empty file name were a silent no-op.

With this change, sed programs with empty filename are rejected with a
clear error:

  $ sed 's/1/2/w'
  sed: -e expression #1, char 7: missing filename in r/R/w/W commands
  $ sed r
  sed: -e expression #1, char 1: missing filename in r/R/w/W commands

* NEWS: Mention change.
* sed/compile.c (get_openfile): Exit with an error message if filename
is missing. (compile_program): Same for 'r' command code.
* testsuite/missing-filename.sh: New test.
* testsuite/local.mk (T): Add new test.
2018-10-18 04:44:19 -06:00
Clint Adams 8e52c0aff0 doc: use @key{TAB} in texinfo
Discussed in https://bugs.gnu.org/22636 .

* doc/sed.texi: Use '@kbd{@key{TAB}}' instead of '@kbd{tab}' to improve
rendering in HTML and info formats.
Copyright-paperwork-exempt: Yes
2018-10-12 11:45:56 -06:00
Bjarni Ingi Gislason 19089418c6 doc: sed.x: some formatting corrections
Reported in https://bugs.gnu.org/30479 .

doc/sed.x: Change a HYPHEN-MINUS (code 0x55, 2D) to a dash (\-, minus)
if it matches " -[:alpha:]" or \(aq-[:alpha:] (for options); Use a pronoun
instead of a repeated noun.
Copyright-paperwork-exempt: Yes
2018-10-12 11:45:56 -06:00
Assaf Gordon 50f598b6a4 sed: update authors
* sed/sed.c (AUTHORS): Update.
* doc/sed.texi (@author): Update.
2018-10-12 11:35:07 -06:00
Assaf Gordon 261c7f145d maint: remove REG_PERL code
Perl-regexp syntax (PCRE) in GNU Sed is shelved indefinitely.
See https://bugs.gnu.org/22801 , https://bugs.gnu.org/22647 .
Remove all (unused) REG_PERL related code.

* sed/sed.c, sed/sed.h, sed/regexp.c, sed/compile.c: Remove REG_PERL code.
2018-08-23 14:34:18 -06:00
Assaf Gordon 41a169a9a1 maint: remove EXPERIMENTAL_DASH_N_OPTIMIZATION code
The optimization was buggy and was never enabled in a released version.

* sed/execute.c: Remove EXPERIMENTAL_DASH_N_OPTIMIZATION code.
2018-08-23 14:34:17 -06:00
Assaf Gordon 30192e7587 maint: replace street address with URL in license text
* basicdefs.h, sed/compile.c, sed/execute.c, sed/mbcs.c, sed/regexp.c,
sed/sed.c, sed/sed.h, sed/utils.c, sed/utils.h, testsuite/get-mb-cur-max.c,
testsuite/test-mbrtowc.c (GPL License notice): Replace FSF's office
address with 'https://www.gnu.org/licenses/'.
2018-08-23 14:34:17 -06:00
Assaf Gordon ac159313c7 build: use system's native sed during build
Use the locally built sed binary only during 'check*' targets,
but not during build/installation targets - it might be buggy, or contain
temporary debugging/ASAN features that can break the build.
See: https://lists.gnu.org/r/sed-devel/2018-08/msg00013.html .

* cfg.mk (PATH): Add the 'sed' directory to the PATH only if this is a
'check' target.
2018-08-23 14:34:09 -06:00
Assaf Gordon 36e3485e7f sed: mark function as attribute(malloc)
Suggested by gcc-8.2.0.

* sed/compile.c (read_label): Mark as _GL_ATTRIBUTE_MALLOC.
2018-08-08 14:14:51 -06:00
Assaf Gordon 665f9a9dc1 maint: add undefined-behavior build target
Using gcc-specific options, a recent gcc is required.
build with:
   make build-ubsan CC=gcc-8.2

* cfg.mk (build-ubsan): New target.
2018-08-08 14:14:51 -06:00
Assaf Gordon 0cc0ade423 maint: add address-sanitizer build target
use 'make build-asan' to rebuild sed with gcc's address sanitizer.

* cfg.mk (build-asan): New target.
2018-08-08 14:14:51 -06:00
Assaf Gordon 9a9f9e34f3 gnulib: update to latest (for regex memory leaks)
Specifically for gnulib commits 66b99e5259,c5e76a9560.
see https://lists.gnu.org/r/bug-gnulib/2018-07/msg00127.html
2018-08-08 14:14:51 -06:00
Assaf Gordon ba8e66de6d sed: fix memory leak
* sed/regexp.c (match_regex): Free the previously allocated regex struct
before re-building the regex during program execution.
2018-08-08 14:14:51 -06:00
Assaf Gordon 35b30b124c sed: free allocated memory under lint, remove DEBUG_LEAKS
Under normal operation there is no need for explicit freeing,
as all memory will be released when sed terminates.
During development (and specifically, valgrind and address-sanitizing)
enabling lint prevents false-positive warnings about memory
leaks.
Lint mode can be enabled with CFLAGS="-Dlint". It is also automatically
enabled by default when building from git (as opposed to tarball).

For consistency all instances of "#ifdef DEBUG_LEAKS" are replaced
with "#ifdef lint".

* sed/sed.h (struct subst): Add member variable to keep the address of
allocated buffer in compile.c:setup_replacement().
(release_regex): Add declaration.
(finish_program): Function now takes an argument: the sed program
vector.
* sed/sed.c (main): Adjust call to finish_program.
* sed/compile.c (finish_program): Release program allocations.
(setup_replacement): Remember the allocated buffer address.
(compile_program): Free temporary array in 'y' command.
* sed/execute.c (execute_program): Free allocated but unused buffer in
'R' command when reaching EOF.
* sed/regexp.c (release_regex): Remove 'static', free the allocated
dfa struct.
* sed/utils.c (panic): Free open files linked-list elements.
2018-08-08 14:14:39 -06:00
Assaf Gordon 08675afe9a sed: do not close stderr on exit
Not needed, and prevents leak-sanitizing from working.

* sed/utils.c (ck_fclose): Do not close stderr.
2018-08-08 14:14:20 -06:00
Jim Meyering 7c6a618904 doc: insert two missing words
doc/sed.texi (BRE vs ERE): Insert "with a".
2018-08-07 03:32:28 -07:00
Assaf Gordon a2cda80ecd maint: syntax-check: prohibit-operator-at-eol
* cfg.mk (sc_prohibit_operator_at_end_of_line): Copied from coreutils'
cfg.mk.
* sed/execute.c (do_subst,execute_program): Adjust as needed.
2018-08-04 22:09:58 -06:00
Assaf Gordon 4766cf6717 maint: syntax-check: fail-0 and fail-1 in shell script tests
* cfg.mk (prohibit_fail_0,sc_prohibit_and_fail_1): Copied from
coreutils' cfg.mk
* testsuite/*.sh: Remove 'fail=0'.
2018-08-04 22:09:58 -06:00