sed: fix extraneous NUL in s///n command
Under certain conditions sed would add an extraneous NUL:
$ echo 0 | sed -e 's/$/a/2' | od -tx1 -An
30 00 0a
This would happen when the regex is an empty (zero-length) match at the
end of the line (e.g. '$' and 'a*$') and the substitute number flag
('n' in s///n) is higher than the number of actual matches (multiple
EOL matches are possible with multiline match, e.g. 's/$/a/3m').
Details:
The comment at the top of 'execute.c:do_subst()' says:
/* The first part of the loop optimizes s/xxx// when xxx is at the
start, and s/xxx$// */
Which refers to lines 1051-3:
1051 /* Copy stuff to the left of this match into the output string. */
1052 if (start < offset)
1053 str_append(&s_accum, line.active + start, offset - start);
The above code appends text to 's_accum' but does *not* update 'start'.
Later on, if the s/// command includes 'n' flag, and if 'matched == 0'
(an empty match), this comparison will be incorrect:
1081 if (start < line.length)
1082 matched = 1;
Will in turn will set 'matched' to 1, and the 'str_append' call that
follows (line 1087) will append an additional character.
Because the empty match is EOL, the appended character is NUL.
More examples that trigger the bug:
echo 0 | sed -e 's/a*$/X/3'
printf "%s\n" 0 0 0 | sed -e 'N;N;s/a*$/X/4m'
Examples that do not trigger the bug:
# The 'a*' empty regex matches at the beginning of the line (in
# addition to the end of the line), and the optimization in line
# 1052 is skipped.
echo 0 | sed -e 's/a*/X/3'
# There are 3 EOLs in the pattern space, s///3 is not too large.
printf "%s\n" 0 0 0 | sed -e 'N;N;s/a*$/X/3m'
This was discovered while investigating bug#32271 reported by bugs@feusi.co
in https://lists.gnu.org/r/bug-sed/2018-07/msg00018.html .
* NEWS: Mention the fix.
* sed/execute.c (do_subst): Update 'start' as needed.
* testsuite/bug-32271-1.sh: New test.
* testsuite/local.mk (T): Add test.
This commit is contained in:
1 parent
0144eeeb3d
commit
2cb09e1463
4 files changed
+53
-1
No files matched your search
@@ -14,6 +14,9 @@ GNU sed NEWS -*- outline -*-
|
||||
sed no longer accesses invalid memory (heap overflow) when given invalid
|
||||
backreferences in 's' command [bug#32082, present at least since sed-4.0.6].
|
||||
|
||||
sed no longer adds extraneous NUL when given s/$//n command.
|
||||
[related to bug#32271, present since sed-4.0.7]
|
||||
|
||||
|
||||
* Noteworthy changes in release 4.5 (2018-03-31) [stable]
|
||||
|
||||
|
||||
+4
-1
@@ -1050,7 +1050,10 @@ do_subst(struct subst *sub)
|
||||
|
||||
/* Copy stuff to the left of this match into the output string. */
|
||||
if (start < offset)
|
||||
str_append(&s_accum, line.active + start, offset - start);
|
||||
{
|
||||
str_append(&s_accum, line.active + start, offset - start);
|
||||
start = offset;
|
||||
}
|
||||
|
||||
/* If we're counting up to the Nth match, are we there yet?
|
||||
And even if we are there, there is another case we have to
|
||||
|
||||
Executable
+45
@@ -0,0 +1,45 @@
|
||||
#!/bin/sh
|
||||
# sed would incorrectly copy internal buffers under certain s/// uses.
|
||||
# Before sed 4.6 these would result in an extraneous NUL at end of lines.
|
||||
#
|
||||
|
||||
# Copyright (C) 2018 Free Software Foundation, Inc.
|
||||
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License as published by
|
||||
# the Free Software Foundation, either version 3 of the License, or
|
||||
# (at your option) any later version.
|
||||
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
. "${srcdir=.}/testsuite/init.sh"; path_prepend_ ./sed
|
||||
print_ver_ sed
|
||||
|
||||
printf '0\n' > in || framework_failure_
|
||||
printf '0\n' > exp || framework_failure_
|
||||
|
||||
# Before sed 4.6, this would result in: 0x30 0x00 0x0a.
|
||||
sed -e 's/$/a/2' in > out 2> err || fail=1
|
||||
|
||||
compare exp out || fail=1
|
||||
compare /dev/null err || fail=1
|
||||
|
||||
# To ease debugging / error reporting (the above 'compare'
|
||||
# will report "binary files differ" - not very helpful here)
|
||||
if test -n "$fail" ; then
|
||||
echo "---- TEST FAILED"
|
||||
echo "out:"
|
||||
od -tx1 out
|
||||
echo "exp:"
|
||||
od -tx1 exp
|
||||
echo "err:"
|
||||
od -tx1 err
|
||||
fi
|
||||
|
||||
|
||||
Exit $fail
|
||||
@@ -44,6 +44,7 @@ LOG_COMPILER = false
|
||||
T = \
|
||||
testsuite/misc.pl \
|
||||
testsuite/bug32082.sh \
|
||||
testsuite/bug32271-1.sh \
|
||||
testsuite/cmd-l.sh \
|
||||
testsuite/cmd-R.sh \
|
||||
testsuite/colon-with-no-label.sh \
|
||||
|
||||
Reference in new issue
Block a user