This commit implements robust detection of entity amplification attacks,
better known as the "billion laughs" attack.
We now limit the size of the document after substitution of entities to
10 times the size before expansion. This guarantees linear behavior by
definition. There already was a similar check before, but the accounting
of "sizeentities" (size of external entities) and "sizeentcopy" (size of
all copies created by entity references) wasn't accurate.
We also need saturation arithmetic since we're historically limited to
"unsigned long" which is 32-bit on many platforms.
A maximum of 10 MB of substitutions is always allowed. This should make
use cases like DITA work which have caused problems in the past.
The old checks based on the number of entities were removed. This is
accounted for by adding a fixed cost to each entity reference.
Entity amplification checks are now enabled even if XML_PARSE_HUGE is
set. This option is mainly used to allow larger text nodes. Most users
were unaware that it also disabled entity expansion checks.
Some of the limits might be adjusted later. If this change turns out to
affect legitimate use cases, we can add a separate parser option to
disable the checks.
Fixes#294.
Fixes#345.
Instead of abusing the LSB of the "checked" member, store the result
of testing for occurrence of '<' character in "flags".
Also use the flags in xmlParseStringEntityRef instead of rescanning
every time.
To check whether an entity was already parsed, the code previously
tested whether "checked" was non-zero or "children" was non-null. The
"children" check could be unreliable because an empty entity also
results in an empty (NULL) node list. Use a separate flag to make this
check more reliable.
This was an experimental and undocumented micro-optimization for
Windows which apparently required different calling conventions for
variable-argument functions, making it impossible to maintain without
domain knowledge.
Define XML_IGNORE_DEPRECATION_WARNINGS and the corresponding XML_POP_WARNINGS
for Visual Studio, and consequently define XML_IGNORE_FPTR_CAST_WARNINGS so
that we do not get a compiler warning on Visual Studio by doing a
__pragma(warning(pop)) without a corresponding __pragma(warning(push)).
Also correct the documentation a bit for XML_POP_WARNINGS.
We can mark APIs as deprecated using __declspec(deprecated) with Visual Studio
2005 and later, so add a definition of that so that we can help users avoid
using deprecated APIs when using Visual Studio as well.
For the existing GCC definition, check whether we are on GCC 3.1+ before
enabling the definition.
Remove inaccurate xmlParseCheckTransition check.
Remove non-incremental xmlParseGetLasts check.
Add functions that check for several boundary constructs more
accurately, keeping track of progress in ctxt->checkIndex.
Fixes#439.
In some cases, for example when using encoders, the read callback was
set to NULL, in other cases it was set to xmlInputReadCallbackNop.
xmlGROW only tested for xmlInputReadCallbackNop, resulting in errors
when parsing large encoded content from memory.
Always use a NULL callback for memory buffers to avoid ambiguities.
Fixes#262.
When using xmlreader, XPointer expressions in XIncludes simply cannot
work. Expressions can reference nodes which weren't parsed yet or which
were already deleted.
After fixing nested XIncludes, we reference includes which were parsed
previously. When streaming, these nodes could have been deleted, leading
to use-after-free errors.
Disallow XPointer expressions and truncate the include table in
streaming mode.
Private functions were previously declared
- in header files in the root directory
- in public headers guarded with IN_LIBXML
- in libxml.h
- redundantly in source files that used them.
Consolidate all private header files in include/private.
Somewhat misleadingly, the DOC_DISABLE directive only disabled warnings.
Now we really stop the documentation generator from indexing.
This results in additional warnings for xmlThrDef* functions. This should
be fixed by documenting or deprecating them.
Commit 4fd69f3e fixed handling of '<' characters not followed by an
ASCII letter. But a '<!' sequence followed by invalid characters should
be treated as bogus comment and skipped.
Fixes#380.
Add a new configuration flag that controls whether the outdated support
for XPointer locations (ranges and points) is enabled.
--with-xptr-locs # Autotools
LIBXML2_WITH_XPTR_LOCS # CMake
The latest spec for what it essentially an XPath extension seems to be
this working draft from 2002:
https://www.w3.org/TR/xptr-xpointer/
The xpointer() scheme is listed as "being reviewed" in the XPointer
registry since at least 2006. libxml2 seems to be the only modern
software that tries to implement this spec, but the code has many bugs
and quality issues.
The flag defaults to "off" and support for this extensions has to be
requested explicitly. The relevant API functions are deprecated.
