For both XML and HTML, the document can provide an encoding
either in XMLDecl in XML, or as a meta element in HTML head.
This adds options to ignore those encodings if the encoding
is known in advace for example if the content had been converted
before being passed to the parser.
* parser.c include/libxml/parser.h: add XML_PARSE_IGNORE_ENC option
for XML parsing
* include/libxml/HTMLparser.h HTMLparser.c: adds the
HTML_PARSE_IGNORE_ENC for HTML parsing
* HTMLtree.c: fix the handling of saving when an unknown encoding is
defined in meta document header
* xmllint.c: add a --noenc option to activate the new parser options
Mostly except we keep support for some older constructs and
don't implement extension or privateuse. It's messy because
it's used mostly by XSD datatype which itself reference RFC 3066
and suggests a lexical space completely different from what
5646 defines.
In xmlInitParser, both __xmlGlobalInitMutexLock and xmlInitGlobals are
called before xmlInitThreads, and both use pthread symbols.
__xmlGlobalInitMutexLock does so directly, without checking if the symbol
exists, and xmlInitGlobals calls xmlNewMutex, which correctly depends on
libxml_is_threaded... except libxml_is_threaded is still -1 by then...
And again, when releasing the global mutex in __xmlGlobalInitMutexUnlock,
the pthread function is called directly.
The patch changes the initialization order and make sure the functions
are available before calling them
if encoding was autodetected, in xmlParseChunk, if initial size is 86 (a
chunk in UTF-16 encoding), the code that tries to read only the first line
will set the size to 90, which eventually leads to a memmove of 90 bytes
(in xmlBufferAdd) which will copy extra random memory bytes, which will
make the parser to fail because of these extra bytes.
xmlParseInNodeContext notices that the enclosing document is
an HTML document, so invoke the HTML parser for that fragment, and
the HTML parser finding a "<p>hello world!</p>" document automatically
augment it with defaulted <html> and <body>. This defaulting should
be turned off in the HTML parser for this to work, but there is no
such HTML parser option. There is an htmlOmittedDefaultValue global
variable that you could use, but really we should not rely on global
variable for processing options anymore, best is to add an
HTML_PARSE_NOIMPLIED.
* include/libxml/HTMLparser.h: add the HTML_PARSE_NOIMPLIED parser flag
* HTMLparser.c: do add implied element if HTML_PARSE_NOIMPLIED is set
* parser.c: add HTML_PARSE_NOIMPLIED to options for xmlParseInNodeContext
on HTML documents
* SAX2.c dict.c error.c hash.c nanohttp.c parser.c python/libxml.c
relaxng.c runtest.c tree.c valid.c xinclude.c xmlregexp.c xmlsave.c
xmlschemas.c xpath.c xpointer.c: mostly removing unneded affectations,
but this led to a few real bugs and some part not yet understood
(relaxng/interleave)
* HTMLparser.c c14n.c debugXML.c entities.c nanohttp.c parser.c
testC14N.c uri.c xmlcatalog.c xmllint.c xmlregexp.c xpath.c:
fix unused variables, or unneeded increments as well as a couple
of space issues
* runtest.c: check for NULL before calling unlink()
* test/utf16bebom.xml: regression test showed that this test case was
broken but previous behaviour would not detect it !
* parser.c: fix 566012 for the push mode of the parser, tricky !
* test/ebcdic_566012.xml result//ebcdic_566012.xml*: add the test to the
regression suite
* encoding.c parser.c parserInternals.c: when we autodetect an encoding
but it's actually not completely compatible with the one declared
great care must be taken to not convert more than just the first line.
Led to some refactoring, more private functions and a bit of cleanup.
* parser.c: when replacing entities and that the entity is CDATA and
reference entities then white space character in replacement text
need to be replaced by 0x20
* result/noent/att10: correct the output of the associated regression
test
* libxml2.syms: the symbols with history, going back to 2.4.30
* Makefile.am configure.in: linking flags detection and use
* parser.c tree.c valid.c xpointer.c: various cleanup of functions
which could be made static or simply discarded, not that many
* SAX2.c parser.c: fix for CVE-2008-4226, a memory overflow
when building gigantic text nodes, and a bit of cleanup
to better handled out of memory problem in that code.
* tree.c: fix for CVE-2008-4225, lack of testing leads to
a busy loop test assuming one have enough core memory.
Daniel
svn path=/trunk/; revision=3803
* configure.in doc/* NEWS: preparing the release of 2.7.2
* dict.c: fix the Solaris portability issue
* parser.c: additional cleanup on #554660 fix
* test/ent13 result/ent13* result/noent/ent13*: added the
example in the regression test suite.
* HTMLparser.c: handle leading BOM in htmlParseElement()
Daniel
svn path=/trunk/; revision=3799
* trionan.c: Borland C fix from Moritz Both
* testapi.c: regenerate, workaround a problem for buffer testing
* xmlIO.c HTMLtree.c: new internal entry point to hide even better
xmlAllocOutputBufferInternal
* tree.c: harden the code around buffer allocation schemes
* parser.c: restore the warning when namespace names are not absolute
URIs
* runxmlconf.c: continue regression tests if we get the expected
number of errors
* Makefile.am: run the python tests on make check
* xmlsave.c: handle the HTML documents and trees
* python/libxml.c: convert python serialization to the xmlSave APIs
and avoid some horrible hacks
Daniel
svn path=/trunk/; revision=3790
* parser.c: a couple more fixes
* nanohttp.c nanoftp.c: patch from Andreas Färber to compile on Haiku
* doc/examples/*: regenerated
daniel
svn path=/trunk/; revision=3784
* parser.c include/libxml/parser.h: completely different fix for
the recursion detection based on entity density, big cleanups
in the entity parsing code too
* result/*.sax*: the parser should not ask for used defined versions
of the predefined entities
* testrecurse.c: automatic test for entity recursion checks
* Makefile.am: added testrecurse
* test/recurse/lol* test/recurse/good*: a first set of tests for
the recursion
Daniel
svn path=/trunk/; revision=3783
* include/libxml/xmlerror.h parser.c: a bit of cleanup and
added checks based on the regression tests of the xmlconf suite
Daniel
svn path=/trunk/; revision=3782
* include/libxml/parser.h parser.c xmllint.c: strengthen some
of the internal parser limits, add an XML_PARSE_HUGE option
to bypass them all. More internal parser limits will still need
to be added.
Daniel
svn path=/trunk/; revision=3777
* include/libxml/entities.h entities.c SAX2.c parser.c: rework
the patch to avoid some ABI issue with people allocating
entities structure directly
Daniel
svn path=/trunk/; revision=3773
