From eb70f93ffaaf94163098f025ac146e4d8278ce92 Mon Sep 17 00:00:00 2001 From: Daniel Veillard Date: Mon, 5 Jul 2004 16:46:09 +0000 Subject: [PATCH] make the push interfaces synchronous added a specific test added the new * parser.c: make the push interfaces synchronous * python/tests/sync.py: added a specific test * python/tests/Makefile.am doc/examples/Makefile.am doc/examples/index.py: added the new test, cleaning up "make tests" output Daniel --- ChangeLog | 8 +++ doc/examples/Makefile.am | 1 + doc/examples/index.html | 2 +- doc/examples/index.py | 1 + parser.c | 49 ++++++++------ python/tests/Makefile.am | 10 ++- python/tests/sync.py | 135 +++++++++++++++++++++++++++++++++++++++ 7 files changed, 183 insertions(+), 23 deletions(-) create mode 100755 python/tests/sync.py diff --git a/ChangeLog b/ChangeLog index 3cf20210..e733dc7f 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,11 @@ +Mon Jul 5 18:43:47 CEST 2004 Daniel Veillard + + * parser.c: make the push interfaces synchronous + * python/tests/sync.py: added a specific test + * python/tests/Makefile.am doc/examples/Makefile.am + doc/examples/index.py: added the new test, cleaning up + "make tests" output + Mon Jul 5 15:09:17 CEST 2004 Daniel Veillard * xmlschemas.c: applied patch from Kasimier to fix some Relax-NG diff --git a/doc/examples/Makefile.am b/doc/examples/Makefile.am index 0d4bf13d..26d143f7 100644 --- a/doc/examples/Makefile.am +++ b/doc/examples/Makefile.am @@ -93,6 +93,7 @@ valgrind: $(MAKE) CHECKER='valgrind -q' tests tests: $(noinst_PROGRAMS) + @(echo '## examples regression tests') @(echo > .memdump) @($(CHECKER) ././xpath1 test3.xml '//child2' > xpath1.tmp ; diff xpath1.tmp xpath1.res ; rm xpath1.tmp) @(grep "MORY ALLO" .memdump | grep -v "MEMORY ALLOCATED : 0" ; exit 0) diff --git a/doc/examples/index.html b/doc/examples/index.html index 1d289711..a84a9ac1 100644 --- a/doc/examples/index.html +++ b/doc/examples/index.html @@ -11,4 +11,4 @@ A:link, A:visited, A:active { text-decoration: underline } of the example:

  • xmlWriter :

  • InputOutput :

    • io1.c: Example of custom Input/Output
    • io2.c: Output to char buffer

  • Tree :

    • tree1.c: Navigates a tree to print element names
    • tree2.c: Creates a tree

  • XPath :

    • xpath1.c: Evaluate XPath expression and prints result node set.
    • xpath2.c: Load a document, locate subelements with XPath, modify said elements and save the resulting document.

  • xmlReader :

    • reader1.c: Parse an XML file with an xmlReader
    • reader2.c: Parse and validate an XML file with an xmlReader
    • reader3.c: Show how to extract subdocuments with xmlReader

  • Parsing :

    • parse1.c: Parse an XML file to a tree and free it
    • parse2.c: Parse and validate an XML file to a tree and free the result
    • parse3.c: Parse an XML document in memory to a tree and free it
    • parse4.c: Parse an XML document chunk by chunk to a tree and free it

Getting the compilation options and libraries dependancies needed to generate binaries from the examples is best done on Linux/Unix by using the xml2-config script which should have been installed as part of make -install step or when installing the libxml2 development package:

gcc -o example `xml2-config --cflags` example.c `xml2-config --libs`

InputOutput Examples

io1.c: Example of custom Input/Output

Demonstrate the use of xmlRegisterInputCallbacks to build a custom I/O layer, this is used in an XInclude method context to show how dynamic document can be built in a clean way.

Includes:

Uses:

Usage:

io1

Author: Daniel Veillard

io2.c: Output to char buffer

Demonstrate the use of xmlDocDumpMemory to output document to a character buffer

Includes:

Uses:

Usage:

io2

Author: John Fleck

Parsing Examples

parse1.c: Parse an XML file to a tree and free it

Demonstrate the use of xmlReadFile() to read an XML file into a tree and and xmlFreeDoc() to free the resulting tree

Includes:

Uses:

Usage:

parse1 test1.xml

Author: Daniel Veillard

parse2.c: Parse and validate an XML file to a tree and free the result

Create a parser context for an XML file, then parse and validate the file, creating a tree, check the validation result and xmlFreeDoc() to free the resulting tree.

Includes:

Uses:

Usage:

parse2 test2.xml

Author: Daniel Veillard

parse3.c: Parse an XML document in memory to a tree and free it

Demonstrate the use of xmlReadMemory() to read an XML file into a tree and and xmlFreeDoc() to free the resulting tree

Includes:

Uses:

Usage:

parse3

Author: Daniel Veillard

parse4.c: Parse an XML document chunk by chunk to a tree and free it

Demonstrate the use of xmlCreatePushParserCtxt() and xmlParseChunk() to read an XML file progressively into a tree and and xmlFreeDoc() to free the resulting tree

Includes:

Uses:

Usage:

parse4 test3.xml

Author: Daniel Veillard

Tree Examples

tree1.c: Navigates a tree to print element names

Parse a file to a tree, use xmlDocGetRootElement() to get the root element, then walk the document and print all the element name in document order.

Includes:

Uses:

Usage:

tree1 filename_or_URL

Author: Dodji Seketeli

tree2.c: Creates a tree

Shows how to create document, nodes and dump it to stdout or file.

Includes:

Uses:

Usage:

tree2 <filename> -Default output: stdout

Author: Lucas Brasilino <brasilino@recife.pe.gov.br>

XPath Examples

xpath1.c: Evaluate XPath expression and prints result node set.

Shows how to evaluate XPath expression and register known namespaces in XPath context.

Includes:

Uses:

Usage:

xpath1 <xml-file> <xpath-expr> [<known-ns-list>]

Author: Aleksey Sanin

xpath2.c: Load a document, locate subelements with XPath, modify said elements and save the resulting document.

Shows how to make a full round-trip from a load/edit/save

Includes:

Uses:

Usage:

xpath2 <xml-file> <xpath-expr> <new-value>

Author: Aleksey Sanin and Daniel Veillard

xmlReader Examples

reader1.c: Parse an XML file with an xmlReader

Demonstrate the use of xmlReaderForFile() to parse an XML file and dump the informations about the nodes found in the process. (Note that the XMLReader functions require libxml2 version later than 2.6.)

Includes:

Uses:

Usage:

reader1 <filename>

Author: Daniel Veillard

reader2.c: Parse and validate an XML file with an xmlReader

Demonstrate the use of xmlReaderForFile() to parse an XML file validating the content in the process and activating options like entities substitution, and DTD attributes defaulting. (Note that the XMLReader functions require libxml2 version later than 2.6.)

Includes:

Uses:

Usage:

reader2 <valid_xml_filename>

Author: Daniel Veillard

reader3.c: Show how to extract subdocuments with xmlReader

Demonstrate the use of xmlTextReaderPreservePattern() to parse an XML file with the xmlReader while collecting only some subparts of the document. (Note that the XMLReader functions require libxml2 version later than 2.6.)

Includes:

Uses:

Usage:

reader3

Author: Daniel Veillard

xmlWriter Examples

testWriter.c: use various APIs for the xmlWriter

tests a number of APIs for the xmlWriter, especially the various methods to write to a filename, to a memory buffer, to a new document, or to a subtree. It shows how to do encoding string conversions too. The resulting documents are then serialized.

Includes:

Uses:

Usage:

testWriter

Author: Alfred Mickautsch

Daniel Veillard

+install step or when installing the libxml2 development package:

gcc -o example `xml2-config --cflags` example.c `xml2-config --libs`

InputOutput Examples

io1.c: Example of custom Input/Output

Demonstrate the use of xmlRegisterInputCallbacks to build a custom I/O layer, this is used in an XInclude method context to show how dynamic document can be built in a clean way.

Includes:

Uses:

Usage:

io1

Author: Daniel Veillard

io2.c: Output to char buffer

Demonstrate the use of xmlDocDumpMemory to output document to a character buffer

Includes:

Uses:

Usage:

io2

Author: John Fleck

Parsing Examples

parse1.c: Parse an XML file to a tree and free it

Demonstrate the use of xmlReadFile() to read an XML file into a tree and and xmlFreeDoc() to free the resulting tree

Includes:

Uses:

Usage:

parse1 test1.xml

Author: Daniel Veillard

parse2.c: Parse and validate an XML file to a tree and free the result

Create a parser context for an XML file, then parse and validate the file, creating a tree, check the validation result and xmlFreeDoc() to free the resulting tree.

Includes:

Uses:

Usage:

parse2 test2.xml

Author: Daniel Veillard

parse3.c: Parse an XML document in memory to a tree and free it

Demonstrate the use of xmlReadMemory() to read an XML file into a tree and and xmlFreeDoc() to free the resulting tree

Includes:

Uses:

Usage:

parse3

Author: Daniel Veillard

parse4.c: Parse an XML document chunk by chunk to a tree and free it

Demonstrate the use of xmlCreatePushParserCtxt() and xmlParseChunk() to read an XML file progressively into a tree and and xmlFreeDoc() to free the resulting tree

Includes:

Uses:

Usage:

parse4 test3.xml

Author: Daniel Veillard

Tree Examples

tree1.c: Navigates a tree to print element names

Parse a file to a tree, use xmlDocGetRootElement() to get the root element, then walk the document and print all the element name in document order.

Includes:

Uses:

Usage:

tree1 filename_or_URL

Author: Dodji Seketeli

tree2.c: Creates a tree

Shows how to create document, nodes and dump it to stdout or file.

Includes:

Uses:

Usage:

tree2 <filename> -Default output: stdout

Author: Lucas Brasilino <brasilino@recife.pe.gov.br>

XPath Examples

xpath1.c: Evaluate XPath expression and prints result node set.

Shows how to evaluate XPath expression and register known namespaces in XPath context.

Includes:

Uses:

Usage:

xpath1 <xml-file> <xpath-expr> [<known-ns-list>]

Author: Aleksey Sanin

xpath2.c: Load a document, locate subelements with XPath, modify said elements and save the resulting document.

Shows how to make a full round-trip from a load/edit/save

Includes:

Uses:

Usage:

xpath2 <xml-file> <xpath-expr> <new-value>

Author: Aleksey Sanin and Daniel Veillard

xmlReader Examples

reader1.c: Parse an XML file with an xmlReader

Demonstrate the use of xmlReaderForFile() to parse an XML file and dump the informations about the nodes found in the process. (Note that the XMLReader functions require libxml2 version later than 2.6.)

Includes:

Uses:

Usage:

reader1 <filename>

Author: Daniel Veillard

reader2.c: Parse and validate an XML file with an xmlReader

Demonstrate the use of xmlReaderForFile() to parse an XML file validating the content in the process and activating options like entities substitution, and DTD attributes defaulting. (Note that the XMLReader functions require libxml2 version later than 2.6.)

Includes:

Uses:

Usage:

reader2 <valid_xml_filename>

Author: Daniel Veillard

reader3.c: Show how to extract subdocuments with xmlReader

Demonstrate the use of xmlTextReaderPreservePattern() to parse an XML file with the xmlReader while collecting only some subparts of the document. (Note that the XMLReader functions require libxml2 version later than 2.6.)

Includes:

Uses:

Usage:

reader3

Author: Daniel Veillard

xmlWriter Examples

testWriter.c: use various APIs for the xmlWriter

tests a number of APIs for the xmlWriter, especially the various methods to write to a filename, to a memory buffer, to a new document, or to a subtree. It shows how to do encoding string conversions too. The resulting documents are then serialized.

Includes:

Uses:

Usage:

testWriter

Author: Alfred Mickautsch

Daniel Veillard

diff --git a/doc/examples/index.py b/doc/examples/index.py index 466cfc98..944c181e 100755 --- a/doc/examples/index.py +++ b/doc/examples/index.py @@ -251,6 +251,7 @@ install-data-local: example, example) Makefile = Makefile + "valgrind: \n\t$(MAKE) CHECKER='valgrind -q' tests\n\n" Makefile = Makefile + "tests: $(noinst_PROGRAMS)\n" + Makefile = Makefile + "\t@(echo '## examples regression tests')\n" Makefile = Makefile + "\t@(echo > .memdump)\n" for test in tests: Makefile = Makefile + "\t@($(CHECKER) %s)\n" % (test) diff --git a/parser.c b/parser.c index 5a39915e..77a12f50 100644 --- a/parser.c +++ b/parser.c @@ -8818,29 +8818,37 @@ xmlParseGetLasts(xmlParserCtxtPtr ctxt, const xmlChar **lastlt, if ((ctxt->progressive != 0) && (ctxt->inputNr == 1)) { tmp = ctxt->input->end; tmp--; - while ((tmp >= ctxt->input->base) && (*tmp != '<') && - (*tmp != '>')) tmp--; + while ((tmp >= ctxt->input->base) && (*tmp != '<')) tmp--; if (tmp < ctxt->input->base) { *lastlt = NULL; *lastgt = NULL; - } else if (*tmp == '<') { - *lastlt = tmp; - tmp--; - while ((tmp >= ctxt->input->base) && (*tmp != '>')) tmp--; - if (tmp < ctxt->input->base) - *lastgt = NULL; - else - *lastgt = tmp; } else { - *lastgt = tmp; - tmp--; - while ((tmp >= ctxt->input->base) && (*tmp != '<')) tmp--; - if (tmp < ctxt->input->base) - *lastlt = NULL; - else - *lastlt = tmp; + *lastlt = tmp; + tmp++; + while ((tmp < ctxt->input->end) && (*tmp != '>')) { + if (*tmp == '\'') { + tmp++; + while ((tmp < ctxt->input->end) && (*tmp != '\'')) tmp++; + if (tmp < ctxt->input->end) tmp++; + } else if (*tmp == '"') { + tmp++; + while ((tmp < ctxt->input->end) && (*tmp != '"')) tmp++; + if (tmp < ctxt->input->end) tmp++; + } else + tmp++; + } + if (tmp < ctxt->input->end) + *lastgt = tmp; + else { + tmp = *lastlt; + tmp--; + while ((tmp >= ctxt->input->base) && (*tmp != '>')) tmp--; + if (tmp >= ctxt->input->base) + *lastgt = tmp; + else + *lastgt = NULL; + } } - } else { *lastlt = NULL; *lastgt = NULL; @@ -9096,7 +9104,7 @@ xmlParseTryOrFinish(xmlParserCtxtPtr ctxt, int terminate) { if (!terminate) { if (ctxt->progressive) { /* > can be found unescaped in attribute values */ - if ((lastlt == NULL) || (ctxt->input->cur >= lastlt)) + if ((lastgt == NULL) || (ctxt->input->cur >= lastgt)) goto done; } else if (xmlParseLookupSequence(ctxt, '>', 0, 0) < 0) { goto done; @@ -9274,7 +9282,8 @@ xmlParseTryOrFinish(xmlParserCtxtPtr ctxt, int terminate) { goto done; if (!terminate) { if (ctxt->progressive) { - if ((lastgt == NULL) || (ctxt->input->cur > lastgt)) + /* > can be found unescaped in attribute values */ + if ((lastgt == NULL) || (ctxt->input->cur >= lastgt)) goto done; } else if (xmlParseLookupSequence(ctxt, '>', 0, 0) < 0) { goto done; diff --git a/python/tests/Makefile.am b/python/tests/Makefile.am index bf048788..259a1bf5 100644 --- a/python/tests/Makefile.am +++ b/python/tests/Makefile.am @@ -33,6 +33,7 @@ PYTESTS= \ readererr.py\ relaxng.py \ thread2.py \ + sync.py \ tstLastError.py \ indexes.py @@ -45,8 +46,13 @@ EXTRA_DIST = $(PYTESTS) $(XMLS) if WITH_PYTHON tests: $(PYTESTS) - -@(PYTHONPATH="..:../.libs:$(srcdir)/..:$$PYTHONPATH" ; export PYTHONPATH; \ - for test in $(PYTESTS) ; do echo "-- $$test" ; $(PYTHON) $(srcdir)/$$test ; done) + echo "## running Python regression tests" + -@(PYTHONPATH="..:../.libs:$(srcdir)/..:$$PYTHONPATH" ; \ + export PYTHONPATH; \ + for test in $(PYTESTS) ; \ + do log=`$(PYTHON) $(srcdir)/$$test` ; \ + if [ "`echo $$log | grep OK`" = "" ] ; then \ + echo "-- $$test" ; echo "$$log" ; fi ; done) else tests: endif diff --git a/python/tests/sync.py b/python/tests/sync.py new file mode 100755 index 00000000..2141875c --- /dev/null +++ b/python/tests/sync.py @@ -0,0 +1,135 @@ +#!/usr/bin/python -u +import sys +import libxml2 + +# Memory debug specific +libxml2.debugMemory(1) + +log = "" + +class callback: + def startDocument(self): + global log + log = log + "startDocument:" + + def endDocument(self): + global log + log = log + "endDocument:" + + def startElement(self, tag, attrs): + global log + log = log + "startElement %s %s:" % (tag, attrs) + + def endElement(self, tag): + global log + log = log + "endElement %s:" % (tag) + + def characters(self, data): + global log + log = log + "characters: %s:" % (data) + + def warning(self, msg): + global log + log = log + "warning: %s:" % (msg) + + def error(self, msg): + global log + log = log + "error: %s:" % (msg) + + def fatalError(self, msg): + global log + log = log + "fatalError: %s:" % (msg) + +handler = callback() + +log="" +chunk="""""" +ctxt = libxml2.createPushParser(handler, None, 0, "test.xml") +ctxt.parseChunk(chunk, len(chunk), 0) +ctxt=None + +reference = "startDocument:startElement foo None:startElement bar2 None:endElement bar2:" +if log != reference: + print "Error got: %s" % log + print "Exprected: %s" % reference + sys.exit(1) + +log="" +chunk="""""" +ctxt = libxml2.createPushParser(handler, None, 0, "test.xml") +ctxt.parseChunk(chunk, len(chunk), 0) +ctxt=None + +reference = "startDocument:startElement foo None:startElement bar2 None:endElement bar2:" +if log != reference: + print "Error got: %s" % log + print "Exprected: %s" % reference + sys.exit(1) + +log="" +chunk="""""" +ctxt = libxml2.createPushParser(handler, None, 0, "test.xml") +ctxt.parseChunk(chunk, len(chunk), 0) +ctxt=None + +reference = "startDocument:startElement foo None:startElement bar2 None:" +if log != reference: + print "Error got: %s" % log + print "Exprected: %s" % reference + sys.exit(1) + +log="" +chunk="""""" +ctxt = libxml2.createPushParser(handler, None, 0, "test.xml") +ctxt.parseChunk(chunk, len(chunk), 0) +ctxt=None + +reference = "startDocument:startElement foo None:startElement bar2 {'a': '1', 'b': '2'}:endElement bar2:" +if log != reference: + print "Error got: %s" % log + print "Exprected: %s" % reference + sys.exit(1) + +log="" +chunk="""""" +ctxt = libxml2.createPushParser(handler, None, 0, "test.xml") +ctxt.parseChunk(chunk, len(chunk), 0) +ctxt=None + +reference = "startDocument:startElement foo None:startElement bar2 {'a': '1', 'b': '2'}:" +if log != reference: + print "Error got: %s" % log + print "Exprected: %s" % reference + sys.exit(1) + +log="" +chunk="""""" +ctxt = libxml2.createPushParser(handler, None, 0, "test.xml") +ctxt.parseChunk(chunk, len(chunk), 0) +ctxt=None + +reference = "startDocument:startElement foo None:startElement bar2 {'a': '1', 'b': '2'}:endElement bar2:" +if log != reference: + print "Error got: %s" % log + print "Exprected: %s" % reference + sys.exit(1) + +log="" +chunk="""