- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/FAQ.html b/doc/FAQ.html index a47314e2..4e1ace2b 100644 --- a/doc/FAQ.html +++ b/doc/FAQ.html @@ -1,294 +1,346 @@ - + -
- Licence(s) -
- Installation -
- Compilation -
- Developer corner +
- Licence(s) +
- Installation +
- Compilation +
- Developer corner
- Licensing Terms for libxml +
-
+Licensing Terms for libxml
libxml is released under 2 (compatible) licences:
--
-
- the LGPL: GNU
+
-
+
- the LGPL: GNU Library General Public License -
- the W3C
+
- the W3C IPR: very similar to the XWindow licence
-
- - Can I embed libxml in a proprietary application ? +
+ - the LGPL: GNU
+
-
+Can I embed libxml in a proprietary application ?
Yes. The W3C IPR allows you to also keep proprietary the changes you made to libxml, but it would be graceful to provide back bugfixes and improvements as patches for possible incorporation in the main development tree
-
+
- Unless you are forced to because your application links with a Gnome +
- Unless you are forced to because your application links with a Gnome library requiring it, Do Not Use libxml1, use libxml2 -
- Where can I get libxml +
-
+Where can I get libxml
?
-
The original distribution comes from rpmfind.net or gnome.org
-Most linux and Bsd distribution includes libxml, this is probably the +
The original distribution comes from rpmfind.net or gnome.org +
+Most linux and Bsd distribution includes libxml, this is probably the safer way for end-users
-David Doolin provides precompiled Windows versions at http://www.ce.berkeley.edu/~doolin/code/libxmlwin32/
-
- - I see libxml and libxml2 releases, which one should I install ?
+
David Doolin provides precompiled Windows versions at http://www.ce.berkeley.edu/~doolin/code/libxmlwin32/ +
+
+ -
+I see libxml and libxml2 releases, which one should I install ?
-
-
- If you are not concerned by any existing backward compatibility +
- If you are not concerned by any existing backward compatibility with existing application, install libxml2 only -
- If you are not doing development, you can safely install both. - usually the packages libxml and libxml2 are +
- If you are not doing development, you can safely install both. + usually the packages libxml and libxml2 are compatible (this is not the case for development packages) -
- If you are a developer and your system provides separate packaging +
- If you are a developer and your system provides separate packaging for shared libraries and the development components, it is possible - to install libxml and libxml2, and also libxml-devel - and libxml2-devel + to install libxml and libxml2, and also libxml-devel + and libxml2-devel too for libxml2 >= 2.3.0 -
- If you are developing a new application, please develop against +
- If you are developing a new application, please develop against libxml2(-devel) -
- - I can't install the libxml package it conflicts with libxml0 + + +
-
+I can't install the libxml package it conflicts with libxml0
You probably have an old libxml0 package used to provide the shared library for libxml.so.0, you can probably safely remove it. Anyway the - libxml packages provided on rpmfind.net provides + libxml packages provided on rpmfind.net provides libxml.so.0
-
- - I can't install the libxml(2) RPM package due to failed + +
-
+I can't install the libxml(2) RPM package due to failed
dependancies
The most generic solution is to refetch the latest src.rpm , and rebuild it locally with
-
-rpm --rebuild libxml(2)-xxx.src.rpmif everything goes well it will generate two binary rpm (one providing +
+rpm --rebuild libxml(2)-xxx.src.rpmif everything goes well it will generate two binary rpm (one providing the shared libs and xmllint, and the other one, the -devel package providing includes, static libraries and scripts needed to build applications with libxml(2)) that you can install locally.
-
+
- What is the process to compile libxml ?
-
As most UNIX libraries libxml follows the "standard":
-
-gunzip -c xxx.tar.gz | tar xvf -
-cd libxml-xxxx
-./configure --helpto see the options, then the compilation/installation proper
-
-./configure [possible options]
-make
-make installAt that point you may have to rerun ldconfig or similar utility to +
-
+What is the process to compile libxml ?
+
As most UNIX libraries libxml follows the "standard":
+
+gunzip -c xxx.tar.gz | tar xvf -
+cd libxml-xxxx
+./configure --helpto see the options, then the compilation/installation proper
+
+./configure [possible options]
+make
+make installAt that point you may have to rerun ldconfig or similar utility to update your list of installed shared libs.
-
- - What other libraries are needed to compile/install libxml ? + +
-
+What other libraries are needed to compile/install libxml ?
Libxml does not requires any other library, the normal C ANSI API should be sufficient (please report any violation to this rule you may find).
-However if found at configuration time libxml will detect and use the +
However if found at configuration time libxml will detect and use the following libs:
--
-
- libz
+
-
+
- +libz : a highly portable and available widely compression library -
- iconv: a powerful character encoding conversion library. It's +
- iconv: a powerful character encoding conversion library. It's included by default on recent glibc libraries, so it doesn't need to - be installed specifically on linux. It seems it's now part - of the official UNIX specification. Here is one implementation - of the library which source can be found here. -
- - libxml does not compile with HP-UX's optional ANSI-C compiler
-
this is due to macro limitations. Try to add " -Wp,-H16800 -Ae" to the + be installed specifically on linux. It seems it's now part + of the official UNIX specification. Here is one implementation + of the library which source can be found here.
+
+ - libz
+
-
+libxml does not compile with HP-UX's optional ANSI-C compiler
+
this is due to macro limitations. Try to add " -Wp,-H16800 -Ae" to the CFLAGS
-you can also install and use gcc instead or use a precompiled version - of libxml, both available from the HP-UX Porting - and Archive Centre
-
- - make check fails on some platforms
+
you can also install and use gcc instead or use a precompiled version + of libxml, both available from the HP-UX Porting + and Archive Centre +
+
+ -
+make check fails on some platforms
Sometime the regression tests results don't completely match the value produced by the parser, and the makefile uses diff to print the delta. On some platforms the diff return breaks the compilation process, if the diff is small this is probably not a serious problem
-
- - I use the CVS version and there is no configure script + +
-
+I use the CVS version and there is no configure script
The configure (and other Makefiles) are generated. Use the autogen.sh script to regenerate the configure and Makefiles, like:
-
-./autogen.sh --prefix=/usr --disable-shared
- - I have troubles when running make tests with gcc-3.0
+
+./autogen.sh --prefix=/usr --disable-shared
+ -
+I have troubles when running make tests with gcc-3.0
It seems the initial release of gcc-3.0 has a problem with the optimizer which miscompiles the URI module. Please use another compiler
-
+
- xmlDocDump() generates output on one line +
-
+xmlDocDump() generates output on one line
libxml will not invent spaces in the content of a document since all spaces in the content of a document are significant. If you build a tree from the API and want indentation:
--
-
- the correct way is to generate those yourself too -
- the dangerous way is to ask libxml to add those blanks to your
+
-
+
- the correct way is to generate those yourself too +
- the dangerous way is to ask libxml to add those blanks to your content modifying the content of your document in the process. The result may not be what you expect. There is NO way to guarantee that such a modification won't - impact other part of the content of your document. See xmlKeepBlanksDefault - () and xmlSaveFormatFile - () -
- - Extra nodes in the document: + impact other part of the content of your document. See xmlKeepBlanksDefault + () and xmlSaveFormatFile + () + +
+ - Extra nodes in the document:
For a XML file as below:
-<?xml version="1.0"?> -<PLAN xmlns="http://www.argus.ca/autotest/1.0/"> -<NODE CommFlag="0"/> -<NODE CommFlag="1"/> +
<?xml version="1.0"?> +<PLAN xmlns="http://www.argus.ca/autotest/1.0/"> +<NODE CommFlag="0"/> +<NODE CommFlag="1"/> </PLAN>
-after parsing it with the function +
after parsing it with the function pxmlDoc=xmlParseFile(...);
-I want to the get the content of the first node (node with the - CommFlag="0")
-so I did it as following;
-xmlNodePtr pode; +
I want to the get the content of the first node (node with the + CommFlag="0")
+so I did it as following;
+xmlNodePtr pode; pnode=pxmlDoc->children->children;
-but it does not work. If I change it to
-pnode=pxmlDoc->children->children->next;
-then it works. Can someone explain it to me.
- -In XML all characters in the content of the document are significant +
but it does not work. If I change it to
+pnode=pxmlDoc->children->children->next;
+then it works. Can someone explain it to me.
++
In XML all characters in the content of the document are significant including blanks and formatting line breaks.
-The extra nodes you are wondering about are just that, text nodes with +
The extra nodes you are wondering about are just that, text nodes with the formatting spaces wich are part of the document but that people tend - to forget. There is a function xmlKeepBlanksDefault + to forget. There is a function xmlKeepBlanksDefault () to remove those at parse time, but that's an heuristic, and its use should be limited to case where you are sure there is no mixed-content in the document.
-
- - I get compilation errors of existing code like when accessing + +
-
+I get compilation errors of existing code like when accessing
root or childs fields of nodes
You are compiling code developed for libxml version 1 and using a libxml2 development environment. Either switch back to libxml v1 devel or - even better fix the code to compile with libxml2 (or both) by following the instructions.
-
- - I get compilation errors about non existing + even better fix the code to compile with libxml2 (or both) by following the instructions. + +
-
+I get compilation errors about non existing
xmlRootNode or xmlChildrenNode
fields
-
The source code you are using has been upgraded to be able to compile with both libxml +
The source code you are using has been upgraded to be able to compile with both libxml and libxml2, but you need to install a more recent version: libxml(-devel) >= 1.8.8 or libxml2(-devel) >= 2.1.0
-
- - XPath implementation looks seriously broken + +
-
+XPath implementation looks seriously broken
XPath implementation prior to 2.3.0 was really incomplete, upgrade to a recent version, the implementation and debug of libxslt generated fixes for most obvious problems.
-
- - The example provided in the web page does not compile + +
-
+The example provided in the web page does not compile
It's hard to maintain the documentation in sync with the code <grin/> ...
-Check the previous points 1/ and 2/ raised before, and send +
Check the previous points 1/ and 2/ raised before, and send patches.
-
- - Where can I get more examples and informations than in the web + +
-
+Where can I get more examples and informations than in the web
page
Ideally a libxml book would be nice. I have no such plan ... But you can:
--
-
- check more deeply the existing - generated doc -
- looks for examples of use for libxml function using the Gnome code
+
-
+
- check more deeply the existing + generated doc + +
- looks for examples of use for libxml function using the Gnome code
for example the following will query the full Gnome CVs base for the
use of the xmlAddChild() function:
-
http://cvs.gnome.org/lxr/search?string=xmlAddChild
-This may be slow, a large hardware donation to the gnome project +
http://cvs.gnome.org/lxr/search?string=xmlAddChild
+This may be slow, a large hardware donation to the gnome project could cure this :-)
-
- - Browse + +
- +Browse the libxml source , I try to write code as clean and documented as possible, so looking at it may be helpful -
- - What about C++ ? +
+ - What about C++ ?
libxml is written in pure C in order to allow easy reuse on a number of platforms, including embedded systems. I don't intend to convert to C++.
-There is however a C++ wrapper provided by Ari Johnson +
There is however a C++ wrapper provided by Ari Johnson <ari@btigate.com> which may fullfill your needs:
-Website: http://lusis.org/~ari/xml++/
- -
- - How to validate a document a posteriori ?
+
Website: http://lusis.org/~ari/xml++/ +
+Download: http://lusis.org/~ari/xml++/libxml++.tar.gz +
+
+ - How to validate a document a posteriori ?
It is possible to validate documents which had not been validated at initial parsing time or documents who have been built from scratch using - the API. Use the xmlValidateDtd() + the API. Use the xmlValidateDtd() function. It is also possible to simply add a Dtd to an existing document:
-xmlDocPtr doc; /* your existing document */ +
xmlDocPtr doc; /* your existing document */ xmlDtdPtr dtd = xmlParseDTD(NULL, filename_of_dtd); /* parse the DTD */ - dtd->name = xmlStrDup((xmlChar*)"root_name"); /* use the given root */ + dtd->name = xmlStrDup((xmlChar*)"root_name"); /* use the given root */ doc->intSubset = dtd; if (doc->children == NULL) xmlAddChild((xmlNodePtr)doc, (xmlNodePtr)dtd); else xmlAddPrevSibling(doc->children, (xmlNodePtr)dtd);-
- - etc ... + +
- etc ...
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/XSLT.html b/doc/XSLT.html index ab9bd57e..55788385 100644 --- a/doc/XSLT.html +++ b/doc/XSLT.html @@ -8,6 +8,7 @@ BODY {font-size: 10pt; font-family: Verdana,Arial,Helvetica; margin-top: 5pt; ma H1 {font-size: 16pt; font-family: Verdana,Arial,Helvetica} H2 {font-size: 14pt; font-family: Verdana,Arial,Helvetica} H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} +A:link, A:visited, A:active { text-decoration: underline } -->
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/architecture.html b/doc/architecture.html index 7ffecc53..707a0083 100644 --- a/doc/architecture.html +++ b/doc/architecture.html @@ -8,8 +8,9 @@ BODY {font-size: 10pt; font-family: Verdana,Arial,Helvetica; margin-top: 5pt; ma H1 {font-size: 16pt; font-family: Verdana,Arial,Helvetica} H2 {font-size: 14pt; font-family: Verdana,Arial,Helvetica} H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} +A:link, A:visited, A:active { text-decoration: underline } --> -
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/bugs.html b/doc/bugs.html index 63130218..c470b153 100644 --- a/doc/bugs.html +++ b/doc/bugs.html @@ -8,6 +8,7 @@ BODY {font-size: 10pt; font-family: Verdana,Arial,Helvetica; margin-top: 5pt; ma H1 {font-size: 16pt; font-family: Verdana,Arial,Helvetica} H2 {font-size: 14pt; font-family: Verdana,Arial,Helvetica} H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} +A:link, A:visited, A:active { text-decoration: underline } -->
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/catalog.html b/doc/catalog.html index 14c49c2f..af7ea69b 100644 --- a/doc/catalog.html +++ b/doc/catalog.html @@ -1,137 +1,174 @@ - + -
- General overview -
- The definition -
- Using catalogs -
- Some examples -
- How to tune catalog usage -
- How to debug catalog processing -
- How to create and maintain catalogs -
- The implementor corner quick review of the
+
- General overview
+- The definition
+- Using catalogs
+- Some examples
+- How to tune catalog usage
+- How to debug catalog processing
+- How to create and maintain catalogs
+- The implementor corner quick review of the API
-- Other resources
+- Other resources
- mapping from "logical" names, the public identifiers and a more +
- mapping from "logical" names, the public identifiers and a more
concrete name usable for download (and URI). For example it can associate
the logical name
-
"-//OASIS//DTD DocBook XML V4.1.2//EN"
-of the DocBook 4.1.2 XML DTD with the actual URL where it can be +
"-//OASIS//DTD DocBook XML V4.1.2//EN"
+of the DocBook 4.1.2 XML DTD with the actual URL where it can be downloaded
-http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd
-
- - remapping from a given URL to another one, like an HTTP indirection
+
http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd
+
+ - remapping from a given URL to another one, like an HTTP indirection
saying that
-
"http://www.oasis-open.org/committes/tr.xsl"
-should really be looked at
-"http://www.oasis-open.org/committes/entity/stylesheets/base/tr.xsl"
-
- - providing a local cache mechanism allowing to load the entities
+
"http://www.oasis-open.org/committes/tr.xsl"
+should really be looked at
+"http://www.oasis-open.org/committes/entity/stylesheets/base/tr.xsl"
+
+ - providing a local cache mechanism allowing to load the entities associated to public identifiers or remote resources, this is a really important feature for any significant deployment of XML or SGML since it allows to avoid the aleas and delays associated to fetching remote resources.
- the older SGML catalogs, the official spec is SGML Open Technical - Resolution TR9401:1997, but is better understood by reading the SP Catalog page from +
- the older SGML catalogs, the official spec is SGML Open Technical + Resolution TR9401:1997, but is better understood by reading the SP Catalog page from James Clark. This is relatively old and not the preferred mode of operation of libxml. -
- XML
+
- +XML Catalogs is far more flexible, more recent, uses an XML syntax and should scale quite better. This is the default option of libxml.
- the default one, global shared by all the application -
- a per-document catalog, this one is built if the document uses the +
- the default one, global shared by all the application +
- a per-document catalog, this one is built if the document uses the
oasis-xml-catalogPIs to specify its own catalog list, it is associated to the parser context and destroyed when the parsing context is destroyed. - You can find an good rant from Norm Walsh about the
+
- You can find an good rant from Norm Walsh about the need for catalogs, it provides a lot of context informations even if I don't agree with everything presented.
-- An old XML +
- An old XML catalog proposal from John Cowan
-- The Resource Directory Description +
- The Resource Directory Description Language (RDDL) another catalog system but more oriented toward providing metadata for XML namespaces.
-- the page from the OASIS Technical Committee on Entity +
- the page from the OASIS Technical Committee on Entity Resolution who maintains XML Catalog, you will find pointers to the specification update, some background and pointers to others tools providing XML Catalog support
-- I have uploaded a +
- I have uploaded a mall tarball containing XML Catalogs for DocBook 4.1.2 which seems to work fine for me
-- The xmlcatalog manual page
- +- The xmlcatalog + manual page +
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet @@ -70,8 +71,12 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- Bjorn Reese, William Brack and Thomas Broyer have provided a number of + patches, Gary Pennington worked on the validation API, threading support + and Solaris port. +
- John Fleck helps maintaining the documentation and man pages.
-
-Ari Johnson
+
provides a C++ wrapper for libxml:
Website: http://lusis.org/~ari/xml++/
diff --git a/doc/docs.html b/doc/docs.html index de067193..6ba089f9 100644 --- a/doc/docs.html +++ b/doc/docs.html @@ -8,6 +8,7 @@ BODY {font-size: 10pt; font-family: Verdana,Arial,Helvetica; margin-top: 5pt; ma H1 {font-size: 16pt; font-family: Verdana,Arial,Helvetica} H2 {font-size: 14pt; font-family: Verdana,Arial,Helvetica} H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} +A:link, A:visited, A:active { text-decoration: underline } -->Documentation @@ -27,8 +28,8 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}Main Menu - Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/downloads.html b/doc/downloads.html index 79ee9f9b..9dc038fc 100644 --- a/doc/downloads.html +++ b/doc/downloads.html @@ -8,6 +8,7 @@ BODY {font-size: 10pt; font-family: Verdana,Arial,Helvetica; margin-top: 5pt; ma H1 {font-size: 16pt; font-family: Verdana,Arial,Helvetica} H2 {font-size: 14pt; font-family: Verdana,Arial,Helvetica} H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} +A:link, A:visited, A:active { text-decoration: underline } -->
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/encoding.html b/doc/encoding.html index 24573e39..41790acd 100644 --- a/doc/encoding.html +++ b/doc/encoding.html @@ -1,100 +1,146 @@ - + -
- What does internationalization support mean ? -
- The internal encoding, how and why -
- How is it implemented ? -
- Default supported encodings -
- How to extend the existing support +
- What does internationalization support + mean ? +
- The internal encoding, how and + why +
- How is it implemented ? +
- Default supported encodings +
- How to extend the existing + support
- the document is properly parsed -
- informations about it's encoding are saved -
- it can be modified -
- it can be saved in its original encoding -
- it can also be saved in another encoding supported by libxml (for +
- the document is properly parsed +
- informations about it's encoding are saved +
- it can be modified +
- it can be saved in its original encoding +
- it can also be saved in another encoding supported by libxml (for example straight UTF8 or even an ASCII form)
- keeping the native encoding in the internal form would force the libxml +
- keeping the native encoding in the internal form would force the libxml users (or the code associated) to be fully aware of the encoding of the - original document, for examples when adding a text node to a document, the - content would have to be provided in the document encoding, i.e. the + original document, for examples when adding a text node to a document, + the content would have to be provided in the document encoding, i.e. the client code would have to check it before hand, make sure it's conformant to the encoding, etc ... Very hard in practice, though in some specific cases this may make sense. -
- the second decision was which encoding. From the XML spec only UTF8 and +
- the second decision was which encoding. From the XML spec only UTF8 and
UTF16 really makes sense as being the two only encodings for which there
is amndatory support. UCS-4 (32 bits fixed size encoding) could be
considered an intelligent choice too since it's a direct Unicode mapping
support. I selected UTF-8 on the basis of efficiency and compatibility
with surrounding software:
-
-
- UTF-8 while a bit more complex to convert from/to (i.e. slightly +
- UTF-8 while a bit more complex to convert from/to (i.e. slightly more costly to import and export CPU wise) is also far more compact than UTF-16 (and UCS-4) for a majority of the documents I see it used for right now (RPM RDF catalogs, advogato data, various configuration @@ -104,42 +150,39 @@ rationale for those choices: caches (main memory/external caches/internal caches) and my take is that this harms the system far more than the CPU requirements needed for the conversion to UTF-8 -
- Most of libxml version 1 users were using it with straight ASCII +
- Most of libxml version 1 users were using it with straight ASCII most of the time, doing the conversion with an internal encoding requiring all their code to be rewritten was a serious show-stopper for using UTF-16 or UCS-4. -
- UTF-8 is being used as the de-facto internal encoding standard for +
- UTF-8 is being used as the de-facto internal encoding standard for related code like the pango upcoming Gnome text widget, and a lot of Unix code (yep another place - where Unix programmer base takes a different approach from Microsoft - - they are using UTF-16) -
+ where Unix programmer base takes a different approach from Microsoft
+ - they are using UTF-16)
+ - xmlChar, the libxml data type is a byte, those bytes must be assembled - as UTF-8 valid strings. The proper way to terminate an xmlChar * string is - simply to append 0 byte, as usual. -
- One just need to make sure that when using chars outside the ASCII set, +
- xmlChar, the libxml data type is a byte, those bytes must be assembled + as UTF-8 valid strings. The proper way to terminate an xmlChar * string + is simply to append 0 byte, as usual. +
- One just need to make sure that when using chars outside the ASCII set, the values has been properly converted to UTF-8
- when a document is processed, we usually don't know the encoding, a +
- when a document is processed, we usually don't know the encoding, a simple heuristic allows to detect UTF-18 and UCS-4 from whose where the ASCII range (0-0x7F) maps with ASCII -
- the xml declaration if available is parsed, including the encoding - declaration. At that point, if the autodetected encoding is different from - the one declared a call to xmlSwitchEncoding() is issued. -
- If there is no encoding declaration, then the input has to be in either +
- the xml declaration if available is parsed, including the encoding + declaration. At that point, if the autodetected encoding is different + from the one declared a call to xmlSwitchEncoding() is issued. +
- If there is no encoding declaration, then the input has to be in either UTF-8 or UTF-16, if it is not then at some point when processing the input, the converter/checker of UTF-8 form will raise an encoding error. You may end-up with a garbled document, or no document at all ! Example: @@ -150,114 +193,104 @@ err.xml:1: error: Input is not proper UTF-8, indicate encoding ! err.xml:1: error: Bytes: 0xE8 0x73 0x3E 0x6C <très>là</très> ^ - -
- xmlSwitchEncoding() does an encoding name lookup, canonalize it, and + +
- xmlSwitchEncoding() does an encoding name lookup, canonalize it, and
then search the default registered encoding converters for that encoding.
If it's not within the default set and iconv() support has been compiled
it, it will ask iconv for such an encoder. If this fails then the parser
will report an error and stops processing:
~/XML -> ./xmllint err2.xml err2.xml:1: error: Unsupported encoding UnsupportedEnc -<?xml version="1.0" encoding="UnsupportedEnc"?> +<?xml version="1.0" encoding="UnsupportedEnc"?> ^-
- - From that point the encoder process progressingly the input (it is - plugged as a front-end to the I/O module) for that entity. It captures and - convert on-the-fly the document to be parsed to UTF-8. The parser itself - just does UTF-8 checking of this input and process it transparently. The - only difference is that the encoding information has been added to the - parsing context (more precisely to the input corresponding to this - entity). -
- The result (when using DOM) is an internal form completely in UTF-8 with - just an encoding information on the document node. + +
- From that point the encoder process progressingly the input (it is + plugged as a front-end to the I/O module) for that entity. It captures + and convert on-the-fly the document to be parsed to UTF-8. The parser + itself just does UTF-8 checking of this input and process it + transparently. The only difference is that the encoding information has + been added to the parsing context (more precisely to the input + corresponding to this entity). +
- The result (when using DOM) is an internal form completely in UTF-8 + with just an encoding information on the document node.
- if no encoding is given, libxml will look for an encoding value +
- if no encoding is given, libxml will look for an encoding value
associated to the document and if it exists will try to save to that
encoding,
otherwise everything is written in the internal form, i.e. UTF-8
-
- - so if an encoding was specified, either at the API level or on the + +
- so if an encoding was specified, either at the API level or on the document, libxml will again canonalize the encoding name, lookup for a converter in the registered set or through iconv. If not found the function will return an error code -
- the converter is placed before the I/O buffer layer, as another kind of +
- the converter is placed before the I/O buffer layer, as another kind of buffer, then libxml will simply push the UTF-8 serialization to through that buffer, which will then progressively be converted and pushed onto the I/O layer. -
- It is possible that the converter code fails on some input, for example +
- It is possible that the converter code fails on some input, for example trying to push an UTF-8 encoded chinese character through the UTF-8 to ISO-8859-1 converter won't work. Since the encoders are progressive they will just report the error and the number of bytes converted, at that point libxml will decode the offending character, remove it from the buffer and replace it with the associated charRef encoding { and resume the convertion. This guarante that any document will be saved - without losses (except for markup names where this is not legal, this is a - problem in the current version, in pactice avoid using non-ascci - characters for tags or attributes names @@). A special "ascii" encoding + without losses (except for markup names where this is not legal, this is + a problem in the current version, in pactice avoid using non-ascci + characters for tags or attributes names @@). A special "ascii" encoding name is used to save documents to a pure ascii form can be used when portability is really crucial
- UTF-8 is supported by default (null handlers) -
- UTF-16, both little and big endian -
- ISO-Latin-1 (ISO-8859-1) covering most western languages -
- ASCII, useful mostly for saving -
- HTML, a specific handler for the conversion of UTF-8 to ASCII with HTML +
- UTF-8 is supported by default (null handlers) +
- UTF-16, both little and big endian +
- ISO-Latin-1 (ISO-8859-1) covering most western languages +
- ASCII, useful mostly for saving +
- HTML, a specific handler for the conversion of UTF-8 to ASCII with HTML predefined entities like © for the Copyright sign.
- int xmlAddEncodingAlias(const char *name, const char *alias); -
- int xmlDelEncodingAlias(const char *alias); -
- const char * xmlGetEncodingAlias(const char *alias); -
- void xmlCleanupEncodingAliases(void); +
- int xmlAddEncodingAlias(const char *name, const char *alias); +
- int xmlDelEncodingAlias(const char *alias); +
- const char * xmlGetEncodingAlias(const char *alias); +
- void xmlCleanupEncodingAliases(void);
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/example.html b/doc/example.html index 14ccaa1c..2cf77634 100644 --- a/doc/example.html +++ b/doc/example.html @@ -8,6 +8,7 @@ BODY {font-size: 10pt; font-family: Verdana,Arial,Helvetica; margin-top: 5pt; ma H1 {font-size: 16pt; font-family: Verdana,Arial,Helvetica} H2 {font-size: 14pt; font-family: Verdana,Arial,Helvetica} H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} +A:link, A:visited, A:active { text-decoration: underline } -->
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/help.html b/doc/help.html index 2d0fb31e..266df691 100644 --- a/doc/help.html +++ b/doc/help.html @@ -8,6 +8,7 @@ BODY {font-size: 10pt; font-family: Verdana,Arial,Helvetica; margin-top: 5pt; ma H1 {font-size: 16pt; font-family: Verdana,Arial,Helvetica} H2 {font-size: 14pt; font-family: Verdana,Arial,Helvetica} H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} +A:link, A:visited, A:active { text-decoration: underline } -->
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/index.html b/doc/index.html index d341eeb0..e13480d6 100644 --- a/doc/index.html +++ b/doc/index.html @@ -8,6 +8,7 @@ BODY {font-size: 10pt; font-family: Verdana,Arial,Helvetica; margin-top: 5pt; ma H1 {font-size: 16pt; font-family: Verdana,Arial,Helvetica} H2 {font-size: 14pt; font-family: Verdana,Arial,Helvetica} H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} +A:link, A:visited, A:active { text-decoration: underline } -->
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet @@ -84,21 +85,13 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- The XML library interfaces
- Entities or no entities
- Namespaces -
- Validation +
- Validation
- DOM principles
- A real example
- Contributions
- upgrade instructions for migrating to - libxml2 -
- libxml Internationalization support -
- libxml Input/Output interfaces -
- libxml Memory interfaces -
- libxml Catalog support -
- a short introduction about DTDs and - libxml
- the libxslt page
- the gdome2 page: a standard DOM interface for libxml2 diff --git a/doc/interface.html b/doc/interface.html index eba19cbc..64e32e91 100644 --- a/doc/interface.html +++ b/doc/interface.html @@ -8,6 +8,7 @@ BODY {font-size: 10pt; font-family: Verdana,Arial,Helvetica; margin-top: 5pt; ma H1 {font-size: 16pt; font-family: Verdana,Arial,Helvetica} H2 {font-size: 14pt; font-family: Verdana,Arial,Helvetica} H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} +A:link, A:visited, A:active { text-decoration: underline } -->
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/intro.html b/doc/intro.html index 6a59c927..4c7dec51 100644 --- a/doc/intro.html +++ b/doc/intro.html @@ -8,6 +8,7 @@ BODY {font-size: 10pt; font-family: Verdana,Arial,Helvetica; margin-top: 5pt; ma H1 {font-size: 16pt; font-family: Verdana,Arial,Helvetica} H2 {font-size: 14pt; font-family: Verdana,Arial,Helvetica} H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} +A:link, A:visited, A:active { text-decoration: underline } -->
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/library.html b/doc/library.html index fc2dd500..e93c098a 100644 --- a/doc/library.html +++ b/doc/library.html @@ -8,8 +8,9 @@ BODY {font-size: 10pt; font-family: Verdana,Arial,Helvetica; margin-top: 5pt; ma H1 {font-size: 16pt; font-family: Verdana,Arial,Helvetica} H2 {font-size: 14pt; font-family: Verdana,Arial,Helvetica} H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} +A:link, A:visited, A:active { text-decoration: underline } --> -
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/namespaces.html b/doc/namespaces.html index 52d7d5b2..a3495400 100644 --- a/doc/namespaces.html +++ b/doc/namespaces.html @@ -8,6 +8,7 @@ BODY {font-size: 10pt; font-family: Verdana,Arial,Helvetica; margin-top: 5pt; ma H1 {font-size: 16pt; font-family: Verdana,Arial,Helvetica} H2 {font-size: 14pt; font-family: Verdana,Arial,Helvetica} H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} +A:link, A:visited, A:active { text-decoration: underline } -->
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/news.html b/doc/news.html index f5c8bc2c..dbcb663e 100644 --- a/doc/news.html +++ b/doc/news.html @@ -8,6 +8,7 @@ BODY {font-size: 10pt; font-family: Verdana,Arial,Helvetica; margin-top: 5pt; ma H1 {font-size: 16pt; font-family: Verdana,Arial,Helvetica} H2 {font-size: 14pt; font-family: Verdana,Arial,Helvetica} H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} +A:link, A:visited, A:active { text-decoration: underline } -->
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/site.xsl b/doc/site.xsl index 67f97b6a..2d794646 100644 --- a/doc/site.xsl +++ b/doc/site.xsl @@ -1,9 +1,104 @@
- Home -
- FAQ
- Introduction +
- FAQ
- Documentation
- Reporting bugs and getting help
- How to help @@ -36,21 +37,21 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
- News
- XML
- XSLT -
- An overview of libxml architecture +
- libxml architecture
- The tree output
- The SAX interface -
- The XML library interfaces +
- Validation & DTDs +
- Memory Management +
- Encodings support +
- I/O Interfaces +
- Catalog support +
- The parser interfaces
- Entities or no entities
- Namespaces -
- Validation, or are you afraid of DTDs ? +
- Upgrading 1.x code
- DOM Principles
- A real example
- Contributions -
- Encodings support -
- Catalogs support -
- I/O interfaces -
- Memory interfaces -
- DTD support
- flat page, stylesheet diff --git a/doc/upgrade.html b/doc/upgrade.html index 721334a1..f14a864a 100644 --- a/doc/upgrade.html +++ b/doc/upgrade.html @@ -1,49 +1,106 @@ - + -
- a general cleanup. A number of mistakes inherited from the very early +
- a general cleanup. A number of mistakes inherited from the very early versions couldn't be changed due to compatibility constraints. Example - the "childs" element in the nodes. -
- Uniformization of the various nodes, at least for their header and link + the "childs" element in the nodes. +
- Uniformization of the various nodes, at least for their header and link parts (doc, parent, children, prev, next), the goal is a simpler programming model and simplifying the task of the DOM implementors. -
- better conformances to the XML specification, for example version 1.x +
- better conformances to the XML specification, for example version 1.x had an heuristic to try to detect ignorable white spaces. As a result the SAX event generated were ignorableWhitespace() while the spec requires character() in that case. This also mean that a number of DOM node containing blank text may populate the DOM tree which were not present before.
- The package name have changed from libxml to libxml2, the library name +
- The package name have changed from libxml to libxml2, the library name is now -lxml2 . There is a new xml2-config script which should be used to select the right parameters libxml2 -
- Node childs field has been renamed +
- Node childs field has been renamed children so s/childs/children/g should be applied - (probablility of having "childs" anywere else is close to 0+ -
- The document don't have anymore a root element it has + (probablility of having "childs" anywere else is close to 0+ +
- The document don't have anymore a root element it has been replaced by children and usually you will get a list of element here. For example a Dtd element for the internal subset and it's declaration may be found in that list, as well as processing @@ -52,89 +109,86 @@ mail: a document. Alternatively if you are sure to not reference Dtds nor have PIs or comments before or after the root element s/->root/->children/g will probably do it. -
- The white space issue, this one is more complex, unless special case of +
- The white space issue, this one is more complex, unless special case of
validating parsing, the line breaks and spaces usually used for indenting
and formatting the document content becomes significant. So they are
reported by SAX and if your using the DOM tree, corresponding nodes are
generated. Too approach can be taken:
-
-
- lazy one, use the compatibility call +
- lazy one, use the compatibility call xmlKeepBlanksDefault(0) but be aware that you are relying on a special (and possibly broken) set of heuristics of libxml to detect ignorable blanks. Don't complain if it breaks or make your application not 100% clean w.r.t. to it's input. -
- the Right Way: change you code to accept possibly unsignificant +
- the Right Way: change you code to accept possibly unsignificant blanks characters, or have your tree populated with weird blank text nodes. You can spot them using the comodity function xmlIsBlankNode(node) returning 1 for such blank nodes. -
Note also that with the new default the output functions don't add any +
- The include path has changed to $prefix/libxml/ and the includes + +
- The include path has changed to $prefix/libxml/ and the includes
themselves uses this new prefix in includes instructions... If you are
using (as expected) the
xml2-config --cflags
-output to generate you compile commands this will probably work out of +
output to generate you compile commands this will probably work out of the box
-
- - xmlDetectCharEncoding takes an extra argument indicating the lenght in + +
- xmlDetectCharEncoding takes an extra argument indicating the lenght in byte of the head of the document available for character detection. - -
- similar include naming, one should use +
- similar include naming, one should use #include<libxml/...> in both cases. -
- similar identifiers defined via macros for the child and root fields: +
- similar identifiers defined via macros for the child and root fields: respectively xmlChildrenNode and - xmlRootNode -
- a new macro LIBXML_TEST_VERSION which should be + xmlRootNode + +
- a new macro LIBXML_TEST_VERSION which should be inserted once in the client code
- install the libxml-1.8.8 (and libxml-devel-1.8.8) packages -
- find all occurences where the xmlDoc root field is - used and change it to xmlRootNode -
- similary find all occurences where the xmlNode childs - field is used and change it to xmlChildrenNode -
- add a LIBXML_TEST_VERSION macro somewhere in your +
- install the libxml-1.8.8 (and libxml-devel-1.8.8) packages +
- find all occurences where the xmlDoc root field is + used and change it to xmlRootNode + +
- similary find all occurences where the xmlNode childs + field is used and change it to xmlChildrenNode + +
- add a LIBXML_TEST_VERSION macro somewhere in your main() or in the library init entry point -
- Recompile, check compatibility, it should still work -
- Change your configure script to look first for xml2-config and fallback +
- Recompile, check compatibility, it should still work +
- Change your configure script to look first for xml2-config and fallback using xml-config . Use the --cflags and --libs ouptut of the command as the Include and Linking parameters needed to use libxml. -
- install libxml2-2.3.x and libxml2-devel-2.3.x (libxml-1.8.y and +
- install libxml2-2.3.x and libxml2-devel-2.3.x (libxml-1.8.y and libxml-devel-1.8.y can be kept simultaneously) -
- remove your config.cache, relaunch your configuration mechanism, and +
- remove your config.cache, relaunch your configuration mechanism, and recompile, if steps 2 and 3 were done right it should compile as-is -
- Test that your application is still running correctly, if not this may +
- Test that your application is still running correctly, if not this may be due to extra empty nodes due to formating spaces being kept in libxml2 contrary to libxml1, in that case insert xmlKeepBlanksDefault(1) in your code before calling the parser (next to LIBXML_TEST_VERSION is a fine place).
- upgrade instructions for migrating to - libxml2 -
- libxml Internationalization support -
- libxml Input/Output interfaces -
- libxml Memory interfaces -
- libxml Catalog support -
- a short introduction about DTDs and - libxml
- the libxslt page
- the gdome2 page: a standard DOM interface for libxml2 @@ -89,6 +81,277 @@ Gnome library requiring it, Do Not Use libxml1, use libxml2 +
- Licence(s) +
- Installation +
- Compilation +
- Developer corner +
- Licensing Terms for libxml
+
libxml is released under 2 (compatible) licences:
+ +
+ - Can I embed libxml in a proprietary application ?
+
Yes. The W3C IPR allows you to also keep proprietary the changes you + made to libxml, but it would be graceful to provide back bugfixes and + improvements as patches for possible incorporation in the main + development tree
+
+ - Unless you are forced to because your application links with a Gnome + library requiring it, Do + Not Use libxml1, use libxml2 +
- Where can I get libxml
+ ?
+
The original distribution comes from rpmfind.net or gnome.org
+Most linux and Bsd distribution includes libxml, this is probably the + safer way for end-users
+David Doolin provides precompiled Windows versions at http://www.ce.berkeley.edu/~doolin/code/libxmlwin32/
+
+ - I see libxml and libxml2 releases, which one should I install ?
+
-
+
- If you are not concerned by any existing backward compatibility + with existing application, install libxml2 only +
- If you are not doing development, you can safely install both. + usually the packages libxml and libxml2 are + compatible (this is not the case for development packages) +
- If you are a developer and your system provides separate packaging + for shared libraries and the development components, it is possible + to install libxml and libxml2, and also libxml-devel + and libxml2-devel + too for libxml2 >= 2.3.0 +
- If you are developing a new application, please develop against + libxml2(-devel) +
+ - I can't install the libxml package it conflicts with libxml0
+
You probably have an old libxml0 package used to provide the shared + library for libxml.so.0, you can probably safely remove it. Anyway the + libxml packages provided on rpmfind.net provides + libxml.so.0
+
+ - I can't install the libxml(2) RPM package due to failed
+ dependancies
+
The most generic solution is to refetch the latest src.rpm , and + rebuild it locally with
+
+rpm --rebuild libxml(2)-xxx.src.rpmif everything goes well it will generate two binary rpm (one providing + the shared libs and xmllint, and the other one, the -devel package + providing includes, static libraries and scripts needed to build + applications with libxml(2)) that you can install locally.
+
+ - What is the process to compile libxml ?
+
As most UNIX libraries libxml follows the "standard":
+
+gunzip -c xxx.tar.gz | tar xvf -
+cd libxml-xxxx
+./configure --helpto see the options, then the compilation/installation proper
+
+./configure [possible options]
+make
+make installAt that point you may have to rerun ldconfig or similar utility to + update your list of installed shared libs.
+
+ - What other libraries are needed to compile/install libxml ?
+
Libxml does not requires any other library, the normal C ANSI API + should be sufficient (please report any violation to this rule you may + find).
+However if found at configuration time libxml will detect and use the + following libs:
+-
+
- libz + : a highly portable and available widely compression library +
- iconv: a powerful character encoding conversion library. It's + included by default on recent glibc libraries, so it doesn't need to + be installed specifically on linux. It seems it's now part + of the official UNIX specification. Here is one implementation + of the library which source can be found here. +
+ - libxml does not compile with HP-UX's optional ANSI-C compiler
+
this is due to macro limitations. Try to add " -Wp,-H16800 -Ae" to the + CFLAGS
+you can also install and use gcc instead or use a precompiled version + of libxml, both available from the HP-UX Porting + and Archive Centre
+
+ - make check fails on some platforms
+
Sometime the regression tests results don't completely match the value + produced by the parser, and the makefile uses diff to print the delta. On + some platforms the diff return breaks the compilation process, if the + diff is small this is probably not a serious problem
+
+ - I use the CVS version and there is no configure script
+
The configure (and other Makefiles) are generated. Use the autogen.sh + script to regenerate the configure and Makefiles, like:
+
+./autogen.sh --prefix=/usr --disable-shared
+ - I have troubles when running make tests with gcc-3.0
+
It seems the initial release of gcc-3.0 has a problem with the + optimizer which miscompiles the URI module. Please use another + compiler
+
+ - xmlDocDump() generates output on one line
+
libxml will not invent spaces in the content of a + document since all spaces in the content of a document are + significant. If you build a tree from the API and want + indentation:
+-
+
- the correct way is to generate those yourself too +
- the dangerous way is to ask libxml to add those blanks to your + content modifying the content of your document in the + process. The result may not be what you expect. There is + NO way to guarantee that such a modification won't + impact other part of the content of your document. See xmlKeepBlanksDefault + () and xmlSaveFormatFile + () +
+ - Extra nodes in the document:
+
For a XML file as below:
+<?xml version="1.0"?> +<PLAN xmlns="http://www.argus.ca/autotest/1.0/"> +<NODE CommFlag="0"/> +<NODE CommFlag="1"/> +</PLAN>
+after parsing it with the function + pxmlDoc=xmlParseFile(...);
+I want to the get the content of the first node (node with the + CommFlag="0")
+so I did it as following;
+xmlNodePtr pode; +pnode=pxmlDoc->children->children;
+but it does not work. If I change it to
+pnode=pxmlDoc->children->children->next;
+then it works. Can someone explain it to me.
+ +In XML all characters in the content of the document are significant + including blanks and formatting line breaks.
+The extra nodes you are wondering about are just that, text nodes with + the formatting spaces wich are part of the document but that people tend + to forget. There is a function xmlKeepBlanksDefault + () to remove those at parse time, but that's an heuristic, and its + use should be limited to case where you are sure there is no + mixed-content in the document.
+
+ - I get compilation errors of existing code like when accessing
+ root or childs fields of nodes
+
You are compiling code developed for libxml version 1 and using a + libxml2 development environment. Either switch back to libxml v1 devel or + even better fix the code to compile with libxml2 (or both) by following the instructions.
+
+ - I get compilation errors about non existing
+ xmlRootNode or xmlChildrenNode
+ fields
+
The source code you are using has been upgraded to be able to compile with both libxml + and libxml2, but you need to install a more recent version: + libxml(-devel) >= 1.8.8 or libxml2(-devel) >= 2.1.0
+
+ - XPath implementation looks seriously broken
+
XPath implementation prior to 2.3.0 was really incomplete, upgrade to + a recent version, the implementation and debug of libxslt generated fixes + for most obvious problems.
+
+ - The example provided in the web page does not compile
+
It's hard to maintain the documentation in sync with the code + <grin/> ...
+Check the previous points 1/ and 2/ raised before, and send + patches.
+
+ - Where can I get more examples and informations than in the web
+ page
+
Ideally a libxml book would be nice. I have no such plan ... But you + can:
+-
+
- check more deeply the existing + generated doc +
- looks for examples of use for libxml function using the Gnome code
+ for example the following will query the full Gnome CVs base for the
+ use of the xmlAddChild() function:
+
http://cvs.gnome.org/lxr/search?string=xmlAddChild
+This may be slow, a large hardware donation to the gnome project + could cure this :-)
+
+ - Browse + the libxml source + , I try to write code as clean and documented as possible, so + looking at it may be helpful +
+ - What about C++ ?
+
libxml is written in pure C in order to allow easy reuse on a number + of platforms, including embedded systems. I don't intend to convert to + C++.
+There is however a C++ wrapper provided by Ari Johnson + <ari@btigate.com> which may fullfill your needs:
+Website: http://lusis.org/~ari/xml++/
+ +
+ - How to validate a document a posteriori ?
+
It is possible to validate documents which had not been validated at + initial parsing time or documents who have been built from scratch using + the API. Use the xmlValidateDtd() + function. It is also possible to simply add a Dtd to an existing + document:
+xmlDocPtr doc; /* your existing document */ + xmlDtdPtr dtd = xmlParseDTD(NULL, filename_of_dtd); /* parse the DTD */ + dtd->name = xmlStrDup((xmlChar*)"root_name"); /* use the given root */ + + doc->intSubset = dtd; + if (doc->children == NULL) xmlAddChild((xmlNodePtr)doc, (xmlNodePtr)dtd); + else xmlAddPrevSibling(doc->children, (xmlNodePtr)dtd); +
+
+ - etc ... +
- General overview +
- The definition +
- Simple rules + + +
- Some examples +
- How to validate +
- Other resources +
- the system string is actually an URI-Reference (as defined in RFC 2396) so you can use a + full URL string indicating the location of your DTD on the Web, this is a + really good thing to do if you want others to validate your document +
- it is also possible to associate a
PUBLICidentifier (a + magic string) so that the DTd is looked up in catalogs on the client side + without having to locate it on the web
+ - a dtd contains a set of elements and attributes declarations, but they
+ don't define what the root of the document should be. This is explicitely
+ told to the parser/validator as the first element of the
+
DOCTYPEdeclaration.
+ - usually the attributes pertaining to a given element are declared in a
+ single expression, but it is just a convention adopted by a lot of DTD
+ writers:
+
<!ATTLIST termdef + id ID #REQUIRED + name CDATA #IMPLIED>
+The previous construct defines both
+idand +nameattributes for the elementtermdef
+ - XML-101 DTD +
- General overview +
- Setting libxml set of memory + routines +
- Cleaning up after parsing +
- Debugging routines +
- General memory requirements +
- libxml does not use the libc memory allocator directly but xmlFree(), + xmlMalloc() and xmlRealloc() +
- those routines can be reallocated to a specific set of routine, by + default the libc ones i.e. free(), malloc() and realloc() +
- the xmlmemory.c module includes a set of debugging routine +
- xmlMemGet () + which return the current set of functions in use by the parser +
- xmlMemSetup() + which allow to set up a new set of memory allocation functions +
- xmlCleanupParser + () + is a centralized routine to free the parsing states. Note that it won't + deallocate any produced tree if any (use the xmlFreeDoc() and related + routines for this). +
- xmlInitParser + () + is the dual routine allowing to preallocate the parsing state which can + be useful for example to avoid initialization reentrancy problems when + using libxml in multithreaded applications +
- xmlMallocLoc() + xmlReallocLoc() + and xmlMemStrdupLoc() + are the memory debugging replacement allocation routines +
- xmlMemoryDump
+ ()
+ dumps all the informations about the allocated memory block lefts in the
+
.memdumpfile
+ - write down the block number xxxx not allocated +
- export the environement variable XML_MEM_BREAKPOINT=xxxx +
- run the program under a debugger and set a breakpoint on + xmlMallocBreakpoint() a specific function called when this precise block + is allocated +
- when the breakpoint is reached you can then do a fine analysis of the + allocation an step to see the condition resulting in the missing + deallocation. +
- the parser itself should work in a fixed amout of memory, except for + information maintained about the stacks of names and entities locations. + The I/O and encoding handlers will probably account for a few KBytes. + This is true for both the XML and HTML parser (though the HTML parser + need more state). +
- If you are generating the DOM tree then memory requirements will grow + nearly lineary with the size of the data. In general for a balanced + textual document the internal memory requirement is about 4 times the + size of the UTF8 serialization of this document (exmple the XML-1.0 + recommendation is a bit more of 150KBytes and takes 650KBytes of main + memory when parsed). Validation will add a amount of memory required for + maintaining the external Dtd state which should be linear with the + complexity of the content model defined by the Dtd +
- If you don't care about the advanced features of libxml like + validation, DOM, XPath or XPointer, but really need to work fixed memory + requirements, then the SAX interface should be used. +
- What does internationalization support + mean ? +
- The internal encoding, how and + why +
- How is it implemented ? +
- Default supported encodings +
- How to extend the existing + support +
- the document is properly parsed +
- informations about it's encoding are saved +
- it can be modified +
- it can be saved in its original encoding +
- it can also be saved in another encoding supported by libxml (for + example straight UTF8 or even an ASCII form) +
- keeping the native encoding in the internal form would force the libxml + users (or the code associated) to be fully aware of the encoding of the + original document, for examples when adding a text node to a document, + the content would have to be provided in the document encoding, i.e. the + client code would have to check it before hand, make sure it's conformant + to the encoding, etc ... Very hard in practice, though in some specific + cases this may make sense. +
- the second decision was which encoding. From the XML spec only UTF8 and
+ UTF16 really makes sense as being the two only encodings for which there
+ is amndatory support. UCS-4 (32 bits fixed size encoding) could be
+ considered an intelligent choice too since it's a direct Unicode mapping
+ support. I selected UTF-8 on the basis of efficiency and compatibility
+ with surrounding software:
+
-
+
- UTF-8 while a bit more complex to convert from/to (i.e. slightly + more costly to import and export CPU wise) is also far more compact + than UTF-16 (and UCS-4) for a majority of the documents I see it used + for right now (RPM RDF catalogs, advogato data, various configuration + file formats, etc.) and the key point for today's computer + architecture is efficient uses of caches. If one nearly double the + memory requirement to store the same amount of data, this will trash + caches (main memory/external caches/internal caches) and my take is + that this harms the system far more than the CPU requirements needed + for the conversion to UTF-8 +
- Most of libxml version 1 users were using it with straight ASCII + most of the time, doing the conversion with an internal encoding + requiring all their code to be rewritten was a serious show-stopper + for using UTF-16 or UCS-4. +
- UTF-8 is being used as the de-facto internal encoding standard for + related code like the pango + upcoming Gnome text widget, and a lot of Unix code (yep another place + where Unix programmer base takes a different approach from Microsoft + - they are using UTF-16) +
+ - xmlChar, the libxml data type is a byte, those bytes must be assembled + as UTF-8 valid strings. The proper way to terminate an xmlChar * string + is simply to append 0 byte, as usual. +
- One just need to make sure that when using chars outside the ASCII set, + the values has been properly converted to UTF-8 +
- when a document is processed, we usually don't know the encoding, a + simple heuristic allows to detect UTF-18 and UCS-4 from whose where the + ASCII range (0-0x7F) maps with ASCII +
- the xml declaration if available is parsed, including the encoding + declaration. At that point, if the autodetected encoding is different + from the one declared a call to xmlSwitchEncoding() is issued. +
- If there is no encoding declaration, then the input has to be in either
+ UTF-8 or UTF-16, if it is not then at some point when processing the
+ input, the converter/checker of UTF-8 form will raise an encoding error.
+ You may end-up with a garbled document, or no document at all ! Example:
+
~/XML -> ./xmllint err.xml +err.xml:1: error: Input is not proper UTF-8, indicate encoding ! +<très>là</très> + ^ +err.xml:1: error: Bytes: 0xE8 0x73 0x3E 0x6C +<très>là</très> + ^
+
+ - xmlSwitchEncoding() does an encoding name lookup, canonalize it, and
+ then search the default registered encoding converters for that encoding.
+ If it's not within the default set and iconv() support has been compiled
+ it, it will ask iconv for such an encoder. If this fails then the parser
+ will report an error and stops processing:
+
~/XML -> ./xmllint err2.xml +err2.xml:1: error: Unsupported encoding UnsupportedEnc +<?xml version="1.0" encoding="UnsupportedEnc"?> + ^
+
+ - From that point the encoder process progressingly the input (it is + plugged as a front-end to the I/O module) for that entity. It captures + and convert on-the-fly the document to be parsed to UTF-8. The parser + itself just does UTF-8 checking of this input and process it + transparently. The only difference is that the encoding information has + been added to the parsing context (more precisely to the input + corresponding to this entity). +
- The result (when using DOM) is an internal form completely in UTF-8 + with just an encoding information on the document node. +
- if no encoding is given, libxml will look for an encoding value
+ associated to the document and if it exists will try to save to that
+ encoding,
+
otherwise everything is written in the internal form, i.e. UTF-8
+
+ - so if an encoding was specified, either at the API level or on the + document, libxml will again canonalize the encoding name, lookup for a + converter in the registered set or through iconv. If not found the + function will return an error code +
- the converter is placed before the I/O buffer layer, as another kind of + buffer, then libxml will simply push the UTF-8 serialization to through + that buffer, which will then progressively be converted and pushed onto + the I/O layer. +
- It is possible that the converter code fails on some input, for example + trying to push an UTF-8 encoded chinese character through the UTF-8 to + ISO-8859-1 converter won't work. Since the encoders are progressive they + will just report the error and the number of bytes converted, at that + point libxml will decode the offending character, remove it from the + buffer and replace it with the associated charRef encoding { and + resume the convertion. This guarante that any document will be saved + without losses (except for markup names where this is not legal, this is + a problem in the current version, in pactice avoid using non-ascci + characters for tags or attributes names @@). A special "ascii" encoding + name is used to save documents to a pure ascii form can be used when + portability is really crucial +
- UTF-8 is supported by default (null handlers) +
- UTF-16, both little and big endian +
- ISO-Latin-1 (ISO-8859-1) covering most western languages +
- ASCII, useful mostly for saving +
- HTML, a specific handler for the conversion of UTF-8 to ASCII with HTML + predefined entities like © for the Copyright sign. +
- int xmlAddEncodingAlias(const char *name, const char *alias); +
- int xmlDelEncodingAlias(const char *alias); +
- const char * xmlGetEncodingAlias(const char *alias); +
- void xmlCleanupEncodingAliases(void); +
- General overview +
- The basic buffer type +
- Input I/O handlers +
- Output I/O handlers +
- The entities loader +
- Example of customized I/O +
- Entities loader, this is a routine which tries to fetch the entities
+ (files) based on their PUBLIC and SYSTEM identifiers. The default loader
+ don't look at the public identifier since libxml do not maintain a
+ catalog. You can redefine you own entity loader by using
+
xmlGetExternalEntityLoader()and +xmlSetExternalEntityLoader(). Check the example.
+ - Input I/O buffers which are a commodity structure used by the parser(s) + input layer to handle fetching the informations to feed the parser. This + provides buffering and is also a placeholder where the encoding + convertors to UTF8 are piggy-backed. +
- Output I/O buffers are similar to the Input ones and fulfill similar + task but when generating a serialization from a tree. +
- A mechanism to register sets of I/O callbacks and associate them with
+ specific naming schemes like the protocol part of the URIs.
+
This affect the default I/O operations and allows to use specific I/O + handlers for certain names.
+
+ - The default entity loader calls
xmlNewInputFromFile()with + the parsing context and the URI string.
+ - the URI string is checked against the existing registered handlers + using their match() callback function, if the HTTP module was compiled + in, it is registered and its match() function will succeeds +
- the open() function of the handler is called and if successful will + return an I/O Input buffer +
- the parser will the start reading from this buffer and progressively + fetch information from the resource, calling the read() function of the + handler until the resource is exhausted +
- if an encoding change is detected it will be installed on the input + buffer, providing buffering and efficient use of the conversion + routines +
- once the parser has finished, the close() function of the handler is + called once and the Input buffer and associed resources are + deallocated. +
- First define a new I/O ouput allocator where the output don't close the
+ file:
+
xmlOutputBufferPtr +xmlOutputBufferCreateOwn(FILE *file, xmlCharEncodingHandlerPtr encoder) { + xmlOutputBufferPtr ret; + + if (xmlOutputCallbackInitialized == 0) + xmlRegisterDefaultOutputCallbacks(); + + if (file == NULL) return(NULL); + ret = xmlAllocOutputBuffer(encoder); + if (ret != NULL) { + ret->context = file; + ret->writecallback = xmlFileWrite; + ret->closecallback = NULL; /* No close callback */ + } + return(ret);+
+ + + +}
+ - And then use it to save the document:
+
FILE *f; +xmlOutputBufferPtr output; +xmlDocPtr doc; +int res; + +f = ... +doc = .... + +output = xmlOutputBufferCreateOwn(f, NULL); +res = xmlSaveFileTo(output, doc, NULL); +
+
+ - General overview +
- The definition +
- Using catalogs +
- Some examples +
- How to tune catalog usage +
- How to debug catalog processing +
- How to create and maintain catalogs +
- The implementor corner quick review of the + API +
- Other resources +
- mapping from "logical" names, the public identifiers and a more
+ concrete name usable for download (and URI). For example it can associate
+ the logical name
+
"-//OASIS//DTD DocBook XML V4.1.2//EN"
+of the DocBook 4.1.2 XML DTD with the actual URL where it can be + downloaded
+http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd
+
+ - remapping from a given URL to another one, like an HTTP indirection
+ saying that
+
"http://www.oasis-open.org/committes/tr.xsl"
+should really be looked at
+"http://www.oasis-open.org/committes/entity/stylesheets/base/tr.xsl"
+
+ - providing a local cache mechanism allowing to load the entities + associated to public identifiers or remote resources, this is a really + important feature for any significant deployment of XML or SGML since it + allows to avoid the aleas and delays associated to fetching remote + resources. +
- the older SGML catalogs, the official spec is SGML Open Technical + Resolution TR9401:1997, but is better understood by reading the SP Catalog page from + James Clark. This is relatively old and not the preferred mode of + operation of libxml. +
- XML + Catalogs + is far more flexible, more recent, uses an XML syntax and should scale + quite better. This is the default option of libxml. +
- the default one, global shared by all the application +
- a per-document catalog, this one is built if the document uses the
+
oasis-xml-catalogPIs to specify its own catalog list, it is + associated to the parser context and destroyed when the parsing context + is destroyed.
+ - You can find an good rant from Norm Walsh about the + need for catalogs, it provides a lot of context informations even if + I don't agree with everything presented. +
- An old XML + catalog proposal from John Cowan +
- The Resource Directory Description + Language (RDDL) another catalog system but more oriented toward + providing metadata for XML namespaces. +
- the page from the OASIS Technical Committee on Entity + Resolution who maintains XML Catalog, you will find pointers to the + specification update, some background and pointers to others tools + providing XML Catalog support +
- I have uploaded a + mall tarball containing XML Catalogs for DocBook 4.1.2 which seems to + work fine for me +
- The xmlcatalog + manual page +
- a general cleanup. A number of mistakes inherited from the very early + versions couldn't be changed due to compatibility constraints. Example + the "childs" element in the nodes. +
- Uniformization of the various nodes, at least for their header and link + parts (doc, parent, children, prev, next), the goal is a simpler + programming model and simplifying the task of the DOM implementors. +
- better conformances to the XML specification, for example version 1.x + had an heuristic to try to detect ignorable white spaces. As a result the + SAX event generated were ignorableWhitespace() while the spec requires + character() in that case. This also mean that a number of DOM node + containing blank text may populate the DOM tree which were not present + before. +
- The package name have changed from libxml to libxml2, the library name + is now -lxml2 . There is a new xml2-config script which should be used to + select the right parameters libxml2 +
- Node childs field has been renamed + children so s/childs/children/g should be applied + (probablility of having "childs" anywere else is close to 0+ +
- The document don't have anymore a root element it has + been replaced by children and usually you will get a + list of element here. For example a Dtd element for the internal subset + and it's declaration may be found in that list, as well as processing + instructions or comments found before or after the document root element. + Use xmlDocGetRootElement(doc) to get the root element of + a document. Alternatively if you are sure to not reference Dtds nor have + PIs or comments before or after the root element + s/->root/->children/g will probably do it. +
- The white space issue, this one is more complex, unless special case of
+ validating parsing, the line breaks and spaces usually used for indenting
+ and formatting the document content becomes significant. So they are
+ reported by SAX and if your using the DOM tree, corresponding nodes are
+ generated. Too approach can be taken:
+
-
+
- lazy one, use the compatibility call + xmlKeepBlanksDefault(0) but be aware that you are + relying on a special (and possibly broken) set of heuristics of + libxml to detect ignorable blanks. Don't complain if it breaks or + make your application not 100% clean w.r.t. to it's input. +
- the Right Way: change you code to accept possibly unsignificant + blanks characters, or have your tree populated with weird blank text + nodes. You can spot them using the comodity function + xmlIsBlankNode(node) returning 1 for such blank + nodes. +
Note also that with the new default the output functions don't add any + extra indentation when saving a tree in order to be able to round trip + (read and save) without inflating the document with extra formatting + chars.
+
+ - The include path has changed to $prefix/libxml/ and the includes
+ themselves uses this new prefix in includes instructions... If you are
+ using (as expected) the
+
xml2-config --cflags
+output to generate you compile commands this will probably work out of + the box
+
+ - xmlDetectCharEncoding takes an extra argument indicating the lenght in + byte of the head of the document available for character detection. +
- similar include naming, one should use + #include<libxml/...> in both cases. +
- similar identifiers defined via macros for the child and root fields: + respectively xmlChildrenNode and + xmlRootNode +
- a new macro LIBXML_TEST_VERSION which should be + inserted once in the client code +
- install the libxml-1.8.8 (and libxml-devel-1.8.8) packages +
- find all occurences where the xmlDoc root field is + used and change it to xmlRootNode +
- similary find all occurences where the xmlNode childs + field is used and change it to xmlChildrenNode +
- add a LIBXML_TEST_VERSION macro somewhere in your + main() or in the library init entry point +
- Recompile, check compatibility, it should still work +
- Change your configure script to look first for xml2-config and fallback + using xml-config . Use the --cflags and --libs ouptut of the command as + the Include and Linking parameters needed to use libxml. +
- install libxml2-2.3.x and libxml2-devel-2.3.x (libxml-1.8.y and + libxml-devel-1.8.y can be kept simultaneously) +
- remove your config.cache, relaunch your configuration mechanism, and + recompile, if steps 2 and 3 were done right it should compile as-is +
- Test that your application is still running correctly, if not this may + be due to extra empty nodes due to formating spaces being kept in libxml2 + contrary to libxml1, in that case insert xmlKeepBlanksDefault(1) in your + code before calling the parser (next to + LIBXML_TEST_VERSION is a fine place). +
- Ari Johnson +
- Bjorn Reese, William Brack and Thomas Broyer have provided a number of + patches, Gary Pennington worked on the validation API, threading support + and Solaris port. +
- John Fleck helps maintaining the documentation and man pages. +
-
provides a C++ wrapper for libxml:
Website: http://lusis.org/~ari/xml++/
@@ -1698,6 +3208,6 @@ Gnome CVS base under gnome-xml/example -$Id: xml.html,v 1.113 2001/10/19 14:50:57 veillard Exp $
+$Id: xml.html,v 1.114 2001/10/24 12:35:52 veillard Exp $
Downloads @@ -27,8 +28,8 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}Main Menu Libxml Internationalization support - - + + +Encodings support - - -Libxml Internationalization support
- -Location: http://xmlsoft.org/encoding.html
- -Libxml home page: http://xmlsoft.org/
- -Mailing-list archive: http://xmlsoft.org/messages/
- -Version: $Revision$
- + +
++ + +
++ The XML C library for Gnome
+Encodings support
+
diff --git a/doc/entities.html b/doc/entities.html index 64ae1fa1..3709136f 100644 --- a/doc/entities.html +++ b/doc/entities.html @@ -8,6 +8,7 @@ BODY {font-size: 10pt; font-family: Verdana,Arial,Helvetica; margin-top: 5pt; ma H1 {font-size: 16pt; font-family: Verdana,Arial,Helvetica} H2 {font-size: 14pt; font-family: Verdana,Arial,Helvetica} H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} +A:link, A:visited, A:active { text-decoration: underline } -->+ + +
+Related links Table of Content:
-
-
What does internationalization support mean ?
- +What does internationalization support mean ?
XML was designed from the start to allow the support of any character set by using Unicode. Any conformant XML parser has to support the UTF-8 and UTF-16 default encodings which can both express the full unicode ranges. UTF8 is a variable length encoding whose greatest point are to resuse the same emcoding for ASCII and to save space for Western encodings, but it is a bit more complex to handle in practice. UTF-16 use 2 bytes per characters (and -sometimes combines two pairs), it makes implementation easier, but looks a bit -overkill for Western languages encoding. Moreover the XML specification allows -document to be encoded in other encodings at the condition that they are -clearly labelled as such. For example the following is a wellformed XML +sometimes combines two pairs), it makes implementation easier, but looks a +bit overkill for Western languages encoding. Moreover the XML specification +allows document to be encoded in other encodings at the condition that they +are clearly labelled as such. For example the following is a wellformed XML document encoded in ISO-8859 1 and using accentuated letter that we French likes for both markup and content:
-<?xml version="1.0" encoding="ISO-8859-1"?> +
<?xml version="1.0" encoding="ISO-8859-1"?> <très>là</très>
-Having internationalization support in libxml means the foolowing:
-
-
Another very important point is that the whole libxml API, with the exception of a few routines to read with a specific encoding or save to a specific encoding, is completely agnostic about the original encoding of the document.
-It should be noted too that the HTML parser embedded in libxml now obbey the same rules too, the following document will be (as of 2.2.2) handled in an internationalized fashion by libxml too:
-<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" - "http://www.w3.org/TR/REC-html40/loose.dtd"> -<html lang="fr"> +
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" + "http://www.w3.org/TR/REC-html40/loose.dtd"> +<html lang="fr"> <head> - <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> </head> <body> <p>W3C crée des standards pour le Web.</body> </html>
- -The internal encoding, how and why
- +The internal encoding, how and why
One of the core decision was to force all documents to be converted to a default internal encoding, and that encoding to be UTF-8, here are the rationale for those choices:
-
-
What does this mean in practice for the libxml user:
-
-
How is it implemented ?
- +How is it implemented ?
Let's describe how all this works within libxml, basically the I18N (internationalization) support get triggered only during I/O operation, i.e. when reading a document or saving one. Let's look first at the reading sequence:
-
-
Ok then what's happen when saving the document (assuming you colllected/built an xmlDoc DOM like structure) ? It depends on the function called, xmlSaveFile() will just try to save in the original encoding, while xmlSaveFileTo() and xmlSaveFileEnc() can optionally save to a given encoding:
-
-
Here is a few examples based on the same test document:
~/XML -> ./xmllint isolat1 -<?xml version="1.0" encoding="ISO-8859-1"?> +<?xml version="1.0" encoding="ISO-8859-1"?> <très>là</très> ~/XML -> ./xmllint --encode UTF-8 isolat1 -<?xml version="1.0" encoding="UTF-8"?> +<?xml version="1.0" encoding="UTF-8"?> <très>là </très> ~/XML ->
-The same processing is applied (and reuse most of the code) for HTML I18N processing. Looking up and modifying the content encoding is a bit more -difficult since it is located in a <meta> tag under the <head>, so -a couple of functions htmlGetMetaEncoding() and htmlSetMetaEncoding() have +difficult since it is located in a <meta> tag under the <head>, +so a couple of functions htmlGetMetaEncoding() and htmlSetMetaEncoding() have been provided. The parser also attempts to switch encoding on the fly when -detecting such a tag on input. Except for that the processing is the same (and -again reuses the same code).
- -Default supported encodings
- -libxml has a set of default converters for the following encodings (located -in encoding.c):
+detecting such a tag on input. Except for that the processing is the same +(and again reuses the same code). +Default supported encodings
+libxml has a set of default converters for the following encodings +(located in encoding.c):
-
-
More over when compiled on an Unix platfor with iconv support the full set -of encodings supported by iconv can be instantly be used by libxml. On a linux -machine with glibc-2.1 the list of supported encodings and aliases fill 3 full -pages, and include UCS-4, the full set of ISO-Latin encodings, and the various -Japanese ones.
- -Encoding aliases
- -From 2.2.3, libxml has support to register encoding names aliases. The goal -is to be able to parse document whose encoding is supported but where the name -differs (for example from the default set of names accepted by iconv). The -following functions allow to register and handle new aliases for existing -encodings. Once registered libxml will automatically lookup the aliases when -handling a document:
+of encodings supported by iconv can be instantly be used by libxml. On a +linux machine with glibc-2.1 the list of supported encodings and aliases fill +3 full pages, and include UCS-4, the full set of ISO-Latin encodings, and the +various Japanese ones. +Encoding aliases
+From 2.2.3, libxml has support to register encoding names aliases. The +goal is to be able to parse document whose encoding is supported but where +the name differs (for example from the default set of names accepted by +iconv). The following functions allow to register and handle new aliases for +existing encodings. Once registered libxml will automatically lookup the +aliases when handling a document:
-
-
How to extend the existing support
- +How to extend the existing support
Well adding support for new encoding, or overriding one of the encoders (assuming it is buggy) should not be hard, just write an input and output conversion routines to/from UTF-8, and register them using @@ -266,23 +299,21 @@ called automatically if the parser(s) encounter such an encoding name (register it uppercase, this will help). The description of the encoders, their arguments and expected return values are described in the encoding.h header.
-A quick note on the topic of subverting the parser to use a different -internal encoding than UTF-8, in some case people will absolutely want to keep -the internal encoding different, I think it's still possible (but the encoding -must be compliant with ASCII on the same subrange) though I didn't tried it. -The key is to override the default conversion routines (by registering null -encoders/decoders for your charsets), and bypass the UTF-8 checking of the -parser by setting the parser context charset (ctxt->charset) to something -different than XML_CHAR_ENCODING_UTF8, but there is no guarantee taht this -will work. You may also have some troubles saving back.
- +internal encoding than UTF-8, in some case people will absolutely want to +keep the internal encoding different, I think it's still possible (but the +encoding must be compliant with ASCII on the same subrange) though I didn't +tried it. The key is to override the default conversion routines (by +registering null encoders/decoders for your charsets), and bypass the UTF-8 +checking of the parser by setting the parser context charset +(ctxt->charset) to something different than XML_CHAR_ENCODING_UTF8, but +there is no guarantee taht this will work. You may also have some troubles +saving back.Basically proper I18N support is important, this requires at least libxml-2.0.0, but a lot of features and corrections are really available only starting 2.2.
- - -$Id$
+Entities or no entities @@ -27,8 +28,8 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}Main Menu A real example @@ -27,8 +28,8 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}Main Menu How to help @@ -27,8 +28,8 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}Main Menu The XML C library for Gnome @@ -27,8 +28,8 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}Main Menu Separate documents:
-
-
The SAX interface @@ -27,8 +28,8 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}Main Menu Introduction @@ -27,8 +28,8 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}Main Menu The XML library interfaces +The parser interfaces @@ -18,7 +19,7 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} The XML C library for Gnome
-The XML library interfaces
+The parser interfaces
@@ -27,8 +28,8 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica} Main Menu Namespaces @@ -27,8 +28,8 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}Main Menu News @@ -27,8 +28,8 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}Main Menu The tree output @@ -27,8 +28,8 @@ H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}Main Menu Upgrading libxml client code from 1.x to 2.x - - + + +Upgrading 1.x code - - -Upgrading libxml client code from 1.x to 2.x
- -Incompatible changes:
- + +
++ + +
++ The XML C library for Gnome
+Upgrading 1.x code
+
diff --git a/doc/valid.html b/doc/valid.html deleted file mode 100644 index 1d61e32e..00000000 --- a/doc/valid.html +++ /dev/null @@ -1,108 +0,0 @@ - - - - - -+ + +
+Related links + Incompatible changes:
Version 2 of libxml is the first version introducing serious backward incompatible changes. The main goals were:
-
-
How to fix libxml-1.x code:
- +How to fix libxml-1.x code:
So client code of libxml designed to run with version 1.x may have to be changed to compile against version 2.x of libxml. Here is a list of changes that I have collected, they may not be sufficient, so in case you find other change which are required, drop me a mail:
-
-
Note also that with the new default the output functions don't add any extra indentation when saving a tree in order to be able to round trip (read and save) without inflating the document with extra formatting chars.
- -Ensuring both libxml-1.x and libxml-2.x compatibility
- +Ensuring both libxml-1.x and libxml-2.x compatibility
Two new version of libxml (1.8.11) and libxml2 (2.3.4) have been released to allow smoth upgrade of existing libxml v1code while retaining compatibility. They offers the following:
-
-
So the roadmap to upgrade your existing libxml applications is the following:
-
-
Following those steps should work. It worked for some of my own code.
-Let me put some emphasis on the fact that there is far more changes from libxml 1.x to 2.x than the ones you may have to patch for. The overall code -has been considerably improved and the conformance to the XML specification -has been drastically improve. Don't take those changes as an excuse to not -upgrade, it may cost a lot on the long term ...
- +has been considerably cleaned up and the conformance to the XML specification +has been drastically improved too. Don't take those changes as an excuse to +not upgrade, it may cost a lot on the long term ... - -$Id: upgrade.html,v 1.9 2001/06/24 12:13:21 veillard Exp $
+Validation, or are you afraid of DTDs ? - - -
-- - -
-- The XML C library for Gnome
-Validation, or are you afraid of DTDs ?
-
- - diff --git a/doc/xml.html b/doc/xml.html index 19f3de84..de3f4043 100644 --- a/doc/xml.html +++ b/doc/xml.html @@ -35,14 +35,6 @@- - -
-Related links - Well what is validation and what is a DTD ?
-Validation is the process of checking a document against a set of -construction rules; a DTD (Document Type Definition) is such -a set of rules.
-The validation process and building DTDs are the two most difficult parts -of the XML life cycle. Briefly a DTD defines all the possibles element to be -found within your document, what is the formal shape of your document tree -(by defining the allowed content of an element, either text, a regular -expression for the allowed list of children, or mixed content i.e. both text -and children). The DTD also defines the allowed attributes for all elements -and the types of the attributes. For more detailed information, I suggest -that you read the related parts of the XML specification, the examples found -under gnome-xml/test/valid/dtd and any of the large number of books available -on XML. The dia example in gnome-xml/test/valid should be both simple and -complete enough to allow you to build your own.
-A word of warning, building a good DTD which will fit the needs of your -application in the long-term is far from trivial; however, the extra level of -quality it can ensure is well worth the price for some sets of applications -or if you already have already a DTD defined for your application field.
-The validation is not completely finished but in a (very IMHO) usable -state. Until a real validation interface is defined the way to do it is to -define and set the xmlDoValidityCheckingDefaultValue -external variable to 1, this will of course be changed at some point:
-extern int xmlDoValidityCheckingDefaultValue;
-...
-xmlDoValidityCheckingDefaultValue = 1;
--
To handle external entities, use the function -xmlSetExternalEntityLoader(xmlExternalEntityLoader f); to -link in you HTTP/FTP/Entities database library to the standard libxml -core.
-@@interfaces@@
- -Separate documents:
-
-
FAQ
+ +Table of Content:
+-
+
Licence(s)
+-
+
Installation
+-
+
Compilation
+-
+
Developer corner
+-
+
Documentation
There are some on-line resources about using libxml:
@@ -909,7 +1172,7 @@ href="http://cvs.gnome.org/lxr/source/libxslt/FEATURES">features supported and the progresses on the Changelog -An overview of libxml architecture
+libxml architecture
Libxml is made of multiple components; some of them are optional, and most of the block interfaces are public. The main components are:
@@ -1051,7 +1314,1170 @@ use of the standard DOM tree build. Note that the DOM tree itself is built by a set of registered default callbacks, without internal specific interface. -The XML library interfaces
+Validation & DTDs
+ +Table of Content:
+-
+
General overview
+ +Well what is validation and what is a DTD ?
+ +DTD is the acronym for Document Type Definition. This is a description of +the content for a familly of XML files. This is part of the XML 1.0 +specification, and alows to describe and check that a given document instance +conforms to a set of rules detailing its structure and content.
+ +Validation is the process of checking a document against a DTD (more +generally against a set of construction rules).
+ +The validation process and building DTDs are the two most difficult parts +of the XML life cycle. Briefly a DTD defines all the possibles element to be +found within your document, what is the formal shape of your document tree +(by defining the allowed content of an element, either text, a regular +expression for the allowed list of children, or mixed content i.e. both text +and children). The DTD also defines the allowed attributes for all elements +and the types of the attributes.
+ +The definition
+ +The W3C XML Recommendation (Tim Bray's annotated version of +Rev1):
+ + +(unfortunately) all this is inherited from the SGML world, the syntax is +ancient...
+ +Simple rules
+ +Writing DTD can be done in multiple ways, the rules to build them if you +need something fixed or something which can evolve over time can be radically +different. Really complex DTD like Docbook ones are flexible but quite harder +to design. I will just focuse on DTDs for a formats with a fixed simple +structure. It is just a set of basic rules, and definitely not exhaustive nor +useable for complex DTD design.
+ +How to reference a DTD from a document:
+ +Assuming the top element of the document is
+ +specand the dtd +is placed in the filemydtdin the subdirectory +dtdsof the directory from where the document were loaded:
+ +<!DOCTYPE spec SYSTEM "dtds/mydtd">Notes:
+-
+
Declaring elements:
+ +The following declares an element
+ +spec:
+ +<!ELEMENT spec (front, body, back?)>it also expresses that the spec element contains one
+ +front, +onebodyand one optionnalbackchildren elements +in this order. The declaration of one element of the structure and its +content are done in a single declaration. Similary the following declares +div1elements:
+ +<!ELEMENT div1 (head, (p | list | note)*, div2*)>means div1 contains one
+ +headthen a series of optional +p,lists andnotes and then an +optionaldiv2. And last but not least an element can contain +text:
+ +<!ELEMENT b (#PCDATA)>
+ +bcontains text or being of mixed content (text and elements +in no particular order):
+ +<!ELEMENT p (#PCDATA|a|ul|b|i|em)*>
+ +pcan contain text ora,ul, +b,ioremelements in no particular +order.Declaring attributes:
+ +again the attributes declaration includes their content definition:
+ +
+ +<!ATTLIST termdef name CDATA #IMPLIED>means that the element
+ +termdefcan have aname+attribute containing text (CDATA) and which is optionnal +(#IMPLIED). The attribute value can also be defined within a +set:
+ +<!ATTLIST list type (bullets|ordered|glossary) +"ordered">means
+ +listelement have atypeattribute with 3 +allowed values "bullets", "ordered" or "glossary" and which default to +"ordered" if the attribute is not explicitely specified.The content type of an attribute can be text (
+ +CDATA), +anchor/reference/references +(ID/IDREF/IDREFS), entity(ies) +(ENTITY/ENTITIES) or name(s) +(NMTOKEN/NMTOKENS). The following defines that a +chapterelement can have an optionalidattribute +of typeID, usable for reference from attribute of type +IDREF:
+ +<!ATTLIST chapter id ID #IMPLIED>The last value of an attribute definition can be
+ +#REQUIRED +meaning that the attribute has to be given,#IMPLIED+meaning that it is optional, or the default value (possibly prefixed by +#FIXEDif it is the only allowed).Notes:
+-
+
Some examples
+ +The directory
+ +test/valid/dtds/in the libxml distribution +contains some complex DTD examples. Thetest/valid/dia.xml+example shows an XML file where the simple DTD is directly included within +the document.How to validate
+ +The simplest is to use the xmllint program comming with libxml. The +
+ +--validoption turn on validation of the files given as input, +for example the following validates a copy of the first revision of the XML +1.0 specification:
+ +xmllint --valid --noout test/valid/REC-xml-19980210.xmlthe -- noout is used to not output the resulting tree.
+ +The
+ +--dtdvalid dtdallows to validate the document(s) against +a given DTD.Libxml exports an API to handle DTDs and validation, check the associated +description.
+ +Other resources
+ +DTDs are as old as SGML. So there may be a number of examples on-line, I +will just list one for now, others pointers welcome:
+-
+
I suggest looking at the examples found under test/valid/dtd and any of +the large number of books available on XML. The dia example in test/valid +should be both simple and complete enough to allow you to build your own.
+ + + +Memory Management
+ +Table of Content:
+-
+
General overview
+ +The module
+xmlmemory.h+provides the interfaces to the libxml memory system:-
+
Setting libxml set of memory routines
+ +It is sometimes useful to not use the default memory allocator, either for +debugging, analysis or to implement a specific behaviour on memory management +(like on embedded systems). Two function calls are available to do so:
+-
+
Of course a call to xmlMemSetup() should probably be done before calling +any other libxml routines (unless you are sure your allocations routines are +compatibles).
+ +Cleaning up after parsing
+ +Libxml is not stateless, there is a few set of memory structures needing +allocation before the parser is fully functionnal (some encoding structures +for example). This also mean that once parsing is finished there is a tiny +amount of memory (a few hundred bytes) which can be recollected if you don't +reuse the parser immediately:
+-
+
Generally xmlCleanupParser() is safe, if needed the state will be rebuild +at the next invocation of parser routines, but be careful of the consequences +in multithreaded applications.
+ +Debugging routines
+ +When configured using --with-mem-debug flag (off by default), libxml uses +a set of memory allocation debugging routineskeeping track of all allocated +blocks and the location in the code where the routine was called. A couple of +other debugging routines allow to dump the memory allocated infos to a file +or call a specific routine when a given block number is allocated:
+-
+
When developping libxml memory debug is enabled, the tests programs call +xmlMemoryDump () and the "make test" regression tests will check for any +memory leak during the full regression test sequence, this helps a lot +ensuring that libxml does not leak memory and bullet proof memory +allocations use (some libc implementations are known to be far too permissive +resulting in major portability problems!).
+ +If the .memdump reports a leak, it displays the allocation function and +also tries to give some informations about the content and structure of the +allocated blocks left. This is sufficient in most cases to find the culprit, +but not always. Assuming the allocation problem is reproductible, it is +possible to find more easilly:
+-
+
I used to use a commercial tool to debug libxml memory problems but after +noticing that it was not detecting memory leaks that simple mechanism was +used and proved extremely efficient until now.
+ +General memory requirements
+ +How much libxml memory require ? It's hard to tell in average it depends +of a number of things:
+-
+
Encodings support
+ +Table of Content:
+-
+
What does internationalization support mean ?
+ +XML was designed from the start to allow the support of any character set +by using Unicode. Any conformant XML parser has to support the UTF-8 and +UTF-16 default encodings which can both express the full unicode ranges. UTF8 +is a variable length encoding whose greatest point are to resuse the same +emcoding for ASCII and to save space for Western encodings, but it is a bit +more complex to handle in practice. UTF-16 use 2 bytes per characters (and +sometimes combines two pairs), it makes implementation easier, but looks a +bit overkill for Western languages encoding. Moreover the XML specification +allows document to be encoded in other encodings at the condition that they +are clearly labelled as such. For example the following is a wellformed XML +document encoded in ISO-8859 1 and using accentuated letter that we French +likes for both markup and content:
+<?xml version="1.0" encoding="ISO-8859-1"?> +<très>là</très>
+ +Having internationalization support in libxml means the foolowing:
+-
+
Another very important point is that the whole libxml API, with the +exception of a few routines to read with a specific encoding or save to a +specific encoding, is completely agnostic about the original encoding of the +document.
+ +It should be noted too that the HTML parser embedded in libxml now obbey +the same rules too, the following document will be (as of 2.2.2) handled in +an internationalized fashion by libxml too:
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" + "http://www.w3.org/TR/REC-html40/loose.dtd"> +<html lang="fr"> +<head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> +</head> +<body> +<p>W3C crée des standards pour le Web.</body> +</html>
+ +The internal encoding, how and why
+ +One of the core decision was to force all documents to be converted to a +default internal encoding, and that encoding to be UTF-8, here are the +rationale for those choices:
+-
+
What does this mean in practice for the libxml user:
+-
+
How is it implemented ?
+ +Let's describe how all this works within libxml, basically the I18N +(internationalization) support get triggered only during I/O operation, i.e. +when reading a document or saving one. Let's look first at the reading +sequence:
+-
+
Ok then what's happen when saving the document (assuming you +colllected/built an xmlDoc DOM like structure) ? It depends on the function +called, xmlSaveFile() will just try to save in the original encoding, while +xmlSaveFileTo() and xmlSaveFileEnc() can optionally save to a given +encoding:
+-
+
Here is a few examples based on the same test document:
+~/XML -> ./xmllint isolat1 +<?xml version="1.0" encoding="ISO-8859-1"?> +<très>là</très> +~/XML -> ./xmllint --encode UTF-8 isolat1 +<?xml version="1.0" encoding="UTF-8"?> +<très>là </très> +~/XML ->
+ +The same processing is applied (and reuse most of the code) for HTML I18N +processing. Looking up and modifying the content encoding is a bit more +difficult since it is located in a <meta> tag under the <head>, +so a couple of functions htmlGetMetaEncoding() and htmlSetMetaEncoding() have +been provided. The parser also attempts to switch encoding on the fly when +detecting such a tag on input. Except for that the processing is the same +(and again reuses the same code).
+ +Default supported encodings
+ +libxml has a set of default converters for the following encodings +(located in encoding.c):
+-
+
More over when compiled on an Unix platfor with iconv support the full set +of encodings supported by iconv can be instantly be used by libxml. On a +linux machine with glibc-2.1 the list of supported encodings and aliases fill +3 full pages, and include UCS-4, the full set of ISO-Latin encodings, and the +various Japanese ones.
+ +Encoding aliases
+ +From 2.2.3, libxml has support to register encoding names aliases. The +goal is to be able to parse document whose encoding is supported but where +the name differs (for example from the default set of names accepted by +iconv). The following functions allow to register and handle new aliases for +existing encodings. Once registered libxml will automatically lookup the +aliases when handling a document:
+-
+
How to extend the existing support
+ +Well adding support for new encoding, or overriding one of the encoders +(assuming it is buggy) should not be hard, just write an input and output +conversion routines to/from UTF-8, and register them using +xmlNewCharEncodingHandler(name, xxxToUTF8, UTF8Toxxx), and they will be +called automatically if the parser(s) encounter such an encoding name +(register it uppercase, this will help). The description of the encoders, +their arguments and expected return values are described in the encoding.h +header.
+ +A quick note on the topic of subverting the parser to use a different +internal encoding than UTF-8, in some case people will absolutely want to +keep the internal encoding different, I think it's still possible (but the +encoding must be compliant with ASCII on the same subrange) though I didn't +tried it. The key is to override the default conversion routines (by +registering null encoders/decoders for your charsets), and bypass the UTF-8 +checking of the parser by setting the parser context charset +(ctxt->charset) to something different than XML_CHAR_ENCODING_UTF8, but +there is no guarantee taht this will work. You may also have some troubles +saving back.
+ +Basically proper I18N support is important, this requires at least +libxml-2.0.0, but a lot of features and corrections are really available only +starting 2.2.
+ +I/O Interfaces
+ +Table of Content:
+-
+
General overview
+ +The module
+xmlIO.hprovides +the interfaces to the libxml I/O system. This consists of 4 main parts:-
+
The general mechanism used when loading http://rpmfind.net/xml.html for +example in the HTML parser is the following:
+-
+
The user defined callbacks are checked first to allow overriding of the +default libxml I/O routines.
+ +The basic buffer type
+ +All the buffer manipulation handling is done using the +
+ +xmlBuffertype define intree.hwhich is a +resizable memory buffer. The buffer allocation strategy can be selected to be +either best-fit or use an exponential doubling one (CPU vs. memory use +tradeoff). The values areXML_BUFFER_ALLOC_EXACTand +XML_BUFFER_ALLOC_DOUBLEIT, and can be set individually or on a +system wide basis usingxmlBufferSetAllocationScheme(). A number +of functions allows to manipulate buffers with names starting with the +xmlBuffer...prefix.Input I/O handlers
+ +An Input I/O handler is a simple structure +
+ +xmlParserInputBuffercontaining a context associated to the +resource (file descriptor, or pointer to a protocol handler), the read() and +close() callbacks to use and an xmlBuffer. And extra xmlBuffer and a charset +encoding handler are also present to support charset conversion when +needed.Output I/O handlers
+ +An Output handler
+ +xmlOutputBufferis completely similar to an +Input one except the callbacks are write() and close().The entities loader
+ +The entity loader resolves requests for new entities and create inputs for +the parser. Creating an input from a filename or an URI string is done +through the xmlNewInputFromFile() routine. The default entity loader do not +handle the PUBLIC identifier associated with an entity (if any). So it just +calls xmlNewInputFromFile() with the SYSTEM identifier (which is mandatory in +XML).
+ +If you want to hook up a catalog mechanism then you simply need to +override the default entity loader, here is an example:
+#include <libxml/xmlIO.h> + +xmlExternalEntityLoader defaultLoader = NULL; + +xmlParserInputPtr +xmlMyExternalEntityLoader(const char *URL, const char *ID, + xmlParserCtxtPtr ctxt) { + xmlParserInputPtr ret; + const char *fileID = NULL; + /* lookup for the fileID depending on ID */ + + ret = xmlNewInputFromFile(ctxt, fileID); + if (ret != NULL) + return(ret); + if (defaultLoader != NULL) + ret = defaultLoader(URL, ID, ctxt); + return(ret); +} + +int main(..) { + ... + + /* + * Install our own entity loader + */ + defaultLoader = xmlGetExternalEntityLoader(); + xmlSetExternalEntityLoader(xmlMyExternalEntityLoader); + + ... +}+ +Example of customized I/O
+ +This example come from a +real use case, xmlDocDump() closes the FILE * passed by the application +and this was a problem. The solution was to redefine a +new output handler with the closing call deactivated:
+-
+
Catalog support
+ +Table of Content:
+-
+
General overview
+ +What is a catalog? Basically it's a lookup mechanism used when an entity +(a file or a remote resource) references another entity. The catalog lookup +is inserted between the moment the reference is recognized by the software +(XML parser, stylesheet processing, or even images referenced for inclusion +in a rendering) and the time where loading that resource is actually +started.
+ +It is basically used for 3 things:
+-
+
The definitions
+ +Libxml, as of 2.4.3 implements 2 kind of catalogs:
+-
+
Using catalog
+ +In a normal environment libxml will by default check the presence of a +catalog in /etc/xml/catalog, and assuming it has been correctly populated, +the processing is completely transparent to the document user. To take a +concrete example, suppose you are authoring a DocBook document, this one +starts with the following DOCTYPE definition:
+<?xml version='1.0'?> +<!DOCTYPE book PUBLIC "-//Norman Walsh//DTD DocBk XML V3.1.4//EN" + "http://nwalsh.com/docbook/xml/3.1.4/db3xml.dtd">
+ +When validating the document with libxml, the catalog will be +automatically consulted to lookup the public identifier "-//Norman Walsh//DTD +DocBk XML V3.1.4//EN" and the system identifier +"http://nwalsh.com/docbook/xml/3.1.4/db3xml.dtd", and if these entities have +been installed on your system and the catalogs actually point to them, libxml +will fetch them from the local disk.
+ +Note: Really don't use this +DOCTYPE example it's a really old version, but is fine as an example.
+ +Libxml will check the catalog each time that it is requested to load an +entity, this includes DTD, external parsed entities, stylesheets, etc ... If +your system is correctly configured all the authoring phase and processing +should use only local files, even if your document stays portable because it +uses the canonical public and system ID, referencing the remote document.
+ +Some examples:
+ +Here is a couple of fragments from XML Catalogs used in libxml early +regression tests in
+test/catalogs:<?xml version="1.0"?> +<!DOCTYPE catalog PUBLIC + "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" + "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd"> +<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"> + <public publicId="-//OASIS//DTD DocBook XML V4.1.2//EN" + uri="http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"/> +...
+ +This is the beginning of a catalog for DocBook 4.1.2, XML Catalogs are +written in XML, there is a specific namespace for catalog elements +"urn:oasis:names:tc:entity:xmlns:xml:catalog". The first entry in this +catalog is a
+publicmapping it allows to associate a Public +Identifier with an URI.... + <rewriteSystem systemIdStartString="http://www.oasis-open.org/docbook/" + rewritePrefix="file:///usr/share/xml/docbook/"/> +...
+ +A
+rewriteSystemis a very powerful instruction, it says that +any URI starting with a given prefix should be looked at another URI +constructed by replacing the prefix with an new one. In effect this acts like +a cache system for a full area of the Web. In practice it is extremely useful +with a file prefix if you have installed a copy of those resources on your +local system.... +<delegatePublic publicIdStartString="-//OASIS//DTD XML Catalog //" + catalog="file:///usr/share/xml/docbook.xml"/> +<delegatePublic publicIdStartString="-//OASIS//ENTITIES DocBook XML" + catalog="file:///usr/share/xml/docbook.xml"/> +<delegatePublic publicIdStartString="-//OASIS//DTD DocBook XML" + catalog="file:///usr/share/xml/docbook.xml"/> +<delegateSystem systemIdStartString="http://www.oasis-open.org/docbook/" + catalog="file:///usr/share/xml/docbook.xml"/> +<delegateURI uriStartString="http://www.oasis-open.org/docbook/" + catalog="file:///usr/share/xml/docbook.xml"/> +...
+ +Delegation is the core features which allows to build a tree of catalogs, +easier to maintain than a single catalog, based on Public Identifier, System +Identifier or URI prefixes it instructs the catalog software to look up +entries in another resource. This feature allow to build hierarchies of +catalogs, the set of entries presented should be sufficient to redirect the +resolution of all DocBook references to the specific catalog in +
+ +/usr/share/xml/docbook.xmlthis one in turn could delegate all +references for DocBook 4.2.1 to a specific catalog installed at the same time +as the DocBook resources on the local machine.How to tune catalog usage:
+ +The user can change the default catalog behaviour by redirecting queries +to its own set of catalogs, this can be done by setting the +
+ +XML_CATALOG_FILESenvironment variable to a list of catalogs, an +empty one should deactivate loading the default/etc/xml/catalog+default catalogHow to debug catalog processing:
+ +Setting up the
+XML_DEBUG_CATALOGenvironment variable will +make libxml output debugging informations for each catalog operations, for +example:orchis:~/XML -> xmllint --memory --noout test/ent2 +warning: failed to load external entity "title.xml" +orchis:~/XML -> export XML_DEBUG_CATALOG= +orchis:~/XML -> xmllint --memory --noout test/ent2 +Failed to parse catalog /etc/xml/catalog +Failed to parse catalog /etc/xml/catalog +warning: failed to load external entity "title.xml" +Catalogs cleanup +orchis:~/XML ->
+ +The test/ent2 references an entity, running the parser from memory makes +the base URI unavailable and the the "title.xml" entity cannot be loaded. +Setting up the debug environment variable allows to detect that an attempt is +made to load the
+ +/etc/xml/catalogbut since it's not present the +resolution fails.But the most advanced way to debug XML catalog processing is to use the +xmlcatalog command shipped with libxml2, it allows to load +catalogs and make resolution queries to see what is going on. This is also +used for the regression tests:
+orchis:~/XML -> ./xmlcatalog test/catalogs/docbook.xml \ + "-//OASIS//DTD DocBook XML V4.1.2//EN" +http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd +orchis:~/XML ->
+ +For debugging what is going on, adding one -v flags increase the verbosity +level to indicate the processing done (adding a second flag also indicate +what elements are recognized at parsing):
+orchis:~/XML -> ./xmlcatalog -v test/catalogs/docbook.xml \ + "-//OASIS//DTD DocBook XML V4.1.2//EN" +Parsing catalog test/catalogs/docbook.xml's content +Found public match -//OASIS//DTD DocBook XML V4.1.2//EN +http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd +Catalogs cleanup +orchis:~/XML ->
+ +A shell interface is also available to debug and process multiple queries +(and for regression tests):
+orchis:~/XML -> ./xmlcatalog -shell test/catalogs/docbook.xml \ + "-//OASIS//DTD DocBook XML V4.1.2//EN" +> help +Commands available: +public PublicID: make a PUBLIC identifier lookup +system SystemID: make a SYSTEM identifier lookup +resolve PublicID SystemID: do a full resolver lookup +add 'type' 'orig' 'replace' : add an entry +del 'values' : remove values +dump: print the current catalog state +debug: increase the verbosity level +quiet: decrease the verbosity level +exit: quit the shell +> public "-//OASIS//DTD DocBook XML V4.1.2//EN" +http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd +> quit +orchis:~/XML ->
+ +This should be sufficient for most debugging purpose, this was actually +used heavily to debug the XML Catalog implementation itself.
+ +How to create and maintain catalogs:
+ +Basically XML Catalogs are XML files, you can either use XML tools to +manage them or use xmlcatalog for this. The basic step is +to create a catalog the -create option provide this facility:
+orchis:~/XML -> ./xmlcatalog --create tst.xml +<?xml version="1.0"?> +<!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" + "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd"> +<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"/> +orchis:~/XML ->
+ +By default xmlcatalog does not overwrite the original catalog and save the +result on the standard output, this can be overridden using the -noout +option. The
+-addcommand allows to add entries in the +catalog:orchis:~/XML -> ./xmlcatalog --noout --create --add "public" \ + "-//OASIS//DTD DocBook XML V4.1.2//EN" \ + http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd tst.xml +orchis:~/XML -> cat tst.xml +<?xml version="1.0"?> +<!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" \ + "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd"> +<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"> +<public publicId="-//OASIS//DTD DocBook XML V4.1.2//EN" + uri="http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"/> +</catalog> +orchis:~/XML ->
+ +The
+ +-addoption will always take 3 parameters even if some of +the XML Catalog constructs (like nextCatalog) will have only a single +argument, just pass a third empty string, it will be ignored.Similarly the
+-deloption remove matching entries from the +catalog:orchis:~/XML -> ./xmlcatalog --del \ + "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd" tst.xml +<?xml version="1.0"?> +<!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" + "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd"> +<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"/> +orchis:~/XML ->
+ +The catalog is now empty. Note that the matching of
+ +-delis +exact and would have worked in a similar fashion with the Public ID +string.This is rudimentary but should be sufficient to manage a not too complex +catalog tree of resources.
+ +The implementor corner quick review of the +API:
+ +First, and like for every other module of libxml, there is an +automatically generated API page for +catalog support.
+ +The header for the catalog interfaces should be included as:
+#include <libxml/catalog.h>
+ +The API is voluntarily kept very simple. First it is not obvious that +applications really need access to it since it is the default behaviour of +libxml (Note: it is possible to completely override libxml default catalog by +using xmlSetExternalEntityLoader to +plug an application specific resolver).
+ +Basically libxml support 2 catalog lists:
+-
+
the document one will be used first if it exists.
+ +Initialization routines:
+ +xmlInitializeCatalog(), xmlLoadCatalog() and xmlLoadCatalogs() should be +used at startup to initialize the catalog, if the catalog should be +initialized with specific values xmlLoadCatalog() or xmlLoadCatalogs() +should be called before xmlInitializeCatalog() which would otherwise do a +default initialization first.
+ +The xmlCatalogAddLocal() call is used by the parser to grow the document +own catalog list if needed.
+ +Preferences setup:
+ +The XML Catalog spec requires the possibility to select default +preferences between public and system delegation, +xmlCatalogSetDefaultPrefer() allows this, xmlCatalogSetDefaults() and +xmlCatalogGetDefaults() allow to control if XML Catalogs resolution should +be forbidden, allowed for global catalog, for document catalog or both, the +default is to allow both.
+ +And of course xmlCatalogSetDebug() allows to generate debug messages +(through the xmlGenericError() mechanism).
+ +Querying routines:
+ +xmlCatalogResolve(), xmlCatalogResolveSystem(), xmlCatalogResolvePublic() +and xmlCatalogResolveURI() are relatively explicit if you read the XML +Catalog specification they correspond to section 7 algorithms, they should +also work if you have loaded an SGML catalog with a simplified semantic.
+ +xmlCatalogLocalResolve() and xmlCatalogLocalResolveURI() are the same but +operate on the document catalog list
+ +Cleanup and Miscellaneous:
+ +xmlCatalogCleanup() free-up the global catalog, xmlCatalogFreeLocal() is +the per-document equivalent.
+ +xmlCatalogAdd() and xmlCatalogRemove() are used to dynamically modify the +first catalog in the global list, and xmlCatalogDump() allows to dump a +catalog state, those routines are primarily designed for xmlcatalog, I'm not +sure that exposing more complex interfaces (like navigation ones) would be +really useful.
+ +The xmlParseCatalogFile() is a function used to load XML Catalog files, +it's similar as xmlParseFile() except it bypass all catalog lookups, it's +provided because this functionality may be useful for client tools.
+ +threaded environments:
+ +Since the catalog tree is built progressively, some care has been taken to +try to avoid troubles in multithreaded environments. The code is now thread +safe assuming that the libxml library has been compiled with threads +support.
+ + + +Other resources
+ +The XML Catalog specification is relatively recent so there isn't much +literature to point at:
+-
+
If you have suggestions for corrections or additions, simply contact +me:
+ +The parser interfaces
This section is directly intended to help programmers getting bootstrapped using the XML library from the C language. It is not intended to be @@ -1400,50 +2826,130 @@ from multiple DTDs will certainly break current validation schemes. I will try to provide ways to do this, but this may not be portable or standardized.
-Validation, or are you afraid of DTDs ?
+Upgrading 1.x code
-Well what is validation and what is a DTD ?
+Incompatible changes:
-Validation is the process of checking a document against a set of -construction rules; a DTD (Document Type Definition) is such -a set of rules.
+Version 2 of libxml is the first version introducing serious backward +incompatible changes. The main goals were:
+-
+
The validation process and building DTDs are the two most difficult parts -of the XML life cycle. Briefly a DTD defines all the possibles element to be -found within your document, what is the formal shape of your document tree -(by defining the allowed content of an element, either text, a regular -expression for the allowed list of children, or mixed content i.e. both text -and children). The DTD also defines the allowed attributes for all elements -and the types of the attributes. For more detailed information, I suggest -that you read the related parts of the XML specification, the examples found -under gnome-xml/test/valid/dtd and any of the large number of books available -on XML. The dia example in gnome-xml/test/valid should be both simple and -complete enough to allow you to build your own.
+How to fix libxml-1.x code:
-A word of warning, building a good DTD which will fit the needs of your -application in the long-term is far from trivial; however, the extra level of -quality it can ensure is well worth the price for some sets of applications -or if you already have already a DTD defined for your application field.
+So client code of libxml designed to run with version 1.x may have to be +changed to compile against version 2.x of libxml. Here is a list of changes +that I have collected, they may not be sufficient, so in case you find other +change which are required, drop me a +mail:
+-
+
The validation is not completely finished but in a (very IMHO) usable -state. Until a real validation interface is defined the way to do it is to -define and set the xmlDoValidityCheckingDefaultValue -external variable to 1, this will of course be changed at some point:
+Ensuring both libxml-1.x and libxml-2.x compatibility
-extern int xmlDoValidityCheckingDefaultValue;
+Two new version of libxml (1.8.11) and libxml2 (2.3.4) have been released +to allow smoth upgrade of existing libxml v1code while retaining +compatibility. They offers the following:
+-
+
...
+So the roadmap to upgrade your existing libxml applications is the +following:
+-
+
xmlDoValidityCheckingDefaultValue = 1;
+Following those steps should work. It worked for some of my own code.
- - -To handle external entities, use the function -xmlSetExternalEntityLoader(xmlExternalEntityLoader f); to -link in you HTTP/FTP/Entities database library to the standard libxml -core.
- -@@interfaces@@
+Let me put some emphasis on the fact that there is far more changes from +libxml 1.x to 2.x than the ones you may have to patch for. The overall code +has been considerably cleaned up and the conformance to the XML specification +has been drastically improved too. Don't take those changes as an excuse to +not upgrade, it may cost a lot on the long term ...
DOM Principles
@@ -1659,7 +3165,11 @@ Gnome CVS base under gnome-xml/exampleContributions
-
-
Libxml Frequently Asked Questions
- -Location: http://xmlsoft.org/FAQ.html
- -Libxml home page: http://xmlsoft.org/
- -Mailing-list archive: http://xmlsoft.org/messages/
- -Version: $Revision$
- + +|
+
+ |
+
|
+
|
|
|
|