Data encoding compatibility problems are one of the most common
difficulties encountered by programmers new to XML in
general and libxml in particular. Thinking
through the design of your application in light of this issue will help
avoid difficulties later. Internally, libxml
stores and manipulates data in the UTF-8 format. Data used by your program
in other formats, such as the commonly used ISO-8859-1 encoding, must be
converted to UTF-8 before passing it to libxml
functions. If you want your program's output in an encoding other than
UTF-8, you also must convert it.
Libxml uses
iconv if it is available to convert
data. Without iconv, only UTF-8, UTF-16 and
ISO-8859-1 can be used as external formats. With
iconv, any format can be used provided
iconv is able to convert it to and from
UTF-8. Currently iconv supports about 150
different character formats with ability to convert from any to any. While
the actual number of supported formats varies between implementations, every
iconv implementation is almost guaranteed to
support every format anyone has ever heard of.
| Warning |
---|
A common mistake is to use different formats for the internal data
in different parts of one's code. The most common case is an application
that assumes ISO-8859-1 to be the internal data format, combined with
libxml, which assumes UTF-8 to be the
internal data format. The result is an application that treats internal
data differently, depending on which code section is executing. The one or
the other part of code will then, naturally, misinterpret the data.
|
This example constructs a simple document, then adds content provided
at the command line to the document's root element and outputs the results
to stdout in the proper encoding. For this example, we
use ISO-8859-1 encoding. The encoding of the string input at the command
line is converted from ISO-8859-1 to UTF-8. Full code: AppendixH, Code for Encoding Conversion Example
The conversion, encapsulated in the example code in the
convert function, uses
libxml's
xmlFindCharEncodingHandler function:
xmlCharEncodingHandlerPtr handler;
size = (int)strlen(in)+1;
out_size = size*2-1;
out = malloc((size_t)out_size);
…
handler = xmlFindCharEncodingHandler(encoding);
…
handler->input(out, &out_size, in, &temp);
…
xmlSaveFormatFileEnc("-", doc, encoding, 1);
| handler is declared as a pointer to an
xmlCharEncodingHandler function. |
| The xmlCharEncodingHandler function needs
to be given the size of the input and output strings, which are
calculated here for strings in and
out. |
| xmlFindCharEncodingHandler takes as its
argument the data's initial encoding and searches
libxml's built-in set of conversion
handlers, returning a pointer to the function or NULL if none is
found. |
| The conversion function identified by handler
requires as its arguments pointers to the input and output strings,
along with the length of each. The lengths must be determined
separately by the application. |
| To output in a specified encoding rather than UTF-8, we use
xmlSaveFormatFileEnc, specifying the
encoding. |