﻿UNICODE
=======

Log4cplus uses the expression "UNICODE" in at least two not so equal
meanings:

1. the [Unicode][unicode] standard as defined by the Unicode Consortium

2. compiler's and/or C++ standard library's support for strings of
   `wchar_t`s and their manipulation

[unicode]: http://unicode.org/


`wchar_t` support
-----------------

Log4cplus is aimed to be portable and to have as little 3rd party
dependencies as possible. To fulfill this goal it has to use
facilities offered by the operating systems and standard libraries it
runs on. To offer the best possible level of support of national
character, it has to support usage of `wchar_t` and it has to use
`wchar_t` support (especially on Windows) provided by operating system
and standard C and C++ libraries.

This approach to portability has some limitations. One of the
limitations is lacking support for C++ locales in various operating
systems and standard C++ libraries. Some standard C++ libraries do not
support other than the "C" and "POSIX" locales. This usually means
that `wchar_t`↔`char` conversion using `std::codecvt<>` facet is
impossible. On such deficient platforms, log4cplus can use either
standard C locale support or `iconv()` (through libiconv or
built--in).


Unicode and file appenders
--------------------------

Another limitation related to Unicode support is the need to convert
between the application's character type and the file's external byte
encoding when writing log files using `FileAppender`. On Unix--like
systems this usually means choosing a suitable locale for the standard
C++ file stream. On Windows, file-based appenders use log4cplus'
Win32-backed file stream and the external file encoding is UTF-8.


Unix--like platforms
--------------------

To support output of non-ASCII characters in `wchar_t` message on
Unix--like platforms, it is necessary to use UTF-8 based locale (e.g.,
`en_US.UTF-8`) and to set up global locale with `std::codecvt<>` facet
or imbue individual `FileAppender`s with that facet. The following
code can be used to get such `std::locale` instance and to set it into
global locale:

~~~~{.cpp}
std::locale::global (     // set global locale
    std::locale (         // using std::locale constructed from
        std::locale (),   // global locale
                          // and codecvt facet from user locale
        new std::codecvt_byname<wchar_t, char, std::mbstate_t>("")));
~~~~


Windows
-------

Windows do not support UTF-8 based locales. With the standard C++ file
streams this used to mean that a `std::locale` instance commonly
converted `wchar_t`s to the current process' code page, and code points
outside that code page could not be written by `FileAppender` without a
custom `std::codecvt_utf8` facet.

File-based appenders no longer use the standard C++ file streams on
Windows. They use log4cplus' Win32-backed file stream, which stores log
files as UTF-8. In UNICODE builds, `wchar_t` log messages are converted
to UTF-8 directly. In non-UNICODE builds, narrow `char` log messages are
interpreted using the stream locale's Windows code page and are then
stored as UTF-8. The `TextMode` property controls text-mode newline and
BOM handling, but the file contents remain UTF-8.
