• GáborSoC2009

BSD-licensed libiconv in base system

Status

The work is mostly completed. The initial code has been obtained from NetBSD's implementation, which was part of the http://citrus.bsdclub.org/. The following items has been completed:

• The Citrus implementation has been ported from NetBSD.
• Some utilities have been added, e.g. there is a conversion table generator, which can compare conversion tables to reference data generated by GNU libiconv. This helps ensuring conversion compatibility.
• UTF-16 surrogate support and some endianness issues have been fixed.
• The rather chaotic Makefiles to build metadata have been refactored and cleaned up, now it is easy to read and it is also easier to add support for new encodings.
• A bunch of new encodings and encoding aliases have been added.

• Support for 1->2, 1->3 and 1->4 mappings, which is needed for transliterating with flying accents as GNU does, like "u. Before this change, only simple mappings were possible, where one character is mapped to exactly one character.

• Lots of warnings have been fixed, the major part of the code is now WARNS=6 clean.
• New section 1 and section 5 manual pages have been added.
• Some GNU-specific calls have been implemented: iconvlist(), iconvctl(), iconv_canonicalize(), iconv_open_into()
• Support for GNU's //IGNORE suffix has been added.
• The "-" argument for stdin is now recognized in iconv(1) as per POSIX.
• The Big5 conversion module has been fixed.
• The iconv.h header files is supposed to be compatible with the GNU version, i.e. sources should build with base iconv.h and GNU libiconv.
• Various cleanups and style(9) fixes.
• Added support for "" and "char" to iconv_open() as aliases for the local encoding.
• GNU violates POSIX by declaring the inbuf parameter as "const char **" instead of "char **". We want POSIX conformance but still want to have compatibility with GNU. This has been solved with some macro magic.

• A http://www.kovesdan.org/files/bsc_iconv.pdf (bachelor thesis) written in Hungarian language

TODO

First phase

Getting BSD iconv into the base system while still letting ports link to GNU libiconv by default.

Second phase

Evaluating if the compatibility level is high enough to use BSD iconv for ports instead of GNU in the following manner. USE_ICONV could be set to yes, gnu or libc. In the first case the port declares that it accepts both implementations and the user could indicate his preference by setting the WANT_ICONV knob either to gnu or to libc. The default would be libc if BSD iconv proves good enough for wide usage. The latter two cases would indicate that the port requires the specified implementation.