SysconsUnicodeProject

For any reader not familiar with Unicode, reading SysconsUnicodeProject/UnicodeBasics is suggested.

This project aims to enhance Unicode support on FreeBSD operating system by implementing support for UTF-8 in the Syscons console driver. This is not about full support for Unicode-based applications (because has no impact on correctness of sorting text strings, for example). Currently system console driver deals with 8-bit character sets only.

News

TODO

Description

Assigned to

Status

Make terminal emulator UTF-8 aware

Ed

Committed (01/01/2009)

Implement unicode-to-8-bit mapper for existing fonts

Alexander

Implemented (29/01/2009)

UTF-8 aware input layer, keymaps.

Ed

Semi-done

UTF-8 aware font renderer

Ed

TBD

Unicode fonts, not limited to 256 chars

Alexander

TBD, not scheduled

Terminal emulator

The old terminal renderer used by Syscons is not suitable to use with UTF-8. Right now we use a terminal type called cons25. The disadvantages of cons25:

This is why we should migrate to xterm at the same time. This adds a little more complexity to the terminal emulator (old VT100 character sets, keypad application mode, etc), but has its advantages.

As of 01/01/2009, the old Syscons emulator has been replaced by libteken. libteken is still configured to be a cons25 emulator, but already supports a lot more escape sequences. This means you should already be able to SSH to your Cisco/HP switch without problems.

If you want to try the new xterm + UTF-8 support, recompile your kernel with TEKEN_XTERM and TEKEN_UTF8 in your kernel configuration file. All non-ASCII characters will still be displayed as a yellow/red question mark. Be sure to replace cons25 with xterm in /etc/ttys!

Input layer

Right now the input layer only has 8-bit keymaps. This should be changed to Unicode. XXX: Extend.

Font rendering

Because Syscons (and the hardware!) is restricted to 8-bit fontmaps, we should add a Unicode-to-8-bit fontmap conversion. An advantage is that we should be able to display exactly 256 different characters now, instead of 240-250 (not sure). We could even map multiple Unicode characters to the same graphical character (German Eszett and Beta?).

We could consider writing only a very small amount of fonts and some kind of file format, describing which characters should be included in the 8-bit map. This means we can generate a lot of fontmaps. I guess it's a little more tricky to add support for full width characters, because these should consume two graphical characters. Maybe we can save some space by just rendering full-width ASCII characters as regular characters, followed by a space?

This is still very premature. WIP. TBD.

Integration

In order to make integration into the operating system as easy as possible, we should at least make UTF-8 support a compile-time switch. Users/developers may still want to migrate their key/fontmaps to the new format; POLA.

Testing

To verify that the project has reached its goals and met all requirements performing the following is necessary:

SysconsUnicodeProject (last edited 2009-08-04 16:34:19 by EdSchouten)