Irssi uses three stages of representation.

256 colours are supported with %x/%X and %z/%Z escape codes from themes or scripts. Formats, Themes and Scripts are the only places that use %-codes.

A demo /CUBES alias is provided to review the %X-colours. Those correspond 1:1 to the xterm 256 colour palette

The %-codes are converted to ^D Codes near immediately. This is the internal control code representation that can also be used in /echo and over IRC. They cannot contain 0-byte (C string terminator) and must be wary of other control codes such as \r or \n (line break)

The TextBuffer uses a more dense representation where formatting commands are introduced by \0 (0-byte)

the parser has to extend for allowing %-codes with arguments. to do this, it has to be altered to return the length everywhere so that the reading pointer can advance as necessary.

the original ^D encoding for colours uses 3 bytes. ^D + foreground + background. Due to the limitation of line protocol, we cannot encode 256 colours by (char)colour: it leads to \0 and \n problems. The encoding is altered to break the 256 colour space into 4 planes: original 16 colour (compatible with original encoding), plane 1-3. Encoding becomes: ^D + plane/is_bg_marker + colour, colour is representable as non-control code ASCII. (c.f format_ext_color)

advantage: the message length remains 3 bytes.

to implement true colour encoding, altering the message length is unavoidable. to save space (bytes) we ignore that it becomes invalid utf8. (invalid utf8 was in use already before, to signal "default" colour.)

The encoding must extend to 6 bytes: ^D + # + r + g + b + remainder. We face same difficulty as above that we cannot use (char)r/g/b to format the colour, so we have to make sure it starts above control codes. Those are marked in the remainder part. (cf. format_24bit_color)

the ANSI parser has to be modified and turns the ANSI 256colour codes into the appropriate signals. the gui_printtext signal alread has (int) bg and fg so no change is necessary here. However, additional flag has to be introduced to differentiate between 24bit or indexed colour. (see GUI_PRINT)

(the missing formatting bits of the ANSI parser are corrected in the course. italics is another topic that should be integrated throughout Irssi)

the signals to print text have to turn ^D encoding into fg, bg, and flags. for flags, if 24bit is supported, additional 24bit flags are set, thus making fg and bg into 2 different meanings.

in %codes, it is %N (this is actually a full reset) in ^D it is (char)0xff, can reset fg or bg separately in Signalling, it is -1 (24bit flag: off)

GUI_PRINT flags are then converted into the TextBuffer encoding.

TextBuffer stores all control codes as "commands", encoding is \0 + command(1 byte) + command args (optional), these are interleaved in the text. It covers some of ^D encoding and has special commands not possible with ^D: copy raw memory pointer continuation

there are separate commands for the colours: 0x01 .. 0x0f for the foreground colours and LINE_COLOR_BG + 0x00 ... for the foreground colours. The Line Commands of the TextBuffer encoding are converted into attrs which are shared with the Terminal encoding

To allow extended colours, command args had to be introduced. This becomes important by TextBuffer chunking! If the codes are separated in the wrong places, illegal memory segments are read and corruption or worse occurs.

Extended codes are LINE_COLOR_EXT(_BG) followed by (char)colour. This time it's Ok, we're not going to produce \0 and also not 0x01..0x0f, thus \n isn't a problem either.

For 24bit, we use LINE_COLOR_24 and the same encoding as for ^D because why invent yet another thing. we'd need one escape bit anyway

The TextBuffer is allocating parts memory and stores continuation memory pointers in-line with the LINE_CMD_CONTINUE. This has implications for multi-byte encodings such as but not limited to utf8, as if you break characters apart and render them to screen independently that will break (see the bug tracker report + patch). c.f text_chunk_append

For the convenience of buf.pl, there is a line2text function implemented in C source that does the reverse encoding of LINE_CMDs into ^D codes.

For support of 24bit colour, attrs storage is not enough, so the text fragment storage must increase by additional fg24 and bg24 storages.

the TextBuffer View then renders the Text Fragment storage onto the terminal by passing the ATTR-Encoding to term_set_color(2). Attrs are a packed view of Reset, Underline, .... The term_set_color2 variant code is the extension to take additional fg24/bg24 parameters that are used only in 24bit mode

attr is altered by the 256colour evolution: Old representation was 4bit for each colour: 0xAABF (A=attrs, B=bg, F=fg), now we extend to 0xAABBFF

Curses uses pair allocation and thus isn't really very amicable to many colours. For now, curses support is NOT implemented. Even full 256 colour support would not be possible with curses, as #pairs is limited to 32768 (but 256**2==65536...)

yup, this one works. 24bit is of course not implemented in terminfo (it's the same underlying database for curses) so it is hard-coded escape in the terminfo driver.

The last_fg/last_bg are ints. They remembered whether the colour is 24bit or indexed by this convention: 0xRRGGBBII (II=indexed)

there are 2 down-conversions: 24bit->256colour. This one is implemented by calculating nearest colours (c.f color_24bit_256). It is called depends on whether we support 24bit in the compilation: either at the %-codes parsing (then only 256colour is stored) or at display time when 24bit setting is turned off (user option)

256colour->indexed: this is a lookup table (term_color256map). It is invoked on display time at term_set_fg/bg when terminfo shows that colour number is more than terminal understands (c.f color256)

IMPORTANT: This means you need to use correct TERM, e.g screen-256color or xterm-256color

themes implement special handling for %n: it's a short-cut that expands to: %n, %(last seen colour here). Only the theme loader does this! The core treats %n and %N the same.

Thus, themes need to have special knowledge how colour codes look, and remember them in a variable. If colour codes are extended (as above), a simple char isn't enough any more. Because C sucks, we have to hide a char[8] inside theme_rm_col. Then, all places of remembering are replaced from char to theme_rm_col, and additional logic to detect (c.f chr_is_valid_...) and remember %x/%z is added

all parts that use themes are affected by this, this means the Python module and Perl module where theme_format_expand is wrapped

because we could keep the message length same for extended colours, it doesn't need to modify here. Exception is for 24bit colours, this needs to be advanced properly.

And before you ask, /hilight IS broken (see the bug report), its approach to storing the "last" colour code is a bit too simplistic... (hilight uses strip_real_length which has been hacked to remember the last seen colour code, but doesn't take into account that this might be a mIRC code, or only specify background or only foreground)