I have entertained ideas like this every since becoming familiar with the original RFC2279 UTF-8 well over two decades ago, before UTF-8 was subsequently restri

I have entertained ideas like this every since becoming familiar with the original RFC2279 UTF-8 well over two decades ago, before UTF-8 was subsequently restricted to 21 bits. Another idea that would work is to start an encoding with 0b10xxxxxx have subsequent 0b10xxxxxx bytes continue the encoding, and end it with 11xxxxxx. 0b1000_0000 (i.e. 128) is not allowed for the first byte of this sequence, and we add 127 to the resulting number (so each code point has only one representation) This way, the ♥ symbol (that’s 0x2665 or 9829 in decimal) would be represented as follows: ♥ → 0b0010_0110_0110_0101 → 0b0010_0101_1110_0110 (subtract 127) → 0b10_00_0010 0b10_0101_11 0b11_10_0110 This allows 18 bits to be encoded in three bytes, and, like UTF-8000, allows arbitrarily long sequences. In the real world, the last time I needed to have a custom encoding, I went the other way and converted Unicode in to a 7-bit ASCII compatible encoding, where most ASCII control characters were converted into glyphs, as follows: 0123456789abcdef 0 .ÁÉÍÑÓÚÜ¡..—..«» 1 •áéíñóúü¿‘’.→“”© 2 .!"#$%&'()*+,-./ 3 0123456789:;<=>? 4 @ABCDEFGHIJKLMNO 5 PQRSTUVWXYZ[\]^_ 6 `abcdefghijklmno 7 pqrstuvwxyz{|}~♥ The reason for this encoding is that it allows me to write stuff in languages I am actually fluent in (English and Spanish), and adds a handful of useful non-ASCII punctuation (smart quotes, etc.). “.” represents a control character here; only “null” (0x00), “line feed” (0x0a), “form feed” (0x0c), “tab” (0x09), “carriage return” (0x0d), and “escape” (0x1b) are control characters; the rest are printable glyphs. I used this character set for my blog, since my blog is processed using HTMLDOC (which I learned the hard way is buggy with full Unicode) and my own Lua script. With Lua, Unicode regexes are not readily supported because its regular expression engine assumes a codepoint is only one byte long when forming character classes such as %u for upper case letters. My Lua code has character classes for the non-ASCII glyphs such as %t for all letters in this custom encoding. I have made this encoding 7 bits because that gives me the option to use the eight bit for future expansion. The real reason for so few glyphs is because, in addition to having things work nicely with code which assumes 8-bit codepoints, with the modern web, I need to send to the viewer the font a page will be rendered with (it would had been nice if Microsoft had open sourced Verdana, Georgia, and the rest of the core fonts, and had those fonts became a part of the browser standards, so we could have cross-platform font stacks, but oh well) [1], so I aggressively subset the font to minimize the page load time. [1] I still wish for the day when Apple, Google, Microsoft, and Mozilla come together, decide to include “Noto” and “Noto Serif” with their browsers, so “font-family: Noto (Serif)” always renders the exact same font without having to download a font over the network when loading a web page.