3 ms·
Many symbols being longer only matters if they are common (and your encoding method doesn't allow you to leave affected sequences plain to avoid increasing thei
by dspillett 12y ago
Many symbols being longer only matters if they are common (and your encoding method doesn't allow you to leave affected sequences plain to avoid increasing their size), and if, moving away from UFT8-a-like encodings which might have been a bad example, you allow context beyond one symbol you can break them into pages making symbols after a page selection smaller (though the next one that requires a new page to be selected will necessarily be larger).
There are various ways you could arrange the symbol encoding and how deal with those parts of the input that can't be encoded effectively (where the encoding would increase size overall instead of decreasing it). I'm pretty sure it would work, though whether it would be truly practical (in terms of the return from making the effort) rather than just being an interesting little thought experiment is definitely a valid discussion point! The idea of a collection of preset initial dictionaries for LZ77 and similar feels much more practical than the single large reference dictionary for some other scheme.