2 ms·
Whenever new compression algs come up I can't help but think they're cheating with their dictionaries by pre-defining most common words and character sequences,
by devwastaken 7y ago
Whenever new compression algs come up I can't help but think they're cheating with their dictionaries by pre-defining most common words and character sequences, such as html tags. I would wonder if it's more ideal for human language to simply have the best dictionary. Perhaps even a negotiation on both ends to define what that dictionary is. If browsers had an efficient dictionary of all English words and phrases, compression would seem to tiny.
- terrelln 7y agoIn this competition, and in similar competitions, the size of the binary used to decompress is taken into account. If you wanted to use a dictionary, you would need to pay for it in binary size. In this competition, the file must be self-decompressing. Dictionaries are powerful tools when compressing small data. But once the data is large enough they stop mattering so much. See the dictionary compression section of https://engineering.fb.com/core-data/zstandard/ https://engineering.fb.com/core-data/zstandard/.