3 ms·
Huffman coding [1] and all derived algorithms do. They find the most common "words" and assign them shorter codes. Language models, among which GPT-2, do essen
by blackbear_ 6y ago
Huffman coding [1] and all derived algorithms do. They find the most common "words" and assign them shorter codes.
Language models, among which GPT-2, do essentially the same: they learn what is likely to appear in the text, albeit in a much more elaborated way.
[1] https://en.wikipedia.org/wiki/Huffman_coding https://en.wikipedia.org/wiki/Huffman_coding
- seanwilson 6y agoYep, but you have to send the created dictionary to the client. I'm asking if there's compression setups that rely on the client already having a massive dictionary downloaded that's reused to decompress downloads from different sources.