TynanSigg

3 ms·

1. 1.▲
   by TynanSigg 5y ago

   Those two metrics will converge as the size of the text being compressed goes to infinity. It's necessary to include model size for things like the Hutter prize that involve compressing a fixed text (to avoid hard-coding) but isn'
2. 2.▲
   by TynanSigg 5y ago

   After a bit more digging, it seems Bellard also has a version using GPT-2: https://bellard.org/libnc/gpt2tc.html Looks much cleaner and faster than my little winter break project, cool to see this implemented properly!
3. 3.▲
   by TynanSigg 5y ago

   Oh hey I did something like this a while ago, but much less polished. I basically made an rANS encoder and hooked it up to GPT-2 (via the excellent transformers library from HuggingFace) to see what compression ratio I could get. Bellard&#x