3 ms·
It's closer to 11% with the large model. I assume it would be the hutter prize leader if the prizes' challenge were large enough to amortize the model.
by nullc 6y ago
It's closer to 11% with the large model. I assume it would be the hutter prize leader if the prizes' challenge were large enough to amortize the model.
- CJefferson 6y agoThe entire of wikipedia's English text is only about 15GB, and GPT-2 is 5GB -- so it couldn't "win". Also, other techniques might get better as size increases.
- xamuel 6y agoEven worse, the Hutter challenge is based on just a 1GB fragment of (one particular snapshot of) English Wikipedia. Further highlighting what has been said elsewhere in this thread, that as far as compression goes this is basically just a variation on compression-via-giant-dictionary
- nullc 6y agoThus the "if the prizes' challenge were large enough". :)