4 ms·
This could be circumvented by _training_ the LLM on the fly on the previously observed file data. This is what Bellard's other NN compressor, nncp, does [1], wh
by gliptic 2y ago
This could be circumvented by _training_ the LLM on the fly on the previously observed file data. This is what Bellard's other NN compressor, nncp, does [1], which is currently #1 on Mahoney's benchmark [2]. Unfortunately this is too slow, especially running on the CPU as Hutter's challenge stipulates IIRC.
[1] https://bellard.org/nncp/ https://bellard.org/nncp/
[2] http://mattmahoney.net/dc/text.html http://mattmahoney.net/dc/text.html
- lifthrasiir 2y agoIn fact, pretty much every adaptive compression algorithm does. The eventual compression ratio would thus be determined by the algorithm (nncp, cmix, ...; also includes smaller tweaks like those typically made by the Hutter Prize winners) and its hyperparameters.
- gliptic 2y agoYes, the only exception is dictionaries used in preprocessing, but I think that's mostly a tradeoff to reduce the runtime.