2 ms·
It does seem as though the training cost of these models ought to be included in their times, because the compressors are useless for any purpose other than com
by jeffbee 2y ago
It does seem as though the training cost of these models ought to be included in their times, because the compressors are useless for any purpose other than compressing English Wikipedia.
- gwern 2y agoIn these cases, the models are so small, and have to run on very limited CPU, that the training time is going to be fairly negligible. The top-listed program, nncp, is just 0.06b parameters! https://bellard.org/nncp/nncp_v2.pdf#page=4 https://bellard.org/nncp/nncp_v2.pdf#page=4
- gliptic 2y agoThe training cost is included because these NN compressors are running the training while they are compressing. They are general compressors, although the specific variants submitted here may be tuned a little to Wikipedia (e.g. the pre-processors).