It's fair because the model itself is included in the compressed size. You're not trying to extrapolate predictions. It makes perfect sense for compression "at

It's fair because the model itself is included in the compressed size. You're not trying to extrapolate predictions. It makes perfect sense for compression "at rest" applications to intentionally overfit on the data you're trying to compress if that gives a smaller total size(model ++ compressed text). If you were going to standardize on a foundational model that gets delivered to billions of devices to be used in network transfers (where you just transfer the compressed text and assume the other side already has the model), then it makes sense to optimize for generalizability. In that case it would make sense to exclude the model itself from the compressed size. Basically like how brotli gets to "cheat" by including a 120KB pre-defined dictionary. Edit: Actually that can't be right. The numbers on https://www.mattmahoney.net/dc/text.html https://www.mattmahoney.net/dc/text.html use that methodology, but the numbers in OP here can't be, since a zip compressed version of rwkv_430M is ~650 MB, so obviously it's not being included or Alice in Wonderland would have an abysmal ratio.