> I'm not sure how you would optimally use an LLM for a lossless compression task. You have the model output probabilities for each token, and then use arithme

> I'm not sure how you would optimally use an LLM for a lossless compression task. You have the model output probabilities for each token, and then use arithmetic encoding using that model. (This uses optimally few bits per token, under the assumption that your model is right.) This is how all the winning models work, AFAIK, except they frequently work per-bit instead of per-token.