It doesn't need to be lossless to be useful. A lossy LLM can just throw all the redundant stuff away and focus on the actual information in the text, instead of

It doesn't need to be lossless to be useful. A lossy LLM can just throw all the redundant stuff away and focus on the actual information in the text, instead of the text word by word. It wouldn't need to keep dozens of languages around for if it could translate on the fly. And it wouldn't need to keep a full text when it can just have the raw facts and generate the text on the fly. Not sure what the state of the art is with LLM text compression, but with StableDiffusion compression ratios of up to 155:1 have been achieved[1] compared to 10:1 for JPEG. [1] https://pub.towardsai.net/stable-diffusion-based-image-compresssion-6f1f0a399202 https://pub.towardsai.net/stable-diffusion-based-image-compr...