Compression and prediction are equivalent, and both measure some function of intelligence and knowledge. Current LLMs are heavy on the knowledge side, being tra

Compression and prediction are equivalent, and both measure some function of intelligence and knowledge. Current LLMs are heavy on the knowledge side, being trained on ~all the text.