Absolutely. These transformer models are trained to autoregressively predict the next token (text, or now audio), and so can be viewed as powerful compression s

Absolutely. These transformer models are trained to autoregressively predict the next token (text, or now audio), and so can be viewed as powerful compression schemes by combining with something like Huffman coding (see https://ml.berkeley.edu/blog/posts/dalle2/ https://ml.berkeley.edu/blog/posts/dalle2/ for a lengthy but great read on how language models can be viewed as sophisticated lossless compression algorithms). The log-likelihood loss function used to train these models can quite directly be interpreted as the average bit-per-token needed to compress the dataset. "Compression is comprehension" - Chaitin