3 ms·

> AI is just compression, and compression is indistinguishable from AI Almost. Compression and AI both revolve around information processing, but their core ob

> AI is just compression, and compression is indistinguishable from AI Almost. Compression and AI both revolve around information processing, but their core objectives diverge. Compression is focused on efficient representation, while AI is built for flexibility and the ability to navigate the unpredictable aspects of real-world data. Compression learns a representation from the same data it encodes, like "testing on the training set". AI models have different training and test data. There are no surprises in compression.

larodi 3y ago
lets say AI is not-so-smart JPEG which has more parts missing, and there is more guesswork when producing restoration. compression is most of the times about finding the minimal grammar that unfolds to the same original material. interestingly Fabrice Bellard somehow found a way to use transformers for compression without loss, and beats xz by significant margin. https://bellard.org/nncp/nncp_v2.1.pdf https://bellard.org/nncp/nncp_v2.1.pdf. it uses "deterministic mode of PyTorch" to make sure both directions work alike which I guess means - it saves the random toss throughout compression, for the decompression to use. note: this paper is still on my to-read list.
kimixa 3y ago
A lot of current compression techniques use prediction followed by some set of correction data to fix mis-predictions. If the prediction is more accurate, you can have a smaller correction set. But you're right the predictor does need to be reproducible - the output must be exactly the same to match encoder and decoder behavior. While I don't think this is big focus right now for many, I don't think there's a fundamental reason why it couldn't be, though probably at the cost of some performance.
jazzyjackson 3y ago
representations' efficiency increases the more flexibility you give it

larodi 3y ago

lets say AI is not-so-smart JPEG which has more parts missing, and there is more guesswork when producing restoration. compression is most of the times about finding the minimal grammar that unfolds to the same original material. interestingly Fabrice Bellard somehow found a way to use transformers for compression without loss, and beats xz by significant margin. https://bellard.org/nncp/nncp_v2.1.pdf https://bellard.org/nncp/nncp_v2.1.pdf. it uses "deterministic mode of PyTorch" to make sure both directions work alike which I guess means - it saves the random toss throughout compression, for the decompression to use. note: this paper is still on my to-read list.

kimixa 3y ago

A lot of current compression techniques use prediction followed by some set of correction data to fix mis-predictions. If the prediction is more accurate, you can have a smaller correction set. But you're right the predictor does need to be reproducible - the output must be exactly the same to match encoder and decoder behavior. While I don't think this is big focus right now for many, I don't think there's a fundamental reason why it couldn't be, though probably at the cost of some performance.

jazzyjackson 3y ago

representations' efficiency increases the more flexibility you give it