3 ms·
It has long been recognised that the state of the art in data compression has much in common with the state of the art in AI, for example: http://prize.hutter1
by jl6 1y ago
It has long been recognised that the state of the art in data compression has much in common with the state of the art in AI, for example:
http://prize.hutter1.net/ http://prize.hutter1.net/
https://bellard.org/nncp/ https://bellard.org/nncp/
- ddtaylor 1y agoSome view these as so interconnected that they will say LLMs are "just" compression.
- pjc50 1y agoWhich is an interesting view when applied to the IP. I think it's relatively uncontroversial that an MP4 file which "predicts" a Disney movie which it was "trained on" is a derived work. Suppose you have an LLM which was trained on a fairly small set of movies and you could produce any one on demand; would that be treated as a derived work? If you have a predictor/compressor LLM which was trained on all the movies in the world, would that not also be infringement?
- mr_toad 1y agoMP4s are compressed data, not a compression algorithm. An MP4 (or any compressed data) is not a “prediction”, it is the difference between what was predicted and what you’re trying to compress. An LLM is (or can be used) as a compression algorithm, but it is not compressed data. It is possible to have an overfit algorithm exactly predict (or reproduce) an output, but it’s not possible for one to reproduce all the outputs due to the pigeonhole principle. To reiterate - LLMs are not compressed data.