3 ms·
LLM training is a compression algorithm, LLM weights are a compressed dataset of the training materials, and executing LLMs is accessing the compressed source m
by a13o 3y ago
LLM training is a compression algorithm, LLM weights are a compressed dataset of the training materials, and executing LLMs is accessing the compressed source material.
That the compression technology relies on parameterizing the copyrighted material, and as a result can produce hallucinations remixing the copyrighted material, is super cool but doesn't change that this is compression at rest.
All the hullabaloo about artificial intelligence is science fiction laundering a (cool new) compression algorithm.
Copyright law shouldn't apply any differently to a LLM as it does to gzip.
- epups 3y agoCan I talk to gzip in natural language and have it produce novel output not contained within any of its source files? If not, I think your comparison is deeply flawed. LLM's are not simply compression algorithms.
- a13o 3y agoMaybe if someone were to build it. You can't talk to LLMs in natural language either. They have a very precise query language. The natural language component is an additional feature bolted onto the front. Also the output of LLMs is not novel, it's a derivative work of the training dataset. LLMs can't produce anything not present in the training set. They are operating on parameterized classifications of artwork instead of the artwork itself, which is the new technology; but they can't do anything except combine those existing Legos in new ways. We also talk to databases in what was considered 'natural language' for that era - SQL. The LLMs are no different, which is why we have prompt engineering same as we have database engineering. Just because the database is processing the dataset and storing it in a proprietary way, approachable via query language, does not change the fact that the original data is still in there, and everything that comes out is derivative.