3 ms·
> there is no "compression of information" in deep learning i would strongly disagree. when you are training a model you are taking the information from a docu
by metalspot 3y ago
> there is no "compression of information" in deep learning
i would strongly disagree. when you are training a model you are taking the information from a document and extracting the relationships between tokens and storing that information conglomerated with the same information from a massive amount of other documents. the model that results is a compressed form of all of the information from all of the documents where you have extracted and stored a synthesis of the relationships between the tokens in all of them. this is a lossy compression, but it does reproduce exact sequences of source documents in some cases, so the original information is stored there.
you can very plausibly argue that an LLM model trained on copyrighted material violates the copyright on every single copyrighted document that was fed to it.