24 ms·

Much of the sales hype and other literature surrounding LLMs specifically obfuscates the role that training data plays in the model. Training data is "learned f

Much of the sales hype and other literature surrounding LLMs specifically obfuscates the role that training data plays in the model. Training data is "learned from", but that's implying the data goes away after the training process ends and you have a model that's solely composed of uncopyrightable knowledge about how to write or draw. If the models are actually retaining training data, and we have a way to extract that data, then the models didn't learn - legally speaking[0], they copied training set data. The idea I had wasn't "LLMs are based on probabilities", it was "what if you benchmarked an LLM against a traditional search index over the training corpus". The linked blog post doesn't completely rip out the LLMs entirely, just the feed-forward layer, but the result is what I thought would happen: an attention-augmented search index that is producing nearly identical probability distributions to the 66% of the model that was removed. [0] Programmers talking about copyright usually get tripped up on this, so I'll spell it out: copyright is a matter of data provenance, not bit-exactness. Just because the weights are harder to inspect does not mean no copyright infringement has occurred. Compression does not launder copyright.

refulgentis 3y ago
Should make sure to establish this up front: People know this, it's not controversial. It's not only known to a few. It's how it works. Also note that this example purposefully minimizes training data down to an absurdity, so it is possible to correlate 1:1 that the next letter's probabilities to the input. The key of the rest of this comment,, and the discussions you reference, is the observation that's vastly harder once the training data is measured in terabytes, to the point the question becomes interesting. The argument of which you're speaking, the people you think are speaking literally are speaking figuratively: they know it reproduces _some_ training data, i.e. 2+2=4 was surely in the training data. Or c.f. NY Times v. OpenAI, where they were able to get it to complete an article given the first ~5 paragraphs of the article. The unsettled question, in US legal parlance, is if LLMs are sufficiently transformative of the training data that it becomes fair use. Eschewing US legal parlance: where, exactly, on the spectrum of "completely original" to "photocopier with perfect recall" LLMs fall, given we know it isn't at either of those extremes? What responsibility does that give someone operating an LLM commercially to the entities who originated the training data?

refulgentis 3y ago

Should make sure to establish this up front: People know this, it's not controversial. It's not only known to a few. It's how it works. Also note that this example purposefully minimizes training data down to an absurdity, so it is possible to correlate 1:1 that the next letter's probabilities to the input. The key of the rest of this comment,, and the discussions you reference, is the observation that's vastly harder once the training data is measured in terabytes, to the point the question becomes interesting. The argument of which you're speaking, the people you think are speaking literally are speaking figuratively: they know it reproduces _some_ training data, i.e. 2+2=4 was surely in the training data. Or c.f. NY Times v. OpenAI, where they were able to get it to complete an article given the first ~5 paragraphs of the article. The unsettled question, in US legal parlance, is if LLMs are sufficiently transformative of the training data that it becomes fair use. Eschewing US legal parlance: where, exactly, on the spectrum of "completely original" to "photocopier with perfect recall" LLMs fall, given we know it isn't at either of those extremes? What responsibility does that give someone operating an LLM commercially to the entities who originated the training data?