5 ms·

Unlearning is described as: "process aims to erase specific knowledge from LLMs while preserving as much model utility as possible." i.e. We know that our mode

Unlearning is described as: "process aims to erase specific knowledge from LLMs while preserving as much model utility as possible." i.e. We know that our model would be useless without your source. So we will take the useful part of your source and obfuscate the rest so that we can charge our users for utility provided by you without having to pay you anything.

short_sells_poo 2y ago
> We know that our model would be useless without your source. So we will take the useful part of your source and obfuscate the rest so that we can charge our users for utility provided by you without having to pay you anything. Isn't this basically the entirety of the latest AI craze? They basically took a public good - the information available on the Internet - and hid behind some thin veneer of "we are not stealing, we just trained an AI on the information" and then they sell it. Note, I'm intentionally not writing "free information available on the Internet", because information is not free. Someone has to pay (in time or money) to generate it and host it. They might have provided it gratis to the public, but nobody asked them if an AI can come along, harvest it all and regurgitate it without a hint of reference to the original source. Much of that information is not even free in the monetary sense, it is supported by ads. The AI will not only not click through the \ds, it won't even generate repeat traffic as once the information is harvested, there's no need to access the source anymore. If you really think about it, it's a brilliant business model. It's a perfect theft, where the affected group is too diffuse and uncoordinated, it's extremely difficult to prove anything anyway, and the "thieves" are flush with investment capital so they sleep well at night. LLMs have undoubtedly great utility as a research tool and I'm not at all against them. I think they (or a model similar in objectives) are the next step in accessing the knowledge humanity has amassed. However, there's a distinct danger that they will simply suck they sources dry and leave the internet itself even more of a wasteland than it has already become. I have no illusions that AI companies will simply regress to the lowest cost solution of simply not giving anything back to whoever created the information in the first place. The fact that they are cutting off the branch that they are sitting on is irrelevant for them, because the current crop of owners will be long gone with their billions by the time the branch snaps.
- cachvico 2y ago
  I think it's fair and reasonable to assume that the AI companies will at some point start licensing their source content. Through gov/legal oversight or not remains to be seen, but OpenAI are already beginning to do so: https://searchengineland.com/openais-growing-list-of-partnerships-442974 https://searchengineland.com/openais-growing-list-of-partner...
  - rvnx 2y ago
    Google is using for 20 years unlicensed source content for their search snippets, they seem to be doing fine with it (with the exception of few news publishers).
    - grugagag 2y ago
      The idea with internet search was to get people to find the source of the information they were searching for. As a matter of fact a lot information indexing was requested at the source. Google did respect the bargain for a while until they started to obfuscate getting to the source with AMP and their info snippets directly in the search, bypassing redirecting to the source. Then they started not displaying all that info at all, not even on the nth page of search results. The broth has been getting sour for a while now. Some people never wanted crawlers indexing and there were numerous discussions about how those robot.txt were ignored. So what I see here is the historical trend broken bargains which is more or less digital theft
  - short_sells_poo 2y ago
    Thanks for the link, I appreciate it. I suppose the issue is that this just further enshittifies the internet into a small handful of walled gardens. Big players get their payday, because they could feasibly sue OpenAI and generate them enough headache. But the vast amount of content on the internet was not built by a small handful of media companies, but rather by masses of small creators. It is their work that OpenAI is profiting from and I have yet to see a credible suggestion on how they will compensate them.
rowanG077 2y ago
That's not entirely true. Retraining is very expensive. If you can train on a very large dataset including proprietary knowledge and then postprocess the model cheaply to forget things saves you retraining for every variation.
seanmcdirmid 2y ago
I thought it was even worse than that: learning any of the corpus verbatim would actually reduce model utility.
- edude03 2y ago
  Yes although how close to verbatim is debatable. For example there are questions that you’d ask that other people have asked many times before that you’d like the exact answer for (e.g. when does daylight saving time end?)
  - startupsfail 2y ago
    > that make it forget specific facts. These are often meant to satisfy copyright claims Facts are not copyrightable. To quote copyright.gov: “Copyright does not protect facts, ideas, systems, or methods of operation, although it may protect the way these things are expressed.”
    - PittleyDunkin 2y ago
      What is a fact without expression? It's not clear under what interpretation might be necessary to get the quoted sentiment to be considered sensical.
    - deleted 2y ago
      [deleted]
  - seanmcdirmid 2y ago
    Wouldn't that be stuffed in the prompt anyways? No reason for the LLM to learn that.
- int_19h 2y ago
  It really depends on which part of the corpus, though. I do expect my LM to be able to reproduce culturally important citations, for example.

short_sells_poo 2y ago

> We know that our model would be useless without your source. So we will take the useful part of your source and obfuscate the rest so that we can charge our users for utility provided by you without having to pay you anything. Isn't this basically the entirety of the latest AI craze? They basically took a public good - the information available on the Internet - and hid behind some thin veneer of "we are not stealing, we just trained an AI on the information" and then they sell it. Note, I'm intentionally not writing "free information available on the Internet", because information is not free. Someone has to pay (in time or money) to generate it and host it. They might have provided it gratis to the public, but nobody asked them if an AI can come along, harvest it all and regurgitate it without a hint of reference to the original source. Much of that information is not even free in the monetary sense, it is supported by ads. The AI will not only not click through the \ds, it won't even generate repeat traffic as once the information is harvested, there's no need to access the source anymore. If you really think about it, it's a brilliant business model. It's a perfect theft, where the affected group is too diffuse and uncoordinated, it's extremely difficult to prove anything anyway, and the "thieves" are flush with investment capital so they sleep well at night. LLMs have undoubtedly great utility as a research tool and I'm not at all against them. I think they (or a model similar in objectives) are the next step in accessing the knowledge humanity has amassed. However, there's a distinct danger that they will simply suck they sources dry and leave the internet itself even more of a wasteland than it has already become. I have no illusions that AI companies will simply regress to the lowest cost solution of simply not giving anything back to whoever created the information in the first place. The fact that they are cutting off the branch that they are sitting on is irrelevant for them, because the current crop of owners will be long gone with their billions by the time the branch snaps.

cachvico 2y ago
I think it's fair and reasonable to assume that the AI companies will at some point start licensing their source content. Through gov/legal oversight or not remains to be seen, but OpenAI are already beginning to do so: https://searchengineland.com/openais-growing-list-of-partnerships-442974 https://searchengineland.com/openais-growing-list-of-partner...
- rvnx 2y ago
  Google is using for 20 years unlicensed source content for their search snippets, they seem to be doing fine with it (with the exception of few news publishers).
  - grugagag 2y ago
    The idea with internet search was to get people to find the source of the information they were searching for. As a matter of fact a lot information indexing was requested at the source. Google did respect the bargain for a while until they started to obfuscate getting to the source with AMP and their info snippets directly in the search, bypassing redirecting to the source. Then they started not displaying all that info at all, not even on the nth page of search results. The broth has been getting sour for a while now. Some people never wanted crawlers indexing and there were numerous discussions about how those robot.txt were ignored. So what I see here is the historical trend broken bargains which is more or less digital theft
- short_sells_poo 2y ago
  Thanks for the link, I appreciate it. I suppose the issue is that this just further enshittifies the internet into a small handful of walled gardens. Big players get their payday, because they could feasibly sue OpenAI and generate them enough headache. But the vast amount of content on the internet was not built by a small handful of media companies, but rather by masses of small creators. It is their work that OpenAI is profiting from and I have yet to see a credible suggestion on how they will compensate them.

rowanG077 2y ago

That's not entirely true. Retraining is very expensive. If you can train on a very large dataset including proprietary knowledge and then postprocess the model cheaply to forget things saves you retraining for every variation.

seanmcdirmid 2y ago

I thought it was even worse than that: learning any of the corpus verbatim would actually reduce model utility.

edude03 2y ago
Yes although how close to verbatim is debatable. For example there are questions that you’d ask that other people have asked many times before that you’d like the exact answer for (e.g. when does daylight saving time end?)
- startupsfail 2y ago
  > that make it forget specific facts. These are often meant to satisfy copyright claims Facts are not copyrightable. To quote copyright.gov: “Copyright does not protect facts, ideas, systems, or methods of operation, although it may protect the way these things are expressed.”
  - PittleyDunkin 2y ago
    What is a fact without expression? It's not clear under what interpretation might be necessary to get the quoted sentiment to be considered sensical.
  - deleted 2y ago
    [deleted]
- seanmcdirmid 2y ago
  Wouldn't that be stuffed in the prompt anyways? No reason for the LLM to learn that.
int_19h 2y ago
It really depends on which part of the corpus, though. I do expect my LM to be able to reproduce culturally important citations, for example.