5 ms·

I truly wonder whether anyone will ultimately go to prison for all the copyrights infringements surrounding OpenAI. Unless being a CEO in the US gives you a fr

I truly wonder whether anyone will ultimately go to prison for all the copyrights infringements surrounding OpenAI. Unless being a CEO in the US gives you a free get-out-of-jail card.

seanmcdirmid 3y ago
Copyright infringement is mostly civil matter in the USA, not a criminal one, there is a very high bar to pass for criminal charges that doesn’t exist for civil ones. Also, I’m not convinced that organizations like the New York Times didn’t accidentally just discover entrapment via prompt injection when making their claims. The courts will sort that out quickly, however.
- DoesntMatter22 3y ago
  What is entrapment via prompt injection?
  - seanmcdirmid 3y ago
    You give pieces of the article during your chat session giving enough context for the LLM to put it together when you ask for it. The other one is just being confused when Bing, who has some sort of deal with the publishers, includes the article in the prompt when you casually use the system to ask for its first ten lines, and get it. It isn’t baked into the model, the attention aspect is just really leaned into. Mostly it’s just people not understanding how LLMs work, and assuming everything is coming from the model when they don’t realize it’s all coming from information in the prompt, and the model is just really good at getting things from the prompt. Again, courts will sort those claims out quickly.
  - __loam 3y ago
    Copyright infringement.
  - gamblor956 3y ago
    There's no such thing as "entrapment via prompt injection" since it still requires the LLM to have the underlying article in its training set. And that is precisely the problem: that the underlying article is in the training set and in the body of reproducible output without permission. This requires an act of copying, and was done without permission, so it's a copyright violation unless a fair use exception applies. (Courts haven't yet ruled on that; it's likely that the NYT case is the case that will ultimately lead to a SCOTUS ruling resolving the issue one way or the other.)
    - seanmcdirmid 3y ago
      This is incorrect. The model does not need to be trained on an article for the LLM to regurgitate it if the article is fed into it (even in small pieces) in the prompt. Again, quit misunderstanding the tech or we can’t argue about it in an intelligent way.
      - gamblor956 3y ago
        OpenAI states directly in their response the NYT lawsuit that an article needs to be part of its training set in order to be regurgitated, and indeed they note that "memorization" can be a big problem if an article shows up multiple times in its training set. They further admit that their LLM is (or was) capable of replicating content from its training set wholesale and that specific safeguards were put in place after complaints to limit that. OpenAI accuses the NYT of using "manipulated prompts" to trigger regurgitation of NYT artices. But so what? For legal purposes, and more relevantly for copyright purposes, it's irrelevant that the LLM needed large prompts to "regurgitate" the entire article. What matters is that, given a prompt that was less than the full article the LLM was able to replicate the entire article. In other words, it distributed a copy of an NYT article without the right to do so. And that is literally what they are being sued for. Whether the use of the article in the training set, or the reproduction of the article in the LLM responses, constitute fair use is still to be decided (and those answers will determine who wins this case.) But it's the legal facts that will determine the winner, not the technical implementation, because those details are irrelevant for purposes of copyright law.
ben_w 3y ago
The only thing I am sure of about copyright law, is that I don't fully understand it. Search engines, for example, necessarily read as much of the internet as possible, in order to create an index, which is a derivative of the original content; when giving you results, many show snippets from those search results, so that users have an understanding of why the link is relevant to their query, which is definitely a copyright violation because Google lost (or settled?) on that exact issue with regard to news sites (and image results?). Search engines are one category of AI. PageRank was a significant improvement over simple indexing, and personalised results are a significant improvement over pure PageRank (and Google's "moat" after Page's patent expired). So the impression I have is that merely downloading a public page and putting into an algorithm isn't a violation by itself, while repeating even short clips of text verbatim totally is. But things aren't static, they are rewritten to respond to the possibilities of the world. Is this going be accepted or not, going forwards, given what AI can now do? I don't know. I can see the two camps, but I can't foresee which will be the victor; and it won't necessarily be the same in all countries, given the balance of interests between creators and users isn't assured to be the same everywhere. (And even that's assuming no good models trained on public domain or suitably licensed documents, even though there are already some such models).
JohnFen 3y ago
As much as I wish it were otherwise, it's far from clear that what OpenAI did constituted copyright infringement under US copyright law. I suspect not. But the courts will clear this up.

I truly wonder whether anyone will ultimately go to prison for all the copyrights infringements surrounding OpenAI. Unless being a CEO in the US gives you a free get-out-of-jail card.

seanmcdirmid 3y ago

Copyright infringement is mostly civil matter in the USA, not a criminal one, there is a very high bar to pass for criminal charges that doesn’t exist for civil ones. Also, I’m not convinced that organizations like the New York Times didn’t accidentally just discover entrapment via prompt injection when making their claims. The courts will sort that out quickly, however.

DoesntMatter22 3y ago
What is entrapment via prompt injection?
- seanmcdirmid 3y ago
  You give pieces of the article during your chat session giving enough context for the LLM to put it together when you ask for it. The other one is just being confused when Bing, who has some sort of deal with the publishers, includes the article in the prompt when you casually use the system to ask for its first ten lines, and get it. It isn’t baked into the model, the attention aspect is just really leaned into. Mostly it’s just people not understanding how LLMs work, and assuming everything is coming from the model when they don’t realize it’s all coming from information in the prompt, and the model is just really good at getting things from the prompt. Again, courts will sort those claims out quickly.
- __loam 3y ago
  Copyright infringement.
- gamblor956 3y ago
  There's no such thing as "entrapment via prompt injection" since it still requires the LLM to have the underlying article in its training set. And that is precisely the problem: that the underlying article is in the training set and in the body of reproducible output without permission. This requires an act of copying, and was done without permission, so it's a copyright violation unless a fair use exception applies. (Courts haven't yet ruled on that; it's likely that the NYT case is the case that will ultimately lead to a SCOTUS ruling resolving the issue one way or the other.)
  - seanmcdirmid 3y ago
    This is incorrect. The model does not need to be trained on an article for the LLM to regurgitate it if the article is fed into it (even in small pieces) in the prompt. Again, quit misunderstanding the tech or we can’t argue about it in an intelligent way.
    - gamblor956 3y ago
      OpenAI states directly in their response the NYT lawsuit that an article needs to be part of its training set in order to be regurgitated, and indeed they note that "memorization" can be a big problem if an article shows up multiple times in its training set. They further admit that their LLM is (or was) capable of replicating content from its training set wholesale and that specific safeguards were put in place after complaints to limit that. OpenAI accuses the NYT of using "manipulated prompts" to trigger regurgitation of NYT artices. But so what? For legal purposes, and more relevantly for copyright purposes, it's irrelevant that the LLM needed large prompts to "regurgitate" the entire article. What matters is that, given a prompt that was less than the full article the LLM was able to replicate the entire article. In other words, it distributed a copy of an NYT article without the right to do so. And that is literally what they are being sued for. Whether the use of the article in the training set, or the reproduction of the article in the LLM responses, constitute fair use is still to be decided (and those answers will determine who wins this case.) But it's the legal facts that will determine the winner, not the technical implementation, because those details are irrelevant for purposes of copyright law.

ben_w 3y ago

The only thing I am sure of about copyright law, is that I don't fully understand it. Search engines, for example, necessarily read as much of the internet as possible, in order to create an index, which is a derivative of the original content; when giving you results, many show snippets from those search results, so that users have an understanding of why the link is relevant to their query, which is definitely a copyright violation because Google lost (or settled?) on that exact issue with regard to news sites (and image results?). Search engines are one category of AI. PageRank was a significant improvement over simple indexing, and personalised results are a significant improvement over pure PageRank (and Google's "moat" after Page's patent expired). So the impression I have is that merely downloading a public page and putting into an algorithm isn't a violation by itself, while repeating even short clips of text verbatim totally is. But things aren't static, they are rewritten to respond to the possibilities of the world. Is this going be accepted or not, going forwards, given what AI can now do? I don't know. I can see the two camps, but I can't foresee which will be the victor; and it won't necessarily be the same in all countries, given the balance of interests between creators and users isn't assured to be the same everywhere. (And even that's assuming no good models trained on public domain or suitably licensed documents, even though there are already some such models).

JohnFen 3y ago

As much as I wish it were otherwise, it's far from clear that what OpenAI did constituted copyright infringement under US copyright law. I suspect not. But the courts will clear this up.