5 ms·

Though I think that training data is totally a violation of copyright, OpenAI really needs to win. Copyright has been unreasonably extended to the point where

Though I think that training data is totally a violation of copyright, OpenAI really needs to win. Copyright has been unreasonably extended to the point where it's untenable and if we needed to track down rights holders and negotiate 'training rights' we'd ensure there would be no open models or competition going forward. The rights holders are going to lose this one and it's probably for the better.

tailspin2019 3y ago
> Though I think that training data is totally a violation of copyright, OpenAI really needs to win. I’d argue that that’s not really how the law is supposed to work.
- TillE 3y ago
  Ultimately it's gonna come down to new legislation, and there is just absolutely no chance of "big tech gets to exploit every copyrighted work on the planet for free". Definitely not in the US or Europe. Big media companies don't want that, zillions of small creators don't want that, the public at large has no great affection for giant tech companies getting even more power.
  - blackoil 3y ago
    > Big media companies don't want that Big Media isn't worried about you drawing Shrek or Donald Duck at home by hand or by GPT. They won't like you distributing it profitably, but they already have tools against that. They'll also like few billions coming their way but that everyone likes. Small Creator are worried about getting the next gig as AI may take away low end of the jobs, and they are not worried about copyright implications of the LLMs. General Public doesn't like Tech companies but loves the products they make, nobody stopped using early Youtube or Telegram because there was piracy happening.
- kzrdude 3y ago
  Well, the objection to current copyright laws is that those laws are not how copyright ought to work.
  - deleted 3y ago
    [deleted]
- thebruce87m 3y ago
  Isn’t that what “jury nullification” is? Deciding that a law is trash and you’re just going to do whatever instead? I realise this requires a jury which isn’t the case for everything.
wvenable 3y ago
Why do you think training data is a violation of copyright? What is being copied?
- sdenton4 3y ago
  Did OpenAI buy a copy of every book in The Pile? Or did they all just fall off the back of a truck? Aside from that question, I tend to agree with the judge - LLM outputs are obviously not derivative works or copies in any sense that we normally use the phrase.
  - sillysaurusx 3y ago
    OpenAI hasn’t trained on The Pile, as far as I know. I think you mean "Did Meta buy a copy of every book in books3?" since llama wasn’t trained on The Pile either. And the answer is no. It seems important for the answer to remain no, otherwise the only entities that can afford to train on sufficient numbers of books will be big companies. No one can afford 190,000 books except huge corporations, so we’ll be surrendering our ability to train our own useful models if the price of training is that high. 190k books isn’t even that much. OpenAI probably trained on millions.
    - tayo42 3y ago
      If the only issue is paying for a copy, thinking out loud, It would be interesting if some ebook lender like libby let you queue up a whole bunch of books to borrow, train on the return it.
- __loam 3y ago
  OpenAI obviously copied the data onto their servers when they collected the training data.
zx8080 3y ago
> Though I think that training data is totally a violation of copyright, OpenAI really needs to win. It will have 2 consequences: 1. No reliable law. 2. No win for humans. Only big corps (openai and corporations behind it) wins. Is it really what we all deserve?
- slyall 3y ago
  And what do you think the consequence will be if Open Ai loses? A: Big media companies will make billions from licensing, AI will only be produced by a few companies that can afford the licenses and creators will get an annual $10 check (see Spotify).
  - anonymousab 3y ago
    You forgot the last point, which is that creators who don't want their work used in the training data for these megacorp LLMs without permission will get what they want.
    - exe34 3y ago
      And we won't have open models.
      - dns_snek 3y ago
        Try training an "open" model on Nintendo's, Disney's, and Elsevier's IP and see how long it takes them to bury you in lawsuits citing copyright infringement. The only way out of this would be to abolish copyright.
        exe34 3y ago
        Or indeed carve out an exception for AI training in recognition for their potential to revolutionize our society.
  - __loam 3y ago
    Let my big tech company fuck you or the big media company will fuck you instead.
throwaway09223 3y ago
"training data is totally a violation of copyright" This really isn't clear because cognition is treated as a special exception to copyright. Every thought we have is derivative of everything we've seen before to some degree; reading a book makes our brains a derivative work. But we recognize that cognition is special. With machines we tend to apply a strict test: Did copyright go in? If so, the output is almost certainly derivative. With human brains, with cognition, it isn't enough to prove that a person has consumed a copywitten work prior to having a thought -- instead we judge every thought individually as to its originality. If we are in a position to apply similar cognitive rules to an LLM then the weights won't be derivative works and we will judge each output as to its originality rather than simply assume.
- Animats 3y ago
  "This really isn't clear because cognition is treated as a special exception to copyright." Actually, no. It's considered a transformative use. If you memorize a copyrighted play or piece of music and then perform in in public, that's a copyright violation. It's the literalness of the copy that matters.
  - throwaway09223 3y ago
    No, that's totally incorrect, we do not consider every observation a "transformative use" as applied to the human mind. If you memorize a copyrighted play and write another play it is NOT inherently a copyright violation of everything which has come before. We just don't do that. The new play is judged as to its originality. People who have seen a play (everybody) are allowed to write new plays which aren't beholden to the copyright of the first play they've ever watched.
- tivert 3y ago
  >> "training data is totally a violation of copyright" > This really isn't clear because cognition is treated as a special exception to copyright. Human cognition; not the latest algorithms and their output, which some enthusiastic software engineers eagerly confuse for cognition. It's actually pretty clear.
hn_acker 3y ago
> Though I think that training data is totally a violation of copyright, OpenAI really needs to win. I'm not sure how to interpret this line. By "violation of copyright" are you referring to the ideal copyright law that's in your mind but doesn't exist yet, or are you referring to the copyright statutes and case law that are currently in effect? And in your opinion is OpenAI violating the latter, the former, or both?
lp0_on_fire 3y ago
Copyright laws need to be reformed but I don't understand why we need to just completely upend the concept of Copyright just because it's inconvenient to these massive tech companies that want to hoover up all the data they can with as little interference and COST to them as possible. Let's not pretend these are some small startups by two employees working out of a garage who took out reverse mortgages on their homes. I can see the argument it might reduce open models, but how is asking for permission (and maybe sometimes paying) of the rights holders going to reduce competition? It would be a level playing field. If you're a company that's working on a LLM, you have to ask permission to ingest data that doesn't belong to you. All companies working on an LLM would have to follow these rules. So how is it anti-competitive?

tailspin2019 3y ago

> Though I think that training data is totally a violation of copyright, OpenAI really needs to win. I’d argue that that’s not really how the law is supposed to work.

TillE 3y ago
Ultimately it's gonna come down to new legislation, and there is just absolutely no chance of "big tech gets to exploit every copyrighted work on the planet for free". Definitely not in the US or Europe. Big media companies don't want that, zillions of small creators don't want that, the public at large has no great affection for giant tech companies getting even more power.
- blackoil 3y ago
  > Big media companies don't want that Big Media isn't worried about you drawing Shrek or Donald Duck at home by hand or by GPT. They won't like you distributing it profitably, but they already have tools against that. They'll also like few billions coming their way but that everyone likes. Small Creator are worried about getting the next gig as AI may take away low end of the jobs, and they are not worried about copyright implications of the LLMs. General Public doesn't like Tech companies but loves the products they make, nobody stopped using early Youtube or Telegram because there was piracy happening.
kzrdude 3y ago
Well, the objection to current copyright laws is that those laws are not how copyright ought to work.
- deleted 3y ago
  [deleted]
thebruce87m 3y ago
Isn’t that what “jury nullification” is? Deciding that a law is trash and you’re just going to do whatever instead? I realise this requires a jury which isn’t the case for everything.

wvenable 3y ago

Why do you think training data is a violation of copyright? What is being copied?

sdenton4 3y ago
Did OpenAI buy a copy of every book in The Pile? Or did they all just fall off the back of a truck? Aside from that question, I tend to agree with the judge - LLM outputs are obviously not derivative works or copies in any sense that we normally use the phrase.
- sillysaurusx 3y ago
  OpenAI hasn’t trained on The Pile, as far as I know. I think you mean "Did Meta buy a copy of every book in books3?" since llama wasn’t trained on The Pile either. And the answer is no. It seems important for the answer to remain no, otherwise the only entities that can afford to train on sufficient numbers of books will be big companies. No one can afford 190,000 books except huge corporations, so we’ll be surrendering our ability to train our own useful models if the price of training is that high. 190k books isn’t even that much. OpenAI probably trained on millions.
  - tayo42 3y ago
    If the only issue is paying for a copy, thinking out loud, It would be interesting if some ebook lender like libby let you queue up a whole bunch of books to borrow, train on the return it.
__loam 3y ago
OpenAI obviously copied the data onto their servers when they collected the training data.

zx8080 3y ago

> Though I think that training data is totally a violation of copyright, OpenAI really needs to win. It will have 2 consequences: 1. No reliable law. 2. No win for humans. Only big corps (openai and corporations behind it) wins. Is it really what we all deserve?

slyall 3y ago
And what do you think the consequence will be if Open Ai loses? A: Big media companies will make billions from licensing, AI will only be produced by a few companies that can afford the licenses and creators will get an annual $10 check (see Spotify).
- anonymousab 3y ago
  You forgot the last point, which is that creators who don't want their work used in the training data for these megacorp LLMs without permission will get what they want.
  - exe34 3y ago
    And we won't have open models.
    - dns_snek 3y ago
      Try training an "open" model on Nintendo's, Disney's, and Elsevier's IP and see how long it takes them to bury you in lawsuits citing copyright infringement. The only way out of this would be to abolish copyright.
      - exe34 3y ago
        Or indeed carve out an exception for AI training in recognition for their potential to revolutionize our society.
- __loam 3y ago
  Let my big tech company fuck you or the big media company will fuck you instead.

throwaway09223 3y ago

"training data is totally a violation of copyright" This really isn't clear because cognition is treated as a special exception to copyright. Every thought we have is derivative of everything we've seen before to some degree; reading a book makes our brains a derivative work. But we recognize that cognition is special. With machines we tend to apply a strict test: Did copyright go in? If so, the output is almost certainly derivative. With human brains, with cognition, it isn't enough to prove that a person has consumed a copywitten work prior to having a thought -- instead we judge every thought individually as to its originality. If we are in a position to apply similar cognitive rules to an LLM then the weights won't be derivative works and we will judge each output as to its originality rather than simply assume.

Animats 3y ago
"This really isn't clear because cognition is treated as a special exception to copyright." Actually, no. It's considered a transformative use. If you memorize a copyrighted play or piece of music and then perform in in public, that's a copyright violation. It's the literalness of the copy that matters.
- throwaway09223 3y ago
  No, that's totally incorrect, we do not consider every observation a "transformative use" as applied to the human mind. If you memorize a copyrighted play and write another play it is NOT inherently a copyright violation of everything which has come before. We just don't do that. The new play is judged as to its originality. People who have seen a play (everybody) are allowed to write new plays which aren't beholden to the copyright of the first play they've ever watched.
tivert 3y ago
>> "training data is totally a violation of copyright" > This really isn't clear because cognition is treated as a special exception to copyright. Human cognition; not the latest algorithms and their output, which some enthusiastic software engineers eagerly confuse for cognition. It's actually pretty clear.

hn_acker 3y ago

> Though I think that training data is totally a violation of copyright, OpenAI really needs to win. I'm not sure how to interpret this line. By "violation of copyright" are you referring to the ideal copyright law that's in your mind but doesn't exist yet, or are you referring to the copyright statutes and case law that are currently in effect? And in your opinion is OpenAI violating the latter, the former, or both?

lp0_on_fire 3y ago

Copyright laws need to be reformed but I don't understand why we need to just completely upend the concept of Copyright just because it's inconvenient to these massive tech companies that want to hoover up all the data they can with as little interference and COST to them as possible. Let's not pretend these are some small startups by two employees working out of a garage who took out reverse mortgages on their homes. I can see the argument it might reduce open models, but how is asking for permission (and maybe sometimes paying) of the rights holders going to reduce competition? It would be a level playing field. If you're a company that's working on a LLM, you have to ask permission to ingest data that doesn't belong to you. All companies working on an LLM would have to follow these rules. So how is it anti-competitive?