4 ms·

Presumably OpenAI and others (for the most part) are checking the licenses for content they use in training? Let’s say for example that they trained on the con

by atonse 3y ago

Presumably OpenAI and others (for the most part) are checking the licenses for content they use in training? Let’s say for example that they trained on the content of the entire archive of the New York Times… isn’t it safe to say they’d have purchased a license for that content from NYT? Wouldn’t “commercial use” cover this? Seems like the publishers just want a piece of the money because they want a piece of the money.

CharlesW 3y ago
OpenAI absolutely does not have licenses for 99%+ of the content used to build their models. They're following the standard tech company model of "negotiate forgiveness rather than ask for permission".
- wing-_-nuts 3y ago
  Yep, my understanding is that one of their datasets is basically the ebook dump of z-lib. Honestly, they're likely to get away with training on copyrighted work unless someone can get the LLMs to spit out whole pages of copyrighted material. I don't even think small excerpts would be breaking fair use
  - wpietri 3y ago
    This is big enough, though, that we'll likely see new laws made in response, the same way the rise of the internet caused previously ambiguous things to get clear laws around it. So I think they also thought they could force the ambiguity to resolve in their direction with a combination of "facts on the ground" and a big budget for lobbying and litigation.
  - srveale 3y ago
    I remember a thread a while back where someone found out that ChatGPT refused to output the "litany against fear" from Dune. https://news.ycombinator.com/item?id=36374429 https://news.ycombinator.com/item?id=36374429
- wpietri 3y ago
  Exactly. When you have enough money, you think laws are for other people, as with Uber or cryptocurrency. The legal status of copyright in this context is unclear, because the laws were written without imitation machines in mind. Presumably they blew right on by "is this right?" and went for "can we get away with this and pay less in fines than we'd make in profits?" As long as there's a 51% chance of that being the case, they saw it as worth a go.
teeray 3y ago
> Wouldn’t “commercial use” cover this? The problem is that companies see that they undersold their content relative to the value it’s providing LLMs, and now want to renegotiate past and future deals with AI companies. Exactly as you said: they just want a bigger slice of pie.
- pasttense01 3y ago
  How big is the pie? How much money are the AI companies actually making?
brucethemoose2 3y ago
> Presumably OpenAI and others (for the most part) are checking the licenses for content they use in training? Ha. There is a difficult to describe culture in the ML community, from the top researchers to casual tinkerers... Everything is bleeding edge and moving at blinding speed. Some devs don't even stop to think about things like licensing or easy reproducibility/packaging, because they are too busy moving onto the next thing. This thread on how a popular repo was unlicensed and violating other licenses for months is a good example: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/2059#issuecomment-1273812099 https://github.com/AUTOMATIC1111/stable-diffusion-webui/issu... OpenAI comes from this culture, even if they are a more commercial company now.
SloopJon 3y ago
I don't think that's a safe presumption at all. Getty, e.g., claims to have found their watermark in the output of Stable Diffusion. More broadly, it's not a settled question of what license you need. The article presents a perspective that LLMs "would turn a Food & Wine review into a simple text recommendation of a bottle of Malbec, without attribution." I doubt that anyone has a license to republish the entire archive of the New York Times without attribution. At the other extreme, I can "train" myself on newspapers I find discarded on the subway with no license at all. I don't need to cite my sources when I state an opinion based on everything I've ever read, seen, or heard. Is that a fair analogy to apply to LLMs?
- pyrale 3y ago
  > At the other extreme, I can "train" myself on... Well, how much harm would you, a single person who also needs to earn a living and probably wishes to write your own original content, be for the original author? Compare that to a datacenter full of servers ready to output millions of articles per minute in the exact style of the original author for a per-article cost close to zero.
eli_gottlieb 3y ago
>Seems like the publishers just want a piece of the money because they want a piece of the money. Sure, but of course, we all know the actual value came from the writers.
- atonse 3y ago
  And the writers were already compensated for their work.
  - eli_gottlieb 3y ago
    Oh were they? So they get paid once and the LLM trainer gets paid unlimited times?

by atonse 3y ago

CharlesW 3y ago

OpenAI absolutely does not have licenses for 99%+ of the content used to build their models. They're following the standard tech company model of "negotiate forgiveness rather than ask for permission".

wing-_-nuts 3y ago
Yep, my understanding is that one of their datasets is basically the ebook dump of z-lib. Honestly, they're likely to get away with training on copyrighted work unless someone can get the LLMs to spit out whole pages of copyrighted material. I don't even think small excerpts would be breaking fair use
- wpietri 3y ago
  This is big enough, though, that we'll likely see new laws made in response, the same way the rise of the internet caused previously ambiguous things to get clear laws around it. So I think they also thought they could force the ambiguity to resolve in their direction with a combination of "facts on the ground" and a big budget for lobbying and litigation.
- srveale 3y ago
  I remember a thread a while back where someone found out that ChatGPT refused to output the "litany against fear" from Dune. https://news.ycombinator.com/item?id=36374429 https://news.ycombinator.com/item?id=36374429
wpietri 3y ago
Exactly. When you have enough money, you think laws are for other people, as with Uber or cryptocurrency. The legal status of copyright in this context is unclear, because the laws were written without imitation machines in mind. Presumably they blew right on by "is this right?" and went for "can we get away with this and pay less in fines than we'd make in profits?" As long as there's a 51% chance of that being the case, they saw it as worth a go.

teeray 3y ago

> Wouldn’t “commercial use” cover this? The problem is that companies see that they undersold their content relative to the value it’s providing LLMs, and now want to renegotiate past and future deals with AI companies. Exactly as you said: they just want a bigger slice of pie.

pasttense01 3y ago
How big is the pie? How much money are the AI companies actually making?

brucethemoose2 3y ago

> Presumably OpenAI and others (for the most part) are checking the licenses for content they use in training? Ha. There is a difficult to describe culture in the ML community, from the top researchers to casual tinkerers... Everything is bleeding edge and moving at blinding speed. Some devs don't even stop to think about things like licensing or easy reproducibility/packaging, because they are too busy moving onto the next thing. This thread on how a popular repo was unlicensed and violating other licenses for months is a good example: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/2059#issuecomment-1273812099 https://github.com/AUTOMATIC1111/stable-diffusion-webui/issu... OpenAI comes from this culture, even if they are a more commercial company now.

SloopJon 3y ago

I don't think that's a safe presumption at all. Getty, e.g., claims to have found their watermark in the output of Stable Diffusion. More broadly, it's not a settled question of what license you need. The article presents a perspective that LLMs "would turn a Food & Wine review into a simple text recommendation of a bottle of Malbec, without attribution." I doubt that anyone has a license to republish the entire archive of the New York Times without attribution. At the other extreme, I can "train" myself on newspapers I find discarded on the subway with no license at all. I don't need to cite my sources when I state an opinion based on everything I've ever read, seen, or heard. Is that a fair analogy to apply to LLMs?

pyrale 3y ago
> At the other extreme, I can "train" myself on... Well, how much harm would you, a single person who also needs to earn a living and probably wishes to write your own original content, be for the original author? Compare that to a datacenter full of servers ready to output millions of articles per minute in the exact style of the original author for a per-article cost close to zero.

eli_gottlieb 3y ago

>Seems like the publishers just want a piece of the money because they want a piece of the money. Sure, but of course, we all know the actual value came from the writers.

atonse 3y ago
And the writers were already compensated for their work.
- eli_gottlieb 3y ago
  Oh were they? So they get paid once and the LLM trainer gets paid unlimited times?