9 ms·

Publishers want billions, not millions, from AI

57 points by herbertl 3y ago

atonse 3y ago
Presumably OpenAI and others (for the most part) are checking the licenses for content they use in training? Let’s say for example that they trained on the content of the entire archive of the New York Times… isn’t it safe to say they’d have purchased a license for that content from NYT? Wouldn’t “commercial use” cover this? Seems like the publishers just want a piece of the money because they want a piece of the money.
- CharlesW 3y ago
  OpenAI absolutely does not have licenses for 99%+ of the content used to build their models. They're following the standard tech company model of "negotiate forgiveness rather than ask for permission".
  - wing-_-nuts 3y ago
    Yep, my understanding is that one of their datasets is basically the ebook dump of z-lib. Honestly, they're likely to get away with training on copyrighted work unless someone can get the LLMs to spit out whole pages of copyrighted material. I don't even think small excerpts would be breaking fair use
    - wpietri 3y ago
      This is big enough, though, that we'll likely see new laws made in response, the same way the rise of the internet caused previously ambiguous things to get clear laws around it. So I think they also thought they could force the ambiguity to resolve in their direction with a combination of "facts on the ground" and a big budget for lobbying and litigation.
    - srveale 3y ago
      I remember a thread a while back where someone found out that ChatGPT refused to output the "litany against fear" from Dune. https://news.ycombinator.com/item?id=36374429 https://news.ycombinator.com/item?id=36374429
  - wpietri 3y ago
    Exactly. When you have enough money, you think laws are for other people, as with Uber or cryptocurrency. The legal status of copyright in this context is unclear, because the laws were written without imitation machines in mind. Presumably they blew right on by "is this right?" and went for "can we get away with this and pay less in fines than we'd make in profits?" As long as there's a 51% chance of that being the case, they saw it as worth a go.
- teeray 3y ago
  > Wouldn’t “commercial use” cover this? The problem is that companies see that they undersold their content relative to the value it’s providing LLMs, and now want to renegotiate past and future deals with AI companies. Exactly as you said: they just want a bigger slice of pie.
  - pasttense01 3y ago
    How big is the pie? How much money are the AI companies actually making?
- brucethemoose2 3y ago
  > Presumably OpenAI and others (for the most part) are checking the licenses for content they use in training? Ha. There is a difficult to describe culture in the ML community, from the top researchers to casual tinkerers... Everything is bleeding edge and moving at blinding speed. Some devs don't even stop to think about things like licensing or easy reproducibility/packaging, because they are too busy moving onto the next thing. This thread on how a popular repo was unlicensed and violating other licenses for months is a good example: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/2059#issuecomment-1273812099 https://github.com/AUTOMATIC1111/stable-diffusion-webui/issu... OpenAI comes from this culture, even if they are a more commercial company now.
- SloopJon 3y ago
  I don't think that's a safe presumption at all. Getty, e.g., claims to have found their watermark in the output of Stable Diffusion. More broadly, it's not a settled question of what license you need. The article presents a perspective that LLMs "would turn a Food & Wine review into a simple text recommendation of a bottle of Malbec, without attribution." I doubt that anyone has a license to republish the entire archive of the New York Times without attribution. At the other extreme, I can "train" myself on newspapers I find discarded on the subway with no license at all. I don't need to cite my sources when I state an opinion based on everything I've ever read, seen, or heard. Is that a fair analogy to apply to LLMs?
  - pyrale 3y ago
    > At the other extreme, I can "train" myself on... Well, how much harm would you, a single person who also needs to earn a living and probably wishes to write your own original content, be for the original author? Compare that to a datacenter full of servers ready to output millions of articles per minute in the exact style of the original author for a per-article cost close to zero.
- eli_gottlieb 3y ago
  >Seems like the publishers just want a piece of the money because they want a piece of the money. Sure, but of course, we all know the actual value came from the writers.
  - atonse 3y ago
    And the writers were already compensated for their work.
    - eli_gottlieb 3y ago
      Oh were they? So they get paid once and the LLM trainer gets paid unlimited times?
brucethemoose2 3y ago
I am out of sympathy for big publishers. They have been greedy and predatory for decades, at the expense of their own content.
- jmclnx 3y ago
  Yes, I agree, they could have invested in the various AI companies, instead the kept bleeding their content creators of all they have. Well the boat sailed, see you later big publishers.
  - brucethemoose2 3y ago
    > they could have invested in the various AI companies Not really what I meant. > instead the kept bleeding their content creators of all they have But yes, that. And their customers too.
- luma 3y ago
  "Publishers want billions (for themselves)" is a headline we could have published at nearly any point in post-Gutenberg history. > “Search was designed to find the best of the internet” Says a publisher who clearly hasn't used search in the past 10 years.
noobermin 3y ago
So this was the outcome I guessed would need to happen, but I didn't think publishers would be the ones to push for it, but I guess they are. The thing is if this upsets you on some gut level, this is just a sign for how valuable information to train on is. May be billions is too much, but hundreds of millions? If your immediate thought is "this must be stopped, this isn't fair" it is a signal of how reliant AI at all is on the data it trains. An untrained nn is useless, it's only useful if it has information to train on. It's not like value creation is zero sum but it certainly doesn't come from zero, and the "something" it comes from is definitely more of the data you train on than the training method you choose.
- 1xb3l 3y ago
  [dead]
- CharlesW 3y ago
  Exactly right. The leaked Google AI "no moat" memo applies to the technology itself, but not the data. Without formal agreements with content owners, OpenAI is just another scraper. One can easily imagine that content cooperatives may ultimately own the GenAI space.
hospitalJail 3y ago
Back when it was OpenAI, I easily could side with the open source software. Now that one company closed it off, is trying to regulate things, and is profiting from it: Let the lawyers win with fees.
- elforce002 3y ago
  Well, "open"ai opened pandora's box. Let the games begin.
skepticATX 3y ago
Don’t worry, Sam will give you a pittance in the form of worldcoin for allowing his company to commandeer everything that humanity has ever or will ever produce. Nothing to see here.
pyrophane 3y ago
I am reminded somewhat of the ride & delivery apps here. While they did use tech to enable some cool things like demand pricing and efficient route planning, a big part of their innovation really came from using that tech to shovel most of the risk and cost of providing the service onto independent contractors. LLMs are a genuinely exciting technology, but I am worried that a part of what they enable will turn out to be a sort of "content laundering." If you train your AI on a bunch of someone elses work, you can make it produce something very simiilar. You can extract whole aspects of an artist or writer's style and replicate their ideas much more easily now, and because it is generated inside this black box you have a sort of deniability when it comes to accusations of copying.
- lotsofpulp 3y ago
  Taxi drivers already were independent contractors.
  - WirelessGigabit 3y ago
    And taxi drivers themselves caused Uber to be so popular. The amount of times I've gotten into a cab in London before Uber and the cabbie was like: "Cash only". What's that machine for then? "Credit card machine down". Yea right... I absolutely hate what Uber did with 'contractors' but I cannot deny the fact that I like being able to travel for work without having to worry about whether the cab has a credit card machine or not.
    - JohnFen 3y ago
      > And taxi drivers themselves caused Uber to be so popular. That entirely depends on where you are. Where I am, ride-share services were never better than taxis in any way except for price.
      - snehk 3y ago
        I wish I could say the same about my location. Where I live they can't even tell you how much the ride will be.
ZephyrBlu 3y ago
Hilarious and desperate. They are in no position to make demands. The current data that has been scraped for AI models is probably good enough to be able to generate synthetic data. Even if it's not, what are publishers going to do about people scraping their content? Absolutely nothing. Trying to force this issue feels like a good way to be disintermediated quicker.
- JKCalhoun 3y ago
  To hell with writers, I for one look forward to our brave new world of fake plastic trees.
  - ZephyrBlu 3y ago
    Large publishers are not writers. It would probably be better if large publishers died and there were more small publishers or independent writers.
TekMol 3y ago
Big media companies go fuck yourself. That nightmare scenario, for Levin, would turn a Food & Wine review into a simple text recommendation of a bottle of Malbec. If there is just one wine review on the web which recommends Malbec, AI will not start recommending it. If there are many such reviews, then yes, AI will tell you "Many reviews recommend Malbec.". Just like a human can tell you about what they learned, our artificial brothers have the same right to learn. Nobody should be allowed to bend the law and extort money from society for writing wine reviews.
- edgyquant 3y ago
  AI is not my “brother” nor is it a human. This is a predictive model controlled by a for profit business and yea, they should be paying out if they are training on data owned by someone.
  - TekMol 3y ago
    It is not a human but it is a fellow intelligent system. Mankind had this discussion before. When Darwin published his theory about evolution. He faced a lot of hatred because it made humans less special. Now we go through the same dance again. This time, intelligence in silicon makes humans less special.
    - wpietri 3y ago
      It is not intelligent, any more than découpé [1] means scissors and paste are intelligent. I understand that people like to anthropomorphize things. That's harmless fun when people are imagining tree spirits or thunder gods or whatever. But just because fancy autocomplete produces semi-intelligible text does not mean there is an intelligence behind it. I believe we will eventually be able to create synthetic minds. But our understanding of actual minds is so poor that it's going to be quite a while before we manage. More than 200 years ago, Mary Shelley told a story of a scientist creating life via the then-new technology of electricity. Since the we keep re-telling the story with the latest technology. It's a good story, and like all good art helps us understand what it means to be human. Which is why I think it's especially bad to use those stories to muddy the many differences between actual humans and something that produces statistically plausible generated text. [1] https://en.wikipedia.org/wiki/Cut-up_technique https://en.wikipedia.org/wiki/Cut-up_technique
bell-cot 3y ago
Easy, humorous fix for the problem: https://www.smbc-comics.com/comic/copyright https://www.smbc-comics.com/comic/copyright
FreshStart 3y ago
Ah, publisher rent seeking. To get independent again. Like they were in the haydays of the cold War or the Iraq war.not. Why not a different rent seeking model.. For pharma companies? If your product cures a patient, all other pharma companies own you a percentage from everything sold ever after.
woah 3y ago
> “The thing that everyone wants to talk about is whether AI is going take over the world to eliminate humans and all that stuff,” IAC CEO Joey Levin "What I want to talk about is whether we will get royalties from the elimination of all humans"
RandomLensman 3y ago
Two rent seekers fighting each other - will be interesting to see what happens.
niemandhier 3y ago
Capitalism needs growth. Growth needs constant new demand. Generating overpriced intangible goods, the only value of which is novelty is almost necessary in capitalism to keep some form of constant demand. The alternative is continuously. burning something, like we do with oil. I wonder how we are going to keep all that money in circulation, once we do not need a trillion dollar industry of entertainment anymore.
dumdumchan 3y ago
First they came for writers and I didn't say anything because I wasn't a writer.
clarity20 3y ago
"Publishers want billions, not millions, from AI" But why ask for billions when you can have ... millions?

57 points by herbertl 3y ago

Presumably OpenAI and others (for the most part) are checking the licenses for content they use in training? Let’s say for example that they trained on the content of the entire archive of the New York Times… isn’t it safe to say they’d have purchased a license for that content from NYT? Wouldn’t “commercial use” cover this? Seems like the publishers just want a piece of the money because they want a piece of the money.

CharlesW 3y ago
OpenAI absolutely does not have licenses for 99%+ of the content used to build their models. They're following the standard tech company model of "negotiate forgiveness rather than ask for permission".
- wing-_-nuts 3y ago
  Yep, my understanding is that one of their datasets is basically the ebook dump of z-lib. Honestly, they're likely to get away with training on copyrighted work unless someone can get the LLMs to spit out whole pages of copyrighted material. I don't even think small excerpts would be breaking fair use
  - wpietri 3y ago
    This is big enough, though, that we'll likely see new laws made in response, the same way the rise of the internet caused previously ambiguous things to get clear laws around it. So I think they also thought they could force the ambiguity to resolve in their direction with a combination of "facts on the ground" and a big budget for lobbying and litigation.
  - srveale 3y ago
    I remember a thread a while back where someone found out that ChatGPT refused to output the "litany against fear" from Dune. https://news.ycombinator.com/item?id=36374429 https://news.ycombinator.com/item?id=36374429
- wpietri 3y ago
  Exactly. When you have enough money, you think laws are for other people, as with Uber or cryptocurrency. The legal status of copyright in this context is unclear, because the laws were written without imitation machines in mind. Presumably they blew right on by "is this right?" and went for "can we get away with this and pay less in fines than we'd make in profits?" As long as there's a 51% chance of that being the case, they saw it as worth a go.
teeray 3y ago
> Wouldn’t “commercial use” cover this? The problem is that companies see that they undersold their content relative to the value it’s providing LLMs, and now want to renegotiate past and future deals with AI companies. Exactly as you said: they just want a bigger slice of pie.
- pasttense01 3y ago
  How big is the pie? How much money are the AI companies actually making?
brucethemoose2 3y ago
> Presumably OpenAI and others (for the most part) are checking the licenses for content they use in training? Ha. There is a difficult to describe culture in the ML community, from the top researchers to casual tinkerers... Everything is bleeding edge and moving at blinding speed. Some devs don't even stop to think about things like licensing or easy reproducibility/packaging, because they are too busy moving onto the next thing. This thread on how a popular repo was unlicensed and violating other licenses for months is a good example: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/2059#issuecomment-1273812099 https://github.com/AUTOMATIC1111/stable-diffusion-webui/issu... OpenAI comes from this culture, even if they are a more commercial company now.
SloopJon 3y ago
I don't think that's a safe presumption at all. Getty, e.g., claims to have found their watermark in the output of Stable Diffusion. More broadly, it's not a settled question of what license you need. The article presents a perspective that LLMs "would turn a Food & Wine review into a simple text recommendation of a bottle of Malbec, without attribution." I doubt that anyone has a license to republish the entire archive of the New York Times without attribution. At the other extreme, I can "train" myself on newspapers I find discarded on the subway with no license at all. I don't need to cite my sources when I state an opinion based on everything I've ever read, seen, or heard. Is that a fair analogy to apply to LLMs?
- pyrale 3y ago
  > At the other extreme, I can "train" myself on... Well, how much harm would you, a single person who also needs to earn a living and probably wishes to write your own original content, be for the original author? Compare that to a datacenter full of servers ready to output millions of articles per minute in the exact style of the original author for a per-article cost close to zero.
eli_gottlieb 3y ago
>Seems like the publishers just want a piece of the money because they want a piece of the money. Sure, but of course, we all know the actual value came from the writers.
- atonse 3y ago
  And the writers were already compensated for their work.
  - eli_gottlieb 3y ago
    Oh were they? So they get paid once and the LLM trainer gets paid unlimited times?

brucethemoose2 3y ago

I am out of sympathy for big publishers. They have been greedy and predatory for decades, at the expense of their own content.

jmclnx 3y ago
Yes, I agree, they could have invested in the various AI companies, instead the kept bleeding their content creators of all they have. Well the boat sailed, see you later big publishers.
- brucethemoose2 3y ago
  > they could have invested in the various AI companies Not really what I meant. > instead the kept bleeding their content creators of all they have But yes, that. And their customers too.
luma 3y ago
"Publishers want billions (for themselves)" is a headline we could have published at nearly any point in post-Gutenberg history. > “Search was designed to find the best of the internet” Says a publisher who clearly hasn't used search in the past 10 years.

noobermin 3y ago

So this was the outcome I guessed would need to happen, but I didn't think publishers would be the ones to push for it, but I guess they are. The thing is if this upsets you on some gut level, this is just a sign for how valuable information to train on is. May be billions is too much, but hundreds of millions? If your immediate thought is "this must be stopped, this isn't fair" it is a signal of how reliant AI at all is on the data it trains. An untrained nn is useless, it's only useful if it has information to train on. It's not like value creation is zero sum but it certainly doesn't come from zero, and the "something" it comes from is definitely more of the data you train on than the training method you choose.

1xb3l 3y ago
[dead]
CharlesW 3y ago
Exactly right. The leaked Google AI "no moat" memo applies to the technology itself, but not the data. Without formal agreements with content owners, OpenAI is just another scraper. One can easily imagine that content cooperatives may ultimately own the GenAI space.

hospitalJail 3y ago

Back when it was OpenAI, I easily could side with the open source software. Now that one company closed it off, is trying to regulate things, and is profiting from it: Let the lawyers win with fees.

elforce002 3y ago
Well, "open"ai opened pandora's box. Let the games begin.

skepticATX 3y ago

Don’t worry, Sam will give you a pittance in the form of worldcoin for allowing his company to commandeer everything that humanity has ever or will ever produce. Nothing to see here.

pyrophane 3y ago

I am reminded somewhat of the ride & delivery apps here. While they did use tech to enable some cool things like demand pricing and efficient route planning, a big part of their innovation really came from using that tech to shovel most of the risk and cost of providing the service onto independent contractors. LLMs are a genuinely exciting technology, but I am worried that a part of what they enable will turn out to be a sort of "content laundering." If you train your AI on a bunch of someone elses work, you can make it produce something very simiilar. You can extract whole aspects of an artist or writer's style and replicate their ideas much more easily now, and because it is generated inside this black box you have a sort of deniability when it comes to accusations of copying.

lotsofpulp 3y ago
Taxi drivers already were independent contractors.
- WirelessGigabit 3y ago
  And taxi drivers themselves caused Uber to be so popular. The amount of times I've gotten into a cab in London before Uber and the cabbie was like: "Cash only". What's that machine for then? "Credit card machine down". Yea right... I absolutely hate what Uber did with 'contractors' but I cannot deny the fact that I like being able to travel for work without having to worry about whether the cab has a credit card machine or not.
  - JohnFen 3y ago
    > And taxi drivers themselves caused Uber to be so popular. That entirely depends on where you are. Where I am, ride-share services were never better than taxis in any way except for price.
    - snehk 3y ago
      I wish I could say the same about my location. Where I live they can't even tell you how much the ride will be.

ZephyrBlu 3y ago

Hilarious and desperate. They are in no position to make demands. The current data that has been scraped for AI models is probably good enough to be able to generate synthetic data. Even if it's not, what are publishers going to do about people scraping their content? Absolutely nothing. Trying to force this issue feels like a good way to be disintermediated quicker.

JKCalhoun 3y ago
To hell with writers, I for one look forward to our brave new world of fake plastic trees.
- ZephyrBlu 3y ago
  Large publishers are not writers. It would probably be better if large publishers died and there were more small publishers or independent writers.

TekMol 3y ago

Big media companies go fuck yourself. That nightmare scenario, for Levin, would turn a Food & Wine review into a simple text recommendation of a bottle of Malbec. If there is just one wine review on the web which recommends Malbec, AI will not start recommending it. If there are many such reviews, then yes, AI will tell you "Many reviews recommend Malbec.". Just like a human can tell you about what they learned, our artificial brothers have the same right to learn. Nobody should be allowed to bend the law and extort money from society for writing wine reviews.

edgyquant 3y ago
AI is not my “brother” nor is it a human. This is a predictive model controlled by a for profit business and yea, they should be paying out if they are training on data owned by someone.
- TekMol 3y ago
  It is not a human but it is a fellow intelligent system. Mankind had this discussion before. When Darwin published his theory about evolution. He faced a lot of hatred because it made humans less special. Now we go through the same dance again. This time, intelligence in silicon makes humans less special.
  - wpietri 3y ago
    It is not intelligent, any more than découpé [1] means scissors and paste are intelligent. I understand that people like to anthropomorphize things. That's harmless fun when people are imagining tree spirits or thunder gods or whatever. But just because fancy autocomplete produces semi-intelligible text does not mean there is an intelligence behind it. I believe we will eventually be able to create synthetic minds. But our understanding of actual minds is so poor that it's going to be quite a while before we manage. More than 200 years ago, Mary Shelley told a story of a scientist creating life via the then-new technology of electricity. Since the we keep re-telling the story with the latest technology. It's a good story, and like all good art helps us understand what it means to be human. Which is why I think it's especially bad to use those stories to muddy the many differences between actual humans and something that produces statistically plausible generated text. [1] https://en.wikipedia.org/wiki/Cut-up_technique https://en.wikipedia.org/wiki/Cut-up_technique

bell-cot 3y ago

Easy, humorous fix for the problem: https://www.smbc-comics.com/comic/copyright https://www.smbc-comics.com/comic/copyright

FreshStart 3y ago

Ah, publisher rent seeking. To get independent again. Like they were in the haydays of the cold War or the Iraq war.not. Why not a different rent seeking model.. For pharma companies? If your product cures a patient, all other pharma companies own you a percentage from everything sold ever after.

woah 3y ago

> “The thing that everyone wants to talk about is whether AI is going take over the world to eliminate humans and all that stuff,” IAC CEO Joey Levin "What I want to talk about is whether we will get royalties from the elimination of all humans"

RandomLensman 3y ago

Two rent seekers fighting each other - will be interesting to see what happens.

niemandhier 3y ago

Capitalism needs growth. Growth needs constant new demand. Generating overpriced intangible goods, the only value of which is novelty is almost necessary in capitalism to keep some form of constant demand. The alternative is continuously. burning something, like we do with oil. I wonder how we are going to keep all that money in circulation, once we do not need a trillion dollar industry of entertainment anymore.

dumdumchan 3y ago

First they came for writers and I didn't say anything because I wasn't a writer.

clarity20 3y ago

"Publishers want billions, not millions, from AI" But why ask for billions when you can have ... millions?