11 ms·

I find it funny that… Literally no one is talking about how all of the LLMs are training on Reddit data via the API. So far… - PaLM - GPT3 - Falcon Yes, R

I find it funny that… Literally no one is talking about how all of the LLMs are training on Reddit data via the API. So far… - PaLM - GPT3 - Falcon Yes, Reddit is closing on its IPO because it’s not a non-profit. But that doesn’t mean they should be forced to provide a low cost service to OpenAI, Microsoft, Google and the entire investor community backing LLM startups. Source: https://huggingface.co/blog/falcon https://huggingface.co/blog/falcon

rcxdude 3y ago
It's quite key to the legal strategy of all the LLMs that they do not need a license for their input data. So if they want reddit data they'll just scrape it (which is also legally protected). And if they do need a license then reddit does not have one to sell to the AI companies, at least for most of the text on their site. I don't believe it's a reasonable strategy for any platform (except maybe marketplaces like shutterstock) to attempt to charge money for data for machine learning.
oldandboring 3y ago
I find it funny also. This fact seemed to have its moment in the sun for about 15 minutes a few months ago when everyone was first talking about ChatGPT and then everyone forgot?
andrewxdiamond 3y ago
Reddit is able to license its content in a way that prevents companies from using it while leaving the API free. In fact, making the API paid does nothing for that goal. Scrapping has been long established as legal so if their license doesn’t change, nothing about LLMs using their content will either
- pierat 3y ago
  > Reddit is able to license its content in a way that prevents companies from using it while leaving the API free. And there's the scam laid bare. It's *NOT* Reddit's content. Sure, by posting you're granting them a whole slew of rights. But you dont transfer ownership to them. And when your content creators leave, blammo, you have no more new content. And I'm already seeing some subreddits drop dramatically in terms of comments and posts. Tech ones, naturally. Turns out Lemmy is actually pretty cool for a fediverse version of reddit. We're already seeing the start of death of Imgur as "reddit image host", oh wait "imgur social network", oh wait "no porn Imgur". We all know how this works out. Same game, just a bit ahead of reddit.
tensor 3y ago
While it seems that some people feel reddit is "owed" something when its data is used by machine learning companies, the reality is that reddit themselves claims that posts are owned by users and so really they are not in any position to be owed anything. They can fully dictate the terms of their API sure, but unless they want to drop off all search indexes there is nothing they can do about people scraping the data the old fashioned way.
- jollofricepeas 3y ago
  I disagree entirely. There’s a real non-negligible cost to Reddit hosting the content. The bottom line is there’s solutions here. - High volume API users pay up and subsidize the cost of everyone else OR - All Reddit users pay a monthly 9.99 subscription and the API stays the same. OR - A not-for-profit let’s say Internet Archive takes ownership and begs the Reddit community for donations (ie. Wikipedia)
  - dingledork69 3y ago
    OR - Everyone will just run web scrapers increasing the load on reddit's servers even more An API actually REDUCES load and lets you manage it.
    - jollofricepeas 3y ago
      Your solution isn’t a solution. Please answer the question: How is Reddit supposed to make money with increased load and demand for its content?
      - dingledork69 3y ago
        Your question isn't the question you should be asking. Please answer the question: How is reddit supposed to prevent scraping when it is legal to do so, and they have incentive to appear in search engines? Considering that scraping is legal and WILL happen, why not opt to reduce the load by offering an API?
BeFlatXIII 3y ago
Isn't this mostly closing the barn door after the horse left? The AI companies already have their data. It may grow stale, but the bulk of the corpus has already been ingested. Either that or they do old-fashioned scraping.
- jollofricepeas 3y ago
  Depends on the model but that HuggingFace blog post on Falcon was just released in the last couple of days where they mention which models are using Reddit. Reddit content is actively being pillaged by LLMs.
  - fkyoureadthedoc 3y ago
    The web is open, the site is available to anyone with a computer and internet connection. LLMs aren't pillaging anything, they're using it as intended. By this definition Google has pillaged the internet far longer and made a lot more money.
  - nkozyra 3y ago
    It's not "Reddit content" it's content posted on Reddit. It's an important distinction as hosting the content is the reason the business exists at all.
tourmalinetaco 3y ago
Perhaps because killing third-party apps is completely unrelated to LLMs? While they’re both related to the API lockdown, the core problem is not LLM-related. It’s Reddit’s own short-sighted incompetence. They can EASILY provide high prices to LLMs and low prices to app devs. They just won’t. It’s in their best interests to remove third parties and force everyone on the inferior, ad-ridden official app. They could even make premium-based account tokens to allow them to use third party apps, but that’s too long-term oriented. They have to pump’n’dump NOW.
grumple 3y ago
Reddit data? It's entirely user generated data, merely hosted by reddit, regardless of what the terms and conditions say.
skinnymuch 3y ago
This isn’t how the world works. We live in a class based society. Reddit could just have LLMs pay more and focus on moderating correct API payment usage. Any one can scrape too. The bigger one is, the more likely they have extensive scraping tools.

rcxdude 3y ago

It's quite key to the legal strategy of all the LLMs that they do not need a license for their input data. So if they want reddit data they'll just scrape it (which is also legally protected). And if they do need a license then reddit does not have one to sell to the AI companies, at least for most of the text on their site. I don't believe it's a reasonable strategy for any platform (except maybe marketplaces like shutterstock) to attempt to charge money for data for machine learning.

oldandboring 3y ago

I find it funny also. This fact seemed to have its moment in the sun for about 15 minutes a few months ago when everyone was first talking about ChatGPT and then everyone forgot?

andrewxdiamond 3y ago

Reddit is able to license its content in a way that prevents companies from using it while leaving the API free. In fact, making the API paid does nothing for that goal. Scrapping has been long established as legal so if their license doesn’t change, nothing about LLMs using their content will either

pierat 3y ago
> Reddit is able to license its content in a way that prevents companies from using it while leaving the API free. And there's the scam laid bare. It's *NOT* Reddit's content. Sure, by posting you're granting them a whole slew of rights. But you dont transfer ownership to them. And when your content creators leave, blammo, you have no more new content. And I'm already seeing some subreddits drop dramatically in terms of comments and posts. Tech ones, naturally. Turns out Lemmy is actually pretty cool for a fediverse version of reddit. We're already seeing the start of death of Imgur as "reddit image host", oh wait "imgur social network", oh wait "no porn Imgur". We all know how this works out. Same game, just a bit ahead of reddit.

tensor 3y ago

While it seems that some people feel reddit is "owed" something when its data is used by machine learning companies, the reality is that reddit themselves claims that posts are owned by users and so really they are not in any position to be owed anything. They can fully dictate the terms of their API sure, but unless they want to drop off all search indexes there is nothing they can do about people scraping the data the old fashioned way.

jollofricepeas 3y ago
I disagree entirely. There’s a real non-negligible cost to Reddit hosting the content. The bottom line is there’s solutions here. - High volume API users pay up and subsidize the cost of everyone else OR - All Reddit users pay a monthly 9.99 subscription and the API stays the same. OR - A not-for-profit let’s say Internet Archive takes ownership and begs the Reddit community for donations (ie. Wikipedia)
- dingledork69 3y ago
  OR - Everyone will just run web scrapers increasing the load on reddit's servers even more An API actually REDUCES load and lets you manage it.
  - jollofricepeas 3y ago
    Your solution isn’t a solution. Please answer the question: How is Reddit supposed to make money with increased load and demand for its content?
    - dingledork69 3y ago
      Your question isn't the question you should be asking. Please answer the question: How is reddit supposed to prevent scraping when it is legal to do so, and they have incentive to appear in search engines? Considering that scraping is legal and WILL happen, why not opt to reduce the load by offering an API?

BeFlatXIII 3y ago

Isn't this mostly closing the barn door after the horse left? The AI companies already have their data. It may grow stale, but the bulk of the corpus has already been ingested. Either that or they do old-fashioned scraping.

jollofricepeas 3y ago
Depends on the model but that HuggingFace blog post on Falcon was just released in the last couple of days where they mention which models are using Reddit. Reddit content is actively being pillaged by LLMs.
- fkyoureadthedoc 3y ago
  The web is open, the site is available to anyone with a computer and internet connection. LLMs aren't pillaging anything, they're using it as intended. By this definition Google has pillaged the internet far longer and made a lot more money.
- nkozyra 3y ago
  It's not "Reddit content" it's content posted on Reddit. It's an important distinction as hosting the content is the reason the business exists at all.

tourmalinetaco 3y ago

Perhaps because killing third-party apps is completely unrelated to LLMs? While they’re both related to the API lockdown, the core problem is not LLM-related. It’s Reddit’s own short-sighted incompetence. They can EASILY provide high prices to LLMs and low prices to app devs. They just won’t. It’s in their best interests to remove third parties and force everyone on the inferior, ad-ridden official app. They could even make premium-based account tokens to allow them to use third party apps, but that’s too long-term oriented. They have to pump’n’dump NOW.

grumple 3y ago

Reddit data? It's entirely user generated data, merely hosted by reddit, regardless of what the terms and conditions say.

skinnymuch 3y ago

This isn’t how the world works. We live in a class based society. Reddit could just have LLMs pay more and focus on moderating correct API payment usage. Any one can scrape too. The bigger one is, the more likely they have extensive scraping tools.