3 ms·
These changes are about exactly one thing: getting LLM companies to pay for data. There's nothing reddit can do to allow app developers to have cheap access and
by smt88 3y ago
These changes are about exactly one thing: getting LLM companies to pay for data. There's nothing reddit can do to allow app developers to have cheap access and to charge, say, OpenAI a higher rate.
- kitsunesoba 3y agoThat’s not going to be much of a roadblock for LLM trainers… they crawl the web just as well as they consume API output. I don’t think this will accomplish anything but anger legitimate users.
- kats 3y agoI think for a big company they would get sued if they try to build an LLM just on a website's data without paying them anything for it. For a search engine they can do that, but the search engine actually benefits the original sites. An LLM doesn't really send any money towards the original websites.
- realusername 3y agoThey already do that, the dataset is called Common Crawl...
- ryanbrunner 3y agoIf your robots.txt allows access to the user agent crawling (which Reddit does except for a very small denylist), and the crawler isn't making any attempt to bypass the site's security, they wouldn't have a leg to stand on legally.
- toomuchtodo 3y agoOpenAI and other LLMs could offer browser extensions that scrape forums and send data off to them in the natural course of web browsing, no API required. No different than Recap for PACER with regards to mechanics. https://free.law/recap https://free.law/recap https://blog.law.cornell.edu/voxpop/2011/02/03/pacer-recap-and-the-movement-to-free-american-case-law/ https://blog.law.cornell.edu/voxpop/2011/02/03/pacer-recap-a...
- paradox460 3y agoSam Altman is on reddit's board. If he wants openai to have access to reddit data, he'll get it
- cratermoon 3y agoSam Altman's OpenAI already has Reddit's data. He is more interested in making sure nobody else can get it.
- throwaway5959 3y agoTOS + require the user to login in to browse Reddit pretty much solves that right?
- kitsunesoba 3y agoGoing login-only would be suicidal. A large chunk of Reddit’s traffic and probably the majority of new users come from search engines, which login walls lock out. It’s not like Twitter where most growth comes from existing users linking tweets and user profiles. TOS as far as I know is not legally enforceable, and even if it is there are people in countries that disregard such things who are training LLMs.
- ryanbrunner 3y agoTechnically they could allowlist Googlebot and deny everything else, but yeah I don't see them really doing that.
- Name_Chawps 3y agoIt's a legal barrier, not a practical one.
- paradox460 3y agoSure there is. Require apps to be approved by reddit, actually approve them, and monitor API usage to see if it's all reads or a mix of reads and writes
- eps 3y agoDuh... just make API users pay for the access rather than apps. That's it, solves all problems, but they aren't interested in that for whatever reason.
- nerdbert 3y agoIf that were true, there are any number of simple solutions that would have allowed third-party mobile apps continued access at a reasonable cost. I think it's much more about collecting data about users' behavior. The official Reddit app already records every keypress, even text that you delete before submitting.
- hakfoo 3y agoTiered pricing? Post/download ratios as a pricing factor? Per-user API keys, where a "human" key is free and restricted to a handful of requests per second, but the "LLM" key opens a full firehose?