27 ms·

APIs for content sites must be free

319 points by NelsonMinar 3y ago

dgrin91 3y ago
I like how he touts Stack Overflow data dumps as an example, even though they also totally sold out and stopped doing the data dumps. They are no better.
- is_true 3y ago
  SO dumps are a great example, it's common to find regurgitated SO data in google search, ranking above the original content.
- NelsonMinar 3y ago
  Do you have more info on a policy change at Stack Overflow? The latest dump on their own post is from March 2023. In the past dumps have mostly been monthly but it's not entirely consistent. https://meta.stackexchange.com/questions/224873/all-stack-exchange-data-dumps/224922#224922 https://meta.stackexchange.com/questions/224873/all-stack-ex...
maxos 3y ago
I think the most important thing is the copyright of the content. If the content belong to the users, the platform (sites) should not charge for access, or at least they need pay to the users as well. If the content is belong to the platform, they certainly can do anything they want, because when you contribute the content for this platform, you agree with it.
SecurityMinded 3y ago
Why do APIs for content sites must be free? Are you paying the cost of upkeep for that social media site? You said no ? Then you should shut up. Every social media site is a for-profit business. You, the user, can not dictate how they run their businesses. If you don't like the way things are going, feel free to pack your stuff up (proverbially speaking) and find yourself a new place. As simple as that.
wbobeirne 3y ago
The title of this post is kind of clickbait-y, the author immediately puts an asterisk in the first sentence and notes that they think charging based on the cost of running the API is justified. The title should really be "APIs for content sites should cost a reasonable amount".
- politelemon 3y ago
  Yes actually I feel the last paragraph completely contradicts the title and tone of the post.
deltree7 3y ago
[dead]
erulabs 3y ago
Content sites must be free to build their products as they see fit. No asterisks required! Confusing Free as in libre and Free as in gratis causes endless turmoil.
dcow 3y ago
Meh the argument is quick and sloppy and caveated. Clickbait title. Move along.
lacker 3y ago
Good luck lobbying YouTube to make their API free. I'd love to have a version of the YouTube app that didn't cost me any money and also didn't have ads.
- dcow 3y ago
  Author is not even arguing that, though I agree with you. Horrible title.
- edrxty 3y ago
  If you're using an open source phone OS (not iOS) Newpipe does exactly that.
- dewey 3y ago
  > I'd love to have a version of the YouTube app that didn't cost me any money and also didn't have ads. If you don't want to pay with money or your attention how are they going to pay for all that traffic end engineering?
  - thulesgold 3y ago
    I think OP was using that ironically as an example of how ridiculous the expectations of the free-API folks are.
- satvikpendem 3y ago
  I use ReVanced and it does exactly what you ask for, including even more features like SponsorBlock. I can't imagine using the regular YouTube app anymore.
  - throwaway32908 3y ago
    Funnily enough there is also Reddit ReVanced, but with enough effort Reddit or YouTube could stop them working, so their existence doesn't change the argument about free/open APIs.
vxNsr 3y ago
What’s interesting is that when you charge for api access calls suddenly you have an incentive to make each api call as inefficient as possible. Everyone has always complained about how terrible reddit’s api is, now they have no incentive to improve it as each call is generating profit.
GRiMe2D 3y ago
Make a completely free API but change data structure every week or so. App makers will get angry and stop at some point
- randomdata 3y ago
  Isn't that effectively what they are doing? The HTML-based API is apparently going to remain free, but makes no guarantees about structure consistency. If you want an API with certain structure stability guarantees then you have to pay up.
dcow 3y ago
The most accurate/charitable interpretation of the content, despite the abysmal title, would be: If users provide content for free, then a platform must allow users to consume that content freely, as in "as they wish", not "for free". The author supposes that this is a fundamental social contract between users and platforms that needs to exist for platforms to work well. The author does not cite any evidence or examples or really even provide any explanation as to why this must be the case, they simply state it as a fact. Personally I find that, while thought provoking, this thesis is not obvious and needs to be defended. And so I don't find much utility in this short essay other than to state the thesis itself.
- com2kid 3y ago
  > If users provide content for free, then a platform must allow users to consume that content freely, as in "as they wish", not "for free". The platform is paying me with distribution and letting me take advantage of network effect. Anyone can setup a free blog and post all they want, and make everything available via RSS, but instead people choose to post on Reddit (and on HN!) because Reddit has built a platform that provides additional value vs posting on one's own blog. Platforms need to make $ to stay alive, Reddit's problem is they took VC money for a product that, it turns out, doesn't generate VC returns. Without the need to pay off VCs, Reddit could likely easily become profitable (as they reportedly were in 2019) and everyone involved could become comfortably-but-not-filthy-rich.
- deafpolygon 3y ago
  The user does not provide content or participate on the platform for free. They get a free account in exchange for doing so, along with allowing their own data to be datamined.
- paxys 3y ago
  > as in "as they wish", not "for free". What's the distinction between the two? If the site is monetized via ads then providing the content ad-free (via an API or otherwise) automatically makes it "for free".
- Zetice 3y ago
  The implication here is that, "...or else users won't participate." Which I agree, needs to be justified. You see this kind of talk a lot when power imbalances exist; the person with no power tries to speak on behalf of everyone involved who they think believes as they do, in order to try and stand against the existing power structure. Sometimes it's highly effective (actually appointed or even de-facto leaders of movements/groups), other times it's kind of laughable and presumptuous. Without justification, this is the latter. That said, I bet there probably is justification, if the author bothered to look for it or provide it! I would suspect it's true that users will not make use of a site like Reddit nearly as much if the content isn't then made available for use "as they wish".
zoogeny 3y ago
I like vehement arguments even when I disagree with them. Based on this man's logic, if I were to run a web forum then I must provide an API or data dump to download the posts created by the users? How about a comment section on my blog? How about a community Discord server? Why not Facebook posts? Or Instagram stories? I often get the feeling that in these circumstances people derive first principles from specific desired outcomes. Some people strongly desire that the content that has been aggregated on Reddit should be available to them. They then attempt to invent moral axioms that they believe will lead to that outcome. They also seem to reinvent history such that there was some "implicit" agreement between Reddit and the users of Reddit. IMO, just because I wrote some shitposts on Reddit does not mean that I am entitled access to every single shitpost that has ever been written on Reddit.
- 11235813213455 3y ago
  but you can access every single shitpost from the website, so why not from API? it's lighter
  - kohlerm 3y ago
    Because others use automatic ways to retrieve the data and derive value from it without paying anything
    - interactivecode 3y ago
      If reddit did that from the start we wouldn’t be in this situation in the first place?
  - David_Axelrod 3y ago
    Most of the time you dont just expose your internal api routes to the world. You need to write curated public facing routes that dont include certain schema or records. Takes time to write and maintain that different set of endpoints
    - 3y ago
moomoo11 3y ago
Uh no.
babbledabbler 3y ago
Is there a reason web scrapers aren't used for public sites with content? It seems like this would avoid the pitfalls of APIs and changing terms and prices?
- Blahah 3y ago
  They were, in a much less constrained way, until companies running platforms started suing people for scraping. And investing heavily in making it difficult technically. The difference is advertising. The vast majority of people do not want to be advertised to, given the choice, nor do they want to have their contract with a platform downgraded (by having to use the UI designed by the company). This situation is some of the users taking a stand about it.
- patmcc 3y ago
  They'll get blocked and/or sued. And break every time the site changes.
88913527 3y ago
Won't machine learning largely solve this problem? Fine, don't provide an API, but I can extract a useful JSON document from your HTML representation.
- boolemancer 3y ago
  As far as I'm aware, Reddit still allows you to append .json to any of their pages and you get the results as a nicely formatted json document. No LLM required.
  - msinclair 3y ago
    The question is: will that still be available after the API is paywalled?
- alecbz 3y ago
  I imagine there's ways to curtail that (like detecting non-human users).
- adrr 3y ago
  If you're referring to reddit you can just add .json to the URL. eg: https://www.reddit.com/r/blog.json https://www.reddit.com/r/blog.json or https://www.reddit.com/r/teslamotors/comments/149ad64/teslas_are_cheaper_than_ever_and_thats_creating_a.json https://www.reddit.com/r/teslamotors/comments/149ad64/teslas...
falcolas 3y ago
My take is slightly more nuanced - if you're paying a company to access their content, it shouldn't matter how you're accessing that content. More concretely, as a (former) subscriber to Reddit Premium, I should continue to get access to Reddit via their APIs for free. But that was never an option, since Reddit only wants to charge the application creators and not the actual consumers.
- iLoveOncall 3y ago
  This is a completely flawed argument. You didn't buy Reddit premium to get access to the content, you bought Reddit premium to not see ads and get free coins, which you did get. Your argument is the same as saying that if I buy a can of Coke I should be allowed to use the CEO's company car for life.
  - throwaway32908 3y ago
    Not seeing ads is always free, just don't use the site, Reddit premium=content-without-ads. It's easy for them to allow premium users API access, they'll still be developed for third-party apps with an accessibility focus amongst other things. Reddit doesn't owe us anything. But given how easy this would be, it's unsurprising that taking the current obtuse approach makes some people not want to use the site.
- wmgries 3y ago
  A different version of this argument is: why should it matter if I'm querying the content using the API directly versus by using the HTML web interface? If it's the case that different consumers of the API might put different load on the system than the web page that the provider can't/doesn't want to support, why isn't rate limiting the solution? This also provides providers a way to fairly monetize their API: we're giving you some basic level of access that matches what you get through the website for free, but if your access pattern is such that you put a load on our system that we can't tolerate, then we'll charge you (by providing a paid tier).
  - zokier 3y ago
MisterBastahrd 3y ago
Some people like the idea of freedom until they encounter the consequences of freedom.
- omgmajk 3y ago
  Freedom is also the choice not to use a platform and the choice to criticize it when they do consumer unfriendly things.
  - MisterBastahrd 3y ago
    Yep. Though they're already going right back to the platform and crying about the ordeal on said platform.
CobrastanJorji 3y ago
The use of the word "must" (in both the title and the body) is uninformative. What on Earth does it mean? Does it mean there's a law? A moral imperative? A strategic necessity? A technical requirement? Is the author claiming some form of jurisdiction? "It must provide APIs" isn't an argument. It can't be taken literally because it's clearly false. And it can't be taken in any other form because the author doesn't tell us what those forms might be. It continues, equally poorly stated: "Millions of users create the content expecting it will be widely available. Locking down an API breaks that social contract." Now that IS an argument, but it's not a good one for a surprisingly large number of reasons, the simplest of which is that websites like Reddit are widely available regardless of a free API. Finally, we get to the only actual statement of merit: "The short sighted thing about these API fees is they will harm the company in the long term." Yes, that's quite likely true. Is that the whole argument? That this is a short sighted plan? Companies are free to make stupid plans. They do it all the time, and you are free to stop doing business with them. "Must" doesn't enter into it.
- johnday 3y ago
  I fail to follow your chain of reasoning. How can the claim both be "clearly false" and unclear in its statement? The use of "should" as the very first line of the body text actually makes the argument extremely clear, which is that there is a non-legal, non-technical imperative for API access [leaving only a moral/social imperative, as you determine]. I think you are probably smarter than the very low reading comprehension bar in order to understand what is being said, so I'm not sure what value you are deriving from pretending not to.
  - CobrastanJorji 3y ago
    "X must do Y," taken literally, would imply that X did Y. But here, X did not do Y. That's the "clearly false" part. and that's fine, because that's how language works. People speak hyperbolically or metaphorically all the time. But the author must be trying to say something when they say "X must do Y," and it's not clear to me what. Are they just using "must" to mean "should?" Or do they mean something stronger? And what does "should" even mean? Do they mean should like "it would be a tactical mistake not to?" Evidently not because they start talking about social contracts and such, which are more about some sort of community unstated rules or traditions or something. Put another way, if I say "X must do Y" to you, and you want to disagree, how would you do that? Would "X is not required to do Y" be a rebuttal? Or "Y would not be very nice?" I can't tell if either assertion would contradict the author's assertions. What are they trying to say? Maybe it's just me being annoyingly pedantic (it's happened before!), but it's the title of the post and even after reading the whole thing I'm no closer to being able to summarize their argument. Is it just "I don't think Reddit should do this because it's a useful tool and it'd be mean," but with the word "must" thrown in randomly for oomph?
iLoveOncall 3y ago
This assumes that the API is the reward provided by the website to "buy" the user-generated content, as if the website did not provide value otherwise. Nobody posts on Reddit just to access the content via an API, it's not a finality of using Reddit, it's a nice to have.
- randomdata 3y ago
  1. It is impossible to use Reddit without an API (a webpage is just another API). 2. No API is ever truly free. There are always tradeoffs that come with a cost. 3. Not all APIs have the same cost. The question is, at what point does an API become a cost too much? There is a lot of indication that changes at Reddit will mean that some APIs that were previously considered to be acceptably priced will no longer be acceptably priced. The cost of using the API will exceed the value derived from using the API. Other APIs are stated to remain unchanged. In theory, users using the APIs in flux could migrate to these price-stable APIs. But, that does not mean that all people find those APIs acceptably priced, even where they did find the previously mentioned APIs in flux to be acceptably priced previously. An API is not why someone uses Reddit, but it is a necessary precondition in order to use Reddit. If one finds the cost of a Reddit API to be too high, one will not use that API. If one finds the cost of all Reddit APIs to be too high, they will not use Reddit. To retain users, is necessary for an API to be "free enough".
  - iLoveOncall 3y ago
    Given that your very first point is arguing semantics, I think it shows how weak any argument that you can bring forward is, so I won't even read the rest.
    - randomdata 3y ago
      1. Defining previously nebulous terms to clearly identify how they will be used as follows is not arguing semantics. To frame it as a semantic argument is erroneous. 2. Comments are always written by the author, for the author. If you do not want to read it, great. It wasn't ever for you.
serhack_ 3y ago
API for the content I write? No thanks. It takes a few seconds for a scammer to copy the content and apply their *** referral links to my content. Let's remember that OpenAI made part of the profit thanks to the data they acquired from crawling the websites (not directly by OpenAI, but some other parties).
alecbz 3y ago
I think the entirety of their argument is: > social media sites don’t produce content. They merely host it. Millions of users create the content expecting it will be widely available. ? But this feels pretty simplistic. Not convinced that people do (or should) upload content expecting they or anyone else can access it however they want. If you create the content then it's obviously yours forever. If you also decide post it to a platform, why is it the platform's social responsibility to allow access to that content for free.
- andrewaylett 3y ago
  This cuts both ways -- if I'm not going to allow access to that content for free, why would you give me your content? The ToS might allow for it, but people are routinely dismayed when companies stick to the letter of what their ToS allows them to do, especially when they start doing things they never did before.
  - alecbz 3y ago
    > if I'm not going to allow access to that content for free, why would you give me your content? A bunch of reasons potentially, wider reach probably being the main one, but maybe also stuff like ways for your audience to interact with you/the content/each other. > The ToS might allow for it "It's in the ToS so it's fine" is a bad argument, but I'm not saying that. I guess there's two things: a) empirically, do users actually expect that content be available "for free" (in general? just to themselves?) from a platform b) is there a "good reason" that this should be a social expectation? a)'s a tricky question with multiple parts but I'm not especially convinced. I imagine a lot of users aren't even really thinking about that kind of thing b) I could imagine being convinced about, but "they didn't make the content" doesn't feel sufficient.
kohlerm 3y ago
Does not make sense to me. Providing Apis costs money.others can make money out of it without paying anything.
kristianc 3y ago
It's been kind of interesting to see this play out in real time: it's clearly a board level reaction at these companies to OpenAI essentially scraping these companies, running a model over their content, and then essentially disintermediating them. And when you ask ChatGPT you're (marginally) less likely to be chided by someone telling you you're an idiot for asking the question in the first place.
notJim 3y ago
This post just asserts the existence of a right that comes out of nowhere. The rights you have on a website are established by agreement between you and the site's creator. You are free to not post on the site if you disagree with the terms of that agreement. If you choose to post, you have apparently come to the conclusion that the exchange is mutually beneficial. If you think that APIs for content sites must be free, I think you need to offer more to justify it then this does.
williamcotton 3y ago
Why not a publicly owned platform instead of forcing private businesses to operate for the good of the general public? Why not stop building public pools and just force people to make their private pools available for anyone to swim? Is it because we have seemingly lost faith in the very notion of a republic?
ashinybrowncoat 3y ago
This seems to be a lost author looking at socialist media platforms, not capitalistic social media. Things that actually have a cost should not necessarily be provided for free, even if one does not understand said cost (such as the author of this story). Providing an API is an add-on and not actually required of any social media platform but it does cost money to maintain the storage of the content as well as to not just build said API but also to maintain it (including keeping it secure and scalable). This author seems to be thinking of some kind of socialist media platforms, not the current capitalist social media platforms we have now. Even so, I would definitely welcome the idea of socialistic media platforms although I am not even sure how one could be built and/or managed. This sounds about the same as "McDonald's should be free" and I don't really see a big difference other than the actual consumption of said good/service.
beders 3y ago
It's stunning to me how little people like the author of this piece know about the economics of running large scale sites like reddit or twitter. Nothing is free, including providing and maintaining APIs. Someone's gotta pay for the ramen, especially if you can't run ads.
- oth001 3y ago
  Read the footnote.
- malfist 3y ago
  If you read the article and not just the headline, the author is in favor of charging a cost based fee for API access. A fee that would cover hosting and serving the API, but not generate profit for the parent company benefiting from the 3P users' content.
  - andersa 3y ago
    There would be no benefit to providing it, then.
    - margalabargala 3y ago
      No first-order benefit, perhaps. It shouldn't be difficult to look to second- and third-order benefits, though, such as increasing your user base and the amount of generated content.
      - andersa 3y ago
        That makes sense.
- thulesgold 3y ago
  I have a theory that governments and corporations across the world are pushing hard for free APIs. If they cost anything, even very little, then it makes astroturfing too expensive.
deleted 3y ago
[deleted]
simonw 3y ago
I've found myself resisting posting comments on Reddit recently - things like answers to simple questions people have about SQLite - partly because I don't want to cross the picket line but also because I realize that I was relying on a very light social contract that was in place there: I would share my knowledge for free, in exchange for which I knew that I was contributing to a larger dataset that myself and others could get back out again. If Reddit are cutting off free API access, that social contract no longer holds. Why should I work to benefit their service if they're hoarding the resulting data and not making it available to me or people like me in the future? I thought I was vanishingly rare in caring about this kind of thing, but given the mass Reddit blackout over the changes to the API apparently I'm not!
- xeromal 3y ago
  I blocked reddit in my hosts file and have found my news through other means. Here and fark.com lol
  - ProllyInfamous 3y ago
    Same on my local PiHole =D It's worked the couple times I instinctively sought reddit advice.
    - stevenhubertron 3y ago
      Same for me on NextDNS
- dvt 3y ago
  > I was relying on a very light social contract that was in place there This was never a social contract so I actually wonder how you or the original blog poster ever got to that idea. In fact, it's explicitly not the social contract if you ever read the Terms of Service of sites like Twitter and/or Reddit.
  - derefr 3y ago
    You're thinking of a legal contract. A social contract is what two sides do in practice in order to not piss the other off.
miahwilde 3y ago
Rather than APIs for everything I'd prefer that we figure out (technically, socially, politically, economically) how to separate data and applications on the "web platform" in the same way that we do on a local OS. Keeping files as distinct from the applications that use them has been a pretty effective design principle.
w0mbat 3y ago
The trouble with a free API is that someone can write a $1 client that hides all the ads (the number one requested feature), and now your website has no income. An ad-supported website has to get paid. On the web there is at least some stuff they can do to detect ad blockers, or show ads in sneaky ways that change over time and are hard to block.
- Blahah 3y ago
  That's a good argument against ads, because ads introduce a fundamental misalignment of incentives between the platform and the user. There are many ways reddit could build a collaborative business from their current situation. Advertising is very clearly not going to work, so they just need to look outside that box.
  - zokier 3y ago
    > There are many ways reddit could build a collaborative business from their current situation Like charging for API access? Yeah, that went well...
    - throwaway32908 3y ago
      The Spotify solution (premium users can use any third-party client) would have made lots of money with less uproar. Another bonus, solves the supposed issues with allowing NSFW content through APIs. Google, for one, take a credit card as proof of being 18+ in the EU (you have a choice between that and scanning an ID card). Or if Reddit doesn't want to handle billing the users directly for whatever reason, require signin and charge the devs per user. Even some sort of grace period and reasonable prices per-request would have made a big difference, Apollo dev's hard blocker seemed to be conflict with existing billing cycles.
kmeisthax 3y ago
Related note in the sidebar: NY State is considering a bill to require social media companies have open API access[0]. I have to wonder if this is something either PIRG or FUTO Tech/Louis Rossman are pushing for - in which case, good on them. [0] https://www.nysenate.gov/legislation/bills/2023/s6686 https://www.nysenate.gov/legislation/bills/2023/s6686
SoftTalker 3y ago
"If a company like Reddit or Twitter derives most of its value from content that users write for free then it must provide APIs for anyone to download and manipulate that content." I disagree that Reddit or Twitter derive value from content. They derive value from eyeballs and showing ads to those eyeballs. If people use APIs as a way to feed their eyeballs without looking at the ads, then Reddit or Twitter gain nothing in exchange for their massive expense of the hardware and staff required to run these services. The other option is that people would pay to use Reddit or Twitter. But they would lose 90% of their users if they took that route, because what they do does not provide enough value that a typical user would pay for it. For most people, Reddit and Twitter are time sinks: mindless entertainment to consume to pass the time. In a prior generation they would be watching TV game shows or soap operas. They wouldn't pay $10 a month to use it, or even $10/year. They would just jump to the next best free platform and continue there.
- uncletammy 3y ago
  > The other option is that people would pay to use Reddit or Twitter. But they would lose 90% of their users if they took that route, because what they do does not provide enough value that a typical user would pay for it. I see this repeated all the time but I've never seen any evidence cited to support it. Also, 90% of the users leaving when the site switches to a paid model doesn't necessarily mean the quality of the site or their profitability will go down. I'd pay a pretty hefty monthly fee for most of the social media sites that I've now stopped using as a result of their heavy handed strategies to maximize ad revenue. Also, since when did we start deciding how much value a thing could potentially provide to an individual? Isn't that what "the free market" is supposed to sort out?
downWidOutaFite 3y ago
Ok then LLMs must be free.
omgmajk 3y ago
>The short sighted thing about these API fees is they will harm the company in the long term. Black-belt in running a public company right there.
ProllyInfamous 3y ago
Why not just make API accessible to paid "Reddit Gold" members?! Then there'd actually be a reason to get gilded up, and 3rd-Party Apps would still work for those paying members.
- drbawb 3y ago
  >Then there'd actually be a reason to get gilded up ... I have to imagine reddit has already thought of this and come up with some justification for why it won't work - because the proposed strategy is just so incredibly lazy. Ultimately I suspect the data somehow suggests (a) the network provider has to provide free access in order to sustain the criticality of the social network, and (b) charging for something that other people are getting for free (a la twitter's 8$ checkmarks, reddit gold, etc.) is just not a very viable product strategy. So Reddit has decided to stick 3rd party devs with the thorny problem of direct-monetization of users, wherein their cost to the developer is usage-billed while the revenue to the developer is presumably fixed; meanwhile Reddit can cash the checks from their mobile ads.
  - deleted 3y ago
    [deleted]
sureglymop 3y ago
But a website is already parsable? When I needed some content from a website without an api I just faked the user agent and parsed it with soup.
jmyeet 3y ago
The problem here is capitalism. You can fight it with high-minded ideals like free API access but that’s really just polishing the brass on the Titanic. UGC sites need to be collectively owned by those who create the value or, in the very least, held in a nonprofit like Wikipedia is. Without such guardrails there will be constant pressure to extract more and more value from the users who create the value.
paxys 3y ago
Such arguments are always just different forms of "everything on the internet must be free, always", which is a valid ideological position to hold, sure, but one should also realize that it is not going to work in the real world.
cm2012 3y ago
All of your labor should be free also. What do you mean you want a six figure salary? How incredibly greedy of you. It's fair to pay you only what you need to survive.
- randomdata 3y ago
  > All of your labor should be free also. Yes, it is, and that's ultimately the contention here. Reddit has taken many millions of man hours of free labor. Now it wants to charge for the fruits of that free labor. Legally, they are within their rights to do so. Socially, however, it's thought to be uncouth to exploit workers like that.
  - pessimizer 3y ago
    > Reddit has taken many millions of man hours of free labor. Reddit has accepted many millions of man-hours of labor. They didn't kick down anybody's door and take it.
    - randomdata 3y ago
      > Reddit has accepted many millions of man-hours of labor. Yes, "accept or receive (someone or something)" is indeed the literal dictionary definition of "take". Thanks for subbing in an alternate word that means the exact same thing. I am sure we have all derived great value from your efforts. Or at least a hearty laugh at the pointlessness of it.
  - memefrog 3y ago
    Shitposting on reddit isn't "labour".
mikequinlan 3y ago
The article disagrees with the headline. The article says >*I don’t mind a site charging a nominal fee for API access. Either to cover the cost of API service itself, or more importantly to encourage API developers to be efficient when making API requests.
Ekaros 3y ago
I just consider anything I post on Internet like this comment to be donation to who ever hosts it... They can do anything they want with it, and take all the responsibility of things like hosting or safe keeping...
frankreyes 3y ago
There's some argument to be made because of Section 230. They are just distributors of user generated content and gatekeeping may go against that.
chaosjevil 3y ago
"Must" [as "be necessary"] and "should" are ultimately meaningless when you're dealing with corporate. Corporate only understands "allowed", "required", and "forbidden". Handle it accordingly.
ghostoftiber 3y ago
[ LAUGHS IN LEXISNEXIS ] Everyone seems to be fixated on the idea that the API should be free and no-one seems to be mentioning that there's some very successful business models built around charging people access to content which the parent company didn't generate. LexisNexis, for legal searches, Elsevier for scientific papers, Facebook, for random graph searches (and occasional political data scraping), Twitter, etc. It feels scummy to me too to take something free and charge people for it, but it's also not this unique thing people seem to be screaming about. Just go somewhere else if the API is that important to you. (Like hackernews). :)
- memefrog 3y ago
  LexisNexis does not just "charge people access to content which the parent company didn't generate". I could access most legislation through the government legislation website that shows the current version of every piece of legislation. I could piece together legislative history by looking through all the various amendments over the years compared to the originally enacted Acts. OR, I could look at LexisNexis, who have done all that work already and also have notes for pretty much every provision linking all that data together with academic and judicial commentary, major cases on those provisions, etc. Similarly, I can look up unreported senior courts cases on the 'Judicial Decisions Online' section of the courts of New Zealand website, for free. But what about the notes written by experienced barristers in the law reports? What about the database cataloguing every judgment that has referred to every other judgment, giving an indication of whether the judgment is still good law? It seems very popular these days to crap on businesses like LexisNexis and it does sometimes feel like they're taking the mickey given their prices and the terrible web interface to their databases. The fact you can't even 'open in new tab' properly is infuriating. But they're not just selling access to something that ought to be free, as many people seem to like to claim. They provide a lot more than just the raw judgments and statutes.
CrampusDestrus 3y ago
>APIs for content sites must be free so..... whose tax money are we gonna use to pay content sites owners for this?
deleted 3y ago
[deleted]
AYBABTME 3y ago
Content sites living off ads revenue with APIs should require API users to display ads. End of story.
Havoc 3y ago
>simple static dump is the bare minimum to fulfill the social contract I don't see how that follows? Would be awesome, but I don't see how posting on a platform creates such an obligation to render such a service (which costs money)
PaulHoule 3y ago
I would frame it as "messaging applications should interoperate". People understand that a T-Mobile customer can send a text to an AT&T customer. They understand that a Samsung owner can call an iPhone owner. Why do they accept that a Slack user can't send a message to a Discord user? Why do they accept that Google has made 11+ different messaging applications that aren't compatible with each other? Why do they accept that you can't call a Skype for business customer with Skype? We've had 30 years of stagnation because messaging applications don't interoperate. That's why the story is always "Hey, remember back when ICQ used to work? You should try Skype, it really works..." Once Skype becomes successful at the two-sided market game the honeymoon is over and the vendor has no incentive to keep it working and they figure you'll keep trying to use it anyway. If there were competition for both the server and the client there would be continuous pressure to keep the products working, and to improve them. So instead of Facebook Messenger being the same as AOL Instant Messenger except for Facebook, we'd have had 30 years of progress and would have apps that look like something out of The Jetsons.
- senko 3y ago
  > People understand that a T-Mobile customer can send a text to an AT&T customer. They understand that a Samsung owner can call an iPhone owner. That was a hard-won victory: https://en.m.wikipedia.org/wiki/Kingsbury_Commitment https://en.m.wikipedia.org/wiki/Kingsbury_Commitment A hundred years later, we've got the same story, except today's Big Tech had wised up.
  - PaulHoule 3y ago
    The EU can do better than spamming us with cookie popups. They'd like to be relevant, why don't they try passing legislation for this?
    - senko 3y ago
      They do try: https://news.ycombinator.com/item?id=36324118 https://news.ycombinator.com/item?id=36324118 But keep in mind that EU legislature (and "government", such as it is) is as dysfunctional as the US one, just in different ways, don't expect much enlightened lawmaking there either.
__MatrixMan__ 3y ago
I haven't been following this much, having quit Reddit many years ago, but I bet that it's a step on the path towards containerized headless browsers being used to provide unauthorized API's.
deleted 3y ago
[deleted]
prima-facie 3y ago
The third party apps were used to create much of the content which they now want to sell. They're trying to get the milk after shooting the cow.
zackmorris 3y ago
I'd go a step further: APIs shouldn't exist. I know, it's a hilarious sentiment in these times. But I really feel that most of the problems today around walled gardens and surveillance capitalism stem from not using the web directly as it was originally envisioned. If we had properly formatted HTML, little/no Javascript, a standard auth mechanism that isn't terrible (HTTP auth or a cookie like the good 'ol days), automatic sitemaps or semantic representations without manual intervention, best practices around site updates and maintaining previous versions, etc etc, then why would we need an API? Since that stuff will never happen, I'd vote for third parties to provide open source APIs for existing sites. Companies could even hire those third parties to maintain APIs for them. Disclaimer: I'm paid to write APIs and I'm just so tired.
- AA-BA-94-2A-56 3y ago
  Eh, a JSON blob is still much more lightweight than HTML/XML. This is important in less developed areas of the world, where internet speed may not be as fast.
tantalor 3y ago
These sites also produce a useful ranking which is separate from the content. They deserve to be compensated for that if you want it.
rosmax_1337 3y ago
Very large websites should be sized by the government as they become a public good, and be subject to control which stems from society at large. The only question is which government should size a website which the entire globe contributed to? Only in a world like that can a reasonable policy like this be enacted. The second thing that should be enacted: these sized websites may only censor content that is proven in court to be unlawful.
cbeach 3y ago
It’s an interesting argument, but I don’t think users provide their content for altruistic purposes. Some do it to benefit from a sense of participating in community, some to build reputation and some to prove their expertise. In all cases, they get something from the site that’s hosting their content. So why should the site that’s hosting content not be allowed to have a business model that pays the bills? Given a choice between the following, which would you rather? * insidious adverts that compromise the impartiality of the platform (because thou shalt not offend the advertiser) * charging for API access, which affects a tiny proportion of users, and would not in any way prevent you accessing your own content (subject access and data portability is guaranteed by data protection law)
agilecobol 3y ago
100 requests per minute will be free. Can someone explain to me why this isn't enough?
- alecbz 3y ago
  100 requests per minute per what? Like if there's a third party app built against the API, is it 100 rpm for each user of the app, or across the entire app's usage?
  - agilecobol 3y ago
    I think it is time for some of those apps to invest into caching. Also APIs are not only about third-party applications. I can think of a dozen ideas for products build around the Reddit API without going over this limit.
    - alecbz 3y ago
      > I think it is time for some of those apps to invest into caching. Even ignoring staleness issues, storing a bunch of reddit's content for them is likely going to get expensive fast as well > Also APIs are not only about third-party applications. Yeah but aren't third-party apps mostly what people are upset about?
robbiemitchell 3y ago
The context is a for-profit company. If a third-party using the API is doing so as a substitute for someone using the ad-supported site, the API will be priced to equal the revenue lost from using the site. If the people switching from UI to API are more valuable than usual, the API will cost more than usual. If the API is being used to support something new -- i.e., not a replacement for a user on the site, but as a new revenue source -- it will be priced to maximize overall API revenue, or overall revenue as an indirect result of growth supported by the API. (Maybe it's one massive customer using it or 100,000 freemium developers.) The idea that a company "must" do something doesn't make any sense. And the assertion that the company is harming itself is ultimately something the Board, acting on behalf of shareholders, is responsible for. (FWIW, I agree Reddit is shooting itself in the foot.)
ktosobcy 3y ago
It doesn't even have to be API... FB/instagram quite often prohibit you from viewing something if you don't have account... :/
Bjorkbat 3y ago
Personally I always considered the social contract to be that the price of admission for a free social network / online community is that you need to put up with ads and attempts to monetize the data, provided the latter isn’t an unreasonable breach of privacy. A free API never crossed my mind. It’s why I have to admit that I am not sympathetic to third-party apps like Apollo. They implicitly violated the social contract I recognize by providing users with an experience that circumvents Reddit’s attempts to make money. Reddit responded by finding a way to put a price on the ad-free experience.
TechBro8615 3y ago
If you don't provide an API for your content, but you're happy to render some HTML containing the content when I browse to your website, then if I want the content, I will get the content. No API is necessary. I will happily waste your resources scraping HTML if it's necessary to get the content I want, so it's in your best interest to provide it via an API, which is ultimately just a more efficient interface to the same data you're already providing me for free.
- gary_0 3y ago
  In the era before JSON and pervasive Javascript, HTML was the API. The Web is an API. The people saying "API" are really just saying "the same data that's in the HTML, but stripped of extra junk". HTML was never supposed to be about presentation, it was data meant for a variety of user agents. Has everyone forgotten? Are they too young to remember? I feel like I'm taking crazy pills!
Willish42 3y ago
> Their main intent is to destroy third party apps that no longer aid the company’s business goals. But they’re also trying to make a few million bucks a year licensing access to data, particularly on the back of AI training. It’s wrong. This seems backwards to me. It's obviously a move to try (and likely fail, assuming screen scraping works well enough) getting in on the AI money train by charging for Reddit's data being used to train LLMs. I don't think Appolo and other 3P clients are the target of this API change, even if the results of it are anti-competitive and should be stopped regardless of intent.
joshsabol46 3y ago
+1 I tweeted exactly this same sentiment just yesterday: > This Reddit / Stack Overflow API drama feels weird to me as a user. It's USER submitted data. R/SO didn't create it. > So what ChatGPT scraped it? They're benefiting the same users who contributed the content on R/SO, and then some. > It's not like ChatGPT is cloning them. > It would be like if Printing Houses tried to block authors from publishing their content as an eBook or Audiobook. > Users contributing to your ecosystem doesn't give you perpetual dominion over that content
ThinkBeat 3y ago
""the social contract"" There is no such thing. Faceboook, Twitter, Reddit, Instagram whatever exist to make money, or whatever those who own it or run it wants. It is your choice. You wish to spend your time adding value to a website someone else runs great. Does that give you rights? Nope. Normally in extremely obfuscated ways, the contract you do enter into by using a site will tell you exactly that you have no rights at all and they own everything you do. I agree that it can be unfair. But you actively decide to provide content. You cant make up the rules for the sites you do use. You can stop whenever you want. You can start your own site. I did so a few years ago when I got sick of this shit. On the other hand, I have few visitors. I can howl in the wilderness. For me it is preferable. If you do engage in one of the big social sites, you are given access to an enormous audience. You can post something that "goes viral". You get fame, site gets $$$$, maybe you get $$$. News can easily quote you from some of the big social sites. How often do you: Thinkbeat today said on his blog thinkbeat.blog that "" Compared to Such and Someone posted on Twitter...
r2vcap 3y ago
It is true that Reddit has problems with the way it operates. But the absurd argument that it should be provided "must be free" is not convincing. Such nonsensical claims are likely only made by socialists.
wg0 3y ago
Screen scrapping is still doable/feasible/economical in 2023? If not legally, technically? What about Cloud flare protections, bot blocking and what not?
dahwolf 3y ago
In my opinion, the author doesn't really think things through, it's just dropping some confident one-dimensional opinions without looking at multiple perspectives. "The key thing here is social media sites don’t produce content. They merely host it. Millions of users create the content expecting it will be widely available. Locking down an API breaks that social contract." "Merely host it" is framed here like it's no big deal. It's a huge fucking deal. Giant social networks are incredibly complicated to run, and it's not only technical aspects that make it so. You can hate these companies all you want, but let's not be ridiculous in suggesting that they offer little to no value. Second, the idea that content producers produce all the real value on the network is incomplete. First because of the reason mentioned above: the network itself (the plumbing and operation) has tremendous value and there would be no content without it. Second, because freely provided content in itself offers no financial value, in fact it starts out as a cost (hosting and moderation). Only when monetized does this content provide financial value. I know that users don't give a shit about that or are hostile to monetization but that doesn't change the fact that a large social network needs to cover costs. Third, the idea that any user of a social network has the right to download/access the entirety of all content ever produced on it, and this being some right or social contract...is made up bullshit. Most people have no idea what an API even is, have no weird need to do data digging, it's not a common need or expectation at all. Fourth, the AI problem is conveniently dodged. At Stackoverflow, moderators angrily demand their data dumps back, completely ignoring the reason why they were shut down in the first place: AI. Yet nobody engages with the point that the entirety of Stackoverflow might cease to exist and become obsolete because of it. That includes all the content those moderators put so much work in to curate. In both cases, you're on a sinking ship and protest to actively sink it faster.
pentagrama 3y ago
But if the product is mostly monetized by ads, how can we expect that they let people use the API for free to create an alternative front end that gets rid of the ads? Reddit isn't Wikipedia.
Avlin67 3y ago
but if internet is only reddit and onlyFans and so one i think we enter the middle age. I miss complexe forums with passionated guys. redeit ends up rants and garbage of newbies, I dont use it so much . Maybe I'll be donwvoted but years ago I did learn and also meet people with forums. Now its redeit full of newbies
wellthisisgreat 3y ago
How about this scenario: Reddit goes ballistic and locks everything behind a 100% paywall under a (flawed) assumption that they have enough critical mass of content for people throw some $5/mo bone their way just to access existing content.
TheMode 3y ago
I may agree with the idea, but it is pure fiction. Instead of begging companies to be nice, we should figure out how to change our entire software paradigm so it becomes impossible.
hoofhearted 3y ago
Hey Nelson, you haven't really been blogging on that domain since 2001 have you? If so, that is pretty amazing, and I'll be adding it to my list of retro sites still in existence.
ygouzerh 3y ago
Putting a price on an API that is just there to parse website content is not a great idea to beginning with. It's open-door for scraping companies and uncontrolled traffic. At least it's possible to put a rate-limit on an API.
memefrog 3y ago
This post makes no argument. It is just a pile of assertions. Simply stating that there is a "social contract" doesn't make it so. What evidence does the author have that "millions of users create the content expecting it will be widely available"? Even if that's true, what evidence is there that they do so with the expectation that it will be widely available through an API or through third party apps? And even if they do expect that, by what logic can you conclude that there is now a social contract that that expectation will be fulfilled? I don't think any of that is true. I don't think people are thinking about third party app users or about the wide availability of content when they post things to Reddit or other social media websites at all. I don't think they're thinking about people using third party apps. I think they're thinking on a much more mundane level. The whole concept of a "social contract" is fraught with difficulty too. It's a relatively niche idea to try to explain why people follow laws, or how laws can be justified. The idea is that we obey the law for the same reason we obey a contract: exchange of consideration for mutual benefit. But that's not actually a very good explanation, for a long list of reasons, and it certainly can't justify laws of universal application given that eg. children can't be bound to contracts they've signed and we apply laws to everyone even if they don't want us to. The concept certainly doesn't make sense in the context of a web API for a social media website. Even if everyone "expects" that their "content" will be widely available, it does not follow that there is any obligation, legal or otherwise, on Reddit to make it widely available. There just is no logical connection there unless you build one. This post does not. It simply asserts that APIs for content sites must be free. Why? Because!
hartator 3y ago
It's what we aim to do at https://serpapi.com https://serpapi.com, making an API for everything that doesn't have an API.
rldjbpin 3y ago
clickbait aside, this seems like a myopic take on social media IMHO. i would like to rather contrast it with the notions of web3 and people wanting ownership of their content, or at least be able to use the platforms without ads or data harvesting. while the real argument in this post seems to be that the api should be priced "fairly", we need to recognize the limited revenue streams the site has been employing till now, including the meaningless awards. the underlying costs might be apple and oranges, but the ads till now have not been super-intrusive à la youtube with its upto 5-6 continuous ads, or more relatably 5 banner ads or autoplay videos like many publications. moreover, given the technically inclined audience here, we can all agree that they have not been fighting back adblockers on their web ui like twitch or news websites. i have no websites with major traffic or ever got blessed with the "HN effect" on my web infrastructure, but those who have can attest to the unaccounted costs of running a public site for free. so i find it ironic that the most backlash has been from this crowd, who also tend to be happy to pay for chatgpt plus for personal use at a monthly rate costlier* than prime and netflix combined.

319 points by NelsonMinar 3y ago

I like how he touts Stack Overflow data dumps as an example, even though they also totally sold out and stopped doing the data dumps. They are no better.

is_true 3y ago
SO dumps are a great example, it's common to find regurgitated SO data in google search, ranking above the original content.
NelsonMinar 3y ago
Do you have more info on a policy change at Stack Overflow? The latest dump on their own post is from March 2023. In the past dumps have mostly been monthly but it's not entirely consistent. https://meta.stackexchange.com/questions/224873/all-stack-exchange-data-dumps/224922#224922 https://meta.stackexchange.com/questions/224873/all-stack-ex...

maxos 3y ago

I think the most important thing is the copyright of the content. If the content belong to the users, the platform (sites) should not charge for access, or at least they need pay to the users as well. If the content is belong to the platform, they certainly can do anything they want, because when you contribute the content for this platform, you agree with it.

SecurityMinded 3y ago

Why do APIs for content sites must be free? Are you paying the cost of upkeep for that social media site? You said no ? Then you should shut up. Every social media site is a for-profit business. You, the user, can not dictate how they run their businesses. If you don't like the way things are going, feel free to pack your stuff up (proverbially speaking) and find yourself a new place. As simple as that.

wbobeirne 3y ago

The title of this post is kind of clickbait-y, the author immediately puts an asterisk in the first sentence and notes that they think charging based on the cost of running the API is justified. The title should really be "APIs for content sites should cost a reasonable amount".

politelemon 3y ago
Yes actually I feel the last paragraph completely contradicts the title and tone of the post.

deltree7 3y ago

[dead]

erulabs 3y ago

Content sites must be free to build their products as they see fit. No asterisks required! Confusing Free as in libre and Free as in gratis causes endless turmoil.

dcow 3y ago

Meh the argument is quick and sloppy and caveated. Clickbait title. Move along.

lacker 3y ago

Good luck lobbying YouTube to make their API free. I'd love to have a version of the YouTube app that didn't cost me any money and also didn't have ads.

dcow 3y ago
Author is not even arguing that, though I agree with you. Horrible title.
edrxty 3y ago
If you're using an open source phone OS (not iOS) Newpipe does exactly that.
dewey 3y ago
> I'd love to have a version of the YouTube app that didn't cost me any money and also didn't have ads. If you don't want to pay with money or your attention how are they going to pay for all that traffic end engineering?
- thulesgold 3y ago
  I think OP was using that ironically as an example of how ridiculous the expectations of the free-API folks are.
satvikpendem 3y ago
I use ReVanced and it does exactly what you ask for, including even more features like SponsorBlock. I can't imagine using the regular YouTube app anymore.
- throwaway32908 3y ago
  Funnily enough there is also Reddit ReVanced, but with enough effort Reddit or YouTube could stop them working, so their existence doesn't change the argument about free/open APIs.

vxNsr 3y ago

What’s interesting is that when you charge for api access calls suddenly you have an incentive to make each api call as inefficient as possible. Everyone has always complained about how terrible reddit’s api is, now they have no incentive to improve it as each call is generating profit.

GRiMe2D 3y ago

Make a completely free API but change data structure every week or so. App makers will get angry and stop at some point

randomdata 3y ago
Isn't that effectively what they are doing? The HTML-based API is apparently going to remain free, but makes no guarantees about structure consistency. If you want an API with certain structure stability guarantees then you have to pay up.

dcow 3y ago

The most accurate/charitable interpretation of the content, despite the abysmal title, would be: If users provide content for free, then a platform must allow users to consume that content freely, as in "as they wish", not "for free". The author supposes that this is a fundamental social contract between users and platforms that needs to exist for platforms to work well. The author does not cite any evidence or examples or really even provide any explanation as to why this must be the case, they simply state it as a fact. Personally I find that, while thought provoking, this thesis is not obvious and needs to be defended. And so I don't find much utility in this short essay other than to state the thesis itself.

com2kid 3y ago
> If users provide content for free, then a platform must allow users to consume that content freely, as in "as they wish", not "for free". The platform is paying me with distribution and letting me take advantage of network effect. Anyone can setup a free blog and post all they want, and make everything available via RSS, but instead people choose to post on Reddit (and on HN!) because Reddit has built a platform that provides additional value vs posting on one's own blog. Platforms need to make $ to stay alive, Reddit's problem is they took VC money for a product that, it turns out, doesn't generate VC returns. Without the need to pay off VCs, Reddit could likely easily become profitable (as they reportedly were in 2019) and everyone involved could become comfortably-but-not-filthy-rich.
deafpolygon 3y ago
The user does not provide content or participate on the platform for free. They get a free account in exchange for doing so, along with allowing their own data to be datamined.
paxys 3y ago
> as in "as they wish", not "for free". What's the distinction between the two? If the site is monetized via ads then providing the content ad-free (via an API or otherwise) automatically makes it "for free".
Zetice 3y ago
The implication here is that, "...or else users won't participate." Which I agree, needs to be justified. You see this kind of talk a lot when power imbalances exist; the person with no power tries to speak on behalf of everyone involved who they think believes as they do, in order to try and stand against the existing power structure. Sometimes it's highly effective (actually appointed or even de-facto leaders of movements/groups), other times it's kind of laughable and presumptuous. Without justification, this is the latter. That said, I bet there probably is justification, if the author bothered to look for it or provide it! I would suspect it's true that users will not make use of a site like Reddit nearly as much if the content isn't then made available for use "as they wish".

zoogeny 3y ago

I like vehement arguments even when I disagree with them. Based on this man's logic, if I were to run a web forum then I must provide an API or data dump to download the posts created by the users? How about a comment section on my blog? How about a community Discord server? Why not Facebook posts? Or Instagram stories? I often get the feeling that in these circumstances people derive first principles from specific desired outcomes. Some people strongly desire that the content that has been aggregated on Reddit should be available to them. They then attempt to invent moral axioms that they believe will lead to that outcome. They also seem to reinvent history such that there was some "implicit" agreement between Reddit and the users of Reddit. IMO, just because I wrote some shitposts on Reddit does not mean that I am entitled access to every single shitpost that has ever been written on Reddit.

11235813213455 3y ago
but you can access every single shitpost from the website, so why not from API? it's lighter
- kohlerm 3y ago
  Because others use automatic ways to retrieve the data and derive value from it without paying anything
  - interactivecode 3y ago
    If reddit did that from the start we wouldn’t be in this situation in the first place?
- David_Axelrod 3y ago
  Most of the time you dont just expose your internal api routes to the world. You need to write curated public facing routes that dont include certain schema or records. Takes time to write and maintain that different set of endpoints
  - 3y ago

moomoo11 3y ago

Uh no.

babbledabbler 3y ago

Is there a reason web scrapers aren't used for public sites with content? It seems like this would avoid the pitfalls of APIs and changing terms and prices?

Blahah 3y ago
They were, in a much less constrained way, until companies running platforms started suing people for scraping. And investing heavily in making it difficult technically. The difference is advertising. The vast majority of people do not want to be advertised to, given the choice, nor do they want to have their contract with a platform downgraded (by having to use the UI designed by the company). This situation is some of the users taking a stand about it.
patmcc 3y ago
They'll get blocked and/or sued. And break every time the site changes.

88913527 3y ago

Won't machine learning largely solve this problem? Fine, don't provide an API, but I can extract a useful JSON document from your HTML representation.

boolemancer 3y ago
As far as I'm aware, Reddit still allows you to append .json to any of their pages and you get the results as a nicely formatted json document. No LLM required.
- msinclair 3y ago
  The question is: will that still be available after the API is paywalled?
alecbz 3y ago
I imagine there's ways to curtail that (like detecting non-human users).
adrr 3y ago
If you're referring to reddit you can just add .json to the URL. eg: https://www.reddit.com/r/blog.json https://www.reddit.com/r/blog.json or https://www.reddit.com/r/teslamotors/comments/149ad64/teslas_are_cheaper_than_ever_and_thats_creating_a.json https://www.reddit.com/r/teslamotors/comments/149ad64/teslas...

falcolas 3y ago

My take is slightly more nuanced - if you're paying a company to access their content, it shouldn't matter how you're accessing that content. More concretely, as a (former) subscriber to Reddit Premium, I should continue to get access to Reddit via their APIs for free. But that was never an option, since Reddit only wants to charge the application creators and not the actual consumers.

iLoveOncall 3y ago
This is a completely flawed argument. You didn't buy Reddit premium to get access to the content, you bought Reddit premium to not see ads and get free coins, which you did get. Your argument is the same as saying that if I buy a can of Coke I should be allowed to use the CEO's company car for life.
- throwaway32908 3y ago
  Not seeing ads is always free, just don't use the site, Reddit premium=content-without-ads. It's easy for them to allow premium users API access, they'll still be developed for third-party apps with an accessibility focus amongst other things. Reddit doesn't owe us anything. But given how easy this would be, it's unsurprising that taking the current obtuse approach makes some people not want to use the site.
wmgries 3y ago
A different version of this argument is: why should it matter if I'm querying the content using the API directly versus by using the HTML web interface? If it's the case that different consumers of the API might put different load on the system than the web page that the provider can't/doesn't want to support, why isn't rate limiting the solution? This also provides providers a way to fairly monetize their API: we're giving you some basic level of access that matches what you get through the website for free, but if your access pattern is such that you put a load on our system that we can't tolerate, then we'll charge you (by providing a paid tier).
- zokier 3y ago

MisterBastahrd 3y ago

Some people like the idea of freedom until they encounter the consequences of freedom.

omgmajk 3y ago
Freedom is also the choice not to use a platform and the choice to criticize it when they do consumer unfriendly things.
- MisterBastahrd 3y ago
  Yep. Though they're already going right back to the platform and crying about the ordeal on said platform.

CobrastanJorji 3y ago

The use of the word "must" (in both the title and the body) is uninformative. What on Earth does it mean? Does it mean there's a law? A moral imperative? A strategic necessity? A technical requirement? Is the author claiming some form of jurisdiction? "It must provide APIs" isn't an argument. It can't be taken literally because it's clearly false. And it can't be taken in any other form because the author doesn't tell us what those forms might be. It continues, equally poorly stated: "Millions of users create the content expecting it will be widely available. Locking down an API breaks that social contract." Now that IS an argument, but it's not a good one for a surprisingly large number of reasons, the simplest of which is that websites like Reddit are widely available regardless of a free API. Finally, we get to the only actual statement of merit: "The short sighted thing about these API fees is they will harm the company in the long term." Yes, that's quite likely true. Is that the whole argument? That this is a short sighted plan? Companies are free to make stupid plans. They do it all the time, and you are free to stop doing business with them. "Must" doesn't enter into it.

johnday 3y ago
I fail to follow your chain of reasoning. How can the claim both be "clearly false" and unclear in its statement? The use of "should" as the very first line of the body text actually makes the argument extremely clear, which is that there is a non-legal, non-technical imperative for API access [leaving only a moral/social imperative, as you determine]. I think you are probably smarter than the very low reading comprehension bar in order to understand what is being said, so I'm not sure what value you are deriving from pretending not to.
- CobrastanJorji 3y ago
  "X must do Y," taken literally, would imply that X did Y. But here, X did not do Y. That's the "clearly false" part. and that's fine, because that's how language works. People speak hyperbolically or metaphorically all the time. But the author must be trying to say something when they say "X must do Y," and it's not clear to me what. Are they just using "must" to mean "should?" Or do they mean something stronger? And what does "should" even mean? Do they mean should like "it would be a tactical mistake not to?" Evidently not because they start talking about social contracts and such, which are more about some sort of community unstated rules or traditions or something. Put another way, if I say "X must do Y" to you, and you want to disagree, how would you do that? Would "X is not required to do Y" be a rebuttal? Or "Y would not be very nice?" I can't tell if either assertion would contradict the author's assertions. What are they trying to say? Maybe it's just me being annoyingly pedantic (it's happened before!), but it's the title of the post and even after reading the whole thing I'm no closer to being able to summarize their argument. Is it just "I don't think Reddit should do this because it's a useful tool and it'd be mean," but with the word "must" thrown in randomly for oomph?

iLoveOncall 3y ago

This assumes that the API is the reward provided by the website to "buy" the user-generated content, as if the website did not provide value otherwise. Nobody posts on Reddit just to access the content via an API, it's not a finality of using Reddit, it's a nice to have.

randomdata 3y ago
1. It is impossible to use Reddit without an API (a webpage is just another API). 2. No API is ever truly free. There are always tradeoffs that come with a cost. 3. Not all APIs have the same cost. The question is, at what point does an API become a cost too much? There is a lot of indication that changes at Reddit will mean that some APIs that were previously considered to be acceptably priced will no longer be acceptably priced. The cost of using the API will exceed the value derived from using the API. Other APIs are stated to remain unchanged. In theory, users using the APIs in flux could migrate to these price-stable APIs. But, that does not mean that all people find those APIs acceptably priced, even where they did find the previously mentioned APIs in flux to be acceptably priced previously. An API is not why someone uses Reddit, but it is a necessary precondition in order to use Reddit. If one finds the cost of a Reddit API to be too high, one will not use that API. If one finds the cost of all Reddit APIs to be too high, they will not use Reddit. To retain users, is necessary for an API to be "free enough".
- iLoveOncall 3y ago
  Given that your very first point is arguing semantics, I think it shows how weak any argument that you can bring forward is, so I won't even read the rest.
  - randomdata 3y ago
    1. Defining previously nebulous terms to clearly identify how they will be used as follows is not arguing semantics. To frame it as a semantic argument is erroneous. 2. Comments are always written by the author, for the author. If you do not want to read it, great. It wasn't ever for you.

serhack_ 3y ago

API for the content I write? No thanks. It takes a few seconds for a scammer to copy the content and apply their *** referral links to my content. Let's remember that OpenAI made part of the profit thanks to the data they acquired from crawling the websites (not directly by OpenAI, but some other parties).

alecbz 3y ago

I think the entirety of their argument is: > social media sites don’t produce content. They merely host it. Millions of users create the content expecting it will be widely available. ? But this feels pretty simplistic. Not convinced that people do (or should) upload content expecting they or anyone else can access it however they want. If you create the content then it's obviously yours forever. If you also decide post it to a platform, why is it the platform's social responsibility to allow access to that content for free.

andrewaylett 3y ago
This cuts both ways -- if I'm not going to allow access to that content for free, why would you give me your content? The ToS might allow for it, but people are routinely dismayed when companies stick to the letter of what their ToS allows them to do, especially when they start doing things they never did before.
- alecbz 3y ago
  > if I'm not going to allow access to that content for free, why would you give me your content? A bunch of reasons potentially, wider reach probably being the main one, but maybe also stuff like ways for your audience to interact with you/the content/each other. > The ToS might allow for it "It's in the ToS so it's fine" is a bad argument, but I'm not saying that. I guess there's two things: a) empirically, do users actually expect that content be available "for free" (in general? just to themselves?) from a platform b) is there a "good reason" that this should be a social expectation? a)'s a tricky question with multiple parts but I'm not especially convinced. I imagine a lot of users aren't even really thinking about that kind of thing b) I could imagine being convinced about, but "they didn't make the content" doesn't feel sufficient.

kohlerm 3y ago

Does not make sense to me. Providing Apis costs money.others can make money out of it without paying anything.

kristianc 3y ago

It's been kind of interesting to see this play out in real time: it's clearly a board level reaction at these companies to OpenAI essentially scraping these companies, running a model over their content, and then essentially disintermediating them. And when you ask ChatGPT you're (marginally) less likely to be chided by someone telling you you're an idiot for asking the question in the first place.

notJim 3y ago

This post just asserts the existence of a right that comes out of nowhere. The rights you have on a website are established by agreement between you and the site's creator. You are free to not post on the site if you disagree with the terms of that agreement. If you choose to post, you have apparently come to the conclusion that the exchange is mutually beneficial. If you think that APIs for content sites must be free, I think you need to offer more to justify it then this does.

williamcotton 3y ago

Why not a publicly owned platform instead of forcing private businesses to operate for the good of the general public? Why not stop building public pools and just force people to make their private pools available for anyone to swim? Is it because we have seemingly lost faith in the very notion of a republic?

ashinybrowncoat 3y ago

This seems to be a lost author looking at socialist media platforms, not capitalistic social media. Things that actually have a cost should not necessarily be provided for free, even if one does not understand said cost (such as the author of this story). Providing an API is an add-on and not actually required of any social media platform but it does cost money to maintain the storage of the content as well as to not just build said API but also to maintain it (including keeping it secure and scalable). This author seems to be thinking of some kind of socialist media platforms, not the current capitalist social media platforms we have now. Even so, I would definitely welcome the idea of socialistic media platforms although I am not even sure how one could be built and/or managed. This sounds about the same as "McDonald's should be free" and I don't really see a big difference other than the actual consumption of said good/service.

beders 3y ago

It's stunning to me how little people like the author of this piece know about the economics of running large scale sites like reddit or twitter. Nothing is free, including providing and maintaining APIs. Someone's gotta pay for the ramen, especially if you can't run ads.

oth001 3y ago
Read the footnote.
malfist 3y ago
If you read the article and not just the headline, the author is in favor of charging a cost based fee for API access. A fee that would cover hosting and serving the API, but not generate profit for the parent company benefiting from the 3P users' content.
- andersa 3y ago
  There would be no benefit to providing it, then.
  - margalabargala 3y ago
    No first-order benefit, perhaps. It shouldn't be difficult to look to second- and third-order benefits, though, such as increasing your user base and the amount of generated content.
    - andersa 3y ago
      That makes sense.
thulesgold 3y ago
I have a theory that governments and corporations across the world are pushing hard for free APIs. If they cost anything, even very little, then it makes astroturfing too expensive.

deleted 3y ago

[deleted]

simonw 3y ago

I've found myself resisting posting comments on Reddit recently - things like answers to simple questions people have about SQLite - partly because I don't want to cross the picket line but also because I realize that I was relying on a very light social contract that was in place there: I would share my knowledge for free, in exchange for which I knew that I was contributing to a larger dataset that myself and others could get back out again. If Reddit are cutting off free API access, that social contract no longer holds. Why should I work to benefit their service if they're hoarding the resulting data and not making it available to me or people like me in the future? I thought I was vanishingly rare in caring about this kind of thing, but given the mass Reddit blackout over the changes to the API apparently I'm not!

xeromal 3y ago
I blocked reddit in my hosts file and have found my news through other means. Here and fark.com lol
- ProllyInfamous 3y ago
  Same on my local PiHole =D It's worked the couple times I instinctively sought reddit advice.
  - stevenhubertron 3y ago
    Same for me on NextDNS
dvt 3y ago
> I was relying on a very light social contract that was in place there This was never a social contract so I actually wonder how you or the original blog poster ever got to that idea. In fact, it's explicitly not the social contract if you ever read the Terms of Service of sites like Twitter and/or Reddit.
- derefr 3y ago
  You're thinking of a legal contract. A social contract is what two sides do in practice in order to not piss the other off.

miahwilde 3y ago

Rather than APIs for everything I'd prefer that we figure out (technically, socially, politically, economically) how to separate data and applications on the "web platform" in the same way that we do on a local OS. Keeping files as distinct from the applications that use them has been a pretty effective design principle.

w0mbat 3y ago

The trouble with a free API is that someone can write a $1 client that hides all the ads (the number one requested feature), and now your website has no income. An ad-supported website has to get paid. On the web there is at least some stuff they can do to detect ad blockers, or show ads in sneaky ways that change over time and are hard to block.

Blahah 3y ago
That's a good argument against ads, because ads introduce a fundamental misalignment of incentives between the platform and the user. There are many ways reddit could build a collaborative business from their current situation. Advertising is very clearly not going to work, so they just need to look outside that box.
- zokier 3y ago
  > There are many ways reddit could build a collaborative business from their current situation Like charging for API access? Yeah, that went well...
  - throwaway32908 3y ago
    The Spotify solution (premium users can use any third-party client) would have made lots of money with less uproar. Another bonus, solves the supposed issues with allowing NSFW content through APIs. Google, for one, take a credit card as proof of being 18+ in the EU (you have a choice between that and scanning an ID card). Or if Reddit doesn't want to handle billing the users directly for whatever reason, require signin and charge the devs per user. Even some sort of grace period and reasonable prices per-request would have made a big difference, Apollo dev's hard blocker seemed to be conflict with existing billing cycles.

kmeisthax 3y ago

Related note in the sidebar: NY State is considering a bill to require social media companies have open API access[0]. I have to wonder if this is something either PIRG or FUTO Tech/Louis Rossman are pushing for - in which case, good on them. [0] https://www.nysenate.gov/legislation/bills/2023/s6686 https://www.nysenate.gov/legislation/bills/2023/s6686

SoftTalker 3y ago

"If a company like Reddit or Twitter derives most of its value from content that users write for free then it must provide APIs for anyone to download and manipulate that content." I disagree that Reddit or Twitter derive value from content. They derive value from eyeballs and showing ads to those eyeballs. If people use APIs as a way to feed their eyeballs without looking at the ads, then Reddit or Twitter gain nothing in exchange for their massive expense of the hardware and staff required to run these services. The other option is that people would pay to use Reddit or Twitter. But they would lose 90% of their users if they took that route, because what they do does not provide enough value that a typical user would pay for it. For most people, Reddit and Twitter are time sinks: mindless entertainment to consume to pass the time. In a prior generation they would be watching TV game shows or soap operas. They wouldn't pay $10 a month to use it, or even $10/year. They would just jump to the next best free platform and continue there.

uncletammy 3y ago
> The other option is that people would pay to use Reddit or Twitter. But they would lose 90% of their users if they took that route, because what they do does not provide enough value that a typical user would pay for it. I see this repeated all the time but I've never seen any evidence cited to support it. Also, 90% of the users leaving when the site switches to a paid model doesn't necessarily mean the quality of the site or their profitability will go down. I'd pay a pretty hefty monthly fee for most of the social media sites that I've now stopped using as a result of their heavy handed strategies to maximize ad revenue. Also, since when did we start deciding how much value a thing could potentially provide to an individual? Isn't that what "the free market" is supposed to sort out?

downWidOutaFite 3y ago

Ok then LLMs must be free.

omgmajk 3y ago

>The short sighted thing about these API fees is they will harm the company in the long term. Black-belt in running a public company right there.

ProllyInfamous 3y ago

Why not just make API accessible to paid "Reddit Gold" members?! Then there'd actually be a reason to get gilded up, and 3rd-Party Apps would still work for those paying members.

drbawb 3y ago
>Then there'd actually be a reason to get gilded up ... I have to imagine reddit has already thought of this and come up with some justification for why it won't work - because the proposed strategy is just so incredibly lazy. Ultimately I suspect the data somehow suggests (a) the network provider has to provide free access in order to sustain the criticality of the social network, and (b) charging for something that other people are getting for free (a la twitter's 8$ checkmarks, reddit gold, etc.) is just not a very viable product strategy. So Reddit has decided to stick 3rd party devs with the thorny problem of direct-monetization of users, wherein their cost to the developer is usage-billed while the revenue to the developer is presumably fixed; meanwhile Reddit can cash the checks from their mobile ads.
- deleted 3y ago
  [deleted]

sureglymop 3y ago

But a website is already parsable? When I needed some content from a website without an api I just faked the user agent and parsed it with soup.

jmyeet 3y ago

The problem here is capitalism. You can fight it with high-minded ideals like free API access but that’s really just polishing the brass on the Titanic. UGC sites need to be collectively owned by those who create the value or, in the very least, held in a nonprofit like Wikipedia is. Without such guardrails there will be constant pressure to extract more and more value from the users who create the value.

paxys 3y ago

Such arguments are always just different forms of "everything on the internet must be free, always", which is a valid ideological position to hold, sure, but one should also realize that it is not going to work in the real world.

cm2012 3y ago

All of your labor should be free also. What do you mean you want a six figure salary? How incredibly greedy of you. It's fair to pay you only what you need to survive.

randomdata 3y ago
> All of your labor should be free also. Yes, it is, and that's ultimately the contention here. Reddit has taken many millions of man hours of free labor. Now it wants to charge for the fruits of that free labor. Legally, they are within their rights to do so. Socially, however, it's thought to be uncouth to exploit workers like that.
- pessimizer 3y ago
  > Reddit has taken many millions of man hours of free labor. Reddit has accepted many millions of man-hours of labor. They didn't kick down anybody's door and take it.
  - randomdata 3y ago
    > Reddit has accepted many millions of man-hours of labor. Yes, "accept or receive (someone or something)" is indeed the literal dictionary definition of "take". Thanks for subbing in an alternate word that means the exact same thing. I am sure we have all derived great value from your efforts. Or at least a hearty laugh at the pointlessness of it.
- memefrog 3y ago
  Shitposting on reddit isn't "labour".

mikequinlan 3y ago

The article disagrees with the headline. The article says >*I don’t mind a site charging a nominal fee for API access. Either to cover the cost of API service itself, or more importantly to encourage API developers to be efficient when making API requests.

Ekaros 3y ago

I just consider anything I post on Internet like this comment to be donation to who ever hosts it... They can do anything they want with it, and take all the responsibility of things like hosting or safe keeping...

frankreyes 3y ago

There's some argument to be made because of Section 230. They are just distributors of user generated content and gatekeeping may go against that.

chaosjevil 3y ago

"Must" [as "be necessary"] and "should" are ultimately meaningless when you're dealing with corporate. Corporate only understands "allowed", "required", and "forbidden". Handle it accordingly.

ghostoftiber 3y ago

[ LAUGHS IN LEXISNEXIS ] Everyone seems to be fixated on the idea that the API should be free and no-one seems to be mentioning that there's some very successful business models built around charging people access to content which the parent company didn't generate. LexisNexis, for legal searches, Elsevier for scientific papers, Facebook, for random graph searches (and occasional political data scraping), Twitter, etc. It feels scummy to me too to take something free and charge people for it, but it's also not this unique thing people seem to be screaming about. Just go somewhere else if the API is that important to you. (Like hackernews). :)

memefrog 3y ago
LexisNexis does not just "charge people access to content which the parent company didn't generate". I could access most legislation through the government legislation website that shows the current version of every piece of legislation. I could piece together legislative history by looking through all the various amendments over the years compared to the originally enacted Acts. OR, I could look at LexisNexis, who have done all that work already and also have notes for pretty much every provision linking all that data together with academic and judicial commentary, major cases on those provisions, etc. Similarly, I can look up unreported senior courts cases on the 'Judicial Decisions Online' section of the courts of New Zealand website, for free. But what about the notes written by experienced barristers in the law reports? What about the database cataloguing every judgment that has referred to every other judgment, giving an indication of whether the judgment is still good law? It seems very popular these days to crap on businesses like LexisNexis and it does sometimes feel like they're taking the mickey given their prices and the terrible web interface to their databases. The fact you can't even 'open in new tab' properly is infuriating. But they're not just selling access to something that ought to be free, as many people seem to like to claim. They provide a lot more than just the raw judgments and statutes.

CrampusDestrus 3y ago

>APIs for content sites must be free so..... whose tax money are we gonna use to pay content sites owners for this?

deleted 3y ago

[deleted]

AYBABTME 3y ago

Content sites living off ads revenue with APIs should require API users to display ads. End of story.

Havoc 3y ago

>simple static dump is the bare minimum to fulfill the social contract I don't see how that follows? Would be awesome, but I don't see how posting on a platform creates such an obligation to render such a service (which costs money)

PaulHoule 3y ago

I would frame it as "messaging applications should interoperate". People understand that a T-Mobile customer can send a text to an AT&T customer. They understand that a Samsung owner can call an iPhone owner. Why do they accept that a Slack user can't send a message to a Discord user? Why do they accept that Google has made 11+ different messaging applications that aren't compatible with each other? Why do they accept that you can't call a Skype for business customer with Skype? We've had 30 years of stagnation because messaging applications don't interoperate. That's why the story is always "Hey, remember back when ICQ used to work? You should try Skype, it really works..." Once Skype becomes successful at the two-sided market game the honeymoon is over and the vendor has no incentive to keep it working and they figure you'll keep trying to use it anyway. If there were competition for both the server and the client there would be continuous pressure to keep the products working, and to improve them. So instead of Facebook Messenger being the same as AOL Instant Messenger except for Facebook, we'd have had 30 years of progress and would have apps that look like something out of The Jetsons.

senko 3y ago
> People understand that a T-Mobile customer can send a text to an AT&T customer. They understand that a Samsung owner can call an iPhone owner. That was a hard-won victory: https://en.m.wikipedia.org/wiki/Kingsbury_Commitment https://en.m.wikipedia.org/wiki/Kingsbury_Commitment A hundred years later, we've got the same story, except today's Big Tech had wised up.
- PaulHoule 3y ago
  The EU can do better than spamming us with cookie popups. They'd like to be relevant, why don't they try passing legislation for this?
  - senko 3y ago
    They do try: https://news.ycombinator.com/item?id=36324118 https://news.ycombinator.com/item?id=36324118 But keep in mind that EU legislature (and "government", such as it is) is as dysfunctional as the US one, just in different ways, don't expect much enlightened lawmaking there either.

__MatrixMan__ 3y ago

I haven't been following this much, having quit Reddit many years ago, but I bet that it's a step on the path towards containerized headless browsers being used to provide unauthorized API's.

deleted 3y ago

[deleted]

prima-facie 3y ago

The third party apps were used to create much of the content which they now want to sell. They're trying to get the milk after shooting the cow.

zackmorris 3y ago

I'd go a step further: APIs shouldn't exist. I know, it's a hilarious sentiment in these times. But I really feel that most of the problems today around walled gardens and surveillance capitalism stem from not using the web directly as it was originally envisioned. If we had properly formatted HTML, little/no Javascript, a standard auth mechanism that isn't terrible (HTTP auth or a cookie like the good 'ol days), automatic sitemaps or semantic representations without manual intervention, best practices around site updates and maintaining previous versions, etc etc, then why would we need an API? Since that stuff will never happen, I'd vote for third parties to provide open source APIs for existing sites. Companies could even hire those third parties to maintain APIs for them. Disclaimer: I'm paid to write APIs and I'm just so tired.

AA-BA-94-2A-56 3y ago
Eh, a JSON blob is still much more lightweight than HTML/XML. This is important in less developed areas of the world, where internet speed may not be as fast.

tantalor 3y ago

These sites also produce a useful ranking which is separate from the content. They deserve to be compensated for that if you want it.

rosmax_1337 3y ago

Very large websites should be sized by the government as they become a public good, and be subject to control which stems from society at large. The only question is which government should size a website which the entire globe contributed to? Only in a world like that can a reasonable policy like this be enacted. The second thing that should be enacted: these sized websites may only censor content that is proven in court to be unlawful.

cbeach 3y ago

It’s an interesting argument, but I don’t think users provide their content for altruistic purposes. Some do it to benefit from a sense of participating in community, some to build reputation and some to prove their expertise. In all cases, they get something from the site that’s hosting their content. So why should the site that’s hosting content not be allowed to have a business model that pays the bills? Given a choice between the following, which would you rather? * insidious adverts that compromise the impartiality of the platform (because thou shalt not offend the advertiser) * charging for API access, which affects a tiny proportion of users, and would not in any way prevent you accessing your own content (subject access and data portability is guaranteed by data protection law)

agilecobol 3y ago

100 requests per minute will be free. Can someone explain to me why this isn't enough?

alecbz 3y ago
100 requests per minute per what? Like if there's a third party app built against the API, is it 100 rpm for each user of the app, or across the entire app's usage?
- agilecobol 3y ago
  I think it is time for some of those apps to invest into caching. Also APIs are not only about third-party applications. I can think of a dozen ideas for products build around the Reddit API without going over this limit.
  - alecbz 3y ago
    > I think it is time for some of those apps to invest into caching. Even ignoring staleness issues, storing a bunch of reddit's content for them is likely going to get expensive fast as well > Also APIs are not only about third-party applications. Yeah but aren't third-party apps mostly what people are upset about?

robbiemitchell 3y ago

The context is a for-profit company. If a third-party using the API is doing so as a substitute for someone using the ad-supported site, the API will be priced to equal the revenue lost from using the site. If the people switching from UI to API are more valuable than usual, the API will cost more than usual. If the API is being used to support something new -- i.e., not a replacement for a user on the site, but as a new revenue source -- it will be priced to maximize overall API revenue, or overall revenue as an indirect result of growth supported by the API. (Maybe it's one massive customer using it or 100,000 freemium developers.) The idea that a company "must" do something doesn't make any sense. And the assertion that the company is harming itself is ultimately something the Board, acting on behalf of shareholders, is responsible for. (FWIW, I agree Reddit is shooting itself in the foot.)

ktosobcy 3y ago

It doesn't even have to be API... FB/instagram quite often prohibit you from viewing something if you don't have account... :/

Bjorkbat 3y ago

Personally I always considered the social contract to be that the price of admission for a free social network / online community is that you need to put up with ads and attempts to monetize the data, provided the latter isn’t an unreasonable breach of privacy. A free API never crossed my mind. It’s why I have to admit that I am not sympathetic to third-party apps like Apollo. They implicitly violated the social contract I recognize by providing users with an experience that circumvents Reddit’s attempts to make money. Reddit responded by finding a way to put a price on the ad-free experience.

TechBro8615 3y ago

If you don't provide an API for your content, but you're happy to render some HTML containing the content when I browse to your website, then if I want the content, I will get the content. No API is necessary. I will happily waste your resources scraping HTML if it's necessary to get the content I want, so it's in your best interest to provide it via an API, which is ultimately just a more efficient interface to the same data you're already providing me for free.

gary_0 3y ago
In the era before JSON and pervasive Javascript, HTML was the API. The Web is an API. The people saying "API" are really just saying "the same data that's in the HTML, but stripped of extra junk". HTML was never supposed to be about presentation, it was data meant for a variety of user agents. Has everyone forgotten? Are they too young to remember? I feel like I'm taking crazy pills!

Willish42 3y ago

> Their main intent is to destroy third party apps that no longer aid the company’s business goals. But they’re also trying to make a few million bucks a year licensing access to data, particularly on the back of AI training. It’s wrong. This seems backwards to me. It's obviously a move to try (and likely fail, assuming screen scraping works well enough) getting in on the AI money train by charging for Reddit's data being used to train LLMs. I don't think Appolo and other 3P clients are the target of this API change, even if the results of it are anti-competitive and should be stopped regardless of intent.

joshsabol46 3y ago

+1 I tweeted exactly this same sentiment just yesterday: > This Reddit / Stack Overflow API drama feels weird to me as a user. It's USER submitted data. R/SO didn't create it. > So what ChatGPT scraped it? They're benefiting the same users who contributed the content on R/SO, and then some. > It's not like ChatGPT is cloning them. > It would be like if Printing Houses tried to block authors from publishing their content as an eBook or Audiobook. > Users contributing to your ecosystem doesn't give you perpetual dominion over that content

ThinkBeat 3y ago

""the social contract"" There is no such thing. Faceboook, Twitter, Reddit, Instagram whatever exist to make money, or whatever those who own it or run it wants. It is your choice. You wish to spend your time adding value to a website someone else runs great. Does that give you rights? Nope. Normally in extremely obfuscated ways, the contract you do enter into by using a site will tell you exactly that you have no rights at all and they own everything you do. I agree that it can be unfair. But you actively decide to provide content. You cant make up the rules for the sites you do use. You can stop whenever you want. You can start your own site. I did so a few years ago when I got sick of this shit. On the other hand, I have few visitors. I can howl in the wilderness. For me it is preferable. If you do engage in one of the big social sites, you are given access to an enormous audience. You can post something that "goes viral". You get fame, site gets $$$$, maybe you get $$$. News can easily quote you from some of the big social sites. How often do you: Thinkbeat today said on his blog thinkbeat.blog that "" Compared to Such and Someone posted on Twitter...

r2vcap 3y ago

It is true that Reddit has problems with the way it operates. But the absurd argument that it should be provided "must be free" is not convincing. Such nonsensical claims are likely only made by socialists.

wg0 3y ago

Screen scrapping is still doable/feasible/economical in 2023? If not legally, technically? What about Cloud flare protections, bot blocking and what not?

dahwolf 3y ago

In my opinion, the author doesn't really think things through, it's just dropping some confident one-dimensional opinions without looking at multiple perspectives. "The key thing here is social media sites don’t produce content. They merely host it. Millions of users create the content expecting it will be widely available. Locking down an API breaks that social contract." "Merely host it" is framed here like it's no big deal. It's a huge fucking deal. Giant social networks are incredibly complicated to run, and it's not only technical aspects that make it so. You can hate these companies all you want, but let's not be ridiculous in suggesting that they offer little to no value. Second, the idea that content producers produce all the real value on the network is incomplete. First because of the reason mentioned above: the network itself (the plumbing and operation) has tremendous value and there would be no content without it. Second, because freely provided content in itself offers no financial value, in fact it starts out as a cost (hosting and moderation). Only when monetized does this content provide financial value. I know that users don't give a shit about that or are hostile to monetization but that doesn't change the fact that a large social network needs to cover costs. Third, the idea that any user of a social network has the right to download/access the entirety of all content ever produced on it, and this being some right or social contract...is made up bullshit. Most people have no idea what an API even is, have no weird need to do data digging, it's not a common need or expectation at all. Fourth, the AI problem is conveniently dodged. At Stackoverflow, moderators angrily demand their data dumps back, completely ignoring the reason why they were shut down in the first place: AI. Yet nobody engages with the point that the entirety of Stackoverflow might cease to exist and become obsolete because of it. That includes all the content those moderators put so much work in to curate. In both cases, you're on a sinking ship and protest to actively sink it faster.

pentagrama 3y ago

But if the product is mostly monetized by ads, how can we expect that they let people use the API for free to create an alternative front end that gets rid of the ads? Reddit isn't Wikipedia.

Avlin67 3y ago

but if internet is only reddit and onlyFans and so one i think we enter the middle age. I miss complexe forums with passionated guys. redeit ends up rants and garbage of newbies, I dont use it so much . Maybe I'll be donwvoted but years ago I did learn and also meet people with forums. Now its redeit full of newbies

wellthisisgreat 3y ago

How about this scenario: Reddit goes ballistic and locks everything behind a 100% paywall under a (flawed) assumption that they have enough critical mass of content for people throw some $5/mo bone their way just to access existing content.

TheMode 3y ago

I may agree with the idea, but it is pure fiction. Instead of begging companies to be nice, we should figure out how to change our entire software paradigm so it becomes impossible.

hoofhearted 3y ago

Hey Nelson, you haven't really been blogging on that domain since 2001 have you? If so, that is pretty amazing, and I'll be adding it to my list of retro sites still in existence.

ygouzerh 3y ago

Putting a price on an API that is just there to parse website content is not a great idea to beginning with. It's open-door for scraping companies and uncontrolled traffic. At least it's possible to put a rate-limit on an API.

memefrog 3y ago

This post makes no argument. It is just a pile of assertions. Simply stating that there is a "social contract" doesn't make it so. What evidence does the author have that "millions of users create the content expecting it will be widely available"? Even if that's true, what evidence is there that they do so with the expectation that it will be widely available through an API or through third party apps? And even if they do expect that, by what logic can you conclude that there is now a social contract that that expectation will be fulfilled? I don't think any of that is true. I don't think people are thinking about third party app users or about the wide availability of content when they post things to Reddit or other social media websites at all. I don't think they're thinking about people using third party apps. I think they're thinking on a much more mundane level. The whole concept of a "social contract" is fraught with difficulty too. It's a relatively niche idea to try to explain why people follow laws, or how laws can be justified. The idea is that we obey the law for the same reason we obey a contract: exchange of consideration for mutual benefit. But that's not actually a very good explanation, for a long list of reasons, and it certainly can't justify laws of universal application given that eg. children can't be bound to contracts they've signed and we apply laws to everyone even if they don't want us to. The concept certainly doesn't make sense in the context of a web API for a social media website. Even if everyone "expects" that their "content" will be widely available, it does not follow that there is any obligation, legal or otherwise, on Reddit to make it widely available. There just is no logical connection there unless you build one. This post does not. It simply asserts that APIs for content sites must be free. Why? Because!

hartator 3y ago

It's what we aim to do at https://serpapi.com https://serpapi.com, making an API for everything that doesn't have an API.

rldjbpin 3y ago

clickbait aside, this seems like a myopic take on social media IMHO. i would like to rather contrast it with the notions of web3 and people wanting ownership of their content, or at least be able to use the platforms without ads or data harvesting. while the real argument in this post seems to be that the api should be priced "fairly", we need to recognize the limited revenue streams the site has been employing till now, including the meaningless awards. the underlying costs might be apple and oranges, but the ads till now have not been super-intrusive à la youtube with its upto 5-6 continuous ads, or more relatably 5 banner ads or autoplay videos like many publications. moreover, given the technically inclined audience here, we can all agree that they have not been fighting back adblockers on their web ui like twitch or news websites. i have no websites with major traffic or ever got blessed with the "HN effect" on my web infrastructure, but those who have can attest to the unaccounted costs of running a public site for free. so i find it ironic that the most backlash has been from this crowd, who also tend to be happy to pay for chatgpt plus for personal use at a monthly rate costlier* than prime and netflix combined.