9 ms·

I like vehement arguments even when I disagree with them. Based on this man's logic, if I were to run a web forum then I must provide an API or data dump to do

I like vehement arguments even when I disagree with them. Based on this man's logic, if I were to run a web forum then I must provide an API or data dump to download the posts created by the users? How about a comment section on my blog? How about a community Discord server? Why not Facebook posts? Or Instagram stories? I often get the feeling that in these circumstances people derive first principles from specific desired outcomes. Some people strongly desire that the content that has been aggregated on Reddit should be available to them. They then attempt to invent moral axioms that they believe will lead to that outcome. They also seem to reinvent history such that there was some "implicit" agreement between Reddit and the users of Reddit. IMO, just because I wrote some shitposts on Reddit does not mean that I am entitled access to every single shitpost that has ever been written on Reddit.

11235813213455 3y ago
but you can access every single shitpost from the website, so why not from API? it's lighter
- kohlerm 3y ago
  Because others use automatic ways to retrieve the data and derive value from it without paying anything
  - interactivecode 3y ago
    If reddit did that from the start we wouldn’t be in this situation in the first place?
- David_Axelrod 3y ago
  Most of the time you dont just expose your internal api routes to the world. You need to write curated public facing routes that dont include certain schema or records. Takes time to write and maintain that different set of endpoints
  - TheCoreh 3y ago
    These are already written and fully functional, and Reddit has essentially been a non-moving target in terms of features since ~2016, when they added first-party image hosting.
    - chc 3y ago
      Reddit actually has added a couple of features since then (e.g. polls), but they just didn't update the API to deal with those, so the API still remained completely stable.
  - TeMPOraL 3y ago
    Not really. Your HTML is, quite literally, a degenerate form of an API. The simplest way to offer a content API is to offer an alternative endpoint that serves the same stuff as your normal one, except without all the bullshit (er, beautifully design, interactive frontend).
    - David_Axelrod 3y ago
      Don't mind the other commentor, spirited discussion is always welcome! I was thinking about this and even hacker news is a good example of how an html view can differ from data model. Hacker news doesn't show you the vote count of every comment despite having that data available. They chose to not even render it into the template so no scraper will ever have access to that information.
    - thulesgold 3y ago
      Any issue with Reddit offering a API only to not maintain it, or uphold any sort of SLA uptime, or not worry about releasing breaking changes every week? Having a public facing API is not trivial or cost free.
      - TeMPOraL 3y ago
        > Reddit offering a API only to not maintain it If they were to offer an API that's just HTML of the website (old.reddit.com specifically, not the new one) but without the cruft that makes for 90% of the markup of a human-facing page, and which exists only to hang styles and scripts off... why wouldn't they maintain it? It's literally the same as what the browser gets, but without the bullshit. > or uphold any sort of SLA uptime Do they uphold any sort of SLA uptime for the webpage itself? The simplest API would be just the meat of the website, so it couldn't possibly be less reliable than the site itself. > not worry about releasing breaking changes every week? Reddit is a stable site. Like most social media platforms, they don't release breaking changes often (they do screw with DOM element ids and CSS classes all the time, but that is to make life harder for ad blockers, which is another topic). Sure, some things may move around, be added or removed - but this is webshit we're talking about. You can't truly rely on any API to have a stable, or well-defined structure[0] - so people are already used to treating schemas as open-ended[1] and keeping up with their changes. > Having a public facing API is not trivial or cost free. Sure. But I'm trying to establish a lower bound here, and it's clear that this is much lower costs and effort than maintaining the human-facing website itself. And I mean, remember the whole "semantic HTML" and "microformats" trends of yore? Or how HTML5 came to be, with all those tags like <em> and <section> and <article>? The whole point of that was to make HTML work as both rendering markup and machine-readable API. Consider also that the alternative isn't no public API - it's scraping. So if your public API is somehow more expensive to serve or maintain than either the website itself, or a decluttered version of it, then you're doing something wrong. -- [0] - Don't get me started on the disaster that is Swagger/OpenAPI. [1] - Something Clojure coding philosophy makes explicit: you pass around maps and arrays, you read and write the keys you know about, and stuff you don't recognize you ignore and pass without changing.
- awestroke 3y ago
  Because you can't include ads in an API response
  - jurimasa 3y ago
    Yet.
  - jodrellblank 3y ago
    Why can’t you? “Sent from my iPhone.”
    - awestroke 3y ago
      Any time I see "Sent from my iPhone" I assume the sender is a fool
      - jodrellblank 3y ago
        Are you missing the point that adverts can be inline with normal content and therefore can be included in an API result?
  - kpw94 3y ago
    You could, but with EU and other legislations you'd have to make it clear it's an ad, so in API-speak, there'd be a "is_sponsored=true". That makes it too easy for API consumers to ignore. (Compared to human eyeballs browsing a webpage) But you could make it part of the ToS: if you're using the API to present posts to users, you MUST show all the posts in the list, including the is_sponsored=true ones.
    - jodrellblank 3y ago
      That's a good point, UK Government page on "Guidance: Hidden ads: Being clear with your audience": https://www.gov.uk/government/publications/social-media-endorsements-guide-for-influencers/social-media-endorsements-being-transparent-with-your-followers https://www.gov.uk/government/publications/social-media-endo...
tmpz22 3y ago
> if I were to run a web forum then I must provide an API Realistically you're using Open source forum software like discourse that itself could provide an API interface at an amortized cost. Expecting individuals to code it themselves (and follow standards etc) is a big ask. I don't think its a hard sticking point and the majority of the encouragement is for large providers.
amelius 3y ago
> I often get the feeling that in these circumstances people derive first principles from specific desired outcomes. But that's exactly how laws are made. E.g. why do we have consumer protection? Because it helps us! It's democracy in action. It's not like nobody would ever build another forum if there were laws like the ones proposed by GP. And if you're worried that every little forum should comply to complicated laws: we could make it so that the laws only apply to large players, e.g. >1M users.
- willcipriano 3y ago
  > laws only apply to large players, e.g. >1M users. What if I don't want to share my user list with the government?
  - interactivecode 3y ago
    Then don’t run a content site?
    - willcipriano 3y ago
      What if I don't want the government to know I'm a user of Yahoo Answers?
      - interactivecode 3y ago
        Don’t connect to any ISP?
  - h4ckerle 3y ago
    You don't have to. You can self report and if you incorrectly report it is a crime. As simple as that. This is also exactly how the definition of very large online platforms (which is a site from 45+million montly active Users) in the EUs new Digital Services Act works [1]. Some companies (primarily porn) are believed to have misreported, and are being investigated for that[2]. If they are found guilty they will have to adhere to the regulation for vlops, and pay a fine of up to 6% of their annual EU revenue... (Obviously not the full sum for a first time offence) [1]https://en.wikipedia.org/wiki/Very_Large_Online_Platform https://en.wikipedia.org/wiki/Very_Large_Online_Platform [2]https://www.politico.eu/article/online-porn-websites-europe-regulation-age/ https://www.politico.eu/article/online-porn-websites-europe-...
duxup 3y ago
I’ve seen a lot of arguments that users are the folks who provide the value/ content of Reddit and so somehow Reddit must respond to their wishes. And I like that idea, a great deal. Although I have to note that users have given Reddit that content to use, for FREE, and continue to do so. They do this elsewhere too, including sites where they say don’t like the site administration, Twitter, etc… So I’m not really sure how this plays out.
- capr 3y ago
  What I think he meant by "specific desired outcomes" is arbitrary rules that cater to special interest groups rather than universal moral principles. And yes, that _is_ democracy in action, where the only rule is that the majority wins, at the expense of the minority who just lost. Your ">1M users" rule is an example of such arbitrary rule that proves his point.
ineedtosleep 3y ago
I don't think your arguments accurately cover the OP's argument at all. The key distinction is literally the first sentence: "Social media businesses should not charge* for APIs." If you have a community [whatever] service that would not be a business. If you have a blog, that's could potentially be a very business-adjacent, but I'd argue it doesn't cover the "social media" qualification, so that also doesn't apply. If you run a business and much of your content is user-generated (because it's a social media site), the OP is arguing that API access should be free/at cost.
- zoogeny 3y ago
  > OP is arguing that API access should be free/at cost. Implicitly he is also arguing that it must exist at all. But let's consider further your own commercial clause. Apple has a discussion board for their support community [1] - and now we are mandating that Apple must both have an API for users to access it and must only charge cost. Who gets to determine the cost? Is it hardware costs? Does it include R&D? Is there a fixed margin set by a regulator? The grey area on this is as wide as an ocean. But I do think it is a little funny that people are arguing about this like it is equivalent to a universal human right. 1. https://discussions.apple.com/ https://discussions.apple.com/
  - derefr 3y ago
    A server-rendered HTML website is an API. If said HTML website is unauthenticated and freely accessible by any user-agent, then it's a de-facto public API, too. This is the case for the Apple discussion boards. You don't have to authenticate with their backend before scraping anything you'd like off of the site. You can build third-party tools to read or even interact with this site, by scraping the HTML. This thing with Reddit is only the big deal that it is, because Reddit's backend blocks "app" user-agents from simply scraping pages from the non-authenticated Reddit HTML website. (If this wasn't true, they'd just do that, and none of this would be an issue.) But these third-party UAs are instead forced to go through the authenticated data API — where they can then be API-credit-limited and forced into paid data-API subscription plans.
  - cal85 3y ago
TheCoreh 3y ago
> Based on this man's logic, if I were to run a web forum then I must provide an API or data dump to download the posts created by the users? How about a comment section on my blog? How about a community Discord server? Why not Facebook posts? Or Instagram stories? Yes? I don't think it's a very radical, unreasonable or difficult thing to provide: this is available as a first class feature on most blogging/forum software (RSS feeds) and wasn't even remotely controversial 10–12 years ago. > They also seem to reinvent history such that there was some "implicit" agreement between Reddit and the users of Reddit. There was an _explicit_ agreement: the site's terms of use have allowed free API access with reasonable quotas for 15 years, and are being changed now with very short notice. Of course the terms of service can change, but users are also free to leave, complain, or demand that they don't change. There are also strong cultural factors involved. For example: just like in the US it's customary for tap water to be free at restaurants, and people would be probably angry if you decided to charge for it, on the open web it's customary for some reasonable level of API access to be free. > IMO, just because I wrote some shitposts on Reddit does not mean that I am entitled access to every single shitpost that has ever been written on Reddit. There's no such asymmetry in scale: _everyone_ wrote the content, _everyone_ is entitled to access it. If you're referring to incredibly resource intensive, mass-access to download several terabytes for AI training I can see how that should cost a price. But they're blocking reasonable use-cases.
- zoogeny 3y ago
  > users are also free to leave, complain, or demand that they don't change Yes, I have no issue with all of those. I'm poking fun at taking that desire for action from a single company in a single instance and extending it to a universal moral principle. My own opinion is that Reddit users were getting something valuable for free and it is being taken away from them. In addition to the totally reasonable protests that they are making, some are taking it a bit far and supporting their tantrum by creating axioms of morality that I believe are a bit excessive.
  - rcoveson 3y ago
    Getting something valuable for free sounds weird in the physical world but it's not weird at all in the information world, where costs can diffuse so thinly that many users never even see them. I see no reason why a non-profit reddit couldn't be free if they didn't try to host video or images (links and comments only). Server costs could absolutely be covered by donations (like "reddit gold") and intra-site advertising (sidebar subreddit ads). Just don't hire many people, don't take VC money, and lean on your community for moderation (which is perfectly morally acceptable for a non-profit entity). It's not freeloading, it's just a bunch of people dedicating so much time and attention to communicating with each other that the server costs are dwarfed by the value of the platform to its users.
jmull 3y ago
I think a lot of the people who contribute to reddit and other sites -- whose content collectively provides a great deal of value to those sites -- want fair access to their content. I think that's a good first principle. I also think "APIs for content sites must be free" is a pretty good attempt to derive a rule from the principle of fairness. So hardly an invented moral axiom, as you put it. Also, claiming a reinvention of history is unfair, since, until now, everyone has been contributing to reddit in a context where API access was free. That's the history. It's not the history that's changed, it's Reddits API access policy. Anyway, it's 100% fine and good that people decide under what circumstances they ought to be willing to invest their time and effort in a social media site. Setting some common principles, like this one, is just a good way to communicate that to the purveyors of social media sites.
ambicapter 3y ago
> I often get the feeling that in these circumstances people derive first principles from specific desired outcomes. Once you learn about motivated reasoning, you start to see it everywhere.
- LodeOfCode 3y ago
  Ironically you only start to see it where you want to see it. Motivated reasoning you agree with is just reasoning
onlyrealcuzzo 3y ago
It would be pretty easy for someone to clone your app and just take out the ads. If you're required to give access to all data with an API for free, it seems it would be hard to run any of these websites without having a massive loss.
senko 3y ago
HTTP GET is an API. All you have to do is not block well-behaved scrapers.
opportune 3y ago
I completely agree. The only real reason to be upset about the Reddit API changes is that Reddit’s official clients suck. The formatting sucks, they stuff them way too full of ads. Apparently they are less accessible for blind users, and inefficient for moderation. It’s weird to be upset about people like the Apollo dev (who could just require payment) or scrapers. They don’t really have any right to API access, and should have to pay for it.
drbawb 3y ago
>if I were to run a web forum then I must provide an API or data dump to download the posts created by the users It's implicitly provided by hosting the forum, blog, etc. at a publicly accessible URL. HTTP is an API, HTML is a data interchange format. There's this weird idea (which I've seen parroted both in this article, and in u/spez's discourse on the topic) that "scraping" is somehow evil and bad, when it's actually an intentional feature of the web. Scraping is quite literally how Google built its search empire. It seems to me, now that the major players are sufficiently centralized and entrenched, they want to turn scraping into a boogeyman so nobody can follow in their wake. The advantage of providing a "real" API is to (a) limit scope narrower than an account/session, and (b) eliminate the overhead implicit in laying out the content for human eyeballs; reducing processing costs for both the server and client. I find it somewhat hypocritical for Reddit et al. to provide the former for free, and charge for the latter, when the latter is explicitly designed to optimize their costs.
- Terr_ 3y ago
  Ultimately this is about DRM again, or at least on the same spectrum of motives and behavior: The corporate middleman wants to be able to force restrictions on how consumers manage or view the content that is being produced by other users. Either forcing you to run a proprietary closed-source client--like their official smartphone app--or else making the desktop rendering so gnarled that you pretty much have to view it exactly the way they intent while having their code pierce your privacy. It's not just about making sure ads are displayed anymore, I'm sure there's some executive who has asked about preventing people from copy-pasting text...
chc 3y ago
> I often get the feeling that in these circumstances people derive first principles from specific desired outcomes. That's a pretty normal way derive principles, isn't it? You start with an observation, and then you recursively ask "why?" It's possible that your answers are wrong, or even that your observation itself is flawed, but the concept of deriving principles from outcomes isn't really flawed. Where else are they going to come from? Should people just pick principles at random and hope they're not terrible?
- tedunangst 3y ago
  People don't seem very good or interested in assessing how their newly derived principle will apply to situations other than the one they want it for.
johannes1234321 3y ago
If you operate in GDPR land there is a requirement to provide all that data (by the user) by law already. It's not unheard of.
- zoogeny 3y ago
  Now that is something I agree with (the principle not the particular law). If we're opening it up to our own opinions - I believe I should have total control over all of the content I generate in all contexts. That would include removing the content I provably generated from any platform at any time. Note that HN doesn't even allow me to do that ... I can't delete posts after some time frame. But my understanding of GDPR is it doesn't go that far. It applies to PII (Personally Identifying Information). That would include things like mailing address, email addresses, full names, phone number, government ids, etc. I may be wrong, but I do not believe it would apply to the content of shitposts on Reddit. Where I disagree with the articles vehement moral stance ... is my right to the publicly generated content of other people. Also, if we want to get all technical, and I'm surprised no one has, there is some analogy to be made between the philosophy of free access to information provided by public libraries and this debate ...
MagicMoonlight 3y ago
Yes. Otherwise you are not entitled to use them to make money. It's not your content.
pessimizer 3y ago
> They then attempt to invent moral axioms that they believe will lead to that outcome. They also seem to reinvent history such that there was some "implicit" agreement between Reddit and the users of Reddit. I notice this also when people are talking about "the spirit of open source." No, open source licenses are licenses, not spirits, and you were contributing huge amounts of time and effort to fill a platform with content that you have absolutely no control over. People don't owe it to your OSS project not to fork it or not to become more successful than you at distributing it as a service, and Reddit can do whatever they want with your content that doesn't violate any obligations to you set out in their ToS. Next time, do AGPL or proprietary, and don't give away any content that you value to someone else's platform for free. If you want them to be obligated to distribute it for free forever, you really should be paying them.
agilecobol 3y ago
Some people perceive Reddit as a platform build with their tax money. The argument usually goes that Reddit is build with the free labor of participants, but I think they forget that they also receive an experience in return.
dahwolf 3y ago
I'm honestly shocked by how naive people are about open data/web. If a platform owner together with a community of enthusiasts produces a valuable set of content/data and then makes it openly available to the world...what exactly do you expect will happen? We've seen what happens during the last 15 years, an era of massive centralization. And now it's AI saying "thanks for everything!". But it doesn't even need to be AI. Allow me to use the free and unlimited Reddit API which apparently allows you to make billions of calls at no charge, and then rehost all that content on an ad farm. This is the real reason the original web3 (semantic web) was dead before it even started and instead of becoming more open, everyone became less open. Because giving away your main assets is suicidal.

11235813213455 3y ago

but you can access every single shitpost from the website, so why not from API? it's lighter

kohlerm 3y ago
Because others use automatic ways to retrieve the data and derive value from it without paying anything
- interactivecode 3y ago
  If reddit did that from the start we wouldn’t be in this situation in the first place?
David_Axelrod 3y ago
Most of the time you dont just expose your internal api routes to the world. You need to write curated public facing routes that dont include certain schema or records. Takes time to write and maintain that different set of endpoints
- TheCoreh 3y ago
  These are already written and fully functional, and Reddit has essentially been a non-moving target in terms of features since ~2016, when they added first-party image hosting.
  - chc 3y ago
    Reddit actually has added a couple of features since then (e.g. polls), but they just didn't update the API to deal with those, so the API still remained completely stable.
- TeMPOraL 3y ago
  Not really. Your HTML is, quite literally, a degenerate form of an API. The simplest way to offer a content API is to offer an alternative endpoint that serves the same stuff as your normal one, except without all the bullshit (er, beautifully design, interactive frontend).
  - David_Axelrod 3y ago
    Don't mind the other commentor, spirited discussion is always welcome! I was thinking about this and even hacker news is a good example of how an html view can differ from data model. Hacker news doesn't show you the vote count of every comment despite having that data available. They chose to not even render it into the template so no scraper will ever have access to that information.
  - thulesgold 3y ago
    Any issue with Reddit offering a API only to not maintain it, or uphold any sort of SLA uptime, or not worry about releasing breaking changes every week? Having a public facing API is not trivial or cost free.
    - TeMPOraL 3y ago
      > Reddit offering a API only to not maintain it If they were to offer an API that's just HTML of the website (old.reddit.com specifically, not the new one) but without the cruft that makes for 90% of the markup of a human-facing page, and which exists only to hang styles and scripts off... why wouldn't they maintain it? It's literally the same as what the browser gets, but without the bullshit. > or uphold any sort of SLA uptime Do they uphold any sort of SLA uptime for the webpage itself? The simplest API would be just the meat of the website, so it couldn't possibly be less reliable than the site itself. > not worry about releasing breaking changes every week? Reddit is a stable site. Like most social media platforms, they don't release breaking changes often (they do screw with DOM element ids and CSS classes all the time, but that is to make life harder for ad blockers, which is another topic). Sure, some things may move around, be added or removed - but this is webshit we're talking about. You can't truly rely on any API to have a stable, or well-defined structure[0] - so people are already used to treating schemas as open-ended[1] and keeping up with their changes. > Having a public facing API is not trivial or cost free. Sure. But I'm trying to establish a lower bound here, and it's clear that this is much lower costs and effort than maintaining the human-facing website itself. And I mean, remember the whole "semantic HTML" and "microformats" trends of yore? Or how HTML5 came to be, with all those tags like <em> and <section> and <article>? The whole point of that was to make HTML work as both rendering markup and machine-readable API. Consider also that the alternative isn't no public API - it's scraping. So if your public API is somehow more expensive to serve or maintain than either the website itself, or a decluttered version of it, then you're doing something wrong. -- [0] - Don't get me started on the disaster that is Swagger/OpenAPI. [1] - Something Clojure coding philosophy makes explicit: you pass around maps and arrays, you read and write the keys you know about, and stuff you don't recognize you ignore and pass without changing.
awestroke 3y ago
Because you can't include ads in an API response
- jurimasa 3y ago
  Yet.
- jodrellblank 3y ago
  Why can’t you? “Sent from my iPhone.”
  - awestroke 3y ago
    Any time I see "Sent from my iPhone" I assume the sender is a fool
    - jodrellblank 3y ago
      Are you missing the point that adverts can be inline with normal content and therefore can be included in an API result?
- kpw94 3y ago
  You could, but with EU and other legislations you'd have to make it clear it's an ad, so in API-speak, there'd be a "is_sponsored=true". That makes it too easy for API consumers to ignore. (Compared to human eyeballs browsing a webpage) But you could make it part of the ToS: if you're using the API to present posts to users, you MUST show all the posts in the list, including the is_sponsored=true ones.
  - jodrellblank 3y ago
    That's a good point, UK Government page on "Guidance: Hidden ads: Being clear with your audience": https://www.gov.uk/government/publications/social-media-endorsements-guide-for-influencers/social-media-endorsements-being-transparent-with-your-followers https://www.gov.uk/government/publications/social-media-endo...

tmpz22 3y ago

> if I were to run a web forum then I must provide an API Realistically you're using Open source forum software like discourse that itself could provide an API interface at an amortized cost. Expecting individuals to code it themselves (and follow standards etc) is a big ask. I don't think its a hard sticking point and the majority of the encouragement is for large providers.

amelius 3y ago

> I often get the feeling that in these circumstances people derive first principles from specific desired outcomes. But that's exactly how laws are made. E.g. why do we have consumer protection? Because it helps us! It's democracy in action. It's not like nobody would ever build another forum if there were laws like the ones proposed by GP. And if you're worried that every little forum should comply to complicated laws: we could make it so that the laws only apply to large players, e.g. >1M users.

willcipriano 3y ago
> laws only apply to large players, e.g. >1M users. What if I don't want to share my user list with the government?
- interactivecode 3y ago
  Then don’t run a content site?
  - willcipriano 3y ago
    What if I don't want the government to know I'm a user of Yahoo Answers?
    - interactivecode 3y ago
      Don’t connect to any ISP?
- h4ckerle 3y ago
  You don't have to. You can self report and if you incorrectly report it is a crime. As simple as that. This is also exactly how the definition of very large online platforms (which is a site from 45+million montly active Users) in the EUs new Digital Services Act works [1]. Some companies (primarily porn) are believed to have misreported, and are being investigated for that[2]. If they are found guilty they will have to adhere to the regulation for vlops, and pay a fine of up to 6% of their annual EU revenue... (Obviously not the full sum for a first time offence) [1]https://en.wikipedia.org/wiki/Very_Large_Online_Platform https://en.wikipedia.org/wiki/Very_Large_Online_Platform [2]https://www.politico.eu/article/online-porn-websites-europe-regulation-age/ https://www.politico.eu/article/online-porn-websites-europe-...

duxup 3y ago

I’ve seen a lot of arguments that users are the folks who provide the value/ content of Reddit and so somehow Reddit must respond to their wishes. And I like that idea, a great deal. Although I have to note that users have given Reddit that content to use, for FREE, and continue to do so. They do this elsewhere too, including sites where they say don’t like the site administration, Twitter, etc… So I’m not really sure how this plays out.

capr 3y ago
What I think he meant by "specific desired outcomes" is arbitrary rules that cater to special interest groups rather than universal moral principles. And yes, that _is_ democracy in action, where the only rule is that the majority wins, at the expense of the minority who just lost. Your ">1M users" rule is an example of such arbitrary rule that proves his point.

ineedtosleep 3y ago

I don't think your arguments accurately cover the OP's argument at all. The key distinction is literally the first sentence: "Social media businesses should not charge* for APIs." If you have a community [whatever] service that would not be a business. If you have a blog, that's could potentially be a very business-adjacent, but I'd argue it doesn't cover the "social media" qualification, so that also doesn't apply. If you run a business and much of your content is user-generated (because it's a social media site), the OP is arguing that API access should be free/at cost.

zoogeny 3y ago
> OP is arguing that API access should be free/at cost. Implicitly he is also arguing that it must exist at all. But let's consider further your own commercial clause. Apple has a discussion board for their support community [1] - and now we are mandating that Apple must both have an API for users to access it and must only charge cost. Who gets to determine the cost? Is it hardware costs? Does it include R&D? Is there a fixed margin set by a regulator? The grey area on this is as wide as an ocean. But I do think it is a little funny that people are arguing about this like it is equivalent to a universal human right. 1. https://discussions.apple.com/ https://discussions.apple.com/
- derefr 3y ago
  A server-rendered HTML website is an API. If said HTML website is unauthenticated and freely accessible by any user-agent, then it's a de-facto public API, too. This is the case for the Apple discussion boards. You don't have to authenticate with their backend before scraping anything you'd like off of the site. You can build third-party tools to read or even interact with this site, by scraping the HTML. This thing with Reddit is only the big deal that it is, because Reddit's backend blocks "app" user-agents from simply scraping pages from the non-authenticated Reddit HTML website. (If this wasn't true, they'd just do that, and none of this would be an issue.) But these third-party UAs are instead forced to go through the authenticated data API — where they can then be API-credit-limited and forced into paid data-API subscription plans.
- cal85 3y ago

TheCoreh 3y ago

> Based on this man's logic, if I were to run a web forum then I must provide an API or data dump to download the posts created by the users? How about a comment section on my blog? How about a community Discord server? Why not Facebook posts? Or Instagram stories? Yes? I don't think it's a very radical, unreasonable or difficult thing to provide: this is available as a first class feature on most blogging/forum software (RSS feeds) and wasn't even remotely controversial 10–12 years ago. > They also seem to reinvent history such that there was some "implicit" agreement between Reddit and the users of Reddit. There was an _explicit_ agreement: the site's terms of use have allowed free API access with reasonable quotas for 15 years, and are being changed now with very short notice. Of course the terms of service can change, but users are also free to leave, complain, or demand that they don't change. There are also strong cultural factors involved. For example: just like in the US it's customary for tap water to be free at restaurants, and people would be probably angry if you decided to charge for it, on the open web it's customary for some reasonable level of API access to be free. > IMO, just because I wrote some shitposts on Reddit does not mean that I am entitled access to every single shitpost that has ever been written on Reddit. There's no such asymmetry in scale: _everyone_ wrote the content, _everyone_ is entitled to access it. If you're referring to incredibly resource intensive, mass-access to download several terabytes for AI training I can see how that should cost a price. But they're blocking reasonable use-cases.

zoogeny 3y ago
> users are also free to leave, complain, or demand that they don't change Yes, I have no issue with all of those. I'm poking fun at taking that desire for action from a single company in a single instance and extending it to a universal moral principle. My own opinion is that Reddit users were getting something valuable for free and it is being taken away from them. In addition to the totally reasonable protests that they are making, some are taking it a bit far and supporting their tantrum by creating axioms of morality that I believe are a bit excessive.
- rcoveson 3y ago
  Getting something valuable for free sounds weird in the physical world but it's not weird at all in the information world, where costs can diffuse so thinly that many users never even see them. I see no reason why a non-profit reddit couldn't be free if they didn't try to host video or images (links and comments only). Server costs could absolutely be covered by donations (like "reddit gold") and intra-site advertising (sidebar subreddit ads). Just don't hire many people, don't take VC money, and lean on your community for moderation (which is perfectly morally acceptable for a non-profit entity). It's not freeloading, it's just a bunch of people dedicating so much time and attention to communicating with each other that the server costs are dwarfed by the value of the platform to its users.

jmull 3y ago

I think a lot of the people who contribute to reddit and other sites -- whose content collectively provides a great deal of value to those sites -- want fair access to their content. I think that's a good first principle. I also think "APIs for content sites must be free" is a pretty good attempt to derive a rule from the principle of fairness. So hardly an invented moral axiom, as you put it. Also, claiming a reinvention of history is unfair, since, until now, everyone has been contributing to reddit in a context where API access was free. That's the history. It's not the history that's changed, it's Reddits API access policy. Anyway, it's 100% fine and good that people decide under what circumstances they ought to be willing to invest their time and effort in a social media site. Setting some common principles, like this one, is just a good way to communicate that to the purveyors of social media sites.

ambicapter 3y ago

> I often get the feeling that in these circumstances people derive first principles from specific desired outcomes. Once you learn about motivated reasoning, you start to see it everywhere.

LodeOfCode 3y ago
Ironically you only start to see it where you want to see it. Motivated reasoning you agree with is just reasoning

onlyrealcuzzo 3y ago

It would be pretty easy for someone to clone your app and just take out the ads. If you're required to give access to all data with an API for free, it seems it would be hard to run any of these websites without having a massive loss.

senko 3y ago

HTTP GET is an API. All you have to do is not block well-behaved scrapers.

opportune 3y ago

I completely agree. The only real reason to be upset about the Reddit API changes is that Reddit’s official clients suck. The formatting sucks, they stuff them way too full of ads. Apparently they are less accessible for blind users, and inefficient for moderation. It’s weird to be upset about people like the Apollo dev (who could just require payment) or scrapers. They don’t really have any right to API access, and should have to pay for it.

drbawb 3y ago

>if I were to run a web forum then I must provide an API or data dump to download the posts created by the users It's implicitly provided by hosting the forum, blog, etc. at a publicly accessible URL. HTTP is an API, HTML is a data interchange format. There's this weird idea (which I've seen parroted both in this article, and in u/spez's discourse on the topic) that "scraping" is somehow evil and bad, when it's actually an intentional feature of the web. Scraping is quite literally how Google built its search empire. It seems to me, now that the major players are sufficiently centralized and entrenched, they want to turn scraping into a boogeyman so nobody can follow in their wake. The advantage of providing a "real" API is to (a) limit scope narrower than an account/session, and (b) eliminate the overhead implicit in laying out the content for human eyeballs; reducing processing costs for both the server and client. I find it somewhat hypocritical for Reddit et al. to provide the former for free, and charge for the latter, when the latter is explicitly designed to optimize their costs.

Terr_ 3y ago
Ultimately this is about DRM again, or at least on the same spectrum of motives and behavior: The corporate middleman wants to be able to force restrictions on how consumers manage or view the content that is being produced by other users. Either forcing you to run a proprietary closed-source client--like their official smartphone app--or else making the desktop rendering so gnarled that you pretty much have to view it exactly the way they intent while having their code pierce your privacy. It's not just about making sure ads are displayed anymore, I'm sure there's some executive who has asked about preventing people from copy-pasting text...

chc 3y ago

> I often get the feeling that in these circumstances people derive first principles from specific desired outcomes. That's a pretty normal way derive principles, isn't it? You start with an observation, and then you recursively ask "why?" It's possible that your answers are wrong, or even that your observation itself is flawed, but the concept of deriving principles from outcomes isn't really flawed. Where else are they going to come from? Should people just pick principles at random and hope they're not terrible?

tedunangst 3y ago
People don't seem very good or interested in assessing how their newly derived principle will apply to situations other than the one they want it for.

johannes1234321 3y ago

If you operate in GDPR land there is a requirement to provide all that data (by the user) by law already. It's not unheard of.

zoogeny 3y ago
Now that is something I agree with (the principle not the particular law). If we're opening it up to our own opinions - I believe I should have total control over all of the content I generate in all contexts. That would include removing the content I provably generated from any platform at any time. Note that HN doesn't even allow me to do that ... I can't delete posts after some time frame. But my understanding of GDPR is it doesn't go that far. It applies to PII (Personally Identifying Information). That would include things like mailing address, email addresses, full names, phone number, government ids, etc. I may be wrong, but I do not believe it would apply to the content of shitposts on Reddit. Where I disagree with the articles vehement moral stance ... is my right to the publicly generated content of other people. Also, if we want to get all technical, and I'm surprised no one has, there is some analogy to be made between the philosophy of free access to information provided by public libraries and this debate ...

MagicMoonlight 3y ago

Yes. Otherwise you are not entitled to use them to make money. It's not your content.

pessimizer 3y ago

> They then attempt to invent moral axioms that they believe will lead to that outcome. They also seem to reinvent history such that there was some "implicit" agreement between Reddit and the users of Reddit. I notice this also when people are talking about "the spirit of open source." No, open source licenses are licenses, not spirits, and you were contributing huge amounts of time and effort to fill a platform with content that you have absolutely no control over. People don't owe it to your OSS project not to fork it or not to become more successful than you at distributing it as a service, and Reddit can do whatever they want with your content that doesn't violate any obligations to you set out in their ToS. Next time, do AGPL or proprietary, and don't give away any content that you value to someone else's platform for free. If you want them to be obligated to distribute it for free forever, you really should be paying them.

agilecobol 3y ago

Some people perceive Reddit as a platform build with their tax money. The argument usually goes that Reddit is build with the free labor of participants, but I think they forget that they also receive an experience in return.

dahwolf 3y ago

I'm honestly shocked by how naive people are about open data/web. If a platform owner together with a community of enthusiasts produces a valuable set of content/data and then makes it openly available to the world...what exactly do you expect will happen? We've seen what happens during the last 15 years, an era of massive centralization. And now it's AI saying "thanks for everything!". But it doesn't even need to be AI. Allow me to use the free and unlimited Reddit API which apparently allows you to make billions of calls at no charge, and then rehost all that content on an ad farm. This is the real reason the original web3 (semantic web) was dead before it even started and instead of becoming more open, everyone became less open. Because giving away your main assets is suicidal.