6 ms·

Good luck. News is the API fees are going to destroy Apollo.

by ubermonkey 3y ago

scottydelta 3y ago
What a poor timing for something like this.
- Aulig 3y ago
  Well, teddit is not new if you mean that. I:ve been using it for a while - only downside is that it's a bit slow.
  - 47thpresident 3y ago
    I've found that the instance hosted by privacytools.io is significantly faster than the official instance. https://teddit.privacytools.io/ https://teddit.privacytools.io/
  - digitallyfree 3y ago
    I personally like the Adminforge instance https://teddit.adminforge.de https://teddit.adminforge.de. It's much quicker than the original teddit.net.
- ajcoll5 3y ago
  Actually, perfect timing. It doesn't use the official API.
seabass-labrax 3y ago
I believe Teddit uses scraping; I run a Teddit instance for myself and haven't needed to set up an API key.
- KMnO4 3y ago
  It would be great for someone to scrape Reddit and expose that information in a format compatible with the official API. So if you call /get-comments/1234 it scrapes post 1234 and returns the JSON object exactly as the official API does. Then third party clients can just point to this endpoint.
  - moneywoes 3y ago
    Won’t they just get rate limited or hit with captchas
  - rovr138 3y ago
    Currently, just add .json to the end of the url. I have no idea what will happen to this with the changes. If what you suggest is done, even a package, we could probably do a distributed pushshift[1] alternative to aggregate the data like the archiveteam's warrior[2] does and keep publishing the monthly data. [1] - reddit.com/r/pushshift [2] - https://wiki.archiveteam.org/index.php?title=ArchiveTeam_Warrior https://wiki.archiveteam.org/index.php?title=ArchiveTeam_War...
- bentcorner 3y ago
  From a very brief skim it doesn't appear to scrape: https://codeberg.org/teddit/teddit/src/branch/main/routes/home.js https://codeberg.org/teddit/teddit/src/branch/main/routes/ho... But I'm not really a web dev so I might be misreading things. Also: https://codeberg.org/teddit/teddit/issues/400 https://codeberg.org/teddit/teddit/issues/400
  - rovr138 3y ago
    You can (currently?) add .json to any url on reddit and it'll provide a json object. - https://codeberg.org/teddit/teddit/src/commit/b0ce6c52a66987543707d3559d50e4dc2d35a3e8/routes/subreddit.js#L379 https://codeberg.org/teddit/teddit/src/commit/b0ce6c52a66987... - https://codeberg.org/teddit/teddit/src/commit/b0ce6c52a66987543707d3559d50e4dc2d35a3e8/routes/subreddit.js#L642 https://codeberg.org/teddit/teddit/src/commit/b0ce6c52a66987... - https://codeberg.org/teddit/teddit/src/commit/b0ce6c52a66987543707d3559d50e4dc2d35a3e8/routes/home.js#L190 https://codeberg.org/teddit/teddit/src/commit/b0ce6c52a66987...
    - imglorp 3y ago
      That's another feature that that seems on its way out, like locking up the api..
      - rovr138 3y ago
        Do you mean that they've said it or just makes sense? Curious if they've said anything about it.
        ffpip 3y ago
        The json thing is an API request, so it will come under the new API rate limits.
- girfan 3y ago
  Right now it seems legal to scrape Reddit. But given their trajectory of making the API fairly expensive to use, do you think it's likely that they would also limit/prohibit scraping (assuming apps like Apollo start scraping as an alternative)?
  - xur17 3y ago
    My understanding is that scraping of public websites is generally legal, isn't it?
    - vcanales 3y ago
      Legal, but probably against ToS
      - Semaphor 3y ago
        That ToS is meaningless if you scrape logged out.
matheusmoreira 3y ago
HTML is a perfectly good API.
- VWWHFSfQ 3y ago
  Reddit is a SPA though right
  - matheusmoreira 3y ago
    Don't know to be honest. I assume new reddit is while old reddit isn't. Truth is the job is even easier in case of an SPA: you just need to use whatever internal APIs they built for themselves. Can't add idiotic restrictions to that without affecting the normal web service.
  - delfinom 3y ago
    SPA being a problem was last decade. Headless chromium is pretty standard for scraping nowadays.
    - VWWHFSfQ 3y ago
      How does that work for clients though. Should Apollo ship a headless chromium in their mobile app
      - rovr138 3y ago
        Can they launch a browser view hidden and scrape it? I have no idea if they can read from it.
    - moneywoes 3y ago
      Isn’t that very expensive to run at scale especially mixing in residential IPs to avoid blocking
      - ipaddr 3y ago
        Each person would spider for their own needs and most would use residential ips
willjp 3y ago
Good to know, good time to donate if I can. I love apollo
grensley 3y ago
$12,000 per 50 million requests according to a post the dev made on Reddit, which he claims translates to $20 million a year.
- grensley 3y ago
  And per user it's about $25 a year
princevegeta89 3y ago
This is the first classic example that I've encountered where a company will uses its power and ownership to completely render smaller, independent products unsustainable
- deleted 3y ago
  [deleted]

Good luck. News is the API fees are going to destroy Apollo.

scottydelta 3y ago

What a poor timing for something like this.

Aulig 3y ago
Well, teddit is not new if you mean that. I:ve been using it for a while - only downside is that it's a bit slow.
- 47thpresident 3y ago
  I've found that the instance hosted by privacytools.io is significantly faster than the official instance. https://teddit.privacytools.io/ https://teddit.privacytools.io/
- digitallyfree 3y ago
  I personally like the Adminforge instance https://teddit.adminforge.de https://teddit.adminforge.de. It's much quicker than the original teddit.net.
ajcoll5 3y ago
Actually, perfect timing. It doesn't use the official API.

seabass-labrax 3y ago

I believe Teddit uses scraping; I run a Teddit instance for myself and haven't needed to set up an API key.

KMnO4 3y ago
It would be great for someone to scrape Reddit and expose that information in a format compatible with the official API. So if you call /get-comments/1234 it scrapes post 1234 and returns the JSON object exactly as the official API does. Then third party clients can just point to this endpoint.
- moneywoes 3y ago
  Won’t they just get rate limited or hit with captchas
- rovr138 3y ago
  Currently, just add .json to the end of the url. I have no idea what will happen to this with the changes. If what you suggest is done, even a package, we could probably do a distributed pushshift[1] alternative to aggregate the data like the archiveteam's warrior[2] does and keep publishing the monthly data. [1] - reddit.com/r/pushshift [2] - https://wiki.archiveteam.org/index.php?title=ArchiveTeam_Warrior https://wiki.archiveteam.org/index.php?title=ArchiveTeam_War...
bentcorner 3y ago
From a very brief skim it doesn't appear to scrape: https://codeberg.org/teddit/teddit/src/branch/main/routes/home.js https://codeberg.org/teddit/teddit/src/branch/main/routes/ho... But I'm not really a web dev so I might be misreading things. Also: https://codeberg.org/teddit/teddit/issues/400 https://codeberg.org/teddit/teddit/issues/400
- rovr138 3y ago
  You can (currently?) add .json to any url on reddit and it'll provide a json object. - https://codeberg.org/teddit/teddit/src/commit/b0ce6c52a66987543707d3559d50e4dc2d35a3e8/routes/subreddit.js#L379 https://codeberg.org/teddit/teddit/src/commit/b0ce6c52a66987... - https://codeberg.org/teddit/teddit/src/commit/b0ce6c52a66987543707d3559d50e4dc2d35a3e8/routes/subreddit.js#L642 https://codeberg.org/teddit/teddit/src/commit/b0ce6c52a66987... - https://codeberg.org/teddit/teddit/src/commit/b0ce6c52a66987543707d3559d50e4dc2d35a3e8/routes/home.js#L190 https://codeberg.org/teddit/teddit/src/commit/b0ce6c52a66987...
  - imglorp 3y ago
    That's another feature that that seems on its way out, like locking up the api..
    - rovr138 3y ago
      Do you mean that they've said it or just makes sense? Curious if they've said anything about it.
      - ffpip 3y ago
        The json thing is an API request, so it will come under the new API rate limits.
girfan 3y ago
Right now it seems legal to scrape Reddit. But given their trajectory of making the API fairly expensive to use, do you think it's likely that they would also limit/prohibit scraping (assuming apps like Apollo start scraping as an alternative)?
- xur17 3y ago
  My understanding is that scraping of public websites is generally legal, isn't it?
  - vcanales 3y ago
    Legal, but probably against ToS
    - Semaphor 3y ago
      That ToS is meaningless if you scrape logged out.

matheusmoreira 3y ago

HTML is a perfectly good API.

VWWHFSfQ 3y ago
Reddit is a SPA though right
- matheusmoreira 3y ago
  Don't know to be honest. I assume new reddit is while old reddit isn't. Truth is the job is even easier in case of an SPA: you just need to use whatever internal APIs they built for themselves. Can't add idiotic restrictions to that without affecting the normal web service.
- delfinom 3y ago
  SPA being a problem was last decade. Headless chromium is pretty standard for scraping nowadays.
  - VWWHFSfQ 3y ago
    How does that work for clients though. Should Apollo ship a headless chromium in their mobile app
    - rovr138 3y ago
      Can they launch a browser view hidden and scrape it? I have no idea if they can read from it.
  - moneywoes 3y ago
    Isn’t that very expensive to run at scale especially mixing in residential IPs to avoid blocking
    - ipaddr 3y ago
      Each person would spider for their own needs and most would use residential ips

willjp 3y ago

Good to know, good time to donate if I can. I love apollo

grensley 3y ago

$12,000 per 50 million requests according to a post the dev made on Reddit, which he claims translates to $20 million a year.

grensley 3y ago
And per user it's about $25 a year

princevegeta89 3y ago

This is the first classic example that I've encountered where a company will uses its power and ownership to completely render smaller, independent products unsustainable

deleted 3y ago
[deleted]