3 ms·
Gotcha - it's times like that that I go with an old-fashioned raw HTTP request and parse it with a DOM parsing engine. Sometimes the "web-facing API" (so-to-spe
by folkhack 7y ago
Gotcha - it's times like that that I go with an old-fashioned raw HTTP request and parse it with a DOM parsing engine. Sometimes the "web-facing API" (so-to-speak) is better than the official API.
Super cool project!
- dang 7y agoYes, an API that is isomorphic to the website itself would be much easier to use. Our plan is for the next version of HN's API to simply serve a JSON version of every page. In the meantime, if you scrape the HTML, please respect https://news.ycombinator.com/robots.txt https://news.ycombinator.com/robots.txt and don't do it more than once per 30 seconds. HN's app server runs on a single core, so we don't have a lot of performance to spare! This is in the works to change as well.
- protomikron 7y ago> HN's app server runs on a single core, so we don't have a lot of performance to spare! That's interesting. Is there any more information about HN's tech stack? I know it's built on top of the Lisp dialect Arc, but I wonder about the server specs and would be happy to be pointed at some resource. Although I experienced HN being down, I guess the overall uptime and availability is pretty decent. Keep up the good work!
- dang 7y agoThis still holds: https://news.ycombinator.com/item?id=16076041 https://news.ycombinator.com/item?id=16076041 Number of requests is closer to 5M per day now.
- protomikron 7y agoNice, thanks! :)
- toomuchtodo 7y agoHN uses Cloudflare, doesn’t it? I would’ve assumed reads would be heavily cached.
- dang 7y agoNot any more.
- HeWhoLurksLate 7y agoWhat made you switch / leave CloudFlare?
- dang 7y agoThe desire to have fewer moving parts.
- toomuchtodo 7y agoI stand corrected.