4 ms·

Looking at the API request counter/how it renders it seems like you're pinging the API for every news item? 43 for just the "news" and "show" sections seems pre

Looking at the API request counter/how it renders it seems like you're pinging the API for every news item? 43 for just the "news" and "show" sections seems pretty high. If that were the case I'd be worried about hitting an API limit, etc. Great concept though! Looks clean =)

ggerganov 7y ago
It's just the way the API works - to get the details for a single story (i.e. title, score, user, time, etc.) you need a single API request. Once the stories are fetched, then you can query the API only for those that have changed. I was wondering what is the API limit - I haven't hit it yet. Will see.
- folkhack 7y ago
  Gotcha - it's times like that that I go with an old-fashioned raw HTTP request and parse it with a DOM parsing engine. Sometimes the "web-facing API" (so-to-speak) is better than the official API. Super cool project!
  - dang 7y ago
    Yes, an API that is isomorphic to the website itself would be much easier to use. Our plan is for the next version of HN's API to simply serve a JSON version of every page. In the meantime, if you scrape the HTML, please respect https://news.ycombinator.com/robots.txt https://news.ycombinator.com/robots.txt and don't do it more than once per 30 seconds. HN's app server runs on a single core, so we don't have a lot of performance to spare! This is in the works to change as well.
    - protomikron 7y ago
      > HN's app server runs on a single core, so we don't have a lot of performance to spare! That's interesting. Is there any more information about HN's tech stack? I know it's built on top of the Lisp dialect Arc, but I wonder about the server specs and would be happy to be pointed at some resource. Although I experienced HN being down, I guess the overall uptime and availability is pretty decent. Keep up the good work!
      - dang 7y ago
        This still holds: https://news.ycombinator.com/item?id=16076041 https://news.ycombinator.com/item?id=16076041 Number of requests is closer to 5M per day now.
        protomikron 7y ago
        Nice, thanks! :)
    - toomuchtodo 7y ago
      HN uses Cloudflare, doesn’t it? I would’ve assumed reads would be heavily cached.
      - dang 7y ago
        Not any more.
        HeWhoLurksLate 7y ago
        What made you switch / leave CloudFlare?
        dang 7y ago
        The desire to have fewer moving parts.
        toomuchtodo 7y ago
        I stand corrected.

ggerganov 7y ago

It's just the way the API works - to get the details for a single story (i.e. title, score, user, time, etc.) you need a single API request. Once the stories are fetched, then you can query the API only for those that have changed. I was wondering what is the API limit - I haven't hit it yet. Will see.

folkhack 7y ago
Gotcha - it's times like that that I go with an old-fashioned raw HTTP request and parse it with a DOM parsing engine. Sometimes the "web-facing API" (so-to-speak) is better than the official API. Super cool project!
- dang 7y ago
  Yes, an API that is isomorphic to the website itself would be much easier to use. Our plan is for the next version of HN's API to simply serve a JSON version of every page. In the meantime, if you scrape the HTML, please respect https://news.ycombinator.com/robots.txt https://news.ycombinator.com/robots.txt and don't do it more than once per 30 seconds. HN's app server runs on a single core, so we don't have a lot of performance to spare! This is in the works to change as well.
  - protomikron 7y ago
    > HN's app server runs on a single core, so we don't have a lot of performance to spare! That's interesting. Is there any more information about HN's tech stack? I know it's built on top of the Lisp dialect Arc, but I wonder about the server specs and would be happy to be pointed at some resource. Although I experienced HN being down, I guess the overall uptime and availability is pretty decent. Keep up the good work!
    - dang 7y ago
      This still holds: https://news.ycombinator.com/item?id=16076041 https://news.ycombinator.com/item?id=16076041 Number of requests is closer to 5M per day now.
      - protomikron 7y ago
        Nice, thanks! :)
  - toomuchtodo 7y ago
    HN uses Cloudflare, doesn’t it? I would’ve assumed reads would be heavily cached.
    - dang 7y ago
      Not any more.
      - HeWhoLurksLate 7y ago
        What made you switch / leave CloudFlare?
        dang 7y ago
        The desire to have fewer moving parts.
      - toomuchtodo 7y ago
        I stand corrected.