2 ms·
HN does have performance / capacity issues, and you'll find that if you're crawling the site rapidly, you'll quickly have your IP banned. I've had that happen
by dredmorbius 3y ago
HN does have performance / capacity issues, and you'll find that if you're crawling the site rapidly, you'll quickly have your IP banned.
I've had that happen even under manual browsing (when logged out). My front-page analytics project hit that limit quickly (within about 30 requests, probably less). Adding in a reasonable delay got around that.
Keep in mind that a lot of Web infrastructure tends over time to operate just at the edge of stability, as capacity costs money.