3 ms·
I don't expect HN give a fuck about the scraping. It's pure HTML, no images, probably cached all to hell for users who aren't logged in anyway. The one thing I
by qingcharles 3y ago
I don't expect HN give a fuck about the scraping. It's pure HTML, no images, probably cached all to hell for users who aren't logged in anyway.
The one thing I see as a future issue is that people are starting to post comments that clearly look like they were manufactured by ChatGPT and friends. Or that could just be the way some people talk and I've spent too long with ChatGPT now and start to smell it everywhere.
- dredmorbius 3y agoHN does have performance / capacity issues, and you'll find that if you're crawling the site rapidly, you'll quickly have your IP banned. I've had that happen even under manual browsing (when logged out). My front-page analytics project hit that limit quickly (within about 30 requests, probably less). Adding in a reasonable delay got around that. Keep in mind that a lot of Web infrastructure tends over time to operate just at the edge of stability, as capacity costs money.
- j5155 3y agoI mean, given that AI might be trained on HN, maybe it’s ChatGPT saying things that sound like HN commenters…