3 ms·
I'm not a web developer, so maybe I'm misremembering, but wasn't there something in HTTP that let you check the last time a page changed? Scraping the poll page
by rcfox 15y ago
I'm not a web developer, so maybe I'm misremembering, but wasn't there something in HTTP that let you check the last time a page changed? Scraping the poll pages every 10 minutes seems like it would cause a lot of unnecessary requests to HN.
Also, you probably consider any poll that's not on the first few pages of the 'news' or 'newest' sections to be complete. Sure, they might change, but they're not going to change by enough to affect your graphs.
- sbashyal 15y agoThe only way to 'listen' to page changes is to scrape it unless the web-page provides a notification service. Since that is not the case with HN, I had to resort to scrapes.
- mooism2 15y agoIf-Modified-Since? You still have to issue the http request.
- icebraining 15y agoThere are ETags too, but they suffer from the same problem (and usually dynamic pages don't serve them).