4 ms·

Why is HN opposed to scraping? I think since the front page is dynamically generated -- a bot would waste resources. The site gets loaded (slow to respond) duri

by sounds 15y ago

Why is HN opposed to scraping? I think since the front page is dynamically generated -- a bot would waste resources. The site gets loaded (slow to respond) during peak times. Beyond that, only pg could say...

mmackh 15y ago
Why would HN refuse to provide a useful RSS feed / a non standard compliant one (number of comments/points/time of submission/id) that could be used to develop apps for a better reading experience? http://api.ihackernews.com/ http://api.ihackernews.com/ is unreliable at best. I'm working on something right now and this limitation causes difficulties. (http://cl.ly/241x3F3d123P0w0L1s3q http://cl.ly/241x3F3d123P0w0L1s3q)
bootload 15y ago
"... Why is HN opposed to scraping? ..." because code written by N hackers scraping HN has a greater effect on the site, compared to using the reliable HNSearch API ~ http://www.hnsearch.com/api http://www.hnsearch.com/api Ask HN: Does HN have an API and if not what's the etiquette for scraping? http://news.ycombinator.com/item?id=2138730 Ask HN: Is there an API for HN? http://news.ycombinator.com/item?id=1107874
- mmackh 15y ago
  Judging from my own experience this API is not reliable.
  - 3pt14159 15y ago
    Mine as well.
  - andres 15y ago
    If you have problems with the API please let me know (andres@octopart.com).
zalew 15y ago
I do HN scrapping of my 'saved stories' for my personal site http://zalew.net/hn/ http://zalew.net/hn/ so I assume pg doesn't mind that as it scraps only once per hour. here's the description of it and source http://zalew.net/2011/12/08/grab-your-hackernews-stories-and-comments-python/ http://zalew.net/2011/12/08/grab-your-hackernews-stories-and...