3 ms·

The robots.txt file doesn't seem to disallow scraping. https://news.ycombinator.com/robots.txt https://news.ycombinator.com/robots.txt

• karangoeluw 13y ago
  Scraping the listing pages seems allowed though.