4 ms·

Ask HN: Why does Hacker News throttle ec2?

2 points by kolodny 13y ago

I'm trying to write a script to scrape data from Hacker News on my ec2 instance but I seem to consistently be throttled at around 5k/s [ec2-user@ip-**-***-**-** ~]$ curl https://news.ycombinator.com/ > /dev/null % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 22237 0 22237 0 0 5195 0 --:--:-- 0:00:04 --:--:-- 5242 Can anyone else verify if this is happening to them or shed any light on this?

hollerith 13y ago
I think PG wants people to use http://www.hnsearch.com/api http://www.hnsearch.com/api
- kolodny 13y ago
  It doesn't seem like that api has an easy way to just get a page be it's post id, which is what I'm trying to do
deleted 13y ago
[deleted]