3 ms·
The best way to get around the ban I think is to have low-latency distributed scraping. Basically, a plugin that submits to RR's site whenever a user is surfin
by adrianwaj 16y ago
The best way to get around the ban I think is to have low-latency distributed scraping.
Basically, a plugin that submits to RR's site whenever a user is surfing on HN. RR's site then parses it as usual.
The plugin could have some setting like max KB submitted per minute, and RR's site would only request the full page if it was needed at the time.