31 ms·
I do crawling from EC2, and yes, I would not like a 1Gbps traffic spike myself. Do you deal with generic webpage crawlers that way, or targeted API abuse? Beca
by kaivi 12y ago
I do crawling from EC2, and yes, I would not like a 1Gbps traffic spike myself.
Do you deal with generic webpage crawlers that way, or targeted API abuse? Because the first ones can be smoothly shaved away with the help of Cloudflare, for instance.