3 ms·
> http://yakamo.org/?p=112 http://yakamo.org/?p=112 "I setup 7 servers with mpi4py and it took 48 hours to scrape 3.2million profiles. Now i was ready to get t
by us0r 10y ago
> http://yakamo.org/?p=112 http://yakamo.org/?p=112
"I setup 7 servers with mpi4py and it took 48 hours to scrape 3.2million profiles. Now i was ready to get the answer and find out how many profiles where public."
I think that works out to around 20 requests/sec. At a certain point it becomes borderline DoS?
- meshko 10y agoAt the point where you are negatively affecting their servers availability and doing so with a malicious intent.
- curiousgal 10y ago20 requests per second per 7 servers so it comes out to 3 requests per second. Not even close to a DoS attack.
- anilgulecha 10y agoIt's 20 rps -- not sure why you divided by 7. A DoS attack is measured by total, not per server.
- curiousgal 10y agoWell if you are going to total might as well add in the traffic from all the other users. And in that grand scheme 20 requests per second won't matter. I only divided by 7 because I thought he wasn't running all the 7 servers simultaneously, come to think of it, I was wrong.
- mistermann 10y agoI can't believe a company as large as okcupid has nothing in place to detect and stop this.
- dreamcompiler 10y agoIt's not fast enough to be a DOS attack by any stretch. But it's easy to detect that a given IP address has downloaded e.g. several hundred profiles in a few minutes, and OKC should have been throttling such obvious crawlers to around one per minute. If the behavior repeated, they should have blocked the address altogether for some period of time.