3 ms·

If you make a shitty site that can't handle a few concurrent requests this is your problem. It's not a DDoS. Not even a DoS. Just trying to request some pages y

If you make a shitty site that can't handle a few concurrent requests this is your problem. It's not a DDoS. Not even a DoS. Just trying to request some pages you make available, and you failing at doing so. Or do you also plan on suing Google once they scrape your site?

fao_ 9y ago
Looks like somebody didn't read the parent. > "scraping" site every 15 minutes with massive ddos Google doesn't scrape every 15 minutes. It was implied that they scraped the site often enough to cause a DDoS.
- tokenizerrr 9y ago
  Looks like someone has never ran a website with even a slight amount of traffic. A few requests every 15 minutes is nothing and your site should be able to handle it. Additionally, it clearly wasn't a DDoS. Quoting from the parent: > Some swedish equal rights server was "scraping" site every 15 minutes with massive ddos. We put capacha for the server ip only (let them scratch their heads now) and let it be If there is only one server ip, it is by definition not distributed. Just a misuse of the term DDoS.
- thephyber 9y ago
  > Google doesn't scrape every 15 minutes. Google has a variable timing crawler. Google News crawls in almost realtime, but all Google crawlers back off if the site responds slow. Additionally, Google respects `/robots.txt` (which allows the site owner to define a crawl delay) and uses sitemaps for hints, so it doesn't have to re-crawl every single HTTP object "every 15 minutes".
DocG 9y ago
We were contacted as outsiders to come in and have a look. We managed to stop site from crashing with configuring but basically the site was bombarded with too many dynamic requests and having no cache it took its toll. Server did not wait for responses, it just sent us bunch of requests all at once.
- tokenizerrr 9y ago
  Any decent webserver can throttle requests per source ip.
robalfonso 9y ago
Yes, agreed though you must also take into account the idea of the "straw that breaks the camels back".

fao_ 9y ago

Looks like somebody didn't read the parent. > "scraping" site every 15 minutes with massive ddos Google doesn't scrape every 15 minutes. It was implied that they scraped the site often enough to cause a DDoS.

tokenizerrr 9y ago
Looks like someone has never ran a website with even a slight amount of traffic. A few requests every 15 minutes is nothing and your site should be able to handle it. Additionally, it clearly wasn't a DDoS. Quoting from the parent: > Some swedish equal rights server was "scraping" site every 15 minutes with massive ddos. We put capacha for the server ip only (let them scratch their heads now) and let it be If there is only one server ip, it is by definition not distributed. Just a misuse of the term DDoS.
thephyber 9y ago
> Google doesn't scrape every 15 minutes. Google has a variable timing crawler. Google News crawls in almost realtime, but all Google crawlers back off if the site responds slow. Additionally, Google respects `/robots.txt` (which allows the site owner to define a crawl delay) and uses sitemaps for hints, so it doesn't have to re-crawl every single HTTP object "every 15 minutes".

DocG 9y ago

We were contacted as outsiders to come in and have a look. We managed to stop site from crashing with configuring but basically the site was bombarded with too many dynamic requests and having no cache it took its toll. Server did not wait for responses, it just sent us bunch of requests all at once.

tokenizerrr 9y ago
Any decent webserver can throttle requests per source ip.

robalfonso 9y ago

Yes, agreed though you must also take into account the idea of the "straw that breaks the camels back".