5 ms·

Github is really great, but I'm a cynic. Indeed, why _would_ anyone want to DDoS github? How can we believe Github that the outages were due to a DDoS? They're

by robbyt 15y ago

Github is really great, but I'm a cynic. Indeed, why _would_ anyone want to DDoS github? How can we believe Github that the outages were due to a DDoS? They're smart people, aren't they using solid load balancesrs that can mitigate DDoS attacks? Why haven't they issued an actual statement describing the supposed attacks in better detail?

mike-cardwell 15y ago
That's not just cynical; that's down right paranoid.
dhconnelly 15y ago
As another poster mentioned, DDOS attacks on successful companies are not uncommon. And there isn't an easy fix for this.
jtchang 15y ago
Load balancers generally don't offer as much mitigation against DDoS as you think. The load balancer will probably fall over if the backend servers don't do so first. The sheer volume of seemingly valid traffic is what kills you. Which is why effective anti-DDoS means working with your upstream provider to figure out how to differentiate between real traffic and fake traffic. The best solution to a DDoS is the phone # of a tech that can implement firewall rules upstream and a good traffic analyzer to tell them exactly how to filter it.
- swalberg 15y ago
  Often you can't pick apart the good traffic from the bad. Fortunately bots are usually pretty stupid. If you can outrun them on bandwidth, then change /victimpage.html to 302 to /victimpage-new.html. The web server or load balancer can send those redirects really fast and it doesn't take much bandwidth either. I have never seen a bot chase that redirect. After a particularly nasty DDOS attack (where our upstream provider just shrugged their shoulders) I wrote an F5 iRule: 1. Check for the IAMNOTABOT cookie 2. If not there, redirect to /cookie-me?oldpage=the_page_you_were_trying_to_access 3. Set IAMNOTABOT=true cookie 4. Redirect to the old page
  - jarcoal 15y ago
    Another, similar, technique is checking for a NOBOT cookie and if it doesn't exist, serve up a page that uses javascript to set the cookie, then reloads. This requires the bot to interpret javascript, and it can easily be configured in nginx and other load balancers.
acdha 15y ago
Ask anyone who's been there and they'll tell you that load balancers only make DDoS attacks harder. Unless you have something like AWS AutoScaling, they'll just generate more traffic - and if you do, it would quickly become a financial DDoS. That's why sites like Yahoo.com have been taken out: you have finite server capacity but the attacker merely has to find another another exploitable system somewhere on the Internet. The way you deal with a DDoS is hard: you can try to identify distinguishing characteristics of the attack so traffic can be blocked as far out as possible and you try to figure out who's behind the attack so you can get law-enforcement involved. Otherwise you're just playing whack-a-mole with compromised home computers all around the globe…
ceejayoz 15y ago
> Indeed, why _would_ anyone want to DDoS github? "Some people just want to watch the world burn." Organized crime extorting sites with DDoSes is certainly not unheard of, either.
mitchellhislop 15y ago
Responding to your last sentence, there was a comment on the blog post that they are being vague to not give anything away to the attackers. If they start describing the attack (even how they perceive it) and/or mitigation steps, they are giving targeting data to the people doing the attack.
brown9-2 15y ago
Arguably, if your load balancer setup is such that traffic can increase 10000x from the minute before and nothing falls over, you were probably incredibly over-provisioned in the first place. Unless your regular traffic pattern has 10000x increases built into it, why would you spend all the extra expense to be able to handle that level of load?

by robbyt 15y ago

mike-cardwell 15y ago

That's not just cynical; that's down right paranoid.

dhconnelly 15y ago

As another poster mentioned, DDOS attacks on successful companies are not uncommon. And there isn't an easy fix for this.

jtchang 15y ago

Load balancers generally don't offer as much mitigation against DDoS as you think. The load balancer will probably fall over if the backend servers don't do so first. The sheer volume of seemingly valid traffic is what kills you. Which is why effective anti-DDoS means working with your upstream provider to figure out how to differentiate between real traffic and fake traffic. The best solution to a DDoS is the phone # of a tech that can implement firewall rules upstream and a good traffic analyzer to tell them exactly how to filter it.

swalberg 15y ago
Often you can't pick apart the good traffic from the bad. Fortunately bots are usually pretty stupid. If you can outrun them on bandwidth, then change /victimpage.html to 302 to /victimpage-new.html. The web server or load balancer can send those redirects really fast and it doesn't take much bandwidth either. I have never seen a bot chase that redirect. After a particularly nasty DDOS attack (where our upstream provider just shrugged their shoulders) I wrote an F5 iRule: 1. Check for the IAMNOTABOT cookie 2. If not there, redirect to /cookie-me?oldpage=the_page_you_were_trying_to_access 3. Set IAMNOTABOT=true cookie 4. Redirect to the old page
- jarcoal 15y ago
  Another, similar, technique is checking for a NOBOT cookie and if it doesn't exist, serve up a page that uses javascript to set the cookie, then reloads. This requires the bot to interpret javascript, and it can easily be configured in nginx and other load balancers.

acdha 15y ago

Ask anyone who's been there and they'll tell you that load balancers only make DDoS attacks harder. Unless you have something like AWS AutoScaling, they'll just generate more traffic - and if you do, it would quickly become a financial DDoS. That's why sites like Yahoo.com have been taken out: you have finite server capacity but the attacker merely has to find another another exploitable system somewhere on the Internet. The way you deal with a DDoS is hard: you can try to identify distinguishing characteristics of the attack so traffic can be blocked as far out as possible and you try to figure out who's behind the attack so you can get law-enforcement involved. Otherwise you're just playing whack-a-mole with compromised home computers all around the globe…

ceejayoz 15y ago

> Indeed, why _would_ anyone want to DDoS github? "Some people just want to watch the world burn." Organized crime extorting sites with DDoSes is certainly not unheard of, either.

mitchellhislop 15y ago

Responding to your last sentence, there was a comment on the blog post that they are being vague to not give anything away to the attackers. If they start describing the attack (even how they perceive it) and/or mitigation steps, they are giving targeting data to the people doing the attack.

brown9-2 15y ago

Arguably, if your load balancer setup is such that traffic can increase 10000x from the minute before and nothing falls over, you were probably incredibly over-provisioned in the first place. Unless your regular traffic pattern has 10000x increases built into it, why would you spend all the extra expense to be able to handle that level of load?