4 ms·

I work for a DDoS protection company and while the advice in the article is indeed high level, it is exactly what we do. - buy a ton of network capacity in m

by gbuk2013 10y ago

I work for a DDoS protection company and while the advice in the article is indeed high level, it is exactly what we do. - buy a ton of network capacity in multiple locations around the world - route incoming traffic to different locations at BGP level This is really the only way to reliably cope with modern volumetric attacks right now. On top of that we: - scrub the traffic using using some expensive fancy appliances(*) - employ a bunch of very skilled people to manage it all and provide support This is what you need to protect against protocol / application level attacks. * which we get a very good deal on by virtue of making said appliances :)

brownbat 10y ago
In some ongoing DDoS attacks, I've heard companies say they're dedicating more resources to combatting the problem. Is active mitigation mainly about sifting through incoming traffic, trying to find signatures and update filters in realtime? Or more replacing dynamic with static content as its targeted? Or are there other ways resources get surged during a crisis? ie, Is it heavy on manual analysis, or are we at the point yet where we can automate mitigation? Just curious. Would recommend an AMA, but that's not this site.
- gbuk2013 10y ago
  (Answer from one of my more knowledgeable colleagues since I work on supporting the infrastructure rather than DDoS mitigation itself): The larger the attack, the simpler the vector. For the Flood (TCP, UDP, DNS, NTP, etc) attacks, creating accurate firewall rules within your cloud-scrubbing provider handles a large portion of this, the remainder can be mitigated by connection rate-limiting or TCP connection mitigations (check to make sure it’s a valid 3WHS before allowing connections to the origin). Complex L7 attacks require more effort and usually shift around in what they are attacking, this takes more analysis to pin down, though the L7 Bot Defenses, intelligent rate limiting and automatic traffic analysis help with this. DDoS mitigation is not explicitly a science, there is an art to it as well that comes from experience and learning day-to-day as attacks evolve. As attacks start up, mitigations may be too strict or too loose, that is the benefit of an expert SOC staff to monitor the situation and adjust as needed. Relying on automation for this will likely leave the customer frustrated with the outcome.
  - brownbat 10y ago
    Fascinating, thanks!
bogomipz 10y ago
>"buy a ton of network capacity in multiple locations around the world" There are estimates that the Dyn attack was 1.2Tbps. What provider do you work for that can absorb that? >"scrub the traffic using using some expensive fancy appliances(*)" Does scrubbing work with a bunch of randomized source ports and source address? How do you find the "signature" in that to scrub? It was my understanding that the traffic in the Dyn attacks was indistinguishable from legitimate traffic. Can you explain how traffic scrubbing works in such scenarios? Or do you just drop stuff on the floor?
- SisypheanLife 10y ago
  I'm unaware of any provider than can protect against a burst of 500 Gbps on a single load balanced endpoint, let alone a sustained attack. 1.2Tbps is basically a death ray.
- bobfunk 10y ago
  The second part is the tricky part - I've gotten offers from DDoS mitigation providers that advertises that level of capacity, but how much it helps depends entirely on how much of that traffic they can scrub. If they only scrub half of that traffic, you're still getting a massive DDoS attack against your origin servers anyway. And the frustrating thing about buying DDoS mitigation is that you have absolutely no way of knowing how much traffic any DDoS mitigation service can actually scrub of a given attack before it hits you...
  - bogomipz 10y ago
    Agreed. Even if they have the capacity if they can only identify a a portion of it as "interesting" and then pass the rest on to you they haven't solved the problem. And like you said you are still getting DDOS and paying dearly for scrubbing. I am curious if the traffic analysis at the scrubbing center is anything more than coarse grain I am assuming it is.
  - gbuk2013 10y ago
    (Answer from one of my more knowledgeable colleagues since I work on supporting the infrastructure rather than DDoS mitigation itself): If the DDoS mitigation provider is doing their job, they will ingest all the traffic, scrub out the bad and return the good. While obvious, there are things a customer should know when entering into that arrangement: - Does the provider have the ingress capacity to absorb all the attack traffic? - Are their scrubbing centres peered with Tier 1 transit providers to reduce carrier congestion? - Do they have a policy on dropping traffic at certain volumes? - Do they charge you based on attack volume or clean traffic? - Do they have rate-limiting in place towards the customer to protect them from high-volume attacks while mitigations are optimized to catch all the attack traffic? Ensuring your provider has these technical and contractual terms in place will make sure they can actually offer value when under attack.

brownbat 10y ago

In some ongoing DDoS attacks, I've heard companies say they're dedicating more resources to combatting the problem. Is active mitigation mainly about sifting through incoming traffic, trying to find signatures and update filters in realtime? Or more replacing dynamic with static content as its targeted? Or are there other ways resources get surged during a crisis? ie, Is it heavy on manual analysis, or are we at the point yet where we can automate mitigation? Just curious. Would recommend an AMA, but that's not this site.

gbuk2013 10y ago
(Answer from one of my more knowledgeable colleagues since I work on supporting the infrastructure rather than DDoS mitigation itself): The larger the attack, the simpler the vector. For the Flood (TCP, UDP, DNS, NTP, etc) attacks, creating accurate firewall rules within your cloud-scrubbing provider handles a large portion of this, the remainder can be mitigated by connection rate-limiting or TCP connection mitigations (check to make sure it’s a valid 3WHS before allowing connections to the origin). Complex L7 attacks require more effort and usually shift around in what they are attacking, this takes more analysis to pin down, though the L7 Bot Defenses, intelligent rate limiting and automatic traffic analysis help with this. DDoS mitigation is not explicitly a science, there is an art to it as well that comes from experience and learning day-to-day as attacks evolve. As attacks start up, mitigations may be too strict or too loose, that is the benefit of an expert SOC staff to monitor the situation and adjust as needed. Relying on automation for this will likely leave the customer frustrated with the outcome.
- brownbat 10y ago
  Fascinating, thanks!

bogomipz 10y ago

>"buy a ton of network capacity in multiple locations around the world" There are estimates that the Dyn attack was 1.2Tbps. What provider do you work for that can absorb that? >"scrub the traffic using using some expensive fancy appliances(*)" Does scrubbing work with a bunch of randomized source ports and source address? How do you find the "signature" in that to scrub? It was my understanding that the traffic in the Dyn attacks was indistinguishable from legitimate traffic. Can you explain how traffic scrubbing works in such scenarios? Or do you just drop stuff on the floor?

SisypheanLife 10y ago
I'm unaware of any provider than can protect against a burst of 500 Gbps on a single load balanced endpoint, let alone a sustained attack. 1.2Tbps is basically a death ray.
bobfunk 10y ago
The second part is the tricky part - I've gotten offers from DDoS mitigation providers that advertises that level of capacity, but how much it helps depends entirely on how much of that traffic they can scrub. If they only scrub half of that traffic, you're still getting a massive DDoS attack against your origin servers anyway. And the frustrating thing about buying DDoS mitigation is that you have absolutely no way of knowing how much traffic any DDoS mitigation service can actually scrub of a given attack before it hits you...
- bogomipz 10y ago
  Agreed. Even if they have the capacity if they can only identify a a portion of it as "interesting" and then pass the rest on to you they haven't solved the problem. And like you said you are still getting DDOS and paying dearly for scrubbing. I am curious if the traffic analysis at the scrubbing center is anything more than coarse grain I am assuming it is.
- gbuk2013 10y ago
  (Answer from one of my more knowledgeable colleagues since I work on supporting the infrastructure rather than DDoS mitigation itself): If the DDoS mitigation provider is doing their job, they will ingest all the traffic, scrub out the bad and return the good. While obvious, there are things a customer should know when entering into that arrangement: - Does the provider have the ingress capacity to absorb all the attack traffic? - Are their scrubbing centres peered with Tier 1 transit providers to reduce carrier congestion? - Do they have a policy on dropping traffic at certain volumes? - Do they charge you based on attack volume or clean traffic? - Do they have rate-limiting in place towards the customer to protect them from high-volume attacks while mitigations are optimized to catch all the attack traffic? Ensuring your provider has these technical and contractual terms in place will make sure they can actually offer value when under attack.