7 ms·

I would bet money that the attack was truly epic... to the point where it was impacting (or was about to impact) other Akamai customers. Because you are righ

by lsc 10y ago

I would bet money that the attack was truly epic... to the point where it was impacting (or was about to impact) other Akamai customers. Because you are right, this is terrible PR, and Akamai knows it. DDoS mitigation is fundamentally a problem of "who has more bandwidth?" - if the attacker has more bandwidth than you (and how much bandwidth you have depends on "to where" ) - it's over. The problem is that the economics, right now, are heavily tipped in favor of the attacker.

masklinn 10y ago
> I would bet money that the attack was truly epic... Seems like it not only was one of the most expensive attacks so far (and by far the biggest one to have ever hit Prolexic, according to them), it also made little use of reflection of amplification making it much harder to mitigate. > to the point where it was impacting (or was about to impact) other Akamai customers. That's exactly the scale it had reached, and Akamai provided free service to Krebs, which was nice of them but only to the extent that it wasn't significantly impacting customers.
- lsc 10y ago
  >That's exactly the scale it had reached, and Akamai provided free service to Krebs, which was nice of them but only to the extent that it wasn't significantly impacting customers. I... wonder if anything different would have been done for a paying customer. I mean, if the attack was big enough to take down other customers, and if Akamai had the choice between kicking one customer and all customers being down?
  - masklinn 10y ago
    I'd expect them to have gone further and probably invested/committed more resources to it.
    - wpietri 10y ago
      Why's that? The theory is that it was impacting other customers. If the notion is that they kick off the low-revenue person to preserve the larger revenue stream, then it would seem that the only way to get real protection from Akamai is to be their largest customer.
      - masklinn 10y ago
        > Why's that? The theory is that it was impacting other customers. If the notion is that they kick off the low-revenue person Krebs was not a "low-revenue person" he was a "no-revenue person" they provided the service pro-bono. With customers they'd have a contract, and while I don't know Akamai's contracts I assume they either have specific service clauses and/or "use clauses" where protection costs get charged to the customer.
        thaeli 10y ago
        Nope. Contract basically says "here are the service levels Akamai commits to meet" and "Customer's sole remedy is to cancel service if Akamai does not meet these levels." Looking at their contract language does not make me feel any better about whether they'd drop a paying customer in similar circumstances. (Quoting from a version available online at http://contracts.onecle.com/akamai/msa.shtml http://contracts.onecle.com/akamai/msa.shtml) 8.1.3 Akamai shall meet or exceed the network availability, capacity and operations levels as set forth in Section 2 above; provided that Customer's sole remedy for the breach of this provision by Akamai shall be the termination rights set forth in Section 10.2 below. 10.2 TERMINATION UPON DEFAULT. Either party may terminate this Agreement in the event that the other party materially defaults in performing any obligation under this Agreement and such default continues unremedied for a period of thirty (30) days following, written notice of default; provided, however, that in the event this Agreement is terminated by Customer due to Akamai's breach of its representations under Section 8.1.3 above and failure to cure, Customer's sole remedy shall be its election to terminate the Agreement without further liability to either party (except for Customer's obligation to pay all accrued and unpaid fees outstanding at the date of termination).
- escapologybb 10y ago
  Having not committed a 620 GBps DDos attack on a blogger I don't like lately, how on earth do you generate that much traffic without reflection or amplification? Asking for a friend. /s
  - nisa 10y ago
    Pure speculation but you can get and click a VPS/dedicated server with a 1Gbit/s pipe almost everywhere. Most of these machines are not regularly patched and run often vulnerable PHP software. I guess if you have a few 0-days for common stuff that is often hosted you can easily!? collect a few thousand servers that blast out traffic. Additionally more machines have faster access to the internet as fiber gets more popular, if you have a decent sized botnet that can pump out traffic on average with 50mbit or so you can also get some serious traffic, not sure about getting 620GBps, through. I guess if you mix all of these and put some amplification attacks on top it's possible.
  - tim333 10y ago
    > There are some indications that this attack was launched with the help of a botnet that has enslaved a large number of hacked so-called “Internet of Things,” (IoT) devices — routers, IP cameras and digital video recorders (DVRs) that are exposed to the Internet and protected with weak or hard-coded passwords. http://webcache.googleusercontent.com/search?q=cache:http://krebsonsecurity.com/2016/09/krebsonsecurity-hit-with-record-ddos/&num=1&strip=1&vwsrc=0 http://webcache.googleusercontent.com/search?q=cache:http://...
  - imaginenore 10y ago
    You hack into a bunch of servers, PCs and smartphones. You don't even need that many with modern connections.
    - lightedman 10y ago
      Yup. Cable subscribers in the USA alone have enough aggregate bandwidth to knock Akamai totally offline if 620 G-bit is something they can't handle. My own uplink is 10 M-bit. 100,000 compromised connections that speed is enough for T-bit scale attacks, and compromising 100,000 machines in the new world of "Idiots of Things" is a no-brainer given almost nobody working in the IoT space even gives a nod towards security.
manarth 10y ago
I would bet money that the attack was truly epic From the article: "The assault has flooded Krebs' site with more than 620 Gbps per second of traffic — nearly double what Akamai has seen in the past." Sounds pretty epic.
- julianwachholz 10y ago
  It's accelerating!
- lightedman 10y ago
  If it takes a mere 620 G-bit to screw Akamai, then they're obviously not much of a content distribution platform. I only need a few thousand compromised machines in the right countries to run 5+ T-bit scale attacks. This isn't the 90s any more, and Akamai has no reason to not have improved bandwidth capabilities, unless they're doing like the Telecoms companies by saving that money and never using it to improve their own infrastructure.
  - daddykotex 10y ago
    I was impressed by that number too, but after reading your comment it seems awfully small. In 2016, what kind of attack do you expect CDN like Akamai to handle?
    - lightedman 10y ago
      Chattanooga has symmetrical G-bit fiber to the home. 5,000 Chat-town residents got uppity, think of what would happen to Akamai if they couldn't deal with 600-ish G-bit.
      - Anasufovic 10y ago
        Yeah if that network was designed to handle continuous 1gbit per home.
        10y ago
Pyxl101 10y ago
> If the attacker has more bandwidth than you (and how much bandwidth you have depends on "to where" ) - it's over. Is there no way to stop such attacks by coordinating with your upstream ISP, or with the sources of the traffic? Why do backbones allow it to be carried? Is the problem that these attacks are too many different individual streams to identify and filter? It seems like there ought to be some way to hierarchically punt the problem to network operators. "Your network is contributing 10 gigabytes per second to this DDOS attack. Identify the sources and shut them down." - times each identifiable traffic stream. Is there no way to capture a list of all IPs involved in the traffic and quickly distribute a "shut off this device" request to the origin network? Maybe a good-faith collaboration of different ISPs could result in quick shut-downs. Or if networks don't cooperate, then the next-nearest border does it for them (and if they don't like the policies under which their neighbor suspends the traffic, they can sign up to do it themselves). Imagine something like a mini automated DMCA type request. "This IP is DDOSing me", signed by the operator of a reputable network, having the effect of suppressing origin traffic from the IP when received by a reputable network. (Any abuse of the mechanism causes the network to lose its privilege to participate, and DMCA requests related to that network fall upon its neighbors.) Perhaps the suppression would be destination-limited so as not to be vulnerable to too much abuse of the mechanism.
- vidarh 10y ago
  It is possible to mitigate, but it takes time. > Is there no way to capture a list of all IPs involved in the traffic and quickly distribute a "shut off this device" request to the origin network? Now hackers have a new attack vector.
  - pbhjpbhj 10y ago
    Without net neutrality economics would likely fix this. "We'll limit our 60GBps traffic to this network block because our peering arrangement makes that expensive." Couldn't an ISP just throttle traffic to a particular network block on receipt of an authorised request. And that seemingly provides a general solution. Record output to particular network blocks and throttle traffic when it peaks beyond statistically normal bounds? Basically, applying damping.

by lsc 10y ago

masklinn 10y ago

> I would bet money that the attack was truly epic... Seems like it not only was one of the most expensive attacks so far (and by far the biggest one to have ever hit Prolexic, according to them), it also made little use of reflection of amplification making it much harder to mitigate. > to the point where it was impacting (or was about to impact) other Akamai customers. That's exactly the scale it had reached, and Akamai provided free service to Krebs, which was nice of them but only to the extent that it wasn't significantly impacting customers.

lsc 10y ago
>That's exactly the scale it had reached, and Akamai provided free service to Krebs, which was nice of them but only to the extent that it wasn't significantly impacting customers. I... wonder if anything different would have been done for a paying customer. I mean, if the attack was big enough to take down other customers, and if Akamai had the choice between kicking one customer and all customers being down?
- masklinn 10y ago
  I'd expect them to have gone further and probably invested/committed more resources to it.
  - wpietri 10y ago
    Why's that? The theory is that it was impacting other customers. If the notion is that they kick off the low-revenue person to preserve the larger revenue stream, then it would seem that the only way to get real protection from Akamai is to be their largest customer.
    - masklinn 10y ago
      > Why's that? The theory is that it was impacting other customers. If the notion is that they kick off the low-revenue person Krebs was not a "low-revenue person" he was a "no-revenue person" they provided the service pro-bono. With customers they'd have a contract, and while I don't know Akamai's contracts I assume they either have specific service clauses and/or "use clauses" where protection costs get charged to the customer.
      - thaeli 10y ago
        Nope. Contract basically says "here are the service levels Akamai commits to meet" and "Customer's sole remedy is to cancel service if Akamai does not meet these levels." Looking at their contract language does not make me feel any better about whether they'd drop a paying customer in similar circumstances. (Quoting from a version available online at http://contracts.onecle.com/akamai/msa.shtml http://contracts.onecle.com/akamai/msa.shtml) 8.1.3 Akamai shall meet or exceed the network availability, capacity and operations levels as set forth in Section 2 above; provided that Customer's sole remedy for the breach of this provision by Akamai shall be the termination rights set forth in Section 10.2 below. 10.2 TERMINATION UPON DEFAULT. Either party may terminate this Agreement in the event that the other party materially defaults in performing any obligation under this Agreement and such default continues unremedied for a period of thirty (30) days following, written notice of default; provided, however, that in the event this Agreement is terminated by Customer due to Akamai's breach of its representations under Section 8.1.3 above and failure to cure, Customer's sole remedy shall be its election to terminate the Agreement without further liability to either party (except for Customer's obligation to pay all accrued and unpaid fees outstanding at the date of termination).
escapologybb 10y ago
Having not committed a 620 GBps DDos attack on a blogger I don't like lately, how on earth do you generate that much traffic without reflection or amplification? Asking for a friend. /s
- nisa 10y ago
  Pure speculation but you can get and click a VPS/dedicated server with a 1Gbit/s pipe almost everywhere. Most of these machines are not regularly patched and run often vulnerable PHP software. I guess if you have a few 0-days for common stuff that is often hosted you can easily!? collect a few thousand servers that blast out traffic. Additionally more machines have faster access to the internet as fiber gets more popular, if you have a decent sized botnet that can pump out traffic on average with 50mbit or so you can also get some serious traffic, not sure about getting 620GBps, through. I guess if you mix all of these and put some amplification attacks on top it's possible.
- tim333 10y ago
  > There are some indications that this attack was launched with the help of a botnet that has enslaved a large number of hacked so-called “Internet of Things,” (IoT) devices — routers, IP cameras and digital video recorders (DVRs) that are exposed to the Internet and protected with weak or hard-coded passwords. http://webcache.googleusercontent.com/search?q=cache:http://krebsonsecurity.com/2016/09/krebsonsecurity-hit-with-record-ddos/&num=1&strip=1&vwsrc=0 http://webcache.googleusercontent.com/search?q=cache:http://...
- imaginenore 10y ago
  You hack into a bunch of servers, PCs and smartphones. You don't even need that many with modern connections.
  - lightedman 10y ago
    Yup. Cable subscribers in the USA alone have enough aggregate bandwidth to knock Akamai totally offline if 620 G-bit is something they can't handle. My own uplink is 10 M-bit. 100,000 compromised connections that speed is enough for T-bit scale attacks, and compromising 100,000 machines in the new world of "Idiots of Things" is a no-brainer given almost nobody working in the IoT space even gives a nod towards security.

manarth 10y ago

I would bet money that the attack was truly epic From the article: "The assault has flooded Krebs' site with more than 620 Gbps per second of traffic — nearly double what Akamai has seen in the past." Sounds pretty epic.

julianwachholz 10y ago
It's accelerating!
lightedman 10y ago
If it takes a mere 620 G-bit to screw Akamai, then they're obviously not much of a content distribution platform. I only need a few thousand compromised machines in the right countries to run 5+ T-bit scale attacks. This isn't the 90s any more, and Akamai has no reason to not have improved bandwidth capabilities, unless they're doing like the Telecoms companies by saving that money and never using it to improve their own infrastructure.
- daddykotex 10y ago
  I was impressed by that number too, but after reading your comment it seems awfully small. In 2016, what kind of attack do you expect CDN like Akamai to handle?
  - lightedman 10y ago
    Chattanooga has symmetrical G-bit fiber to the home. 5,000 Chat-town residents got uppity, think of what would happen to Akamai if they couldn't deal with 600-ish G-bit.
    - Anasufovic 10y ago
      Yeah if that network was designed to handle continuous 1gbit per home.
      - 10y ago

Pyxl101 10y ago

> If the attacker has more bandwidth than you (and how much bandwidth you have depends on "to where" ) - it's over. Is there no way to stop such attacks by coordinating with your upstream ISP, or with the sources of the traffic? Why do backbones allow it to be carried? Is the problem that these attacks are too many different individual streams to identify and filter? It seems like there ought to be some way to hierarchically punt the problem to network operators. "Your network is contributing 10 gigabytes per second to this DDOS attack. Identify the sources and shut them down." - times each identifiable traffic stream. Is there no way to capture a list of all IPs involved in the traffic and quickly distribute a "shut off this device" request to the origin network? Maybe a good-faith collaboration of different ISPs could result in quick shut-downs. Or if networks don't cooperate, then the next-nearest border does it for them (and if they don't like the policies under which their neighbor suspends the traffic, they can sign up to do it themselves). Imagine something like a mini automated DMCA type request. "This IP is DDOSing me", signed by the operator of a reputable network, having the effect of suppressing origin traffic from the IP when received by a reputable network. (Any abuse of the mechanism causes the network to lose its privilege to participate, and DMCA requests related to that network fall upon its neighbors.) Perhaps the suppression would be destination-limited so as not to be vulnerable to too much abuse of the mechanism.

vidarh 10y ago
It is possible to mitigate, but it takes time. > Is there no way to capture a list of all IPs involved in the traffic and quickly distribute a "shut off this device" request to the origin network? Now hackers have a new attack vector.
- pbhjpbhj 10y ago
  Without net neutrality economics would likely fix this. "We'll limit our 60GBps traffic to this network block because our peering arrangement makes that expensive." Couldn't an ISP just throttle traffic to a particular network block on receipt of an authorised request. And that seemingly provides a general solution. Record output to particular network blocks and throttle traffic when it peaks beyond statistically normal bounds? Basically, applying damping.