4 ms·

Why weren't they running any monitoring on their instances? A simple graph of incoming traffic should have revealed the massive influx of packets and helped tra

by bcl 17y ago

Why weren't they running any monitoring on their instances? A simple graph of incoming traffic should have revealed the massive influx of packets and helped track things down more quickly. Just because a system is 'in the cloud' is no reason not to setup normal monitoring on the system. I wrote http://www.brianlane.com/software/systemhealth/ http://www.brianlane.com/software/systemhealth/ to try and help with this situation, it is simple to install, doesn't require anything other then rrdtool and a web server to serve up the graphs.

tghw 17y ago
Jesper addressed this in the IRC channel. Basically, the traffic was not actually reaching the box, it was just saturating the available bandwidth to the EBS store. Moreover, since the attack was against EBS and not EC2, I'm not sure that you could actually get any monitoring in place on the volume. I'm kind of surprised your first assumption is that they have no monitoring in place. I like BitBucket, so I'm biased, but I can't really see anything about this instance that would make assume it was caused by some sort of incompetence on their part.
- bcl 17y ago
  Sorry, I'm not familiar with how EBS connects to the EC2 instances so I assumed the incoming traffic was hitting their EC2 instance. Are you are saying that EBS is accessible from outside and that the attackers were able to discover its IP and attack it directly? This doesn't seem like a very safe way to configure things. I also wonder why Amazon didn't notice this with any of their monitoring of either traffic or system performance and latency. Assuming they have such monitoring in place of course.
rbranson 17y ago
They may have been, but I imagine it's difficult to discern the difference between legitimate traffic and a DoS attack if you've got NAS volumes mounted over the same interface. You'd need to do a more thorough network analysis.
- deleted 17y ago
  [deleted]
- drusenko 17y ago
  They claimed in many places that their "pipe" was "maxed out". If the attack was large enough to use up all of their available bandwidth, that should be easily noticeable in any basic bandwidth graphs.

by bcl 17y ago

tghw 17y ago

Jesper addressed this in the IRC channel. Basically, the traffic was not actually reaching the box, it was just saturating the available bandwidth to the EBS store. Moreover, since the attack was against EBS and not EC2, I'm not sure that you could actually get any monitoring in place on the volume. I'm kind of surprised your first assumption is that they have no monitoring in place. I like BitBucket, so I'm biased, but I can't really see anything about this instance that would make assume it was caused by some sort of incompetence on their part.

bcl 17y ago
Sorry, I'm not familiar with how EBS connects to the EC2 instances so I assumed the incoming traffic was hitting their EC2 instance. Are you are saying that EBS is accessible from outside and that the attackers were able to discover its IP and attack it directly? This doesn't seem like a very safe way to configure things. I also wonder why Amazon didn't notice this with any of their monitoring of either traffic or system performance and latency. Assuming they have such monitoring in place of course.

rbranson 17y ago

They may have been, but I imagine it's difficult to discern the difference between legitimate traffic and a DoS attack if you've got NAS volumes mounted over the same interface. You'd need to do a more thorough network analysis.

deleted 17y ago
[deleted]
drusenko 17y ago
They claimed in many places that their "pipe" was "maxed out". If the attack was large enough to use up all of their available bandwidth, that should be easily noticeable in any basic bandwidth graphs.