6 ms·
No one likes DDOSes from China. One can plead Amazon as much as one wants. Pay or get booted, there are probably 2 engineers paid 6 figures a year by Amazon get
by dreamins 12y ago
No one likes DDOSes from China. One can plead Amazon as much as one wants. Pay or get booted, there are probably 2 engineers paid 6 figures a year by Amazon getting paged for this DDOS, someone must pay for the time they spend tuning DDOS protection instead of their primary project to make attacked website accessible for everyone else.
Source: worked for AWS, was oncall during similar attacks. Nasty things with those they tend to start around 6-7PM (guess when does working day start in China).
- simonebrunozzi 12y agoI used to work at AWS too. Where were you? Seattle / support?
- dreamins 12y agoSoftware engineer, I am pretty easy trackable in internets too :]
- toomuchtodo 12y agoI chuckled, because when everyone tells me "AWS is practically the internet" I can point out "The Internet is resilient at a far lower cost than Amazon".
- dreamins 12y agoInternet as a whole yes, making a single attacked web service resilient to DDOSes at a low cost is quite a challenge.
- powertower 12y agoWhy don't providers just set up a system that creates a country-level null route for a given destination IP? And have a UI with a checkbox for the user to do it, for any selected country. It would mitigate the issue, and once it's over, the user can un-restrict traffic / or just keep blocking if it's a non-valuable source. I know you can do this on the server, using many different techniques. But this does not help as the traffic still reaches you (that you have to pay for). You can also do this with Geo DNS (and get much less of a bill). And the ISPs, datacenters, and anyone with a router can block ASIA or China allocated IP ranges. Especially if it's not the type of a flood that's designed to attack the routers (instead of the web-server). So what's stopping Amazon?
- toomuchtodo 12y agoPlease, don't perceive this as being rude, it's not meant to be. Having provided IP transit at a largish network provider in a previous life, you have no idea at the complexity involved what you're asking for. It could be done, but the costs involved are non-trivial. If you're honestly interested in the complexity involved, start reading about BGP, dynamic routing protocols, router/switch fabrics, control plane integration, autonomous systems, peering agreements, etc.
- unethical_ban 12y agoI feel it shouldn't be unreasonable to expect AWS/Cloudflare/Akamai to have policy-based routing to blackhole a lot of these source subnets. Of course it's complex, but these are some of the largest hosting providers in the world.
- cookiecaper 12y agoI've found this is a common thing to say with AWS employees. One of them insisted that Amazon's ridiculous ephemeral storage policy (immediate, permanent, and irrevocable deletion on any halt or stop event, making accidental data loss a real possibility) had to be that way because it would just take too much hardware to allow a cooldown period before the drives were wiped. There's no way I believe that. I think Amazon is just used to intimidating customers with exactly that line of reasoning: "No offense, but you have no idea how hard the cloud is", and people buy it because "the cloud" is the new hotness.
- jldugger 12y agoI've had RAID 6 fail. It should be extremely rare, but isn't. And at AWS's scale, It's not hard to imagine servers going offline regularly. Ephemeral storage as a policy makes sense to me in the sense that you can separate out that what's important from that which is ephemeral, and provide cheaper storage than a more HA solution like ganeti.
- saywhst 12y ago
- dpweb 12y agoHow do we know people inside China are responsible?
- dreamins 12y agoPeople? No. IP/AS numbers. Where traffic lands first, on which POPs... Its pretty obvious.
- sbov 12y agoYou mean more than I already am paying them? The two colo centers we've hosted in have always helped us with DDOS issues free of charge. Maybe that's not normal, but even a former employee telling us to GTFO looks bad on Amazon to me.
- shaneofalltrad 12y agoInteresting when your product can be spiked, and make significant increases in profit. This looks like numbers that could potentially knock a business out of business. Reminds me of old phone bills.
- dreamins 12y agoIf product revenue doesn't grow faster or even along with traffic (expenses) it will eventually knock itself out of business one way or another.
- creshal 12y agoTurning sustained DDoS attacks into revenue sounds like an intriguing business schemes.
- dreamins 12y agoWhat do you think. Sustained DDoS attack must at least generate enough revenue to cover sustained expenses if they are incurred or no?
- dreamins 12y agoAlso usually AWS doesn't turn attack into revenue, they push customer up the "support tier" (gold/platinum whatever they are called now) and strip the DDOS traffic from expenses as much as possible. Those tiers are quite expensive though, but are fixed support costs more or less. My general point is: AWS is a business, and it operates as one. There are no hollywood style bad guys sitting there in cubicle dungeons on chests filled with gold thinking how to extract money, quite the contrary. It is understandable that customer cannot pay unlimited (from customers perspective) charges, but AWS pretty much incurs them, as customer being ddosed is consuming resources that would be otherwise be sold to others, or engineer time that would be put into developing new features and attracting new customers.
- TrevorJ 12y agoI'm pretty sure they don't make 30k a day though.
- dreamins 12y agoIn my experience Amazon will most probably write off their bill if they refuse to pay, but will refuse further service as well.
- justindz 12y agoI was at a product management event once and met a guy who managed a product in this space. A group of us went out for drinks after the event and he ended up explaining what he did. At some point he mentioned "Chinese hackers." Another guy in the group called him on it, wondering why he just assumed it was Chinese. He laughed and said that the near constant level of activity they see goes basically flat on Chinese New Year. I suppose if you're not a Chinese hacker, it might pay to pretend you are by tailoring your working hours and days.
- dreamins 12y agoYep it is THAT obvious...