7 ms·
Well, I had Linode shortlisted for an upcoming project. I hate to take them off the list because it is not their fault, but I don't want this kind of unreliabil
by i_feel_great 10y ago
Well, I had Linode shortlisted for an upcoming project. I hate to take them off the list because it is not their fault, but I don't want this kind of unreliability.
- tomschlick 10y agoAtlanta is their smallest datacenter IIRC with their lowest bandwidth capacity. I would still recommend them as we have had great luck with their Dallas DC. It seems to be the most resilient of them all in terms of random network outages and also DDOS attacks. However the one in December did take them offline for a few hours but after which they apparently implemented more DDOS protection. Keep in mind this could happen to anyone (Digital Ocean, Vultr, etc) and mostly their mitigation techniques seem to be to kill your VM until the attack is over.
- reefoctopus 10y ago>the one in December did take them offline for a few hours That is a bit of an understatement. Our servers were down every couple of hours for a week, and the attack bounced around to pretty much all of their datacenters.
- tomschlick 10y agoSorry I should have clarified. Our servers were down for a total of 4-5 hours spread across a week or so. After the first incident it only took a few minutes for them to come online each time, although I hear others had varying downtimes while they mitigated that DDOS. Thankfully it was over Christmas time so our clients were mostly offline.
- poopythecat 10y agoHow do you know atlanta is their smallest datacenter? Is there a list somewhere that shows what the capacity of each of their datacenters is? Atlanta got trashed the worst back in December but they beefed it up a lot. But how much? Is Dallas still bigger? Im considering migrating everything to dallas.
- tomschlick 10y agoI think it was a blog post that was talking about why Atlanta had so many capacity issues with the DDOS. I just tried to find it and now cant. Dallas has been superb though, definitely go with that if you can.
- poopythecat 10y agoHow was the recent attack in Dallas? Labor day weekend Atlanta went down for about 25 minutes until they squashed it. Then I saw dallas came under attack. How did it handle the DDOS this time?
- tomschlick 10y agoWe had a few seconds of latency tacked onto our normal response time for about 40 mins but everything stayed up. Here is a graph from our monitoring tool (grafana with WorldPing) http://tomschlick-screenshots.s3.amazonaws.com/RKRIJQ9Q http://tomschlick-screenshots.s3.amazonaws.com/RKRIJQ9Q
- poopythecat 10y agoVery interesting. Looks like Dallas can still handle a DDOS better. Got news that Atlanta is increasing their bandwidth 6X real soon. Its scheduled this month. They handled this DDOS pretty good. Although we actually had downtime. Adding 6X bandwidth should make a huge difference. Not sure which datacenter should be my primary though after the 6X upgrade. Oh wait, linode just responded to me. They are recommending atlanta for ddos over dallas after the 6x upgrade. Hmmmmm.
- tomschlick 10y agoGood to know! We are looking into a secondary DC setup soon so we will have to look at Atlanta after their capacity bump.
- delroth 10y ago> it is not their fault I don't understand this line of reasoning. It's not like DDoS attacks are some kind of 0-day failure mode that nobody has seen before. Would you also say "it is not their fault" if their uplink provider had a fiber cut and they didn't have redundant uplinks? I'm guessing not: it's well understood that has a service provider you need to plan for this kind of unavailability and pay more money for redundant links. So it seems really weird to have this double standard for a different kind of availability failure mode. Just like network availability or datacenter power availability, you need to invest technical and financial resources into DDoS defenses if you want to be resilient to incidents. If you don't do that as a hosting provider, I definitely won't feel sad for you.
- toomuchtodo 10y agoThere are a handful of environments that can sustain a large, coordinated DDOS attack. Can you sink 10-20Gb/s of traffic forever? Not cost effectively.
- ryanlol 10y agoIf you live in 2002, nah. In 2016? Yes. Bandwidth is cheap and linode has high margins.
- toomuchtodo 10y agoInfrastructure to service requests is not (comparitively). Rarely is it trivial to distinguish DDoS traffic from legitimate traffic.
- ryanlol 10y agoUsually it is and according to linode it is so in this case too. Edit: Not sure what alternate reality the downvoter lives in, but vast majority of the attacks these days are just "dumb" packet floods or even easier to filter reflection attacks. (Linode clarified this to be a mix of dns and ntp traffic on IRC) But hey, go on and find me a layer 7 attack that'll take down entire datacenters :)
- julianz 10y agoI have a Linode VPS that is fantastically reliable. It's showing no downtime or problems during this DDOS. They've been great to deal with.