8 ms·
The page was slashdotted for me, here's a copy: "Effective immediately, Linode Manager passwords have been expired. You will be prompted to set a new password
by wiremine 11y ago
The page was slashdotted for me, here's a copy:
"Effective immediately, Linode Manager passwords have been expired. You will be prompted to set a new password on your next login. We regret this inconvenience, however this is a necessary precaution.
"A security investigation into the unauthorized login of three accounts has led us to the discovery of two Linode.com user credentials on an external machine. This implies user credentials could have been read from our database, either offline or on, at some point. The user table contains usernames, email addresses, securely hashed passwords and encrypted two-factor seeds. The resetting of your password will invalidate the old credentials.
"This may have contributed to the unauthorized access of the three Linode customer accounts mentioned above, which were logged into via manager.linode.com. The affected customers were notified immediately. We have found no other evidence of access to Linode infrastructure, including host machines and virtual machine data.
"The entire Linode team has been working around the clock to address both this issue and the ongoing DDoS attacks. We’ve retained a well-known third-party security firm to aid in our investigation. Multiple Federal law enforcement authorities are also investigating and have cases open for both issues. When the thorough investigation is complete, we will share an update on the findings.
"You may be wondering if the same person or group is behind these malicious acts. We are wondering the same thing. At this point we have no information about who is behind either issue. We have not been contacted by anyone taking accountability or making demands. The acts may be related and they may not be.
"The security of your data, the functionality of your servers, and your confidence in Linode are extremely important to all of us. While we feel victimized ourselves, we understand it is our responsibility, and our privilege as your host, to provide the best possible security and service. You can help further enhance the security of your account by always using strong passwords, enabling two-factor authentication, and never using the same password at multiple services.
"We sincerely apologize for the recent disruptions in your Linode service. Thank you for your patience, understanding and ongoing trust in Linode."
- madaxe_again 11y agoSadly I trust them as far as I can throw them, and we moved everything important from them to AWS a few years back. We had left a few static sites there but after the shenanigans over the holidays, we're moving our remaining stuff. Sad. We spent >$10k/month with them for a while, before their shit started falling apart. They didn't appear to care at all when we left, so I suppose they have an awful lot of large customers who just don't mind their stack disintegrating beneath them every sixty days at most.
- jpea 11y agoSo, what happened during the AWS outage this past Fall? Or did you restructure to provide failover beyond just moving platforms?
- madaxe_again 11y agoWell, at linode you can't have a structure that is immune to failover, as they have single points of failure within their infrastructure, apart from anything else - all their London kit for instance lives in Telehouse East, in a few adjacent racks. Once we'd done the initial up sticks and move to AWS, our first priority was to use their redundancy and failover to the fullest (six months of sleepless nights due to linode made this rather front and foremost in our minds) - so nothing that's happened at AWS has ever been anything more than an inconvenience - we've managed five nines since the move - before, we managed one.
- artursapek 11y agoOnly one 9? Even through this crap during the holidays I've managed 3 9's on my service hosted on several servers in Linode Dallas (the most hard-hit region in this DDoS attack). I would have moved to AWS by now if Linode didn't have such cheaper bandwidth.
- madaxe_again 11y agoYeah... our issues mostly arose from the fact that at the time, they were advertising 1Gbps node interconnect - which actually turned out to be 1Gbps HOST interconnect, with the nodes actually throttled down to 50Mbps. We use memcached extensively, and this was absolutely crippling for performance. It didn't help that they furiously denied that they were throttling until we demonstrated it beyond all doubt. They did obligingly increase these caps when we begged them to do so, but at that point the writing was on the wall, and we kept on bumping into other weird and wonderful limitations and issues, such as the fact that someone running an intense job on the same host could bring our VPS's to an absolute crawl. It really is a shame, as we desperately wanted to make it work, as we liked Chris's hands on approach (very much like ours), but ultimately our confidence in them was so eroded by the point that things started going genuinely wrong on their end that we had no choice but to leave. As I said, we kept random small single-server stuff (wordpress sites mostly) there, as if you're not dealing with their networking, performance is generally OK - but the network limitations were an absolute clincher for us, and it was at one point literally every day that we'd find that one of their switches had broken, or we couldn't ARP IP's for no apparent reason, etc. etc.
- Nyr 11y ago> The page was slashdotted for me Likely the server hosting the blog is being DDoSed too. I feel bad for them, but they should know a bit better already...