7 ms·
AWS is having widespread issues
- ramshanker 9y agoHere comes the rarest opportunity of a live AWS outage postmortom. Wait... it should be called present-mortom.
- mkempe 9y agoIt's mortem, a latin word (accusative singular of "mors"). I can't think of any latin declension or any latin word that would have ended in "om".
- KyeRussell 9y agoOkay
- 65827 9y agoI wonder how many people die because of their smart homes being dependent on some network thing this time around. I'm 80% kidding, I think?
- celim307 9y agoIf you have critical systems completely reliant on the cloud, then that's just darwin awards in action
- macintux 9y agoThe problem is that the people dying aren't going to be the people who implemented the solution.
- marcosdumay 9y agoThey are going to be the people that brought them. It's still their decision. That said, we have governments for a reason.
- c17r 9y agohttps://webcache.googleusercontent.com/search?q=cache:DCdzsnXXFZMJ:https://forums.aws.amazon.com/thread.jspa%3FthreadID%3D65649+&cd=1&hl=en&ct=clnk&gl=us https://webcache.googleusercontent.com/search?q=cache:DCdzsn...
- celim307 9y agoholy shit. I really really really hope noone died due to that person's colossal stupidity.
- sidlls 9y agoI have a hard time having sympathy for someone who puts together something that critical with infrastructure meant primarily to sell to modern day gold diggers disguised as technologists.
- hoschicz 9y agoOnly one availability zone is down.
- RomanPushkin 9y agohttps://github.com/ro31337/awesome-aws-alternatives https://github.com/ro31337/awesome-aws-alternatives
- kennydude 9y agoServices mentioned here don't require a manual to read before use and don't come with a list of quirks. Good list
- ceejayoz 9y agoWhen AWS is having widespread issues half the internet seems to stop working. This looks like a 500 error on the console. Is there any actual indication of AWS issues beyond one random person's tweet? edit: Ah, https://twitter.com/ylastic https://twitter.com/ylastic just went on a retweeting tear. Looks like us-east-1?
- tbcj 9y agoAt my company we are seeing issues in us-east-1 involving KMS and EC2.
- Aqueous 9y agoInteresting - I use both those services in us-east-1 and have not experienced issues. https://status.aws.amazon.com/ https://status.aws.amazon.com/ also shows a sea of green, although I'm not sure this page is even functional, because even when I know AWS has been having issues it's a sea of green.
- ceejayoz 9y agoIn the last major AWS outage, it stayed green because updating it depended on some services affected by the outage. Even when they can update it, it seems to be a manual process.
- Bartweiss 9y agoI think they've scrapped the AWS dependencies there, which were awfully silly. But it doesn't really seem to update regardless, and when it does it's a cute little 'i' on the green checkmark to inform you that everything is fine except for the 'actually working' part.
- edgesrazor 9y agoAfter nearly 10 years working with AWS, I've learned to never trust that page.
- 9y ago
- snewman 9y agoThere was definitely an issue. Around 25% of our servers in one availability zone of us-east-1 fell off the network for 15 minutes or so, starting around 13:47 GMT. They're back now. During this time period, we were also unable to access the console (500 errors).
- edgesrazor 9y agoWhich AZ?
- snewman 9y ago"C", but that's meaningless because AWS scrambles the zone names for each account. (Presumably to prevent everyone from putting all their servers in "A".)
- edgesrazor 9y agoInteresting - I had no idea they did that. Well, I guess checking my instances in C won't be any help to you. Sorry!
- mewm 9y agoFYI, it's elaborated here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-re...
- 1001101 9y agoHaha - I didn't know that. Makes sense. I've got a dropdown in one of my Cloudformation scripts for AZs, and every time I get to it, I spend way more time thinking about it than I should. You've saved me some time.
- deleted 9y ago[deleted]
- Bartweiss 9y ago
- i_cant_speel 9y agoI've been working at my first software dev job for a few months now. I sat down at work today and, for the first time, I had to launch and configure an EC2 instance. Of course, within the first few minutes of getting started AWS starts having issues.
- aerovistae 9y agoGreat, you broke it.
- joeskyyy 9y agoThank you for a good laugh to start my day haha
- i_cant_speel 9y agoIt's called manual testing and I clearly did my job.
- aerovistae 9y ago<redacted>
- true_religion 9y agoIt's his first job, he doesn't have the experience necessary to instinctively find this funny.
- i_cant_speel 9y agoI was joking as well :)
- aerovistae 9y agoOhhhh I lose
- conanbatt 9y ago
- CoffeeDregs 9y agoFrom my dashboard: EC2 VPC network health intra AZ issue The issue that began at Thu, 27 Jul 2017 13:53:00 GMT has been resolved and the service is operating normally. Start time July 27, 2017 at 9:53:00 AM UTC-4 End time July 27, 2017 at 10:08:00 AM UTC-4
- jjfine 9y agoWe're seeing high error rates writing to Kinesis
- bryanh 9y agoAt Zapier we saw half the internet on AWS blip out for a bit (us too), but it seems to have been short lived. Approximately Jul 27, 2017 13:47:45 to Jul 27, 2017 13:59:33 (UTC) as far as we could tell.
- ending 9y agoFrom our EC2 dashboard in us-east-1: [RESOLVED] Network Connectivity 07:28 AM PDT Between 6:47 AM and 7:10 AM PDT we experienced increased launch failures for EC2 Instances, degraded EBS volume performance and connectivity issues for some instances in a single Availability Zone in the US-EAST-1 Region. edit: looks like this message is now on the status page
- forrestbrazeal 9y agoNow getting Lambda provisioning errors in us-east-1: LAMBDA_FAILED: ServiceException: We currently do not have sufficient capacity in the region you requested. Our system will be working on provisioning additional capacity. You can avoid getting this error by temporarily reducing your request rate. I wonder if they had to take part of their fleet offline due to the issues
- twunde 9y agoJira is now down too. Not bitbucket though!
- lugg 9y agoIf only it would stay down.
- holydude 9y agoThis is why I am really scared about companies owning too much market share. I mean literally who is not running or using anything that runs on AWS ?
- micael_dias 9y agoGoogle
- chimeracoder 9y ago> I mean literally who is not running or using anything that runs on AWS ? Google and Microsoft both run their own equivalents to AWS (Google Cloud and Azure, respectively). They don't have as much market share as AWS does, but they're a lot larger than you might expect.
- sly010 9y agohttps://xkcd.com/908/ https://xkcd.com/908/
- pc86 9y agoEvery company I've ever worked for, for one. There are huge swaths of the internet not affected by AWS, just as there are (other) huge swaths not affected by Google or Azure.
- ufmace 9y agoWe're heavily dependent on AWS, and haven't seen any issues yet today.
- i_cant_speel 9y agoIt seems like there are only issues on US-EAST-1
- scierama 9y agoI have external scripts monitoring my Lightsail instance, there was no downtime for Lightsail. Edit: The instance is in Ohio.
- iUsedToCode 9y agoRight now it seems that my $1/month shared hosting has less downtime than AWS this year.
- KyeRussell 9y agoIf AWS was running "$1/month shared hosting" I'm sure they'd have better uptime too. Apple and oranges.
- mrmondo 9y agoIf there's "elevated packet loss" to/from EBS, which is your disk - does that mean that people had to rebuild or redeploy instances using EBS storage?