4 ms·
Ass covering-wise, you are probably better off going down with everyone else on us-east-1. The not so fun alternative: being targeted during an RCA explaining
by yibers 9mo ago
Ass covering-wise, you are probably better off going down with everyone else on us-east-1. The not so fun alternative: being targeted during an RCA explaining why you chose some random zone no one ever heard of.
- riffic 9mo agohow about following the well-architected framework and building something with a suitable level of 9s where you can justify your decisions during a blameless postmortem (please stamp your buzzword bingo card for a prize.)
- paradox460 9mo agoWe vibe code everything in flavor of the month node frameworks, tyvm, because elixir is too hard to hire for (or some equally inane excuse)
- DANmode 9mo agoI agree with your post conceptually. However: Don’t underestimate community support (in the areas you’re likely to want it) when comparing development stacks.
- paradox460 9mo agoConversely, a community means nothing if they flit from one "best practice" to another
- transcriptase 9mo agoI look forward to the eventual launch of a new and improved version of your app using electron. What’s the point in having 64 Gb of DDR5 and 16 cores @ 4.2 GHz if not to be able to have a couple electron apps sitting at idle yet somehow still using the equivalent computational resources of the most powerful supercomputer on earth in the mid 1990s.
- paradox460 9mo agoWe also plan to incorporate a full local llm, to ensure we fill the memory up. It will be used to direct people to our online knowledge base, which will always be empty
- transcriptase 9mo agoMake sure another LLM summarizes pages upon loading, but doesn’t load any content before that completes. Each page should have a few megs of JS tracking scripts siphoning the users CPU to create massive logs on AWS that nobody will ever use to improve anything. Oh and put everything behind the strictest cloudflare settings you can, so that even a whiff of anything that’s not a Windows 11 laptop or iPhone on a major U.S. network residential or mobile IP gets non-stop bot checks!
- thejosh 9mo agoBandwidth cost is also another major reason.
- rconti 9mo agoPlaces nobody's ever heard of like "Ohio" or "Oregon"? Yeah, I'm not worried about being targeted in an RCA and pointedly asked why I chose a region with way better uptime than `us-tirefire-1`. What _is_ worth considering is whether your more carefully considered region will perform better during an actual outage where some critical AWS resource goes down in Virginia, taking my region with it anyway.
- xingped 9mo agoIIRC, some AWS services are solely deployed on and/or entirely dependent on us-east-1. I don't recall which ones, but I very distinctly remember this coming up once.
- cj 9mo agoAWS IAM has caused multiple cross-region outages.
- nothrabannosir 9mo agoCloudFront certificates
- nexus-uw 9mo agoIAM
- technicalape 9mo agoEverything new basically, like the AI services.
- paulddraper 9mo agoIAM and Route53 have dependencies on us-east-1. AWS Organizations/Account management is us-east-1. And if you want a CDN with a custom hostname and want TLS…you have to use us-east-1.
- TonyCoffman 9mo agoThe Route53 control plane is in us-east-1, with an optional temporary auto-failover to us-west-2 during outages. The data plane for public zones is globally distributed and highly resilient, with a 100% SLA. It continues to serve DNS records during regular control plane outages in us-east-1, but access to make changes is lost during outages. CloudFront CDN has a similar setup. The SSL certificate and key have to be hosted in us-east-1 for control plane operations but once deployed, the public data plane is globally or regionally dispersed. There is no auto failover for the cert dependency yet. The SLA is only three 9s. Also depends on Route53. The elephant in the room for hyperscalers is the potential for rogue employees or a cyber attack on a control plane. Considering the high stakes and economic criticality of these platforms, both are inevitable and both have likely already happened.
- throwawaysleep 9mo agoThis to me was the real lesson of the outage. A us-east-1 outage is treated like bad weather. A regional outage can be blamed on the dev. us-east-1 is too big to get blamed, which is why it should be the region of choice for an employee.
- dontdoxxme 9mo agoWhy aren't you using IBM cloud?
- deleted 9mo ago[deleted]
- throwawaysleep 9mo agoIf IBM still had a good reputation, I probably would.
- skissane 9mo agoI’ve seen people go with IBM Cloud because their salespeople were willing to discount more heavily than AWS/GCP/Azure were. Tier 2 players can be hungrier for your business than tier 1 are. And here I’m talking about completely mainstream workloads (Linux, K8S, etc) Separately from that, if you are trying to move certain types of non-mainstream IBM workloads to cloud (AIX, IBM i, z/OS) then IBM is tier 1 in that case
- Esophagus4 9mo agoBizarre way of making decisions. us-east-2 is objectively a better region to pick if you want US east, yet you feel safer picking use1 because “I’m safer making a worse decision that everyone understands is worse, as long as everyone else does it as well.”
- nemomarx 9mo agoIt's about risk profile. The question isn't "which region goes down the least" but "how often will I be blamed for an outage." If you never get blamed for a US east outage, that's better than us-east-2 if that could get you blamed 0.5% of the time when it goes down and us1 isn't down or etc
- kristianc 9mo agoI find it funny that we see complaints about why software quality has got worse alongside people advocating to choose objectively risky AWS regions for career risk and blame minimisation reasons.
- deleted 9mo ago[deleted]
- goalieca 9mo agoThis was always the case. The OG saying was “no one got fired for buying IBM”. Then it was changed to Microsoft. And so on..
- throwawaysleep 9mo agoThey are for the same reason. How do customers react to either? If us-east-1 fails, nobody complains. If Microsoft uses a browser to render components on Windows and eats all of your RAM, nobody complains.
- bigstrat2003 9mo agoOh, people complain. The companies responsible have just gotten to the point where they are so entrenched that they don't need to care at all about customer complaints.
- zx8080 9mo agoIt all sticks with the 'monopoly' scent.
- zx8080 9mo agoThe value now is not really money from customers, but a company's share price or valuation. That, together with the hard push for subscriptions from every single app and service, devaluated customer experience and feedback. Because not many will go through the hell of unsubscribing process even after the outage or serious issues like private data stolen. There's just not much motivation left to do better systems.
- nothrabannosir 9mo ago> being targeted during an RCA explaining why you chose some random zone no one ever heard of. “Duh, because there’s an AZ in us-east-1 where you can’t configure EBS volumes for attachment to fargate launch type ECS tasks, of course. Everybody knows that…” :p
- jordanb 9mo agoIstr major resource unavailability in US-East-2 during one of the big US-East-1 outages because people were trying to fail over. Then a week later there was a US-East-2 outage that didn't make the news. So if you tried to be "smart" and set up in Ohio you got crushed by the thundering herd coming out of Virginia and then bit again because aws barely cares about you region and neither does anyone else. The truth is Amazon doesn't have any real backup for Virginia. They don't have the capacity anywhere else and the whole geographic distribution scheme is a chimera.
- Fhch6HQ 9mo agoThis is an interesting point. As recently as mid-2023 us-east-2 was 3 campuses with a 5 building design capacity at each. I know they've expanded by multiples since, but us-east-1 would still dwarf them. Makes one wonder, does us-west-2 have the capacity to take on this surge?
- redditor98654 9mo agous-west-2 is indeed very large, but will still not be able to take a full failover from us-east-1
- g947o 9mo ago> explaining why you chose some random zone no one ever heard of Is this from real experience of something that actually happened, or just imagined? The only things that matter in a decision are: * Services that are available in the region * (if relevant and critical) Latency to other services * SLAs for the region Everything else is irrelevant. If you think AWS is so bad that their SLAs are not trustworthy, that's a different problem to solve.