25 ms·
Tell HN: Azure outage
Azure is down for us, we can't even access the azure portal. Are other experiencing this? Our services are located in Canada/Central and US-East 2
https://downdetector.ca/status/windows-azure/ https://downdetector.ca/status/windows-azure/
https://azure.status.microsoft/en-gb/status https://azure.status.microsoft/en-gb/status
- zelias 1y agoAnyone have betting odds on when Google will go down next? Are we looking at all 3 providers having outages in the span of 3 weeks?
- seinecle 1y agoCan't connect to Claude
- zzake 1y agoPortal is now accessible, bypassing FDN
- deleted 1y ago[deleted]
- llimos 1y agoYep, down from here too (in Israel). Services too, not just the portal.
- andoma 1y agoCan confirm
- andhuman 1y agoI bet it’s DNS.
- deleted 1y ago[deleted]
- andhuman 1y ago“ Starting at approximately 16:00 UTC, we began experiencing DNS issues resulting in availability degradation of some services. Customers may experience issues accessing the Azure Portal. We have taken action that is expected to address the portal access issues here shortly. We are actively investigating the underlying issue and additional mitigation actions. More information will be provided within 60 minutes or sooner. This message was last updated at 16:35 UTC on 29 October 2025”
- pbhjpbhj 1y agoThat was my bet too, then I looked at ISC and noticed there were PoCs released for critical BIND9 vulns yesterday ... might be related?
- paj33t 1y ago[dead]
- joaomoreno 1y agoYup, see it as well.
- xuf 1y agoDown here too (region West Europe)
- voidpointer2000 1y agoDown in Sweden Central as well (all our production systems are down)
- chemodax 1y agoFor me the same. It's very confusing that status page [1] is green [1]: https://azure.status.microsoft/en-us/status https://azure.status.microsoft/en-us/status
- martini333 1y agoThat status page is never red. Absolutely useless. > There are currently no active events. Use Azure Service Health to view other issues that may be impacting your services. Links to a page on Azure Portal which is down...
- endianswap 1y agoIt's red right now.
- Sharparam 1y agoOnly for the Azure Portal, despite Front Door also being down but showing as green on the status page.
- 12_throw_away 1y agoHeh, now it says Front Door and "Network Infrastructure" are down. That second one seems bad.
- kylecazar 1y agoThey added a message at the same time as your comment: "We are investigating an issue with the Azure Portal where customers may be experiencing issues accessing the portal. More information will be provided shortly."
- reid 1y agoThis is impacting the Azure CDN at azureedge.net. DNS A records for azureedge.net tenants are taking 2-6 seconds and often return nothing.
- etyhhgfff 1y agoIt's always DNS, unless it's not DNS.
- uuuubbbb 1y agoIntune, Azure, Entra down in Switzerland
- elFarto 1y agoWe saw all incoming traffic to our app drop to zero at about 15:45. I wonder how long this one will take to fix.
- sech8420 1y agoSame exact time for us as well.
- vincebowdren 1y agoUK, and other regions too; our APAC installation in Australia is affected.
- kierenj 1y agoOuch, and login.microsoftonline.com too - i.e. SSO using MS accounts. We'd just rolled that out across most (all?) of our internal systems... And microsoft.com too - that's gotta hurt
- ocdtrekkie 1y agoI am still stunned people choose to do this, considering major Office 365 outages are basically a weekly thing now.
- NetMageSCW 1y agoWe are very dependent on Azure and Microsoft Authentication and Microsoft 365 and haven’t had weekly or even monthly issues. I can think of maybe three issues this year.
- parliament32 1y agoSSO and 365 are working fine for us, but admin portals for Azure/365 are down. Our workloads in Azure don't seem to be impacted.
- planewave 1y agoIt is interesting to see the differential across different tenants in different geographies: - on a US tenant I am unable to access login.microsoftonline.com and the login flow stalls on any SSO authentication attempt. - on a European tenant, probably germany-west, I am able to login and access the Azure portal.
- manbitesdog 1y agoGuess you have NASSO now (Not A Single Sign On)
- btbuildem 1y agoIt's Safe and Secure!
- patching-trowel 1y agoAs of now Azure Status page still shows no incident. It must be manually updated, someone has to actively decide to acknowledge an issue, and they're just... not. It undermines confidence in that status page.
- charles_f 1y agoIt shows that some people have issues accessing the portal.
- baconbrand 1y agoI have never noticed that page being updated in a timely manner.
- reid 1y agoPortal and Azure CDN are down here in the SF Bay Area. Tenant azureedge.net DNS A queries are taking 2-6 seconds and most often return nothing. I got a couple successful A response in the last 10 minutes. Edit: As of 9:19 AM Pacific time, I'm now getting successful A responses but they can take several seconds. The web server at that address is not responding.
- bronco21016 1y agoUnable to access the portal and any hit to SSO for other corporate accesses is also broken. Seems like there's something wrong in their Identity services.
- blenderob 1y agohttps://azure.status.microsoft/en-us/status https://azure.status.microsoft/en-us/status says everything's fine! Any place I can read more about this outage?
- sbergot 1y agonow there is an information about "Azure Portal Access Issues". No word about front door being down.
- reid 1y agoYou're looking at it. I couldn't find any discussion elsewhere yet...
- sbergot 1y agoofficial status pages are useless most of the time.
- jeffrallen 1y agoI work for a cloud provider which is serious about transparency. Our customers know they are going to get the straight story from our status page. When you find an honest vendor, cherish them. They are rare, and they work hard to earn and keep your confidence.
- The_President 1y ago[dead]
- kryogen1c 1y agodowndetector reports coincident cloudflare outage. is microsoft using cloudflare for management plane, or is there common infra? data center problem somewhere, maybe fiber backbone? BGP?
- chemodax 1y agoIt seems Azure FrontDoor is affected, because our private VM works fine in different regions.
- rluhar 1y agoLooks like AWS is also impacted?
- zavec 1y agoYeah the graph for that one looks exactly the same shape. I wonder if they were depending on some azure component somehow, or maybe there were things hosted on both and the azure failure made enough things failover to AWS that AWS couldn't cope? If that was the case I'd expect to see something similar with GCP too though. Edit: nope looks like there's actually a spike on GCP as well
- estel 1y agoIt's possibly more likely that people mis-attribute the cause of an outage to the wrong providers when they use downdetector.
- zavec 1y agoDefinitely also a strong possibility. I wish I had paid more attention during the AWS one earlier to see what other things looked like on there at the time.
- kryogen1c 1y agodowndetector reports coincident cloudflare outage. is microsoft using cloudflare for management plane, or is there common infra? data center problem somewhere, maybe fiber backbone? BGP?
- Mr_Bees69 1y agonope, dont see any cf issues.
- kierenj 1y agoSorry - my bad. I literally just connected an old XP VM to the internet to activate it.
- NDizzle 1y agoMy best guess at the moment is something global like the CDN is having problems affecting things everywhere. I'm able to use a legacy application we have that goes directly to resources in uswest3, but I'm not able to use our more modern application which uses APIM/CDN networks at all.
- ThatManulTheCat 1y agoAzure portal currently mostly not working (UK)... Downdetector reporting various Microsoft linked services are out (Minecraft, Microsoft 365, Xbox...)
- mystcb 1y agoUpdated 16:35 UTC Azure Portal Access Issues Starting at approximately 16:00 UTC, we began experiencing DNS issues resulting in availability degradation of some services. Customers may experience issues accessing the Azure Portal. We have taken action that is expected to address the portal access issues here shortly. We are actively investigating the underlying issue and additional mitigation actions. More information will be provided within 60 minutes or sooner. This message was last updated at 16:35 UTC on 29 October 2025 ---- Azure Portal Access Issues We are investigating an issue with the Azure Portal where customers may be experiencing issues accessing the portal. More information will be provided shortly. This message was last updated at 16:18 UTC on 29 October 2025 -- From the Azure status page
- mystcb 1y agoUpdate 16:57 UTC: Azure Portal Access Issues Starting at approximately 16:00 UTC, we began experiencing Azure Front Door issues resulting in a loss of availability of some services. In addition. customers may experience issues accessing the Azure Portal. Customers can attempt to use programmatic methods (PowerShell, CLI, etc.) to access/utilize resources if they are unable to access the portal directly. We have failed the portal away from Azure Front Door (AFD) to attempt to mitigate the portal access issues and are continuing to assess the situation. We are actively assessing failover options of internal services from our AFD infrastructure. Our investigation into the contributing factors and additional recovery workstreams continues. More information will be provided within 60 minutes or sooner. This message was last updated at 16:57 UTC on 29 October 2025 --- Update: 16:35 UTC: Azure Portal Access Issues Starting at approximately 16:00 UTC, we began experiencing DNS issues resulting in availability degradation of some services. Customers may experience issues accessing the Azure Portal. We have taken action that is expected to address the portal access issues here shortly. We are actively investigating the underlying issue and additional mitigation actions. More information will be provided within 60 minutes or sooner. This message was last updated at 16:35 UTC on 29 October 2025 --- Azure Portal Access Issues We are investigating an issue with the Azure Portal where customers may be experiencing issues accessing the portal. More information will be provided shortly. This message was last updated at 16:18 UTC on 29 October 2025 --- Message from the Azure Status Page: https://azure.status.microsoft/en-gb/status https://azure.status.microsoft/en-gb/status
- jdc0589 1y agoyea its not just the portal. microsoft.com is down too
- mystcb 1y agoYeah, I am guessing it's just a placeholder till they get more info. I thought I saw somewhere that internally within Microsoft it's seen as a "Sev 1" with "all hands on deck" - Annoyingly I can't remember where I saw it, so if someone spots it before I do, please credit that person :D Edit: Typo!
- somerandomness 1y agoyep having trouble logging into https://entra.microsoft.com/ https://entra.microsoft.com/ as well
- baconbrand 1y agoAll of our sites went down. This is my company’s busiest time of year. Hooray.
- deleted 1y ago[deleted]
- a_f 1y agoLooks like MyGet is impacted too. Seems like they use Azure: >What is required to be able to use MyGet? ... MyGet runs its operations from the Microsoft Azure in the West Europe region, near Amsterdam, the Netherlands.
- Sharparam 1y agoThe learning modules on https://learn.microsoft.com/ https://learn.microsoft.com/ also seem to have a lot of issues properly loading.
- alt227 1y agoCant access certain banking websites in the UK, I am assuming it because of this. https://www.natwest.com/ https://www.natwest.com/
- baconbrand 1y agoOur Azure DevOps site is still functioning and our Azure hosted databases are accessible. Everything else is cooked.
- tyfon 1y agoSeems to be down in Norway. Even the national digital id service is down.
- hexbin010 1y ago> Even the national digital id service is down. Can't help but smirk as my country is ramming through "Digital ID" right now
- bombcar 1y agoSomeone somewhere thought that "national digital ID service" should absolutely rely on a cloud provider in and from another country. What a time to be alive.
- llama052 1y agoJust another day with microsoft. Honestly pretty tiring as something is always generally broken.
- PacketPundit 1y ago[dead]
- ZeroConcerns 1y agoOh, well, I'm sure Azure will be given the same pass that AWS got here recently when they had their 12-hour outage...
- taeric 1y agoI didn't realize AWS got a pass?
- graemep 1y agoHave repeated outages lost them customers? has it lost them any money in any way? That is a pass.
- taeric 1y agoApologies, but this just reads like a low effort critique of big things. To be clear, they should get criticism. They should be held liable for any damage they cause. But that they remain the biggest cloud offering out there isn't something you'd expect to change from a few outages that, by most all evidence, potential replacements have, as well? More, a lot of the outages potential replacements have are often more global in nature.
- graemep 1y agoI would say you are explaining why they get a free pass so they still get one - they are bad but their main competitors are even worse! I thought one of the major selling points of the big cloud providers was that they were more reliable than running your own stuff (by which i mean anything from a VPS to multiple data centres depending on your scale. Compared to those alternatives they seem to be less reliable in practice! The solution is to have a multi-region, or even multi-cloud setup, but then bang goes the "they do all the work for you" argument (which i doubt anyway).
- taeric 1y agoThat isn't a free pass. You have no data showing how many people did go to competitors over this. You are asserting it is zero, but why do you think that? Going on the talks here, you can find plenty of folks that opted not to go with or stay on them. You are further asserting that these outages prove they are not still more reliable than home spun. Is that the case? More than a few people aren't ready for a single hard drive to crash on the stuff they are doing.
- thewisenerd 1y agothey recently had an incident with front door reachability, wonder if it's back. QNBQ-5W8
- djeastm 1y agoI'm mid-deployment, but thankfully it seems to be running ok so far. Just the portal is not working so my visibility is not good.
- nartaczact 1y agoSounds like Shrodinger's Deploy
- deleted 1y ago[deleted]
- siva7 1y agoauth services are down
- barpol 1y agostill down
- Jamie452 1y agoCurrently standing in a half closed supermarket because the tills are down and they cant take payments
- SoftTalker 1y agoMind-boggling that any retailer would not have the capability to at least run the checkout stations offline.
- withinboredom 1y agoI knew an old guy in the '00s who specialized in cobal/fortran for working on tiller software. Guess he retired and they couldn't maintain it
- computerdork 1y agoAnyone remember Bob's number?? Bob?! Oh the humanity! We're all gonna be canned!
- tcmart14 1y agoYou can, but it's all about risk mitigation. Most processors have some form of store and forward (and it can have limitations like only X number of transactions). Some even have controls to limit the amount you can store-and-forward (for instance, only charges under $50). But ultimately, it's still risk mitigation. You can store-and-forward, but you're trusting that the card/account has the funds. If it doesn't, you loose and ain't shit you can do about it. If you can't tolerate any risk, you don't turn on store and forward systems and then you can't process cards offline. Its not the we are not capable. Its, is the business willing to assume the risk?
- bombcar 1y agoMost retailers trust their cashiers a bit less than they trust the customers. They'd rather shut down during a power/Internet failure than give any autonomy to the worker drones.
- 1y ago
- twodave 1y agoAppears to be an issue in Front Door. Our back end stuff is fine but FD is bouncing everything.
- NDizzle 1y agoYeah, I have non prod environments that don't use FD that are functioning. Routing through FD does not work. And a different app, nonprod doesn't use FD (and is working) but loads assets from the CDN (which is not working). FD and CDN are global resources and are experiencing issues. Probably some other global resources as well. Hate to say it, but DNS is looking like it's still the undisputed champ.
- borg16 1y agoi guess folks in azure wanted to show some solidarity with aws brethren (couldn't resist adding it. i acknowledge this comment adds no value to the discussion)
- aurumque 1y agoAzure goes down all the time. On Friday we had an entire regional service down all day. Two weeks ago same thing different region. You only hear about it when it's something everyone uses like the portal, because in general nobody uses Azure unless they're held hostage.
- Mr_Bees69 1y agoYeah, im regretting my decision to buy an xbox now. Every once in a while, everything goes down.
- 8cvor6j844qw_d6 1y agoQuite close to the recent AWS outage. Let me take a look if its a major one similar to AWS. Any guess on what's causing it? In hindsight, I guess the foresight of some organizations to go multi-cloud was correct after all.
- conroydave 1y agocost cutting attempts
- iAMkenough 1y agoTrusting AI without sufficient review and oversight of changes to production.
- shepherdjerred 1y agoDo you have any insight or do you just dislike AI? Incidents like this happened long before AI generated code
- Capricorn2481 1y agoI don't think it's meant to be serious. It's a comment on Microsoft laying off their staff and stuffing their Azure and Dotnet teams with AI product managers.
- whynotminot 1y agoYeah, these things never happened when humans were trusted without sufficient review and oversight of changes to production.
- stuff4ben 1y agoIt's always freakin DNS...
- jcims 1y agoWe're multi-cloud and it really saved a few workloads last week with the AWS issue. It's not easy though.
- tartieret 1y agoMicrosoft posted an update on X: https://x.com/AzureSupport/status/1983569891379835372?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet https://x.com/AzureSupport/status/1983569891379835372?ref_sr... "We’re investigating an issue impacting Azure Front Door services. Customers may experience intermittent request failures or latency. Updates will be provided shortly."
- llama052 1y agoAlways fun when you can't trust the main status page but have to go to some opinionated social medial website to see the actual problem.
- drjasonharrison 1y agohttps://www.cbc.ca/news/investigates/tesla-grok-mom-9.6956930 https://www.cbc.ca/news/investigates/tesla-grok-mom-9.695693... This mom’s son was asking Tesla’s Grok AI chatbot about soccer. It told him to send nude pics, she says xAI, the company that developed Grok, responds to CBC: 'Legacy Media Lies'
- vanviegen 1y agoMany (all?) LinkedIn profiles are also down for me. Luckily the frontpage still works. ;-) Go cloud!
- AznHisoka 1y agoLuckily?
- SoftTalker 1y agoWe're on Office 365 and so far it's still responding. At least Outlook and Teams is.
- jeffdn 1y agoThey don't run on Azure!
- rcarmo 1y agoAre you absolutely sure?
- jansper39 1y agoThey don't, however authentication for those services relies on Entra ID which seems to be affected.
- rcarmo 1y agoI'd say DNS/Front Door (or some carrier interconnect) is the thing affected, since I can auth just fine in a few places. (I'm at MS, but not looped into anything operational these days, so I'm checking my personal subscription).
- RajT88 1y agoThey definitely do run on Azure. Probably not 100%, but at least some footprint of those services do.
- ape4 1y ago2026: the year of your own metal in a rack
- drewnick 1y agoI've been doing it since 1998 in my bedroom with a dual T1 (and on to real DCs later). While I've had some outages for sure it makes me feel better I am not that divergent in uptime in the long run vs big clouds.
- Aperocky 1y agoI'd predict the year of linux desktop instead.
- hshdhdhehd 1y agoAt least YOLD is possible. Is there capacity in the world for everyone to ditch clouds.
- 0xbadcafebee 1y ago2027: the year of migrating from your own metal to a managed provider 2028: the year of migrating from a managed provider to the cloud 2029: the year of migrating from the cloud to your own metal in a rack People keep thinking the solution to their problems is to do something new (that they don't fully understand). TIL it's called Nirvana Fallacy
- majnata 1y agoThe Azure API is still working though.
- m_fayer 1y agoAnd there goes https://www.microsoft.com/ https://www.microsoft.com/
- rawgabbit 1y agoMeanwhile the layoffs continue https://www.entrepreneur.com/business-news/microsoft-ceo-explains-recent-layoffs-in-internal-memo/495027 https://www.entrepreneur.com/business-news/microsoft-ceo-exp...
- ctoth 1y agoLayoffs will continue until uptime improves!
- onraglanroad 1y agoNow that is actually funny!
- FeteCommuniste 1y ago> [Satya Nadella] said that the company’s future opportunity was to bring AI to all eight billion people on the planet. But what if I don't want AI brought to me?
- ryandrake 1y agoLike most technology initiative these tech CEOs dream up: You're going to get it and swallow it, whether you want it or not.
- mring33621 1y agoSounds like someone has a case of the 'Mondays'...
- binarymax 1y agoThe mondAIs
- gcanyon 1y agoReal life Pluribus https://en.wikipedia.org/wiki/Pluribus_(TV_series) https://en.wikipedia.org/wiki/Pluribus_(TV_series)
- Steven_Vellon 1y agoFor us, it looks like most services are still working (eastus and eastus2). Our AKS cluster is still running and taking requests. Failures seem limited to management portal.
- vinyl7 1y agoVibe coded internet keeps getting better
- avgDev 1y agoQuick find someone who can actually read documentation and code!
- the_af 1y agoYou just paste the outage error codes back to the LLM and pray it's still working and can fix whatever went wrong!
- m_fayer 1y agoWhen all the people forget to code for themselves, every LLM will code itself out of existence with that one last bug. One, after another.
- rvz 1y agoLooking forward to the post mortem.
- internet_points 1y ago> What went wrong and why? > An inadvertent tenant configuration change within Azure Front Door (AFD) triggered a widespread service disruption affecting both Microsoft services and customer applications dependent on AFD for global content delivery. The change introduced an invalid or inconsistent configuration state that caused a significant number of AFD nodes to fail to load properly, leading to increased latencies, timeouts, and connection errors for downstream services. > As unhealthy nodes dropped out of the global pool, traffic distribution across healthy nodes became imbalanced, amplifying the impact and causing intermittent availability even for regions that were partially healthy. We immediately blocked all further configuration changes to prevent additional propagation of the faulty state and began deploying a ‘last known good’ configuration across the global fleet. Recovery required reloading configurations across a large number of nodes and rebalancing traffic gradually to avoid overload conditions as nodes returned to service. This deliberate, phased recovery was necessary to stabilize the system while restoring scale and ensuring no recurrence of the issue. > The trigger was traced to a faulty tenant configuration deployment process. Our protection mechanisms, to validate and block any erroneous deployments, failed due to a software defect which allowed the deployment to bypass safety validations. Safeguards have since been reviewed and additional validation and rollback controls have been immediately implemented to prevent similar issues in the future. So, so far they're saying it's a combination of bad config + their config-validator had a bug. Would love more details.
- Aldipower 1y agoWe have some trouble with the AFD in Germany too.
- rcarmo 1y agoNot seeing it. I have VMs in US East and Netherlands and they're up.
- tgv 1y agoI tried to look some things up on their support pages before 1600Z, and it timed-out. The Dutch railways are also affected (they're an MS shop, IIRC).
- giantg2 1y agoCompare the comments and news coverage on this compared to the AWS outage... pretty telling.
- Uehreka 1y agoI noticed that Starbucks mobile ordering was down and thought “welp, I guess I’ll order a bagel and coffee on Grubhub”, then GrubHub was down. My next stop was HN to find the common denominator, and y’all did not disappoint.
- pants2 1y agoGood thing HN is hosted on a couple servers in a basement. Much more reliable than cloud, it seems!
- parliament32 1y agoAlways has been.
- lysace 1y agoIt was on AWS at least (for a while) in 2022. https://news.ycombinator.com/item?id=32030400 https://news.ycombinator.com/item?id=32030400
- jjice 1y agoYeah looks like they're back on M5. dang saying it's temporary: https://news.ycombinator.com/item?id=32031136 https://news.ycombinator.com/item?id=32031136 $ dig news.ycombinator.com ; <<>> DiG 9.10.6 <<>> news.ycombinator.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54819 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;news.ycombinator.com. IN A ;; ANSWER SECTION: news.ycombinator.com. 1 IN A 209.216.230.207 ;; Query time: 79 msec ;; SERVER: 100.100.100.100#53(100.100.100.100) ;; WHEN: Wed Oct 29 13:59:29 EDT 2025 ;; MSG SIZE rcvd: 65 And that IP says it's with M5 again.
- dang 1y agoJust don't use genetically identical hardware: https://news.ycombinator.com/item?id=32031639 https://news.ycombinator.com/item?id=32031639 https://news.ycombinator.com/item?id=32032235 https://news.ycombinator.com/item?id=32032235 Edit: wow, I can't believe we hadn't put https://news.ycombinator.com/item?id=32031243 https://news.ycombinator.com/item?id=32031243 in https://news.ycombinator.com/highlights https://news.ycombinator.com/highlights. Fixed now.
- avgDev 1y agoI am having a bunch of issues. It looks like their sites and azure are both affected. I also got weird notification in VS2022 that my license key was upgraded to Enterprise, but we did not purchase anything.
- ThatManulTheCat 1y agoFree upgrade
- Mr_Bees69 1y agoMight be a failsafe, if you cant get a license status, and you're aware that MS is down, just default to the highest tier.
- vs4vijay 1y agoService Status: https://status.cloud.microsoft/ https://status.cloud.microsoft/ and https://azure.status.microsoft/en-us/status https://azure.status.microsoft/en-us/status
- ipsum2 1y agoStatus page (first link) is down for me. Second one works
- charv 1y agooh the irony, the status link being down too
- karateka01 1y agostatus page being affected by the same issue is so lame
- millzlane 1y agoIt begs the question from a noob like me... Where should they host the status page? Surely it shouldn't be on the same infra that it's supposed to be monitoring. Am I correct in thinking that?
- deleted 1y ago[deleted]
- aftergibson 1y agoLooks like the status page is overloaded...
- flumpcakes 1y agoPretty much all Azure services seem to be down. Their status page says it's only the portal since 16:00. It would be nice if these mega-companies could update their status page when they take down a large fraction of the Internet and thousands of services that use them.
- kierenj 1y agoFWIW, all of our databases, VMs, AKS clusters, services, jobs etc - are all working fine. Which services are down for you, maybe we can build a list?
- reknih 1y agoFront Door is down for us (as Azure‘s Twitter account confirms)
- parliament32 1y agoAll of our Azure workloads are up, but we don't use Azure Front Door. That seems to be the only impacted product, apart from the management portal.
- flumpcakes 1y agoWe're using Application Gateway for ingress, that seems to be effected.
- jayw_lead 1y agoSame playbook for AWS. When they admitted that Dynamo was inaccessible, they failed to provide context that their internal services are heavily dependent on Dynamo It's only after the fact they are transparent about the impact
- wbsun 1y agoDoes their status page depend on something that is down already, so the page just fails static now hence no new updates?
- mythz 1y agoHigh availability is touted as a reason for their high prices, but I swear I read about major cloud outages far more than I experience any outages at Hetzner.
- bad_haircut72 1y agoSame with DigitalOcean. I run one box and it hasnt gone down for like 2 years
- yabones 1y agoDO has been shockingly reliable for me. I shut down a neglected box almost 900 days uptime the other day. In that time AWS has randomly dropped many of my boxes with no warning requiring a manual stop/start action to recover them... But everybody keeps telling me that DO isn't "as reliable" as the big three are.
- robotnikman 1y agoSame here, I run a few droplets for personal projects and never had any issues with then.
- ipdashc 1y agoTo be fair, in the AWS/Azure outages, I don't think any individual (already created) boxes went down, either. In AWS' case you couldn't start up new EC2 instances, and presumably same for Azure (unless you bypass the management portal, I guess). And obviously services like DynamoDB and Front Door, respectively, went down. Hetzner/DO don't offer those, right? Or at least they're not very popular.
- bongodongobob 1y agoIt's just the admin portal.
- NDizzle 1y agoIt's absolutely not only the admin portal. It's CDN and FrontDoor at least.
- dlcarrier 1y agoYesterday Amazon, today Microsoft. Are Google's cloud services going down tomorrow?
- m_fayer 1y agoAnd if they don't, we'll know who the culprit is.
- shishcat 1y agoWho?
- deleted 1y ago[deleted]
- Insanity 1y agoMaybe they are and no one realized yet.. :P That said, I don't hear about GCP outages all that often. I do think AWS might be leading in outages, but that's a gut feeling, I didn't look up numbers.
- xenolithis 1y agofairly certain they had a significant multi region outage within the past few years. I'll try to find some details to link. Few customers....few voices to complain as well.
- Mr_Bees69 1y agoas a victim of xbox, azure is down 'bout as often as its up
- luhn 1y agoThey had a pretty massive one earlier this year. https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1S... This isn't GCP's fault, but the outage ended up taking down Cloudflare too, so in total impact I think that takes the cake.
- vpears87 1y agoAt least MSFT is consistent: https://www.microsoft.com/en-us/ https://www.microsoft.com/en-us/ is down as well
- CommanderData 1y agoLikely behind Azure Front Door. Much of Xbox is behind that too.
- dlcarrier 1y agoWe're quickly learning who's relying on a single cloud provider.
- Insanity 1y agoMulti cloud is really hard to get right at scale, and honestly not worth the effort for the majority of companies and use-case.
- shagie 1y agoLike AWS or GCP? https://downdetector.com/status/aws-amazon-web-services/ https://downdetector.com/status/aws-amazon-web-services/ - https://downdetector.com/status/google-cloud/ https://downdetector.com/status/google-cloud/
- MiguelHudnandez 1y agoWhen you look at the scale of the reports, you find they are much lower than Azure's. seeing a bunch of 24-hour sparkline type graphs next to each other can make it look like they are equally impacted, but AWS has 500 reports and Azure has 20,000. The scale is hidden by the choice of graph. In other words, people reporting outages at AWS are probably having trouble with microsoft-run DNS services or caching proxies. It's not that the issues aren't there, it's that the internet is full of intermingled complexity. Just that amount of organic false-positives can make it look like an unrelated major service is impacted.
- macshome 1y agoI just tried to check the Xbox services status page and it never even loaded.
- chokolad 1y agoMajority of actual Xbox services are working fine, xbox.com itself is busted.
- Jarwain 1y agoOn our end, our VMs are still working, so our gitlab instance is still up. Our services using Azure App Services are available through their provided url. However, Front Door is failing to resolve any domains that it was responsible for.
- ThatManulTheCat 1y agoYudkowsky's feared Superintellignece holding Azure hostage
- LaserToy 1y agoAzure portal still insists the issue is jsut with Console. We had to bypass the Frontdoor
- ksec 1y ago>Last week AWS, now this. This is not the first or second time this happened, multiple Hyperscaler failed one by one.
- cbovis 1y agoLooks to be affecting our pipelines that rely on Playwright as they download images from Azure e.g. https://playwright.azureedge.net/builds/chromium/1124/chromium-linux.zip https://playwright.azureedge.net/builds/chromium/1124/chromi... which aren't currently resolving.
- deleted 1y ago[deleted]
- kierenj 1y agomicrosoft.com is back - edit: it worked once, then died again. So I guess - some resolvers, or FD servers may be working!
- improbableinf 1y agoWhat a time to be alive!
- speckx 1y agoFYI: https://status.cloud.microsoft/ https://status.cloud.microsoft/
- Boxersteavee 1y ago503 Service Unavailable
- speckx 1y agoFYI: https://status.cloud.microsoft/ https://status.cloud.microsoft/
- chuckadams 1y agoWhich itself is^H^H was down. Wow.
- jacquesm 1y agoIt is much more than azure. One of my kids needs a key for their laptop and can't reach that either. Great excuse though, 'Azure ate my homework'. What a ridiculous world we are building. Fuck MS and their account requirements for windows.
- thimkerbell 1y agoDoes (should, could) DownDetector also say what customer-facing services are down, when some infrastructure is unworking? Or is that the info that the malefactors are seeking?
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- improbableinf 1y agoAccording to downtector.com - both AWS and GCP are down as well. Interesting
- jasonjmcghee 1y agoDon't visit this address.
- bernardo786 1y agonow aws down again?
- pred8er 1y agothings seem to be coming back up now
- MangoCoffee 1y agoThe Internet is supposed to be decentralized. The big three seem to have all the power now (Amazon, Microsoft, and Google) plus Cloudflare/Oracle. How did we get here? Is it because of scale? Going to market in minutes by using someone else's computers instead of building out your own, like co-location or dedicated servers, like back in the day.
- TacticalCoder 1y ago[dead]
- kube-system 1y agoIt still is very decentralized. We are discussing this via the internet right now.
- deleted 1y ago[deleted]
- Mr_Bees69 1y agoYeah, but MyChart is down.
- acedTrex 1y agobig, if true
- deleted 1y ago[deleted]
- kbelder 1y agoI need to drop AWS and start passing data through encrypted HN posts.
- chasd00 1y agoWhen AWS was down we were talking about it here, now Azure is down and we're still talking about it here. Where does HN actually live?
- AdmiralAsshat 1y agoSome exec at Microsoft told the Azure guys to ape everything Amazon does and they took it literally.
- dboreham 1y agoThis is funny but also possibly true because: business/MBA types see these outages as a way to prove how critical some services are, leading to investors deciding to load up on the vendor's stock.
- alt227 1y agoI may or may not have been known to temporarily take a database down in the past to make a point to management about how unreliable some old software is.
- Telemakhos 1y agoOr, the NSA needed to upgrade their access at both.
- embedding-shape 1y agoDo Microsoft still say "If the government has a broader voluntary national security program to gather customer data, we don't participate in it" today (which PRISM proved very false), or are they at least acknowledging they're participating in whatever NSA has deployed today?
- terminalshort 1y agoPRISM wasn't voluntary. Also there are 3 levels here: 1. Mandatory 2. "Voluntary" 3. Voluntary And I suspect that very little of what the NSA does falls into category 3. As Sen Chuck Schumer put it "you take on the intelligence community, they have six ways from Sunday at getting back at you"
- 1y ago
- almosthere 1y agoReports of Azure and AWS down on the same day? Infrastructure terrorism?
- reaperducer 1y agoReports of Azure and AWS down on the same day? Infrastructure terrorism? > We have confirmed that an inadvertent configuration change as the trigger event for this issue. Save the speculation for Reddit. HN is better than that.
- 12_throw_away 1y ago> Infrastructure terrorism? Unless that's a euphemism for "vibe coding", no.
- vachina 1y agomicrosoft.com and some subdomains (answers.microsoft.com) has no A and AAA records. They screwed up big time. https://archive.is/Q4izZ https://archive.is/Q4izZ
- 0xbadcafebee 1y agoThat specific subdomain has issues with propagation: https://dnschecker.org/#A/answers.microsoft.com https://dnschecker.org/#A/answers.microsoft.com (only four resolvers return records) The root zone and www. do not: https://dnschecker.org/#A/microsoft.com https://dnschecker.org/#A/microsoft.com (all resolvers return records) And querying https://www.microsoft.com/ https://www.microsoft.com/ results in HTTP 200 on the root document, but the page elements return errors (a 504 on the .css/.js documents, a 404 on some fonts, Name Not Resolved on scripts.clarity.ms, Connection Timed Out on wcpstatic.microsoft.com and mem.gfx.ms). That many different kinds of errors is actually kind of impressive. I'm gonna say this was a networking/routing issue. The CDN stayed up, but everything else non-CDN became unroutable, and different requests traveled through different paths/services, but each eventually hit the bad network path, and that's what created all the different responses. Could also have been a bad deploy or a service stopped running and there's different things trying to access that service in different ways, leading to the weird responses... but that wouldn't explain the failed DNS propagation.
- Aperocky 1y agowow, right after AWS suffered a similar thing. I wonder if this is microsoft "learning" to "prevent" such an issue and instead triggered it... "One often meets his destiny on the path he takes to avoid it" -- Master Oogway
- pred8er 1y agolooks like MS completed a failover and things are be recovering slowly
- Mr_Bees69 1y agoMS website seems to be up but really slow. Think xbox might still be down, Bing works for some reason tho!?
- opengrass 1y agoGithub Actions and Codespaces degraded.
- tecleandor 1y agoLinkedIn has been acting funny for an hour or so, and some pages in the learn.microsoft.com domain have been failing for me too...
- philipallstar 1y agoCan't get to microsoft.com even.
- gianpaj 1y agoCan't download VSCode :D Error: visual-studio-code: Download failed on Cask 'visual-studio-code' with message: Download failed: https://update.code.visualstudio.com/1.105.1/darwin-arm64/stable https://update.code.visualstudio.com/1.105.1/darwin-arm64/st...
- robotnikman 1y agoAlso cant do anything right now with the repo's we have in Azure Devops, how lovely...
- loopduplicate 1y agoget vscodium then
- progmetaldev 1y agoI have had intermittent issues with winget today. I use UniGetUI for a front-end, and anything tied to Microsoft has failed for me. Judging by the logs, it's mostly retrieving the listing of versions (I assume similar to what 'apt-get update' does, I'm fairly new to using winget for Windows package management).
- ApolloFortyNine 1y agoThey admit in their update blurb azure front door is having issues but still report azure front door as having no issues on their status page. And it's very clear from these updates that they're more focused on the portal than the product, their updates haven't even mentioned fixing it yet, just moving off of it, as if it's some third party service that's down.
- consp 1y ago> as having no issues on their status page Unsubstantiated idea: So the support contract likely says there is a window between each reporting step and the status page is the last one and the one in the legal documents giving them several more hours before the clauses trigger.
- empath75 1y agoFriend of mine at MSFT says it's a Sev-0 outage and they can't even get to the ticket tracking system.
- jimmyl02 1y agopretty interesting how datadog's uptime tracker (https://updog.ai/ https://updog.ai/) says all the sites are fully available. if that's true then it's a sign that Azure's control / data plane separation is doing it's job! at least for now
- jonathanlydall 1y agoOur Azure hosted dotnet App Service is working fine, but our docs site served via Front Door went down. Can’t access anything through the Portal.
- layer8 1y agoMaybe they need a downtime tracker. ;)
- syntaxing 1y agoI absolutely love the utility aspect of LLMs but part of me is curious if moving faster by using AI is going to make these sorts of failure more and more often.
- monkaiju 1y agoIf true then what "utility" is there?
- 1718627440 1y agoMore visibility for the general person to see how brittle software is?
- givemeethekeys 1y agoSurely more vibecoding will fix this problem. Time to fire more staff
- agency 1y agoSo that's why I can't check in for my Alaska Airlines flight... https://news.microsoft.com/source/features/digital-transformation/how-alaska-airlines-uses-technology-to-ensure-its-passengers-have-a-seamless-journey-from-ticket-purchase-to-baggage-pickup/ https://news.microsoft.com/source/features/digital-transform...
- kurttheviking 1y agoI am unable to load this article...presumably for related reasons
- Shuddown 1y agoPretty much every single Microsoft domain I've tried to access loads for a looooong time before giving me some bare html. I wonder if someone can explain why that's happening.
- sodafountan 1y agoI was wondering the same thing
- MangoCoffee 1y ago"BREAKING: Alaska Airlines' website, app impacted amid Microsoft Azure outage" https://www.youtube.com/watch?v=YJVkLP57yvM https://www.youtube.com/watch?v=YJVkLP57yvM
- okokwhatever 1y agoThis cannot be a coincidence
- hypeatei 1y agoAll of my employers things are hosted on Azure and running just fine and didn't go down at all. Portal access has been fixed. Doesn't seem to be too bad of an outage unless you were relying on Azure Front Door.
- randomsofr 1y agoSSO is down, Azure Portal Down and more, seems like a major outage. Already a lot of services seem to be affected: banks, airlines, consumer apps, etc.
- gmassman 1y agoI’ve been migrating our services off of Azure slowly for the past couple of years. The last internet facing things remaining are a static assets bucket and an analytics VM running Matomo. Working with Front Door has been an abysmal experience, and today was the push I needed to finally migrate our assets to Cloudflare. I feel pretty justified in my previous decisions to move away from Azure. Using it feels like building on quicksand…
- alt227 1y agoAll the clouds hav had major outages this year. At this point I dont believe that any one of them is any better or reliable than the others.
- btmiller 1y agoNever let a good disaster go to waste ;)
- not_a_bot_4sho 1y ago> I feel pretty justified in my previous decisions to move away from Azure I felt this way about AWS last week
- amluto 1y agovscode.dev appears to be down. I think this will be my excuse to find an alternative -- I never really liked vscode.dev anyway. (Coder is currently at the top of the experiment list. Any other suggestions?)
- wingless_angel 1y agoPlease sort it out, I'll be out of a job tomorrow.
- glzone1 1y agoWasn't the saying "It's always DNS" floating around somewhere? Be interesting to understand cause here. Pretty big impact on services we use
- mikestew 1y agoCould be DNS, I'm seeing SERVFAIL trying to resolve what look to be MS servers when I'm hitting (just one example) mygoodtogo.com (trying to pay a road toll bill, and failing).
- I_am_tiberius 1y agoShouldn't regions be completely independent?
- glzone1 1y agoI remember the saying "It's always DNS". I'm old. Kind of mindboggling it's still sometimes DNS maybe.
- alt227 1y agoThat saying is just as alive today as it ever was. https://isitdns.com/ https://isitdns.com/
- 0000000000100 1y agoYeah just took down the prod site for one of our clients since we host the front-end out of their CDN. Just got wrapped up panic hosting it somewhere else for the past hour, very quickly reminds you about the pain of cookies...
- alt227 1y ago... and DNS caching, and browser file cache, and sessions... Moving a website quickly is never fun.
- anon025 1y agoIt's the DNS https://dnschecker.org/#A/get.helm.sh https://dnschecker.org/#A/get.helm.sh is unreachable
- I_am_tiberius 1y agoWhy are Azure App Services still working?
- howard941 1y agoTook out the archive.ph and .is sites too?
- LouisLazaris 1y agoThe VS Code website is down: https://code.visualstudio.com/ https://code.visualstudio.com/ And so is Microsoft: http://www.microsoft.com/ http://www.microsoft.com/
- codethief 1y agohttps://www.microsoft.com https://www.microsoft.com works for me (with the www subdomain).
- deleted 1y ago[deleted]
- bossyTeacher 1y agoI noticed issues on Azure so I went to the status page. It said everything was fine even though the Azure Portal was down. It took more than 10 minutes for that status page to update. How can one of the richest companies in the world not offer a better service?
- Ylpertnodi 1y ago>How can one of the richest companies in the world not offer a better service? Better service costs money.
- everfrustrated 1y agoGitHub runners (specifically the "larger" runner types) are all down for us. These are known to be hosted on Azure.
- deleted 1y ago[deleted]
- pred8er 1y agoon the line with msft, they said 4 hours is what they are thinking. a workaround they are saying is to use traffic manager,
- ApolloFortyNine 1y agoTwo hours after the initial outage, they have finally updated the Front Door status on their status page.
- port11 1y agoSo much of Belgium runs on Azure… it's honestly baffling how many services are down, there's no resilience built into (even large) companies anymore.
- move-on-by 1y agoInstead of cyber security awareness month, we should rename it to cloud availability awareness month.
- martijnvds 1y agoThis probably explains why paying for street parking in Cologne by phone/web didn't work (eternal spinner) then
- ukblewis 1y agoGitHub also seems to be having trouble for me
- jacquesclouseau 1y agoMy bet is on a bad config change.
- croemer 1y agoThey already announced that.
- hedayet 1y agoThe sad thing is - $MSFT isn't even down by 1%. And IIRC, $AMZN actually went up during their previous outage. So if we look at these companies' bottom lines, all those big wigs are actually doing something right. Sales and lobbying capacity is way more effective than reliability or good engineering (at least in the short term).
- navane 1y agoLook how important we are, is what these failures show
- marcosdumay 1y agoWhat do you mean? That IT isn't important for Microsoft and Amazon? That's certainly not the right conclusion.
- alt227 1y agoI think he was implying that those companies think they are so important that it doesnt matter they are down, they wont loose any customers over it because they are too big and important.
- cyberax 1y agoSo we can look forward to "accidental" cloud outages just to show their importance? I guess the GCP is next.
- Arrath 1y ago"They'll learn their lesson and be rock solid after this! I better invest now!"
- locusofself 1y agoAMZN went up almost 4 percent between the day of the outage and the day after. Crazy market.
- 1y ago
- rsolva 1y agoSo that's why all of our municipality's digital services are down ... utter chaos at the political meeting I attended just now.
- irusensei 1y agoI was working when I saw the portal page showing only resource groups and lots of items missing. I thought it was a weird browser cache issue. The actual stuff I was working on (App Insights, Function App) that was still open was operational.
- AtNightWeCode 1y agoEarnings report today. A coincidence? I can at least login to Azure. But several MS sites are down.
- tonyhart7 1y agoWtf happen with US east????
- bob1029 1y agoFor some reason an Azure outage does not faze me in the same way that an AWS outage does. I have never had much confidence in Azure as a cloud provider. The vertical integration of all the things for a Microsoft shop was initially very compelling. I was ready to fight that battle. But, this fantasy was quickly ruined by poor execution on Microsoft's part. They were able to convince me to move back to AWS by simply making it difficult to provision compute resources. Their quota system & availability issues are a nightmare to deal with compared to EC2. At this point I'd rather use GCP over Azure and I have zero seconds of experience with it. The number of things Microsoft gets right in 2025 can be counted single-handedly. The things they do get right are quite good, but everything else tends to be extremely awful.
- major505 1y agoI like azure. I think they are more intuitive tua aws and good, and they have good prices for startups ( essentially free for a whole year )
- redwood 1y agoI read "Microsoft shop" as "Microsoft slop". Fitting. But at least they open source wash themselves so much they're practically a charity right?
- xmcp123 1y agoMany years back was the first time I used Azure, evaluating it for a client. I remember I at one point had expanded enough menus that it covered the entirety of the screen. Never before have I felt so lost in a cloud product.
- jeffrallen 1y agoAmazon: here's two buttons, some check boxes and a random popup. MSFT : Hold my beer...
- WorldMaker 1y agoThe "Blades" experience [0] where instead of navigating between pages it just kept opening things to the side and expanding horizontally? Yeah, that had some fun ideas but was way more confusing than it needed to be. But also that was quite a few years back now. The Portal ditched that experience relatively quickly. Just long enough to leave a lot of awful first impressions, but not long enough for it to be much more than a distant memory at this point, several redesigns later. [0] The name "Blades" for that came from the early years of the Xbox 360, maybe not the best UX to emulate for a complex control panel/portal.
- AtNightWeCode 1y agoFrom Azure status page: "Customers can consider implementing failover strategies with Azure Traffic Manager, to fail over from Azure Front Door to your origins". What a terrible advise.
- amir734jj 1y agoIt's DNS
- worik 1y agoAn important quality of the cloud is that it is always available. Except that it is not! Interesting times...
- btbuildem 1y agohttps://login.microsoftonline.com/ https://login.microsoftonline.com/ is down, so that's fun
- basfo 1y agoWe’re 100% on Azure but so far there’s no impact for us. Luckily, we moved off Azure Front Door about a year ago. We’d had three major incidents tied to Front Door and stopped treating it as a reliable CDN. They weren’t global outages, more like issues triggered by new deployments. In one case, our homepage suddenly showed a huge Microsoft banner about a “post-quantum encryption algorithm” or something along those lines. Kinda wild that a company that big can be so shaky on a CDN, which should be rock solid.
- Aperocky 1y agoOutages are one thing, but having your content polluted seems like a more serious problem? Unless you subscribed to microsoft banners somehow.
- basfo 1y agoAnd it was HUGE, the microsoft logo was like 50% of the screen.
- qiller 1y agoWe battled https://learn.microsoft.com/en-us/answers/questions/1331370/front-door-responds-with-origintimeout-after-4-sec https://learn.microsoft.com/en-us/answers/questions/1331370/... for over a year, and finally decided to move off since there was no any resolution. Unfortunately our API servers were still behind AFD so they were affected by today's stuff...
- perks_12 1y agoThank you. I was wondering what was going on at a company whose web app I need to access. I just checked with BuiltWith and it seems they are on Azure.
- redwood 1y agoIs it Cosmos DB? If so the symmetry with AWS/Dynamo would be very eerie.
- chrisgeleven 1y ago"Front Door" has to be the worst product name for a CDN I've ever heard of. I used to work for a CDN too.
- unethical_ban 1y agoI wonder if many Germans are eager to sign up for AFD. But seriously I thought it would be the console, not a CDN.
- jeffrallen 1y agoFront Door (tm), with Back Door access for the FBI included free with your subscription! ;)
- oliyoung 1y agoWe should've never let marketing in the door honestly, all of the product names for the big three are awful. Microsoft CDN There, that's it. You're selling it to (hopefully) technical people
- joquarky 1y agoIt so strongly implies a counterpart.
- montague27 1y agoGuess when/who has the next outage!
- mattdecker100 1y agoUnable to use Ona's GitPod through VSCode SSH - Unable to download code server from https://update.code.visualstudio.com https://update.code.visualstudio.com
- smithkl42 1y agoThe iron law of uptime: "The mandatory single point of failure in every possible system is configuration."
- aftbit 1y agoI still can't log into Azure Gov Cloud with https://microsoft.com/deviceloginus https://microsoft.com/deviceloginus Seems like they migrated the non-Gov login but not the Gov one. C'mon Microsoft, I've got a deadline in a few days.
- alt227 1y agoMicrosoft have started putting customer status pages up on windows.net, so it must be really really bad! For example when I try to log into our payroll provider Brightpay, it sends me here: https://bpuk1prod1environment.blob.core.windows.net/host-properties/index.html https://bpuk1prod1environment.blob.core.windows.net/host-pro...
- qmr 1y agoAlways in these large provider outages you see people who have forgotten the old ways.
- udev4096 1y agoLuckily, no one uses azure and it's fully expected from azure to go down all the time! Keep it up!
- CKMo 1y agoReasons to not use hyperscalers, exhibit 654 There's a lot of outages this month!
- Imustaskforhelp 1y agoGoogle cloud run or cloudflare workers it is. Personally I am thinking more and more about hetzner, yes I know its not an apples to orange comparison. But its honestly so good Someone had created a video where they showed the underlying hardware etc., I am wondering if there is something like https://vpspricetracker.com/ https://vpspricetracker.com/ but with geek-benchmarks as well. This video was affiliated with scalahosting but still I don't think that there was too much bias of them and they showed at around 3:37 a graph comparison with prices https://www.youtube.com/watch?v=9dvuBH2Pc1g https://www.youtube.com/watch?v=9dvuBH2Pc1g Now it shows how contabo has better hardware but I am pretty sure that there might be some other issues, and honestly I feel a sense of trust with hetzner I am not sure about others. Either hetzner or self hosting stuff personally or just having a very cheap vps and going to hetzner if need be but hetzner already is pretty cheap or I might use some free service that I know of are good as well.
- TiredOfLife 1y agoOne of recent (4 months ago) Cloudflare outages (I think it was even workers) was caused by Google Cloud being down and Cloudflare hosting an essential service there
- kentonv 1y agoIt was Workers KV (an optional storage add-on to Workers), and we fixed it, it no longer depends on GCP: https://blog.cloudflare.com/rearchitecting-workers-kv-for-redundancy/ https://blog.cloudflare.com/rearchitecting-workers-kv-for-re...
- Imustaskforhelp 1y agoHm it seemed that they hosted a critical service for cloudflare kv on google itself, but I wonder about the update. Personally I just trust cloudflare more than google, given how their focus is on security whereas google feels googly... I have heard some good things about google cloud run and the google's interface feels the best out of AWS,Azure,GCloud but I still would just prefer cloudflare/hetzner iirc Another question: Has there ever been a list of all major cloud outages, like I am interested how many times google cloud and all cloud providers went majorly down I guess y'know? is there a website/git project that tracks this?
- sedatk 1y agoThe paradox of cloud provider crashes is that if the provider goes down and takes the whole world with it, it's actually good advertisement. Because, that means so many things rely on it, it's critically important, and has so many big customers. That might be why Amazon stock went up after AWS crash. If Azure goes down and nobody feels it, does Azure really matter?
- thewebguyd 1y agoPeople feel it, but usually not general consumers like they do when AWS goes down. If Azure goes down, it's mostly affecting internal stuff at big old enterprises. Jane in accounting might notice, but the customers don't. Contrast with AWS which runs most of the world's SaaS products. People not being able to do their jobs internally for a day tends not to make headlines like "100 popular internet services down for everyone" does.
- whalesalad 1y agoYikes, http://schemas.xmlsoap.org/soap/encoding/ http://schemas.xmlsoap.org/soap/encoding/ is running on Azure and it's down. So any SOAP/WSDL api's are dead in the water. HTTPSConnectionPool(host='schemas.xmlsoap.org', port=443): Max retries exceeded with url: /soap/encoding/ (Caused by SSLError(CertificateError("hostname 'schemas.xmlsoap.org' doesn't match '*.azureedge.net'"))) A service we rely on that isn't even running on Azure is inaccessible due to this issue. For an asset that probably never changes. Wild for that to be the SPOF. 160k+ results on GitHub: https://github.com/search?q=http%3A%2F%2Fschemas.xmlsoap.org%2Fwsdl&type=code https://github.com/search?q=http%3A%2F%2Fschemas.xmlsoap.org...
- delf 1y agoThe outage impacted GitSocial minor version bump release: https://marketplace.visualstudio.com/items?itemName=GitSocial.gitsocial https://marketplace.visualstudio.com/items?itemName=GitSocia... There's no way to tell, and after about 30 minutes, the release process on VS Code Marketplace failed with a cryptic message: "Repository signing for extension file failed.". And there's no way to restart/resume it.
- zingababba 1y agoThis brings to mind this -> https://thenewstack.io/github-will-prioritize-migrating-to-azure-over-feature-development/ https://thenewstack.io/github-will-prioritize-migrating-to-a...
- amaccuish 1y agoSeeing users having issues with the "Modern Outlook", specifically empty accounts. Switching back to the "Legacy Outlook" which functions largely without the help of the cloud fixes the issue. How ironic.
- _andrei_ 1y agohttps://www.reddit.com/r/cscareerquestions/comments/1ojbebq/just_pushed_my_first_pr_for_my_new_job_at_azure/ https://www.reddit.com/r/cscareerquestions/comments/1ojbebq/...
- user3939382 1y agoI know how to fix this but this community is too close minded and argumentative egocentric sensitive pedantic threatened angry etc to bother discussing it
- ycombinator_acc 1y agoAww man you got me curious for a sec there.
- user3939382 1y agoI’ll roll it out
- deleted 1y ago[deleted]
- tpl 1y agoPart of this outage involves outlook hanging and then blaming random addins. Pretty terrible practice by Microsoft to blame random vendors for their own outage.
- bragma 1y agoThey suggest to use Traffic Manager to route around failing CDNs. But DNS is not working too, making the suggestion another fail.
- foresterre 1y agoIt still surprises me how much essential services like public transport are completely reliant on cloud providers, and don't seem to have backups in place. Here in The Netherlands, almost all trains were first delayed significantly, and then cancelled for a few hours because of this, which had real impact because today is also the day we got to vote for the next parlement (I know some who can't get home in time before the polls close, and they left for work before they opened).
- conductr 1y agoIs voting there a one day only event? If not, I feel the solution to that particular problem is quite clear. There’s a million things that could go wrong causing you to miss something when you try to do it in a narrow time range (today after work before polls close) If it’s a multi day event, it’s probably that way for a reason. Partially the same as the solution to above.
- klardotsh 1y agoWashington State having full vote-by-mail (there is technically a layer of in-person voting as a fallback for those who need it for accessibility reasons or who missed the registration deadline) has spoiled me rotten, I couldn't imagine having to go back to synchronous on-site voting on a single day like I did in Illinois. Awful. Being able to fill my ballot at my leisure, at home, where I can have all the research material open, and drive it to a ballot drop box whenever is convenient in a 2-3 week window before 20:00 on election night, is a game-changer for democracy. Of course this also means that people who serve to benefit from disenfranchising voters and making it more difficult to vote, absolutely hate our system and continually attack it for one reason or another.
- vanviegen 1y agoAs a Dutchman, I have to go vote in person on a specific day. But to be honest: I really don't mind doing so. If you live in a town or city, there'll usually be multiple voting locations you can choose from within 10 minutes walking distance. I've never experienced waiting times more than a couple of minutes. Opening times are pretty good, from 7:30 til 21:00. The people there are friendly. What's not to like? (Except for some of the candidates maybe, but that's a whole different story. :-))
- bragma 1y agoThey suggest to use Traffic Manager to router around failing FrontDoor CDN, but DNS is failing too, making the suggestion another failure.
- asciii 1y agoYeah they're suggesting to use CLI but then my Frontdoor deployment failed. Welp.
- rodolphoarruda 1y agoI could not access MS Clarity the entire day.
- Shuddown 1y agoGithub Codespaces (for the 5 people that use them) are also still down.
- _pdp_ 1y agoWith all the recent outages considered, it is time to move off the cloud.
- tonymet 1y agoHello fellow boomers! I noticed that winget is also down eg. winget upgrade fabric Failed in attempting to update the source: winget An unexpected error occurred while executing the command: InternetOpenUrl() failed. 0x80072ee7 : unknown error
- _oleksandr_ 1y agoBased on the delay in resolving the issue, it appears MC attempted to rehire some of the DevOps engineers whom AI had previously replaced.
- jeffrallen 1y agoThey probably hired the ones AWS laid off, causing the AWS outage. Institutional knowledge matters. Just has to be the right institution is all.
- m_a_g 1y agoIt’s not DNS There is no way it’s DNS It was DNS
- eeasss 1y agoDeglobalization in geopolitics should be followed by deglobalization in cloud providers as well. Viva la local vendors.
- senderista 1y agoEven if the cloud providers have much better reliability than most on-prem infra, the failure correlation they induce negates much of the benefit.
- tonymet 1y agoAny healthcare IT admins care to chime in? A predominantly MS industry with critical workloads.
- progmetaldev 1y agoI was having issues a few hours ago. I'm now able to access the portal, although I get lots of errors in the browser console, and things are loading slowly. I have services in the US-East region. I have been having issues with GitHub and the winget tool for updates throughout the day as well. I imagine things are pulling from the same locations on Azure for some of the software I needed to update (NPM dependencies, and some .NET tooling).
- acd 1y agoPutting all your eggs software in one basket
- widikidiw 1y ago[flagged]
- zbowling 1y agoAlaska Airlines is redircting folks to their slimmed down international site and you can't check in on mobile.
- ycombinatornews 1y agoSo that’s why CapitalOne is out today. Even though their (incorrect) status page says all systems operational.
- journal 1y agoone day these outages will cause a starvation.
- udfalkso 1y agoOpenAI Clip python library fails because the model download is a hardcoded azure cdn url :(
- ChuckMcM 1y ago"On Prem" is looking better and better :-).
- jasonthorsness 1y agoAhh it got me, Alaska air web site has an Azure outage banner
- jmspring 1y agoThe outage was really weird. For me, parts of the portal worked, other parts didn't. I had access to a couple of resource groups, but no resources visible in those groups. Azure Devops Pipelines that needed do download from packages.microsoft.com didn't work. The Microsoft status page mostly referenced the portal outage, but it was more than that.
- bombcar 1y agoI hate these failures because you end up with things that keep working fine because the login credentials are cached, etc; but if you restart or otherwise refresh, you're doomed.
- xer0x 1y agoWow, they are still down 12 hours later. :/
- croemer 1y agoNot officially - status page says all healthy
- nextworddev 1y agoFascinating timing given the APEC summit ;)
- croemer 1y agoPreliminary post incident review: https://azure.status.microsoft/en-gb/status/history/ https://azure.status.microsoft/en-gb/status/history/ Timeline 15:45 UTC on 29 October 2025 – Customer impact began. 16:04 UTC on 29 October 2025 – Investigation commenced following monitoring alerts being triggered. 16:15 UTC on 29 October 2025 – We began the investigation and started to examine configuration changes within AFD. 16:18 UTC on 29 October 2025 – Initial communication posted to our public status page. 16:20 UTC on 29 October 2025 – Targeted communications to impacted customers sent to Azure Service Health. 17:26 UTC on 29 October 2025 – Azure portal failed away from Azure Front Door. 17:30 UTC on 29 October 2025 – We blocked all new customer configuration changes to prevent further impact. 17:40 UTC on 29 October 2025 – We initiated the deployment of our ‘last known good’ configuration. 18:30 UTC on 29 October 2025 – We started to push the fixed configuration globally. 18:45 UTC on 29 October 2025 – Manual recovery of nodes commenced while gradual routing of traffic to healthy nodes began after the fixed configuration was pushed globally. 23:15 UTC on 29 October 2025 - PowerApps mitigation of dependency, and customers confirm mitigation. 00:05 UTC on 30 October 2025 – AFD impact confirmed mitigated for customers.
- onionisafruit 1y agoAt 16:04 “Investigation commenced”. Then at 16:15 “We began the investigation”. Which is it?
- not_a_bot_4sho 1y agoI read it as the second investigation being specific to AFD. The first more general.
- onionisafruit 1y agoI think you’re right. I missed that subtlety on first reading.
- ssss11 1y agoQuick coffee run before we get stuck in mate
- unit149 1y ago[dead]
- razodactyl 1y agoAWS, now Azure - wasn't this a plot point in Terminator where SkyNet was causing computer systems to have issues much before it finally become self-aware? Funnily enough, AI has been training on its own data as generated by users writing AI conversations back to the internet - there's a feedback loop at play.
- major505 1y agoSomewhere, an ex microsoft engineer that where layoff during the last week, is saying to himself “thank god, this shit is not my problem anymore”
- zimpenfish 1y ago"Microsoft Azure will serve as the backbone of Asda’s digital infrastructure"[0] Oh, that'll be why Scan & Go was down yesterday evening. I thought it was another instance of an iOS 26 update breaking their crappy code. [0] https://corporate.asda.com/newsroom/2025/22/09/asda-announces-renewed-ai-and-cloud-collaboration-with-microsoft https://corporate.asda.com/newsroom/2025/22/09/asda-announce...
- sherinjosephroy 1y ago[flagged]
- Aldipower 1y agoHetzner, Netcup, OVH, BunnyCDN, ClouDNS, Postmark You name them. Other good providers you have experience with? There is no reason for an expensive cloud. Never has been, but decision makers tried to keep their pants dry.
- sherinjosephroy 1y ago[flagged]
- buttscicles 1y agoInteresting that everybody knows when AWS goes down but Azure needs a "Tell HN" :) Best of luck to the teams responding to this incident.
- tartieret 1y agoI was a little puzzled as we got notified our apps were down, and then I tried to login in the Azure portal with no success. But the Azure status page reported no incident, so I posted here and quickly confirmed that others were impacted! They did a pretty bad job with their status page as the front door service was shown green all along
- jammo 1y agoWe all need to move away from these big cloud providers. Two medium size smaller providers is enough. -Cloudflare for R2 (object storage) and CDN (Fastly+backblaze also available). -Two VPS/Server providers with a decent reputation and mid-size (using a comparison site like https://serversearcher.com https://serversearcher.com or look directly into people like Hetzner or latitude) -PlanetScale or Neon for database if you don't co-locate it, though better to use someone like digital ocean, vultr or latitude who offer databases too)
- dspillett 1y ago> We all need to move away from these big cloud providers. But then who do we blame when things are down? If we manage our own infrastructure we have to stay late to fix it when it breaks instead of saying “sorry, Microsoft, nothing we can do” and magically our clients accepting that…
- stbenjam 1y agoAh yes, let's put our multibillion dollar ecommerce site on... checks notes Hetzner. Lol
- nflekkhnnn 1y agoShut the front door!
- DeathArrow 1y agoBuy cloud because you're always safe! Until you aren't.
- mnau 1y agoIt doesn't matter whether you actually are safe or not. What matters is that you are in compliance.
- kure256 1y agoWe’ve been experimenting with multi-cluster failover for Kubernetes workloads, and one open-source project that actually works really well is k8gb . It acts as a GSLB controller inside Kubernetes — doing DNS-level health checks, region awareness, and automatic failover between clusters when one goes down. It integrates with ExternalDNS and supports multiple DNS providers (Infoblox, Route53, Azure DNS, NS1, etc.), so it can handle failover across both on-prem and cloud clusters. It’s not a silver bullet for every architecture, but it’s one of the few OSS projects that make multi-region failover actually manageable in practice.
- dedi089 1y ago[dead]
- dedi089 1y ago[dead]
- dedi089 1y ago[flagged]
- tartieret 1y agoit took a good half hour after we detected the problem to see a notification on the Azure status page. Thanks to those who responded to my question as it validated the issue was global and we contacted our users t right away
- chalo124 1y ago[dead]
- zaoui_amine 1y agoYeah, Azure is a mess today. Can't do anything without the portal.
- zaoui_amine 1y agoLanguage models aren't perfect; they can still generate similar outputs. Invertibility is a stretch.
- altcognito 1y agowrong thread
- FrostKiwi 1y agoSurprised to see the situation getting worse, what the hell. Had some Frontdoor operations timing out, but now I'm straight up denied with "Message: All Changes to Azure Frondoor Configuration are blocked currently." What a mess.
- shivenigma 1y agowhat's happening? self hosting advocate groups attacking all cloud to prove their point?