3 ms·
Tell HN: Google banned Railway's account. Everything down
Everything down including railway.com - this is a bad outage. :(
- gnabgib 5mo agoSome discussion started an hour ago (12 points, 3 comments) https://news.ycombinator.com/item?id=48200827 https://news.ycombinator.com/item?id=48200827
- deleted 5mo ago[deleted]
- jonnyasmar 5mo ago[flagged]
- mcontrerazCL 5mo agoIdentified Google Cloud has blocked our account, making some Railway services unavailable. We have escalated this directly with Google. The Railway Platform team has since confirmed access to Google Cloud and is working on restoring access to all workloads. We have access to some of our Google Cloud–hosted infrastructure and are working to restore the rest of the service. We apologize for the disruption.
- ErystelaThevale 5mo agoRailway had a similar reliability issue two weeks ago when an AI agent deleted a customer's production database via their API — no confirmation step, no environment scoping. Now this. Both incidents suggest the same pattern: infrastructure decisions made without thinking through failure modes, fixed reactively after damage is done.
- choilive 5mo agoI wouldn't blame that incident on Railway.. you can delete your prod database on AWS just as easily with their API.
- sikozu 5mo agoThat incident wasn't Railways fault at all. Don't use AI in your staging and prod tools.
- dankwizard 5mo agoAgree. The author of that article took 0 responsbility and despite the warnings of "Hey, AI with power in prod is a bad idea" thought "This wouldn't happen to me!" and then when it does "HOW COULD IT DO THIS?!"
- jesterson 5mo agoIf everyone can delete your prod database via API or by any other means - you need to sack CTO without severance package.
- ashahin 5mo ago[flagged]
- lotsoweiners 5mo agoSo the solution is the same as it has been for over a decade. Don’t do business with Google.
- scorpioxy 5mo agoI believe the problem was always the process that Google has where they tend to want to automate everything from the start and make it very difficult to reach a human and explain the situation. The service itself seems solid but if you make it difficult to address any problems when(not if) they occur then I won't be comfortable doing business with you. I stay away from their services for anything serious for this reason and always recommend to others to do the same. It's going to be interesting to watch this unfold. Google's automation vs LLM agents, no humans from either side.
- jesterson 5mo ago> Don’t do business with Google. If you think other big tech or bank won't do it for you if you "don't do anything wrong" - you might be delusional. The only way to fight it is redundancy. Dont vendor lock in all you stuff in aws/google/cf/whatever. They can (and will) fuck you up at some point.
- happymellon 5mo agoIt is a weekly occurrence for large named organisations and Google. It isn't for other big tech/banks. It's just a risk assessment, and Google are a high risk partner.
- jesterson 5mo agoJust because you don't see it, it doesn't meant it doesn't happen. I deal with it daily. If i would start writing, I would spend half day just writing. It's in no way to say Google is any good - it's just equally bad as any reasonably big organisation. No more and no less.
- happymellon 5mo ago
- Leena-ch 5mo agoThe fix isn't don't use GCP. It's never letting one cloud account suspension take down your control plane and customer workloads simultaneously
- moezd 5mo agoExactly. Whatever happened to cloud just being "someone else's computer" and "cheap scaling on speed dial"?
- Shanov23 5mo agoWe'll rapidly deploy a reliable backup website with proper hosting to get Railway back online within 48 hours
- Shanov23 5mo agoReply at hello@fourfivestudio.in