3 ms·
> the primary engineer discovered that the isolated servers supporting the Status Site were running in the same data center as the primary App Engine serving
by zmimon 17y ago
> the primary engineer discovered that the
isolated servers supporting the Status Site were running in the same
data center as the primary App Engine serving cluster.
Hilarious - they were running the status site for App Engine on the App Engine infrastructure so when it went down they couldn't post it's status as being down. I'm sure people will be quite philosophical about this outage and how difficult distributed systems are to manage, but this seems kind of an obvious mistake to make...
- moe 17y agoYes, that was an obvious mistake. A brown-paper-bag bug so to say. The difference between google and many other companies is: They are not afraid to admit it and I'm sure that will be fixed before the next outage. Most other hosting providers I've dealt with would not even tell me the exact cause for an outage. Or at most in blurry terms, preferably handing off the blame to someone else. You wouldn't believe how often these pesky core routers at my datacenters supposedly had a problem! I also don't know many other providers with a status page as detailed as google's in first place. So, yes, this was a funny screw-up. But on a very high niveau. This is a bit like mocking someone who drove his Ferrari into a wall - at least he had a Ferrari to drive into a wall... ;-)