3 ms·
Then hopefully everything is HA in your company.
by chefkoch 5y ago
Then hopefully everything is HA in your company.
- fullstop 5y agoIt absolutely is, BGP with geographically diverse paths, databases, app servers, etc, are all redundant. It's hosted in-house, so there is a cold standby database in AWS which would only be used if, say, an aircraft crashed into our server rooms.
- jnpatel 5y agoCould you share what domain your company is in? Just curious what has such HA requirements
- ikiris 5y agoIt always amazes me when someone says they can't ever be down, and then says they only serve from one physical location actively. One of these things is not like the other.
- fullstop 5y agoWe have everything in place to run from AWS if needed but do not operate from there because of cost. This goes both ways -- there have been many AWS outages which have not affected us. I hear what you're saying, but we've had only one instance of extended (hours) downtime in the last 20 years.
- xcambar 5y agoThe more experienced I become, the more such down to earth solutions seem OMG SO MUCH MORE reasonable than all the bells and whistles of "modern" engineering practices.
- ohyeshedid 5y agoOut of curiosity, what do you use in AWS to keep a running live backup of the data you'd need to migrate in the worst case scenario?
- fullstop 5y agoFor postgres we send WAL files to a server in AWS which processes them. To bootstrap the database initially we sent zfs snapshots, and those WAL files are applied on an ongoing basis. If our data center were to die a horrendous fiery death, we could lose, at most, about 3 minutes of data although monitoring shows that it's closer to 30s under normal operating conditions. For the app servers, saltstack is used and we synchronize that repository with what is needed to reproduce a production environment in AWS. Obviously we'd have to provision servers, etc, but it's all possible in a worst-case scenario.