3 ms·
Try that when your deployment requires a schema change of a database table with a billion rows :-) That being said, we've automated deploy and rollback on erro
by jwatte 10y ago
Try that when your deployment requires a schema change of a database table with a billion rows :-)
That being said, we've automated deploy and rollback on error to the point where it's a script, and we run it 50 times a day. The "immune system" from a few minutes of partial deploy saves us weekly.
- kelnos 10y agoDB schema updates don't change your rollback ability. Need to add a column? Go for it. Then do your deployment that uses the new column. If you have to roll back, you can, and the new column will just sit there, again unused. Have a column you don't need anymore? Do the deployment that stops using it, and let it sit for a while. After you're convinced it's safe, drop the column. Have a column that needs to change size/purpose/etc.? Nope, don't do that. Just create a new one, and deal with differences between the old and new column in code until you can safely drop the old column, as above. Does your ALTER statement take too long to run and cause downtime? Do it on a replica that isn't serving traffic, and then promote that to be the new master. Actually, in general, you should be doing this for any kind of change anyway, just in case you screw up the schema update.
- sqeaky 10y agoSpin up a second database VM and point a fresh frontend VM at it. Then atomically switch the frontend VM like I was describing before. If that doesn't work come up with some kind, any kind, of rollback solution. There is no server problem that cannot have a rollback solution made for it. Some places are cheap and would rather just upgrade in place in the middle of the night. Those places are cheap and bad at engineering. Do not work for or do business with them if you actually care about uptime.