4 ms·
Just, no! This is good for a hobby or B2C, but not for serious stuff that must work 24/7. First, about my impression from the title: it's not only the code put
by 1996 6y ago
Just, no! This is good for a hobby or B2C, but not for serious stuff that must work 24/7.
First, about my impression from the title: it's not only the code put into production that matters: it's the experience and history of all the code that was decommissioned from production because of issues, or the code that almost made it to production but didn't because of some critical issue found at the last minute.
Maybe I'm the exception, but I often leave that code, commented out, in the sources - and I keep adding more and more!
I know it's not proper in this day in age (git, documentation etc. should be the place it goes) but that's the only place where I'm 100% positive a pair of eyeball WILL see that FOR SURE.
It's a way to avoid institutional knowledge loss when working in teams, but also a way to avoid forgetting what you did when you work on projects spanning multiple years.
Now for the points raised, (2) is bad: what you buy you don't understand when or how it may fail. Bitten once, twice shy...
So yes, I also go for manual QC (5), and staging environment (8), actually with production split in 4 : both running different version of the code (current, and previous), each with their own backup, because for what I do, (9) is unacceptable: if there's a break in 24/7 operation, the business closes.
Consequently, for (3), deployments are voluntarily made NOT EASY and are manual. It doesn't add much extra friction, because code reaching a production server will at least have been reviewed by eyeballs forced to read what failed before (in comments), who'll then have had to (5) quality control themselves to avoid feeling too sure about themselves, after which the code will have to prove its worth in (8) a staging environment for a few weeks.
Then, if something bad happens, back to the design board and eyeballing. If not, the code is just "good enough" to be deployed on half the fleet, replacing the oldest "known good" version first on the backup servers, then on the main servers.
And... that's how it stays until the next main version.
If some unforeseen problem is discovered, the previous "known good" version is still available on half the fleet. If the server has a hardware problem, the backup server with the N-1 version of the code is the final thing that remains between business and usual and "the 24/7 contract is broken, clients leave, the business closes".
I sell data feed with SLA guaranteeing 24/7 operation, and latency parameters. I've been going on for 3 years with only 1 failure that came close to interrupting production... but didn't. Each lesson was dearly learned.