3 ms·

That's not a startup approach. That's an enterprise approach. Believe me, I've fought tons of resistance in automating deployment operations in the enterprise.

by beat 10y ago

That's not a startup approach. That's an enterprise approach. Believe me, I've fought tons of resistance in automating deployment operations in the enterprise. There's a perception that automation is dangerous, and you need human checkpoints. In practice, I've worked on projects much, much larger than Knight Capital, where the deployment process was driven by a huge spreadsheet, and orchestrated by non-technical overseers telling techs what commands to run based on the spreadsheet in front of them. It's incredibly vulnerable to human error like "Oops, forgot to deploy to one of the eight servers in the cluster". In the enterprise, this is called "mature" and is a sign of great sophistication.

bpchaps 10y ago
Yeah, the amount of resistance towards automation at some larger companies is a total mindfuck. My first ops job back in 2008 was at a large exchange's NOC where we shut down and clean the application environment every day. Every Friday, we would have to take a backup of the ~20 or so production databases - by hand, in an ancient CDE based UI with a . Right click -> menu -> submenu -> backup database. Very little room for error, and you weren't allowed to do it without somebody else watching you. Throughout the weekend, customers would then run tests against the production databases. Once testing was done, we'd restore the prod databases back to their original state to wipe out test data. At one point, I asked my boss if it was alright if I automated it after showing him a POC and was rejected because, "We don't trust automation to do it accurately every single time." Mind boggling. In mild fairness, in the 15 or so years they were doing that, I don't think anyone did it wrong.. which is an enormous miracle in itself. (That was a strange company. My boss was a JW who'd worked there for 30 years regularly tried to convert me and would spend four hours a day on spreadsheets for his church. We'd also manually kick off stock split processing from a ~10" CRT monitor from the early nineties.)

by beat 10y ago

bpchaps 10y ago

Yeah, the amount of resistance towards automation at some larger companies is a total mindfuck. My first ops job back in 2008 was at a large exchange's NOC where we shut down and clean the application environment every day. Every Friday, we would have to take a backup of the ~20 or so production databases - by hand, in an ancient CDE based UI with a . Right click -> menu -> submenu -> backup database. Very little room for error, and you weren't allowed to do it without somebody else watching you. Throughout the weekend, customers would then run tests against the production databases. Once testing was done, we'd restore the prod databases back to their original state to wipe out test data. At one point, I asked my boss if it was alright if I automated it after showing him a POC and was rejected because, "We don't trust automation to do it accurately every single time." Mind boggling. In mild fairness, in the 15 or so years they were doing that, I don't think anyone did it wrong.. which is an enormous miracle in itself. (That was a strange company. My boss was a JW who'd worked there for 30 years regularly tried to convert me and would spend four hours a day on spreadsheets for his church. We'd also manually kick off stock split processing from a ~10" CRT monitor from the early nineties.)