3 ms·

This approach doesn't work well in the long term. I've worked as a successor in one of the author's previous company. All the "do-nothing scripts" diverges from

This approach doesn't work well in the long term. I've worked as a successor in one of the author's previous company. All the "do-nothing scripts" diverges from the reality as time goes by. The scripts became a false prophet, not a single source of truth. When I came in, the fully automated parts are doing fairly well. On the other hands, most the one-off scripts doesn't work. Falsy docs are worse than no documentation. We've tried looking at the scripts, but small divergents in the scripts destroyed our confidence in the scripts. Instead we figure how to do task by looking at the source code and actual deployments. The lesson we learned from this is we need to treat scripts as a real software instead. We make scripts idempotent and run them periodically to see if anything breaks. For infrastructures we make them immutable and embraced the cattle mindset(vs pet mindset).

NationOfJoe 5y ago
We make scripts idempotent does this mean change the commands that run to be idempotent? so the script/echo concept stays the same but you make sure the commands it echo's wont break anything if run more then once?
- dbmikus 5y ago
  Not the parent, but I'm guessing it's the idea that the script always moves the system to the same steady state. Similar to how Ansible tries to be and how Kubernetes controller loops work.
hollasch 5y ago
> All the "do-nothing scripts" diverges from the reality as time goes by. Not sure I follow. In my experience, we have _tons_ of process that's written up in random pages in our company wiki. I assure you, nothing diverges as quickly as a set (regrettably) of duplicated wiki pages outlining a process. If you're making the argument that a fully automated script is superior to a checklist, then ... well, sure. But while you're spending the resources to fully automate one process, duplicated wiki pages are being written for the 100 other checklists (at least in my experience). The key here is that I fully agree with you on the divergence problem. But if a scripted checklist diverges, people complain and it's trivial to update the script. If you don't have these, then people either add a comment to one of the wiki pages, or a side note, or worse, write up a "better" wiki page with the real actual this-time-for-sure steps. I believe the key here is to implement a consistent tactical approach to incremental development, capturing divergence along the way. If you just let them lie fallow, then yes, that's bad. But surely a fully-automated script allowed to lie fallow and diverge from reality is no better off.

NationOfJoe 5y ago

We make scripts idempotent does this mean change the commands that run to be idempotent? so the script/echo concept stays the same but you make sure the commands it echo's wont break anything if run more then once?

dbmikus 5y ago
Not the parent, but I'm guessing it's the idea that the script always moves the system to the same steady state. Similar to how Ansible tries to be and how Kubernetes controller loops work.

hollasch 5y ago

> All the "do-nothing scripts" diverges from the reality as time goes by. Not sure I follow. In my experience, we have _tons_ of process that's written up in random pages in our company wiki. I assure you, nothing diverges as quickly as a set (regrettably) of duplicated wiki pages outlining a process. If you're making the argument that a fully automated script is superior to a checklist, then ... well, sure. But while you're spending the resources to fully automate one process, duplicated wiki pages are being written for the 100 other checklists (at least in my experience). The key here is that I fully agree with you on the divergence problem. But if a scripted checklist diverges, people complain and it's trivial to update the script. If you don't have these, then people either add a comment to one of the wiki pages, or a side note, or worse, write up a "better" wiki page with the real actual this-time-for-sure steps. I believe the key here is to implement a consistent tactical approach to incremental development, capturing divergence along the way. If you just let them lie fallow, then yes, that's bad. But surely a fully-automated script allowed to lie fallow and diverge from reality is no better off.