6 ms·

> only potential downside is if someone decides to "engineer" this method It can be engineered, if you follow a gradual process. On servers, I keep a log of w

by csdvrx 5y ago

> only potential downside is if someone decides to "engineer" this method It can be engineered, if you follow a gradual process. On servers, I keep a log of what was deployed in a root directory following the sequential number _ goal format (ex: 00_partitions ... 90_web_server) It is not fancy, most of the logs are not even scripts: many are just ASCII text files, that will only be used as a checklist if the same "goal" has to be achieved again. For example, 00_partition may be "gdisk /dev/nvme0n1" followed by a copy-pasted list of the partitions and some quick description about why it was done that way. But that's on the first iteration only: the next iteration turns that into "do-nothing" script, the next iteration into a better script with basic checks (supporting both /dev/nvme0n1 and /dev/sda), then exception handling (if partitions already exist, etc), and so on: this gradual complexification process avoids the "premature optimization" of creating infrastructure-as-code for what you rarely need, while optimizing and fine tuning the parts you most often need. Someone will certainly mention Terraform, or Ansible, or something else - yes, they exist and they are nice, but if you are doing everything there, you are over-engineering and wasting time: not everything needs your equal attention! If you only install a webserver once in a blue moon, make a .txt checklist of the steps you followed. But if you leave and breathe nginx options and certificate deployment, fully automatize all that, including the obscure details of what may fail if you use let's encrypt with some specific DNS configuration! And if you don't know yet which is which, start small (a .txt checklist will cost you a few minutes) and the next time you find yourself doing the same thing, do it better using the previous artifact (the .txt file) to create a better one (a script, then a better script etc)

rambambram 5y ago
Oh, what do I love .txt files. I use them for all kinds of simple checklists, logs, and basically everything. I had a manager once (not in the software field though) and she asked me annoyed why I kept using these strange files, and if I wanted to "just use Word and .doc files because that was more safe and compatible". I wasn't able to explain that text files were there and will be there for a long time. She also didn't understand the difference between a text file and a document. Not even when I pointed her to the .txt in the filename.
throwaway20371 5y ago
> If you only install a webserver once in a blue moon, make a .txt checklist of the steps you followed. This brings up a very important point about checklists that I don't think gets enough attention. The problem happens when somebody "updates" that web server in-place. If they try to record what changes they made in the middle of the checklist, eventually when someone tries the whole checklist from the beginning, they'll find it's now broken; the steps aren't working as expected. This happens to me when I try to record changes in my VirtualBox configuration after I add a new system package or something; later I try to re-deploy my vbox, and it breaks. So checklists should be considered immutable. Once you create them, don't assume they will work again if modified. Instead, if you make any change to the checklist, you must follow all the steps from beginning to end. This way you catch the unexpected problems and confirm the checklist still works for the next person.
- csdvrx 5y ago
  > The problem happens when somebody "updates" that web server in-place. Imagine this is 28-nginx : I would create another script 29-nginx-update only recording the update, even if it: "echo apt-get update; apt-get upgrade nginx ; echo "make sure to fix variable $foo" Next time I have to do that, I will integrate that into 28-nginx and remove 29-nginx-update > eventually when someone tries the whole checklist from the beginning, they'll find it's now broken; the steps aren't working as expected. Maybe I don't understand the issue, but my scripts or text files are simple and meant to be used in sequence. If I hack the scripts, I make sure it still works as expected - and given my natural laziness, I only ever update scripts when deploying to a new server or VM, so I get an immediate feedback if they stop working Still, sometimes something may work as expected (ex: above, maybe $foo depends on a context?), but it only means I need to generalize the previous solution - and since the script update only happen in the context of a new deployment, everything is still fresh in my head, so I can do it easily To help me with that, I also use zfs snapshots at important steps, to be able to "observe" what the files looked like on the other server at a specific time. The snapshots conveniently share the same name (ex etc@28-nginx) so comparing the files to create one ot more scripts can be easily done with diff -Nur using .zfs/snapshot/ cf https://docs.oracle.com/cd/E19253-01/819-5461/gbiqe/index.html https://docs.oracle.com/cd/E19253-01/819-5461/gbiqe/index.ht... Between that + a sqlite database containing the full history of commands types (including in which directory, and their return code), I rarely have such issues Shameless plug for that bash history in sqlite: https://github.com/csdvrx/bash-timestamping-sqlite https://github.com/csdvrx/bash-timestamping-sqlite > So checklists should be considered immutable. Once you create them, don't assume they will work again if modified. Instead, if you make any change to the checklist, you must follow all the steps from beginning to end. I agree: if I don't have time to fix 28-nginx, I write 29-nginx-update instead, with the goal next time to integrate it. But I don't try to tweak 28-nginx if I know I won't have the time to test it.
chousuke 5y ago
I would argue that all deployments (no matter how small) should have configuration management. In the simplest case, a deployment consists of: 1. Install packages 2. Install configuration files, possibly from templates. 3. Configure services to start on boot. This kind of automation is trivial to do with almost any tool, but there's no reason not to use something like Ansible that's designed for infrastructure automation, because you get encrypted secrets, templating and idempotency with zero effort and the result can be stored in a git repository somewhere. The Ansible playbook is as fast to write as installing and configuring things manually, and after some practice, it's faster and the resulting system is of higher quality. Even if you end up using the automation you write only once, it still has value because it doubles as a formalized description of what you did, and can be stored together with additional documentation. Over time you will also accumulate a library of bits and pieces that you can copy over to new setups, further improving your speed and quality.
- Arch-TK 5y ago
  This is assuming you already know Ansible.
  - chousuke 5y ago
    Sure, but then again, typing shell commands into text files is assuming you already know shell commands. You have to spend time to learn your tools at some point. For simple configuration management, Ansible is a straight upgrade to most shells because of idempotency alone, never mind the fancier features like the more advanced modules, multi-node orchestration, or encrypted vaults. The YAML syntax is dumb and it has its issues, for sure, but it still does even the simple things much better than plain old shell. Anyone who has any familiarity at all with UNIXy systems can learn Ansible from zero well enough in a day or two for it to start becoming truly useful, and if you don't have the foundation for that... why on Earth are you setting up a web server? I mean, it's of course fine to tinker with things for learning, but I was assuming a real deployment scenario.
    - Arch-TK 5y ago

by csdvrx 5y ago

rambambram 5y ago

Oh, what do I love .txt files. I use them for all kinds of simple checklists, logs, and basically everything. I had a manager once (not in the software field though) and she asked me annoyed why I kept using these strange files, and if I wanted to "just use Word and .doc files because that was more safe and compatible". I wasn't able to explain that text files were there and will be there for a long time. She also didn't understand the difference between a text file and a document. Not even when I pointed her to the .txt in the filename.

throwaway20371 5y ago

> If you only install a webserver once in a blue moon, make a .txt checklist of the steps you followed. This brings up a very important point about checklists that I don't think gets enough attention. The problem happens when somebody "updates" that web server in-place. If they try to record what changes they made in the middle of the checklist, eventually when someone tries the whole checklist from the beginning, they'll find it's now broken; the steps aren't working as expected. This happens to me when I try to record changes in my VirtualBox configuration after I add a new system package or something; later I try to re-deploy my vbox, and it breaks. So checklists should be considered immutable. Once you create them, don't assume they will work again if modified. Instead, if you make any change to the checklist, you must follow all the steps from beginning to end. This way you catch the unexpected problems and confirm the checklist still works for the next person.

csdvrx 5y ago
> The problem happens when somebody "updates" that web server in-place. Imagine this is 28-nginx : I would create another script 29-nginx-update only recording the update, even if it: "echo apt-get update; apt-get upgrade nginx ; echo "make sure to fix variable $foo" Next time I have to do that, I will integrate that into 28-nginx and remove 29-nginx-update > eventually when someone tries the whole checklist from the beginning, they'll find it's now broken; the steps aren't working as expected. Maybe I don't understand the issue, but my scripts or text files are simple and meant to be used in sequence. If I hack the scripts, I make sure it still works as expected - and given my natural laziness, I only ever update scripts when deploying to a new server or VM, so I get an immediate feedback if they stop working Still, sometimes something may work as expected (ex: above, maybe $foo depends on a context?), but it only means I need to generalize the previous solution - and since the script update only happen in the context of a new deployment, everything is still fresh in my head, so I can do it easily To help me with that, I also use zfs snapshots at important steps, to be able to "observe" what the files looked like on the other server at a specific time. The snapshots conveniently share the same name (ex etc@28-nginx) so comparing the files to create one ot more scripts can be easily done with diff -Nur using .zfs/snapshot/ cf https://docs.oracle.com/cd/E19253-01/819-5461/gbiqe/index.html https://docs.oracle.com/cd/E19253-01/819-5461/gbiqe/index.ht... Between that + a sqlite database containing the full history of commands types (including in which directory, and their return code), I rarely have such issues Shameless plug for that bash history in sqlite: https://github.com/csdvrx/bash-timestamping-sqlite https://github.com/csdvrx/bash-timestamping-sqlite > So checklists should be considered immutable. Once you create them, don't assume they will work again if modified. Instead, if you make any change to the checklist, you must follow all the steps from beginning to end. I agree: if I don't have time to fix 28-nginx, I write 29-nginx-update instead, with the goal next time to integrate it. But I don't try to tweak 28-nginx if I know I won't have the time to test it.

chousuke 5y ago

I would argue that all deployments (no matter how small) should have configuration management. In the simplest case, a deployment consists of: 1. Install packages 2. Install configuration files, possibly from templates. 3. Configure services to start on boot. This kind of automation is trivial to do with almost any tool, but there's no reason not to use something like Ansible that's designed for infrastructure automation, because you get encrypted secrets, templating and idempotency with zero effort and the result can be stored in a git repository somewhere. The Ansible playbook is as fast to write as installing and configuring things manually, and after some practice, it's faster and the resulting system is of higher quality. Even if you end up using the automation you write only once, it still has value because it doubles as a formalized description of what you did, and can be stored together with additional documentation. Over time you will also accumulate a library of bits and pieces that you can copy over to new setups, further improving your speed and quality.

Arch-TK 5y ago
This is assuming you already know Ansible.
- chousuke 5y ago
  Sure, but then again, typing shell commands into text files is assuming you already know shell commands. You have to spend time to learn your tools at some point. For simple configuration management, Ansible is a straight upgrade to most shells because of idempotency alone, never mind the fancier features like the more advanced modules, multi-node orchestration, or encrypted vaults. The YAML syntax is dumb and it has its issues, for sure, but it still does even the simple things much better than plain old shell. Anyone who has any familiarity at all with UNIXy systems can learn Ansible from zero well enough in a day or two for it to start becoming truly useful, and if you don't have the foundation for that... why on Earth are you setting up a web server? I mean, it's of course fine to tinker with things for learning, but I was assuming a real deployment scenario.
  - Arch-TK 5y ago