4 ms·

It's nice that you've decided to throw off the golden veil of "declarative" that so many of the tools try to sell themselves as, while in practice the user stil

by nurple 3y ago

It's nice that you've decided to throw off the golden veil of "declarative" that so many of the tools try to sell themselves as, while in practice the user still ends up attempting to write imperative logic while harangued by the declarative representation. Couple things I personally take issue with: String-embedding of code and templates is just straight up a nonstarter for me. I've seen enough dev/ops people struggle with tools like nomad and helm to know any sufficiently complex setup will inevitably become huge, opaque, usability mess. Finally, I just don't believe a "mutate the bag of state" strategy is the future of config management or DevOps tech. It's shown itself time and time again to be the weakest part of systems like salt, ansible, and puppet. I can't count the number of hours of my life that have been wasted writing code that attempted to, poorly I might add, perpetuate the lie of determinism this tooling tries to sell on top of the mutable bag of state that is an OS.

candiddevmike 3y ago
> String-embedding of code and templates is just straight up a nonstarter for me Can you expand on this? > Finally, I just don't believe a "mutate the bag of state" strategy is the future of config management or DevOps tech. Unfortunately that is how operating systems work (today). Even Nix struggles with state (though that's typically a bug). Properly controlling/preventing side effects with configuration management would require a new OS paradigm IMO. > I can't count the number of hours of my life that have been wasted writing code that attempted to, poorly I might add, perpetuate the lie of determinism One nice thing of Etcha is it makes testing your stuff really simple. Testing is baked into the tool, and you don't really have to write separate tests, the test is if your check passes/fails correctly after a command is changed/removed. Checkout the docs on it here for more info, it really helps with validation: https://etcha.dev/docs/guides/testing-patterns https://etcha.dev/docs/guides/testing-patterns
- nurple 3y ago
  Re embedding, just to take one snippet from an example on your site: { always: true, change: 'echo %s > /work/helloworld' % n.getEnv('HOSTNAME'), id: 'write a file', onChange: [ 'read file', ], } This may be quite clear in its simplicity, and particularly to those familiar with etcha/jsonnet. However, what invariably happens is that you start seeing things like vars embedded in templates, embedded in the config mgmt implementation language, embedded in strings. The source of the values becomes incredibly difficult to reason about and to make changes against; does the value come from the target host's env, from the runner's env, from the packaging step's env, from a network request made by one of these stages (eg to a secret server), etc. Take a look at an example of what, IMO, is an absolutely horrid helm chart that Gitlab ships for installing their CI/CD runner: https://gitlab.com/gitlab-org/charts/gitlab-runner/-/blob/main/templates/configmap.yaml https://gitlab.com/gitlab-org/charts/gitlab-runner/-/blob/ma... Bash code, in yaml, in golang template. Besides even the most advanced IDEs failing to grok such a freak of technical nature, there's no way I would believe any dev that told me they understand what the state of their system will be given some input to this morass. In a recent position I was asked to try and make an existing nomad installation viable in a pretty standard corporate environment (not some special operational space e.g. cloudflare), and it was even worse; some configuration expansion was 5 layers deep, with 3 different templating engines, once consul templates were involved in generating an app's config, and the nomad config being env-generalized through generation by a higher-level helm-like tooling. Re state bag: I'm glad you mentioned nix, as I think it, and to a looser extent containers, really approach the issue in the only humanly-tenable fashion (again IMO): starting mutation from a known state. In a lot of cases that state is "nothing" as it's the simplest known state not only to position the beginning of some configuration flow at, but also the most straight-forward from which to deterministically derive a desired end state from. I definitely applaud having tests as a core component of your system, the problem is that you can not derive determinism from nondeterminism even with the best tests. Because you are operating over a nondeterministic bag of state, you can never guarantee that your tests provide a representation of a transfrom from any potential state to the desired end state, only for some particular input state (or set) which may or may not representative of what is found on the actual targets.
linsomniac 3y ago
Your comment really resonates with me, because I'm experimenting with a project to do "ansible, but with Python syntax" [1]. It has some declarative abilities, but also full imperative abilities, and does away with "coding in YAML", so I get full LSP abilities. I've considered the question: What is the difference between declaratively providing an Apache config file, and just copying that file over unconditionally? The big benefits I've come up with are: - Detecting the change so I can trigger an Apache restart only when the file has changed. - Ability to do "--check" and "--diff" to tell me what will change during a dry run. With the downside that the "tasks" become more complicated to build. I had previously built a version of my tool that was just a simpler, local ansible-like tool, with a YAML syntax and realized: I really hate coding Ansible playbooks in YAML. So I started imagining what it might look like if it was in code. I picked Python syntax because I really didn't want to write a parser for a DSL and wanted to see if I could get close enough without it. [1] https://github.com/linsomniac/uplaybook https://github.com/linsomniac/uplaybook
- krab 3y ago
  That looks great! I'm trying to do a similar thing but I'd like to be compatible with the existing Ansible tasks. Still trying to figure out the API, especially for cases like calling a playbook from playbook with some parameters.
  - linsomniac 3y ago
    I am looking at adding the ability to call Ansible tasks (community.apt, for example) from uPlaybook, but that would not involve calling YAML playbooks I'm not thinking... But maybe that would be possible. I did try some other YAML control structures in the original version of uPlaybook, what I'm calling uPlaybook1 [1] now, which I think has some more natural control structures than Ansible. If you are doing something similar, may be worth looking at what I came up with: - I ended up using "block:" as a way of grouping tasks, and you could add conditionals or loops on those: block: tasks: - copy: [args here] - command: [more args here] loop: - vars: basename: XXX - vars: basename YYY - Conditionals were also blocks and could be nested within loops: - if: condition: "x < 1" tasks: - copy: [args here] - elif: condition: "x >= 1 and x < 10" tasks: [tasks here] - else: tasks: [more tasks here] [1] https://github.com/linsomniac/uplaybook1 https://github.com/linsomniac/uplaybook1

by nurple 3y ago

candiddevmike 3y ago

> String-embedding of code and templates is just straight up a nonstarter for me Can you expand on this? > Finally, I just don't believe a "mutate the bag of state" strategy is the future of config management or DevOps tech. Unfortunately that is how operating systems work (today). Even Nix struggles with state (though that's typically a bug). Properly controlling/preventing side effects with configuration management would require a new OS paradigm IMO. > I can't count the number of hours of my life that have been wasted writing code that attempted to, poorly I might add, perpetuate the lie of determinism One nice thing of Etcha is it makes testing your stuff really simple. Testing is baked into the tool, and you don't really have to write separate tests, the test is if your check passes/fails correctly after a command is changed/removed. Checkout the docs on it here for more info, it really helps with validation: https://etcha.dev/docs/guides/testing-patterns https://etcha.dev/docs/guides/testing-patterns

nurple 3y ago
Re embedding, just to take one snippet from an example on your site: { always: true, change: 'echo %s > /work/helloworld' % n.getEnv('HOSTNAME'), id: 'write a file', onChange: [ 'read file', ], } This may be quite clear in its simplicity, and particularly to those familiar with etcha/jsonnet. However, what invariably happens is that you start seeing things like vars embedded in templates, embedded in the config mgmt implementation language, embedded in strings. The source of the values becomes incredibly difficult to reason about and to make changes against; does the value come from the target host's env, from the runner's env, from the packaging step's env, from a network request made by one of these stages (eg to a secret server), etc. Take a look at an example of what, IMO, is an absolutely horrid helm chart that Gitlab ships for installing their CI/CD runner: https://gitlab.com/gitlab-org/charts/gitlab-runner/-/blob/main/templates/configmap.yaml https://gitlab.com/gitlab-org/charts/gitlab-runner/-/blob/ma... Bash code, in yaml, in golang template. Besides even the most advanced IDEs failing to grok such a freak of technical nature, there's no way I would believe any dev that told me they understand what the state of their system will be given some input to this morass. In a recent position I was asked to try and make an existing nomad installation viable in a pretty standard corporate environment (not some special operational space e.g. cloudflare), and it was even worse; some configuration expansion was 5 layers deep, with 3 different templating engines, once consul templates were involved in generating an app's config, and the nomad config being env-generalized through generation by a higher-level helm-like tooling. Re state bag: I'm glad you mentioned nix, as I think it, and to a looser extent containers, really approach the issue in the only humanly-tenable fashion (again IMO): starting mutation from a known state. In a lot of cases that state is "nothing" as it's the simplest known state not only to position the beginning of some configuration flow at, but also the most straight-forward from which to deterministically derive a desired end state from. I definitely applaud having tests as a core component of your system, the problem is that you can not derive determinism from nondeterminism even with the best tests. Because you are operating over a nondeterministic bag of state, you can never guarantee that your tests provide a representation of a transfrom from any potential state to the desired end state, only for some particular input state (or set) which may or may not representative of what is found on the actual targets.

linsomniac 3y ago

Your comment really resonates with me, because I'm experimenting with a project to do "ansible, but with Python syntax" [1]. It has some declarative abilities, but also full imperative abilities, and does away with "coding in YAML", so I get full LSP abilities. I've considered the question: What is the difference between declaratively providing an Apache config file, and just copying that file over unconditionally? The big benefits I've come up with are: - Detecting the change so I can trigger an Apache restart only when the file has changed. - Ability to do "--check" and "--diff" to tell me what will change during a dry run. With the downside that the "tasks" become more complicated to build. I had previously built a version of my tool that was just a simpler, local ansible-like tool, with a YAML syntax and realized: I really hate coding Ansible playbooks in YAML. So I started imagining what it might look like if it was in code. I picked Python syntax because I really didn't want to write a parser for a DSL and wanted to see if I could get close enough without it. [1] https://github.com/linsomniac/uplaybook https://github.com/linsomniac/uplaybook

krab 3y ago
That looks great! I'm trying to do a similar thing but I'd like to be compatible with the existing Ansible tasks. Still trying to figure out the API, especially for cases like calling a playbook from playbook with some parameters.
- linsomniac 3y ago
  I am looking at adding the ability to call Ansible tasks (community.apt, for example) from uPlaybook, but that would not involve calling YAML playbooks I'm not thinking... But maybe that would be possible. I did try some other YAML control structures in the original version of uPlaybook, what I'm calling uPlaybook1 [1] now, which I think has some more natural control structures than Ansible. If you are doing something similar, may be worth looking at what I came up with: - I ended up using "block:" as a way of grouping tasks, and you could add conditionals or loops on those: block: tasks: - copy: [args here] - command: [more args here] loop: - vars: basename: XXX - vars: basename YYY - Conditionals were also blocks and could be nested within loops: - if: condition: "x < 1" tasks: - copy: [args here] - elif: condition: "x >= 1 and x < 10" tasks: [tasks here] - else: tasks: [more tasks here] [1] https://github.com/linsomniac/uplaybook1 https://github.com/linsomniac/uplaybook1