4 ms·

I think Jinja-in-YAML is very much an antipattern. It seems to come from not designing for enough programmability from the start (and then from people copying s

I think Jinja-in-YAML is very much an antipattern. It seems to come from not designing for enough programmability from the start (and then from people copying successful projects that took that route?). The page mentions some alternatives like Dhall and Jsonnet. Two more to consider: 1. Write a configuration library for a real programming language. Have this library generate a JSON configuration file that is treated purely as an inspectable artifact, not something to edit by hand. The user will have their configuration as tooling-enabled code checked into version control (hopefully). That it will be harder to make an emergency fix directly on the server has its downsides and upsides. (What was the first prominent piece of software to implement this idea? I kind of got it by osmosis.) 2. Starlark. It is a non-Turing-complete language derived from Python that was originally developed for the Bazel build system. It has several implementations (I am not sure how deeply compatible they are) and now Python bindings (https://github.com/caketop/python-starlark-go https://github.com/caketop/python-starlark-go for starlark-go, https://github.com/inducer/starlark-pyo3 https://github.com/inducer/starlark-pyo3 for starlark-rust).

baq 3y ago
> 1. Write a configuration library for a real programming language. Have this library generate a JSON configuration file that is treated purely as an inspectable artifact, not something to edit by hand. The user will have their configuration as tooling-enabled code checked into version control (hopefully). That it will be harder to make an emergency fix directly on the server has its downsides and upsides. this is the route Amazon chose with CDK. it kinda works... but it really feels like you're building a rube goldberg machine to do anything non-trivial. how much of it is the CDK and how much is cloudformation plainly sucking I don't know.
mdaniel 3y ago
I also think Jinja in YAML is an antipattern, but I think that due to them using the default block and expression characters of "{%" and "{{" both of which are yaml characters and thus every single such occurrence needs to be quoted, versus the very sane "${{" used by GHA or even "<%" and "<<" (running the risk that, yes, "<<:" is a yamlism but is not legal jinja) If you meant that having any executable inside yaml (e.g. not just picking on jinja, but "a yaml document is not code") then I fear that ship has sailed because people realized that inverting the literal parts and only sometimes having executable parts makes for a great way to ASP/JSP/PHP up some content If I wanted start a sibling flame war within this thread, we'd talk about HCL and for_each :rolling_eyes: (but, for clarity, please don't talk about it, or at least not here, submit your own noHCL.com link)
kimbernator 3y ago
> I think Jinja-in-YAML is very much an antipattern. The thought of working with Ansible again is my nightmare due to this. Apparently they had an actual Python DSL you could use instead of YAML for a while, but discontinued it? So now loops and if-thens are these awful drawn-out messes of YAML that interpret jinja in completely unpredictable ways.
- networked 3y ago
  Do you mean the Ansible Runner interface? It is apparently still available. It was featured more prominently in early documentation (http://web.archive.org/web/20120314070643/http://ansible.github.com/api.html http://web.archive.org/web/20120314070643/http://ansible.git...). The discontinued OpsMop by the original author of Ansible (https://github.com/opsmop/opsmop https://github.com/opsmop/opsmop) used a Python DSL. I would also like to avoid writing Ansible playbooks when I need to manage servers again. Something I want to try out is Pyinfra (https://pyinfra.com/ https://pyinfra.com/). It is like Ansible in that it is agentless and works over SSH, but with configuration in Python instead of YAML and Jinja. There is also itamae (Ruby, works over SSH or on the local machine, https://github.com/itamae-kitchen/itamae https://github.com/itamae-kitchen/itamae) and mitamae (mruby, local-only, https://github.com/itamae-kitchen/mitamae https://github.com/itamae-kitchen/mitamae).
remram 3y ago
Is there a standalone Starlark tool? Similar to `jsonnet`, something that I can use to run Starlark code to generate JSON or YAML?
- networked 3y ago
  I once tried to find a simple standalone interpreter to play with and couldn't. There is a list of projects that use Starlark at https://github.com/bazelbuild/starlark#users https://github.com/bazelbuild/starlark#users. It lists https://github.com/carvel-dev/ytt https://github.com/carvel-dev/ytt for YAML templating and https://github.com/jathu/starfig https://github.com/jathu/starfig for generating JSON and YAML configs. (I have not used either.)
pacha-- 3y ago
I agree. I find working with templated YAML so cumbersome that I ended up creating a tool (Cels - https://github.com/pacha/cels https://github.com/pacha/cels) because of it. I like Jsonnet and Starlark but in practice I don’t usually need a new programming language for most use cases. Most of the time I just want to create a base document and apply patches to do modifications. That simplifies everything a lot. The experience of writing pure YAML is actually not that bad (a couple of things about the format are really questionable, but it is workable). I find that problems appear from the complexity of the solutions you have to add to adapt documents to different environments.

baq 3y ago

> 1. Write a configuration library for a real programming language. Have this library generate a JSON configuration file that is treated purely as an inspectable artifact, not something to edit by hand. The user will have their configuration as tooling-enabled code checked into version control (hopefully). That it will be harder to make an emergency fix directly on the server has its downsides and upsides. this is the route Amazon chose with CDK. it kinda works... but it really feels like you're building a rube goldberg machine to do anything non-trivial. how much of it is the CDK and how much is cloudformation plainly sucking I don't know.

mdaniel 3y ago

I also think Jinja in YAML is an antipattern, but I think that due to them using the default block and expression characters of "{%" and "{{" both of which are yaml characters and thus every single such occurrence needs to be quoted, versus the very sane "${{" used by GHA or even "<%" and "<<" (running the risk that, yes, "<<:" is a yamlism but is not legal jinja) If you meant that having any executable inside yaml (e.g. not just picking on jinja, but "a yaml document is not code") then I fear that ship has sailed because people realized that inverting the literal parts and only sometimes having executable parts makes for a great way to ASP/JSP/PHP up some content If I wanted start a sibling flame war within this thread, we'd talk about HCL and for_each :rolling_eyes: (but, for clarity, please don't talk about it, or at least not here, submit your own noHCL.com link)

kimbernator 3y ago

> I think Jinja-in-YAML is very much an antipattern. The thought of working with Ansible again is my nightmare due to this. Apparently they had an actual Python DSL you could use instead of YAML for a while, but discontinued it? So now loops and if-thens are these awful drawn-out messes of YAML that interpret jinja in completely unpredictable ways.

networked 3y ago
Do you mean the Ansible Runner interface? It is apparently still available. It was featured more prominently in early documentation (http://web.archive.org/web/20120314070643/http://ansible.github.com/api.html http://web.archive.org/web/20120314070643/http://ansible.git...). The discontinued OpsMop by the original author of Ansible (https://github.com/opsmop/opsmop https://github.com/opsmop/opsmop) used a Python DSL. I would also like to avoid writing Ansible playbooks when I need to manage servers again. Something I want to try out is Pyinfra (https://pyinfra.com/ https://pyinfra.com/). It is like Ansible in that it is agentless and works over SSH, but with configuration in Python instead of YAML and Jinja. There is also itamae (Ruby, works over SSH or on the local machine, https://github.com/itamae-kitchen/itamae https://github.com/itamae-kitchen/itamae) and mitamae (mruby, local-only, https://github.com/itamae-kitchen/mitamae https://github.com/itamae-kitchen/mitamae).

remram 3y ago

Is there a standalone Starlark tool? Similar to `jsonnet`, something that I can use to run Starlark code to generate JSON or YAML?

networked 3y ago
I once tried to find a simple standalone interpreter to play with and couldn't. There is a list of projects that use Starlark at https://github.com/bazelbuild/starlark#users https://github.com/bazelbuild/starlark#users. It lists https://github.com/carvel-dev/ytt https://github.com/carvel-dev/ytt for YAML templating and https://github.com/jathu/starfig https://github.com/jathu/starfig for generating JSON and YAML configs. (I have not used either.)

pacha-- 3y ago

I agree. I find working with templated YAML so cumbersome that I ended up creating a tool (Cels - https://github.com/pacha/cels https://github.com/pacha/cels) because of it. I like Jsonnet and Starlark but in practice I don’t usually need a new programming language for most use cases. Most of the time I just want to create a base document and apply patches to do modifications. That simplifies everything a lot. The experience of writing pure YAML is actually not that bad (a couple of things about the format are really questionable, but it is workable). I find that problems appear from the complexity of the solutions you have to add to adapt documents to different environments.