4 ms·

YAML templating tool that works on YAML structure instead of text

10 points by alexellisuk 7y ago

Spivak 7y ago
How did we get to the point of templating YAML files when it’s a superset of JSON? Why not just generate a JSON object? At what point do we get so deep into templates that we just admit that anything short of a programming language just isn’t expressive enough for a config file? I mean that probably wouldn’t solve it either — people would just template the script.
- _frkl 7y ago
  I don't think that's necessarily a bad thing. In a lot of situations a plain yaml file is good enough for templating, and people seem to prefer writing those over json, for several reasons. Then, there are a lot of situations where templating yaml files is good enough. So, I do think having tools to do that sorta thing is good. With more complex system like for example Kubernetes those solutions start to not be good enough any ore, which is why things like jsonette are being adopted. And sooner later 'real' languages. Which, evemtually, will help develop abstractions that have a simple enough interface so we are able to use plain key value pairs for config again. And so on. I guess the point I'm trying to make is that it's not black and white, and a lot of the solutions that appear in this space have a good reason to exist. Well, obviously, since someone took the time to solve a problem they thought was not solved well enough with existing tools.
- jacques_chester 7y ago
  > At what point do we get so deep into templates that we just admit that anything short of a programming language just isn’t expressive enough for a config file? At the point where I'm debugging configuration instead of declaring it. There is no sweeping answer, because the scope of configuration varies according to the tool. Systems like Kubernetes or Concourse accept records rather than configuration per se. At least conceptually, these are projections of an underlying model. They are not the model itself and won't be the most compact representation. Because they are projections, they will inevitably fail to fulfill some degree of schematic normalisation. This means repetition, which for most engineers is interpreted as "I need a loop" instead of "I need a set notation". I prefer ytt to most alternatives I've seen because it's at least trying to avoid introducing unwanted Turing completeness. Where possible, your configuration language shouldn't be an attack vector. I'd prefer a more relational or logical style of configuration (I expect CUE folks will be here shortly), but given the continuing efforts to disguise and hide even SQL from programmers, I'm not going to hold my breath.
- krapp 7y ago
  >At what point do we get so deep into templates that we just admit that anything short of a programming language just isn’t expressive enough for a config file? I want to say that if you need config files to be that expressive, you're not doing configuration, you're still just building the application and probably have some architectural or separation of concerns issues that need to be addressed. I would assume most YAML config files could be made simple enough that they don't need to be Turing complete, and that they could be written more directly rather than through layers of abstraction.
jacques_chester 7y ago
I think a more useful link is https://get-ytt.io/ https://get-ytt.io/, which includes an interactive playground.
birchb 7y ago
These days devops workers often have no choice but to write scads of YAML for tools like GoCD, Concourse, K8s etc etc. To be efficient we turn to tools like Helm. But I recommend everyone write their own TC YAML tool - it does not take long and it's fun. If you don't have time, you can use mine. I wrote two 'macro processor's for YAML. Yamp (in Python, https://github.com/birchb1024/yamp https://github.com/birchb1024/yamp) and GoYamp (GoLang, https://github.com/birchb1024/goyamp https://github.com/birchb1024/goyamp). GoYamp is fast. Some of my users have generated thousands of YAML files with them. IMO ytt is not as good because it seems to embed its syntax in comments. You can use tools like m4 and Jinja2 but you'll be counting indentations all day. Yamp and GoYamp are 100% YAML so indentation just works. Start here: https://github.com/birchb1024/goyamp https://github.com/birchb1024/goyamp !
- brigandish 7y ago
  TC?
  - birchb 7y ago
    Turing Complete

10 points by alexellisuk 7y ago

How did we get to the point of templating YAML files when it’s a superset of JSON? Why not just generate a JSON object? At what point do we get so deep into templates that we just admit that anything short of a programming language just isn’t expressive enough for a config file? I mean that probably wouldn’t solve it either — people would just template the script.

_frkl 7y ago
I don't think that's necessarily a bad thing. In a lot of situations a plain yaml file is good enough for templating, and people seem to prefer writing those over json, for several reasons. Then, there are a lot of situations where templating yaml files is good enough. So, I do think having tools to do that sorta thing is good. With more complex system like for example Kubernetes those solutions start to not be good enough any ore, which is why things like jsonette are being adopted. And sooner later 'real' languages. Which, evemtually, will help develop abstractions that have a simple enough interface so we are able to use plain key value pairs for config again. And so on. I guess the point I'm trying to make is that it's not black and white, and a lot of the solutions that appear in this space have a good reason to exist. Well, obviously, since someone took the time to solve a problem they thought was not solved well enough with existing tools.
jacques_chester 7y ago
> At what point do we get so deep into templates that we just admit that anything short of a programming language just isn’t expressive enough for a config file? At the point where I'm debugging configuration instead of declaring it. There is no sweeping answer, because the scope of configuration varies according to the tool. Systems like Kubernetes or Concourse accept records rather than configuration per se. At least conceptually, these are projections of an underlying model. They are not the model itself and won't be the most compact representation. Because they are projections, they will inevitably fail to fulfill some degree of schematic normalisation. This means repetition, which for most engineers is interpreted as "I need a loop" instead of "I need a set notation". I prefer ytt to most alternatives I've seen because it's at least trying to avoid introducing unwanted Turing completeness. Where possible, your configuration language shouldn't be an attack vector. I'd prefer a more relational or logical style of configuration (I expect CUE folks will be here shortly), but given the continuing efforts to disguise and hide even SQL from programmers, I'm not going to hold my breath.
krapp 7y ago
>At what point do we get so deep into templates that we just admit that anything short of a programming language just isn’t expressive enough for a config file? I want to say that if you need config files to be that expressive, you're not doing configuration, you're still just building the application and probably have some architectural or separation of concerns issues that need to be addressed. I would assume most YAML config files could be made simple enough that they don't need to be Turing complete, and that they could be written more directly rather than through layers of abstraction.

jacques_chester 7y ago

I think a more useful link is https://get-ytt.io/ https://get-ytt.io/, which includes an interactive playground.

birchb 7y ago

These days devops workers often have no choice but to write scads of YAML for tools like GoCD, Concourse, K8s etc etc. To be efficient we turn to tools like Helm. But I recommend everyone write their own TC YAML tool - it does not take long and it's fun. If you don't have time, you can use mine. I wrote two 'macro processor's for YAML. Yamp (in Python, https://github.com/birchb1024/yamp https://github.com/birchb1024/yamp) and GoYamp (GoLang, https://github.com/birchb1024/goyamp https://github.com/birchb1024/goyamp). GoYamp is fast. Some of my users have generated thousands of YAML files with them. IMO ytt is not as good because it seems to embed its syntax in comments. You can use tools like m4 and Jinja2 but you'll be counting indentations all day. Yamp and GoYamp are 100% YAML so indentation just works. Start here: https://github.com/birchb1024/goyamp https://github.com/birchb1024/goyamp !

brigandish 7y ago
TC?
- birchb 7y ago
  Turing Complete