3 ms·

What is this for? YAML in my experience is used for writing and maintaining config at some supposed sweet-spot between human-readable and machine-readable struc

What is this for? YAML in my experience is used for writing and maintaining config at some supposed sweet-spot between human-readable and machine-readable structure. In my experience also, YAML is already too feature-laden for it's own good in this regard, when what people usually want is just a more human-readable JSON. Using Yglu for config seems like a bad idea (the whole point of config is that it's not code!) - so what is it for?

throwaway894345 7y ago
For large, repetitive configurations (Kubernetes configs, CloudFormation templates, CI job configurations, etc) you often want ways to abstract and/or DRY your configurations. Kubernetes folks use mustache/jinja/Go templates via Helm, which is a real mess. CloudFormation builds its own (shitty) evaluation language on top of YAML. This presumably aims to improve on those sort of hacky solutions. I feel strongly that the "right" solution is either (1) generate a YAML config with Python/JS/etc or (2) use a terminating expression language (I think that may be what this is). Since (2) implies a much steeper learning curve for average developers, I opt for (1).
- fredley 7y ago
  I agree with you that the job that Yglu is trying to do would be much better done (and a hell of a lot more readable) as Python code.
- escardin 7y ago
  That's what I've been finding a lot lately. Kubenetes manifests have very complete schemas, but there's not really any official concise way to express them or relations between them. I've come to the conclusion that helm/templating is awful. Every helm chart I've ever seen suffers from the kitchen sink problem. Dependencies don't really address the problem either, because that's not really resolving a whole lot in the end. What I've found so far is that Kustomize strikes a solid balance between an off the shelf working config provided from official sources, and making changes myself to suit my environment. Instead of the developer maintaining a kitchen sink approach, they can have something (or a set of somethings) that are straight forward, and I can maintain a set of tweaks that work for me. Unfortunately Kustomize's documentation is not that great, and it is incredibly awkward to work with. I'm hoping that solid operators come out for most applications instead of helm charts. I'm not 100% sure that they're going to be substantially better than helm, but I'm hoping between operators and kustomize I can link applications together in a way that I get properly maintained applications for minimal effort (ala AWS), without having to maintain a large code base linking all the bits together. I do think I need to look into pulumi as it might be in the direction I want to go, but I remain skeptical. Terraform has some good ideas, but is too rigid for kubernetes, and I feel like I'm giving up the flexibility of kubectl. I like the idea of having a state and resolving differences between desired and actual state, but not enough to rewrite everything in HCL.
  - cppforlife 7y ago
    > I like the idea of having a state and resolving differences between desired and actual state, but not enough to rewrite everything in HCL. i authored https://get-kapp.io https://get-kapp.io based on my previous experiences managing iaas resources as cattle. it strongly revolves around idea of managing chnageset between actual and desired configuration. it was also built as a tool that solely focuses on deployment to k8s, leaving configuration building (ie templating) to other tools. take a look at it, would love to hear what you think. on a side note, someone recently asked me to also wrap kapp as a terraform provider: https://github.com/k14s/terraform-provider-k14s https://github.com/k14s/terraform-provider-k14s
- smitty1e 7y ago
  > This presumably aims to improve on those sort of hacky solutions. Alas, the ship of toothpaste has sailed out of barn squeezed all over the back of the horse. We must retreat to Tom's Obvious, Minimal Language[1] and just ventilate anyone trying to throw it into the metaphorical blender. [1] https://github.com/toml-lang/toml https://github.com/toml-lang/toml
bitbang 7y ago
Automated config management. As for YAML itself, it is handy to have some advanced functionality in a non-turing complete serialization format for the sake of things like security. It's not perfectly analogous, but PostScript is an example of a format that ideally should have been just a descriptive format for the purposes it was intended for, but being Turing complete makes it a security risk
- cppforlife 7y ago
  https://get-ytt.io https://get-ytt.io (i am one of the authors), one of the tools that inspired yglu, has been designed from ground up with an eye towards security even though its turing complete. it is based on starlark which is a runtime that does not have facilities to do networking, fs access, etc. building on that, ytt does not allow/provide any kind of non-deterministic operations (access to time, disk, network, random, etc). it expects user to specify explicitly which files to load via -f flag (ie ytt template cannot load random template from fs). there are several computation attacks that are currently possible (eg infinite loop) but that could be easily addressed thru global timeout for example. check out interactive playground at https://get-ytt.io https://get-ytt.io

throwaway894345 7y ago

For large, repetitive configurations (Kubernetes configs, CloudFormation templates, CI job configurations, etc) you often want ways to abstract and/or DRY your configurations. Kubernetes folks use mustache/jinja/Go templates via Helm, which is a real mess. CloudFormation builds its own (shitty) evaluation language on top of YAML. This presumably aims to improve on those sort of hacky solutions. I feel strongly that the "right" solution is either (1) generate a YAML config with Python/JS/etc or (2) use a terminating expression language (I think that may be what this is). Since (2) implies a much steeper learning curve for average developers, I opt for (1).

fredley 7y ago
I agree with you that the job that Yglu is trying to do would be much better done (and a hell of a lot more readable) as Python code.
escardin 7y ago
That's what I've been finding a lot lately. Kubenetes manifests have very complete schemas, but there's not really any official concise way to express them or relations between them. I've come to the conclusion that helm/templating is awful. Every helm chart I've ever seen suffers from the kitchen sink problem. Dependencies don't really address the problem either, because that's not really resolving a whole lot in the end. What I've found so far is that Kustomize strikes a solid balance between an off the shelf working config provided from official sources, and making changes myself to suit my environment. Instead of the developer maintaining a kitchen sink approach, they can have something (or a set of somethings) that are straight forward, and I can maintain a set of tweaks that work for me. Unfortunately Kustomize's documentation is not that great, and it is incredibly awkward to work with. I'm hoping that solid operators come out for most applications instead of helm charts. I'm not 100% sure that they're going to be substantially better than helm, but I'm hoping between operators and kustomize I can link applications together in a way that I get properly maintained applications for minimal effort (ala AWS), without having to maintain a large code base linking all the bits together. I do think I need to look into pulumi as it might be in the direction I want to go, but I remain skeptical. Terraform has some good ideas, but is too rigid for kubernetes, and I feel like I'm giving up the flexibility of kubectl. I like the idea of having a state and resolving differences between desired and actual state, but not enough to rewrite everything in HCL.
- cppforlife 7y ago
  > I like the idea of having a state and resolving differences between desired and actual state, but not enough to rewrite everything in HCL. i authored https://get-kapp.io https://get-kapp.io based on my previous experiences managing iaas resources as cattle. it strongly revolves around idea of managing chnageset between actual and desired configuration. it was also built as a tool that solely focuses on deployment to k8s, leaving configuration building (ie templating) to other tools. take a look at it, would love to hear what you think. on a side note, someone recently asked me to also wrap kapp as a terraform provider: https://github.com/k14s/terraform-provider-k14s https://github.com/k14s/terraform-provider-k14s
smitty1e 7y ago
> This presumably aims to improve on those sort of hacky solutions. Alas, the ship of toothpaste has sailed out of barn squeezed all over the back of the horse. We must retreat to Tom's Obvious, Minimal Language[1] and just ventilate anyone trying to throw it into the metaphorical blender. [1] https://github.com/toml-lang/toml https://github.com/toml-lang/toml

bitbang 7y ago

Automated config management. As for YAML itself, it is handy to have some advanced functionality in a non-turing complete serialization format for the sake of things like security. It's not perfectly analogous, but PostScript is an example of a format that ideally should have been just a descriptive format for the purposes it was intended for, but being Turing complete makes it a security risk

cppforlife 7y ago
https://get-ytt.io https://get-ytt.io (i am one of the authors), one of the tools that inspired yglu, has been designed from ground up with an eye towards security even though its turing complete. it is based on starlark which is a runtime that does not have facilities to do networking, fs access, etc. building on that, ytt does not allow/provide any kind of non-deterministic operations (access to time, disk, network, random, etc). it expects user to specify explicitly which files to load via -f flag (ie ytt template cannot load random template from fs). there are several computation attacks that are currently possible (eg infinite loop) but that could be easily addressed thru global timeout for example. check out interactive playground at https://get-ytt.io https://get-ytt.io