6 ms·
What about the option of just writing a regular, one-off program in a regular programming language, the output of which is your baked YAML config; and then havi
by derefr 6y ago
What about the option of just writing a regular, one-off program in a regular programming language, the output of which is your baked YAML config; and then having a pipeline that involves running that config-generator program, piping its output to your orchestrator of choice?
Nearly every programming language has a YAML serialization library†. And before that serialization happens, your config can be expressed using the regular-ass coding features of your program, however you like. (For optimal clarity, I personally would suggest creating a builder DSL and using it.)
† Technically a language doesn't even need a YAML serialization library to emit valid YAML; because valid JSON is also valid YAML. You can just serialize to JSON on your end, and feed the result into anything that's expecting YAML.
- throwaway894345 6y agoYeah, I do this for quite a few things. It works pretty well. In my mind that was implicit in the fourth point, but I didn’t spell it out.
- jhhh 6y agoAfter trying to write complex loop statements and conditionals in yaml for ansible I had this thought as well. It's nice when declarative configurations work, but once they don't and you have to try to write a real program in yaml you'll want to pull your hair out.
- dorfsmay 6y agoThis was my exact experience as well. I was an early user if sensible and loved that it was yaml. Then had to deal with jinja inside yaml. And finally a syntax for loops appeared! I have run into the same type of issues with salt.
- lmm 6y agoAt that point why use YAML at all? If it's generated by a program and fed to a program, you're better off using protobuf or something like that. In fact, since you're probably using the same language on both ends, why not just write a regular value in your language? This probably sounds like a strawman, but it's not. It's how a lot of e.g. Python projects are configured - the "config" file is just a normal bit of code that gets run to produce a value. Unless you're using a programming language that absolutely sucks at expressing plain values (e.g. C or Java), it's much better than separate config files, IMO.
- derefr 6y ago> At that point why use YAML at all? Ideological answer: For the same reason HTTP/2.0’s binary protocol didn’t instantly obviate/deprecate HTTP/1.0’s text protocol. Text has advantages: text is debuggable, and prototypable. If the interface between two programs is a text based declarative language, you can audit that text, diff that text, edit that text to see how changes affect the result, mock one side or the other by producing or consuming that text, etc. “GitOps” style config management would never work if config was all opaque binary blobs. These are all reasons that major software projects standardize on YAML or other widely-supported textual data serialization formats for their config. Pragmatic answer: because we’re talking about production configuration management, here, which is, 99% of the time, about writing configuring and managing the third-party black-box components in your stack, not your own components. Your own business layer usually can be configured conventionally, with minimal explicit config, for your use case, since you built it to work idiomatically for that use-case. It’s all the third-party stuff that has an impedance mismatch to your use-case assumptions, translating to needing tons of config to do what you need. And, obviously, if you don’t control the other end, you don’t decide how the other end does its config. Usually, these days, it’s YAML (or TOML) — for the ideological reasons mentioned above. Example: Kubernetes. Big consumer of complex YAML. Many people try to template that YAML. Much simpler and less error-prone to just write a program to generate said YAML. No reason to assume you’re writing in whatever language the k8s orchestrator is written in. (In fact, there are multiple orchestrators, written in different languages, and the shared YAML resource spec is the only formal interface they share.)
- lmm 6y ago> Ideological answer: For the same reason HTTP/2.0’s binary protocol didn’t instantly obviate/deprecate HTTP/1.0’s text protocol. Text has advantages: text is debuggable, and prototypable. If the interface between two programs is a text based declarative language, you can audit that text, diff that text, edit that text to see how changes affect the result, mock one side or the other by producing or consuming that text, etc. I can see the argument for using a textual format (although I think it's weaker than you say; if we're generating this config with code then we don't want to diff or edit the generated config), but YAML seems like a singularly poor choice if you want reliable diffs and editing; it's like picking tag-soup HTML. Straight JSON (ideally with a schema), TOML or even XML seems like a better bet if you're generating it programmatically. > And, obviously, if you don’t control the other end, you don’t decide how the other end does its config. Right, in that case it's all moot. I took GP to be talking about what formats these tools should use. IMO if the tool is intended to consume a machine-generated config then it would be better to use a machine-oriented config format. I think the option of something like protobuf (which is language-independent) is underappreciated, but even restricting ourselves to textual options, something stricter than YAML seems like a better bet.