12 ms·
How I learned to stop worrying and love the YAML
- jamesrom 4y agoCUE is the most interesting language in this space right now https://cuelang.org/ https://cuelang.org/
- vosper 4y agoIt’s a lot more complicated than YAML though. What’re the advantages? Who’s actually using it?
- mirekrusin 4y agoTerseness - you can describe exactly what you want without repetition at arbitrary cross-sections Intuitive behavior - unification by design is unordered (ie. it doesn't matter in which order input is processed, it'll produce the same result), you get what you see. Verification - ie. you can do one-liner policies easily to have some structure with desired types/values etc. It's just well designed (based on good theory), practical language for configuration. We use it in critical project to describe services on different envs, it works very well.
- mdaniel 4y agoThe new docker-based CICD system uses cue: https://docs.dagger.io/1235/what#how-does-it-work https://docs.dagger.io/1235/what#how-does-it-work
- moderation 4y agoThere are few things more complicated that YAML - https://www.arp242.net/yaml-config.html#its-pretty-complex https://www.arp242.net/yaml-config.html#its-pretty-complex > YAML may seem ‘simple’ and ‘obvious’ when glancing at a basic example, but turns out it’s not. The YAML spec is 23,449 words; for comparison, TOML is 3,339 words, JSON is 1,969 words, and XML is 20,603 words. Grafana Labs, Datadog's Vector, the Istio Service Mesh project, Dagger, sigstore, the SLSA framework all use CUE in some fashion. Dagger wrote a great blog post on CUE [0] 0. https://docs.dagger.io/1215/what-is-cue/ https://docs.dagger.io/1215/what-is-cue/
- mirekrusin 4y agoExactly, it's kind of strange reading this type of blog posts months after months, this is solved problem and it's called cue.
- masklinn 4y agoYes nothing says “baseline infrastructure” more than “you have to invoke a go binary to work with the thing”.
- richardwhiuk 4y agoIt's depressing how that's almost not sarcastic.
- zamalek 4y agoI was on the Dhall bandwagon up until Dagger made their announcement recently, Cue is definitely a nice configuration language. My main critique of it is that there's only one implementation.
- whatshisface 4y agoTOML is a better version of the same idea.
- voidfunc 4y agoTOML is not much better either. Large TOML documents suck to work with.
- vaylian 4y agoWhich format do you prefer for large documents/configurations?
- inferiorhuman 4y agoFor me JSON comes close to ideal, if comments were part of the spec I'd be hard pressed to want anything else for general purpose config files. HCL or libucl probably come in second, not that I particularly care much. TOML is less than ideal with (as was pointed out) nested tables – the problem I ran into is that some valid TOML cannot be serialized with serde.
- forgotmypw17 4y agoProblem with json is editing it manually, especially from a shell
- stubish 4y ago> For me JSON comes close to ideal, if comments were part of the spec I'd be hard pressed to want anything else for general purpose config files. ie. YAML. Write your JSON with comments and parse it with a YAML parser. Most decent libraries spit out readable YAML or even JSON if you tweak it to always quote strings.
- inferiorhuman 4y agoNo, definitely not YAML. There is way too much complexity with YAML.
- voidfunc 4y ago> We all know one of those people who’ll tell you that the answer to all infrastructure problems is a bunch of EC2 instances, a golden AMI and a load balancer. Those people might be right, but if take a look at your infrastructure right now, could you solve the problems in your organization by going back to building AMIs and sticking them behind a load balancer? Even if you can, do you really want to? No, I thought not. Other than the slow pace to bake and test golden images I still prefer this approach. It is simple and a proven design and easy to build-out on every single cloud provider. I might be biased tho, this is how we manage the host's for our several thousand Kubernetes clusters. I'm just kind of used to it.
- keithnz 4y agoI learned to stop worrying and don't really care, I'll happily use xml, json, yaml or toml . To me, it doesn't really make a lot of difference, and yes, there are a bunch of reasons why various formats have different advantages, but it really doesn't take much of my life, things don't really go wrong with any of them, and while I have a preference, pragmatically, I just don't care.
- xcambar 4y agoIn a team of engineers, find one that complains about the configuration formats or linting rules, and with 100% confidence, you know who will ruin your meetings. Source: empirical study from me Thanks for not being that person.
- ResearchCode 4y agoThe one that sent out the invitation
- IshKebab 4y agoI'm that person and I don't ruin meetings. I find it's the opposite - there are one or two people that obstinately refuse to allow any improvements and derail meetings with endless naysaying and stuck-in-the-muddery. It's wearing.
- JonChesterfield 4y agoBeware the cost of those improvements. I never want to have another discussion about whitespace layout.
- einherjae 4y agoAs a Norwegian I’m in the camp of «just say NO to YAML»
- gorgoiler 4y agoI wanted a no*-op script the other day: script: - true # oh no* No* pun intended. *Pun also intended. (To the uninitiated: YAML automatically casts the bare string “true” into a boolean for you. Fairly reasonable. But it does the same with “yes” and, per the parent comment, “no” too. This is problematic if you have a list of ISO language codes including Norwegian.)
- Gare 4y agoYAML 1.2 fixed that, but almost nobody supports or uses it (in my experience).
- notreallyserio 4y ago"y" is similarly afflicted.
- marcosdumay 4y agoPersonally, I think configuration languages should only accept strings. I still haven't met a data parser that didn't create more problems than it solves. The application knows exactly the schema of its data, let it do the parsing. But yeah, the YAML one is crazy.
- 4y ago
- rl3 4y agotl;dr: Author mentions they work for Pulumi halfway through the article. Pulumi has recently added YAML support.
- arein3 4y agohehe
- TeddyDD 4y agoAuthor's examples of „good” use of YAML still would be more readable as Lua[1] or JS code. Return one table/object and call two functions instead of clunky YAML stuff like Fn::Invoke. [1]: http://ix.io/3XTk http://ix.io/3XTk
- thangalin 4y ago> It might just be personal preference here, but I look at that thing and shudder. Can you imagine authoring that? Maintaining it? Isn't the problem more the tooling than the configuration language specification? Can you imagine authoring office documents by hand? Or maintaining them? My text editor, KeenWrite, allows users to reference YAML variables as paths while editing documents. The editor includes a collapsible tree view of the variables, which makes navigation and maintenance a breeze. https://i.ibb.co/qp0vj1Q/yaml-editor.png https://i.ibb.co/qp0vj1Q/yaml-editor.png Also, when inserting a variable into the document, the YAML tree expands automatically to the location the variable is defined. From the end user perspective, the data format is irrelevant. Here's an example that produces a user-friendly UI for a JSON schema: https://json-editor.github.io/json-editor/ https://json-editor.github.io/json-editor/
- jerf 4y ago"My text editor, KeenWrite, allows users to reference YAML variables as paths while editing documents." Another thing that would help in these template languages is if they'd pervasively track the sources of things, and let you view the final output of the templating process with some mechanism for viewing those annotations. Then when you look at the final result of some templating operation and can't figure out why "AWS_ENV_VAR" is "lol don't deploy this", you have some chance of figuring out where it came from. This is yet another entry in the class of things that aren't too hard to add from the beginning, but good luck retrofitting it on after the fact.
- winrid 4y agoI'm on my phone right now but maybe one if you can double check: since YAML is a superset of JSON you can parse JSON with a YAML parser right? Does that mean you can add YAML comments to JSON and parse the JSON with the YAML parser, to get JSON-with-comments?
- deleted 4y ago[deleted]
- samus 4y agoShould work, but various JSON parsers (Jackson for sure) can be configured to also deal with (JS-style) comments. If all you will realistically have to process is JSON+comments, I'd stick to a parser that does just that. Also, YAML has more features, and YAML parsers have to be a whole lot more complicated as a result. This is important if you use that parser at a security boundary!
- formerly_proven 4y agoIn hindsight, defining JSON as a subset of JS syntax without comments is the by far biggest design flaw of JSON, in particular because JS comments are very easy to parse.
- winrid 4y agoJSON was meant as a human readable communication protocol, not for configuration files.
- ungawatkt 4y agoThe last I remember thinking about that idea, the answer was yes. And that I could probably switch out the parser to a yml parser, not tell anyone, and get a few months free of parsing errors in our hand written json (don't ask) before anyone noticed. But then I imagined dealing with the inevitable yml parse errors when people figured out the new constraints, and decided not to take that particular path to code sins. I can't remember if I actually tested it for real though.
- rocqua 4y agoSo the idea is to define a component in a general language for great configurability, and then make it possible to consume / instantiate it with YAML. What if the way you wrote the component does not allow some sort of dynamic config you need through YAML. Now you are working in two separate files and two different languages trying to make a change. Things almost get worse when the writer of the component and the consumer of the component are different people. Either massive horrible cludges happen to the YAML, or a weird back and forth needs to happen.
- mst 4y agoThis is just a specific case of "what if the program I'm using doesn't have a config file option for X" though. I usually run into it with webserver config files where (in both apache and nginx) I have to do something moderately contorted to get the end result I had in mind, but in the end the general case is just another trade-off.
- jaxxstorm 4y agoThis isn't how Multi-Language components work. The components only expose the inputs and outputs needed to the downstream consumer, you never have to "work in two separate files"
- forrestthewoods 4y agoso YAML is great because it can be converted into actual source code? That’s a pretty strange argument.
- jaxxstorm 4y agoThis isn't the point of the article at all...
- forrestthewoods 4y agoI mean if you read the summary section it kind of is? What you say the point of the article is? Submarine story is an acceptable answer.
- orthoxerox 4y agoBah, a clickbaity title that amounts to: - YAML is fine when you can use it for last mile configuration and don't have to use it for everything else Well, I agree. Take Helm, for example. values.yaml is a perfectly nice configuration file that you can even enjoy editing. The chart itself is an abomination of endless boilerplate (not Helm's fault, but inherited from k8s) and the worst possible templating mechanism bolted onto it (totally Helm's fault).
- karlicoss 4y agoI stopped worrying and in most cases just write configs for my tools in python. Then you can just import/exec it and you're done. I can use all the operations/primitives I already know in Python: string interpolation & operations, loops, Pathlib, imports etc. I can use mypy and all the other existing linting tools to make sure my configuration is correct without having to write a custom linter (and basically reimplement 10% of mypy). Shameless plug if you wanna read a longer analysis: https://beepb00p.xyz/configs-suck.html https://beepb00p.xyz/configs-suck.html A great example is pyinfra https://github.com/Fizzadar/pyinfra#readme https://github.com/Fizzadar/pyinfra#readme Think Ansible but instead of YAML you write Python. It provides a set of primitives/DSL and some rules you need to adhere to, but otherwise you just write regular python code.
- lysecret 4y agoSame. Using Boto3. But your packages sound good as well. Also curious about Pulumi. Boto3 really isnt that good :D
- baq 4y agoXML is better than YAML. change my mind. disclaimer: hate both
- dagw 4y agoI can teach most people (including non-programmers) enough of a subset of yaml for them to edit/create working config files (for a particular app) in a text editor by themselves. You cannot do that with XML If you're only reading and writing the files via a library, then XML is probably better.
- postalrat 4y agoI can teach people enough csv to write config files and much more. CSV FTW?
- Thiez 4y agoSince YAML is (almost?) a superset of JSON, you can treat it as "JSON with comments". Then it no longer has significant whitespace and strings won't magically coerce to booleans because the quotes are mandatory in JSON... so if you use YAML that way it's pretty good. But YAML, as it is "intended" to be written? I think I'd rather have XML as well...
- maximus-decimus 4y agoAt least YAML doesn't make my eyes bleed.
- Viliam1234 4y agoThis is a valid YAML document! --- --- - --- --- --- \ \ / / / \ | \/ | | | \ / / ^ \ | \ / | | | / / / / \ \ | |\/| | | ---| ---- --- --- --- --- ------ (At least http://www.yamllint.com/ http://www.yamllint.com/ says so.)
- baq 4y agonotice of meme acquisition no action required
- sigmonsays 4y agowho else loves templating yaml? We really need a language and not serialization format to describe configuration.
- mst 4y agoI keep meaning to experiment with https://yglu.io/ https://yglu.io/ (it has support for env vars via a $env builtin -if- you enable it at processing time)
- silon42 4y agoxslt to generate xml from xml
- DonHopkins 4y agoyaml2xml | xslt | xml2yaml
- kkfx 4y agoHonestly I don't... YAML is more compact but still hyper verbose and hyper rigid respect of XML witch is the same respects the older SGML. The old new trend is going through real programming languages instead of dedicated data-formats, from those who use Python or DHall or those who prefer lisp to be in a more civilized land... Just compare Ansible/Salt YAML+python (because you can't do anything YAML-only) to Guix System, just see k*s YAML or Home Assistant ones: they are horrible to manage by a human. They are not much effective respect of anything else for a machine. Long story short IMVHO it's about time to recognize that the old "data and language together" is the way to go, erasing common DSLs/formats/parsers etc.
- notreallyserio 4y agoThe thing that most frustrates me about YAML is probably not the language itself but the poor IDE and GitHub tooling I have been using. Trying to figure out the hierarchy of a key in a YAML file (local or GH) is incredibly frustrating. With formats like JSON at least (if nothing else) I can just open the file in vi, find the next close-curly and hit % to find the parent -- easy. Significant whitespace is also a problem but I think that war has been lost.
- gilbetron 4y agoI've been using YAML for years and years, and will always hate it. I understand why it exists, but I still can't stand it, nor do I have a desire to force myself to love it. I just generally hate white-space defined scopes. I love Python, but still hate it has white-space defined scopes, too! You can't make me love YAML! (but I'll still use it and put my hatred to the side!)
- vagrantJin 4y ago> I love Python, but still hate it Well. This makes a whole lot of sense.
- deleted 4y ago[deleted]
- cosmiccatnap 4y agoI just find that YAML doesn't improve anything over JSON. Having recently migrated from cloudformation to terraform on a project the use of HCL is wonderful in comparison and when I'm all ready to commit I just type terraform fmt and everything is aligned properly. Considering it's just as trivial to do this in any JSON prettifier...I don't understand the need for yaml besides pure asthetics which is why you either love it or hate it, because it doesn't help or hurt it just annoys you or it doesn't.
- AlwaysRock 4y agoBlah. Right now at work I'm using YAML with HAML ruby files and its... Not great. Really difficult to search for in our code base. Its hard to tell what is routing where due to helper methods often pointing to certain YAML files based on certain conditions. I don't have a better solution but it just feels needlessly complex for text.
- nicoburns 4y agoI've been thinking about what make a good language for configuration files, and the conclusion I've come to is that it ought to have: - Local variables - Pure functions with basic math and string manipulation capabilities Possibly with an extended version that also has: - Input variables (that must be provided by the code reading the file) - Exportable functions that are callable by the host code This extended version would be perfect for use cases like defining CI pipelines. A language that did this but was otherwise simple and allowed no access to the host system, and had widely available library support across most common languages would be just awesome.
- tylergetsay 4y agoJavaScript tools and their .js config files work great for this That being said, if you're only evaluating the file one time, just write a script to spit out your config.
- wvenable 4y agoIn my own applications, I've moved away from configuration files as much as possible. Instead, I provide an API and all my configuration is just code. I think creating an entirely new Turing-complete language just for configuration is a waste of brain cells for everyone involved.
- marcosdumay 4y agoSomething like Dhal? https://dhall-lang.org/ https://dhall-lang.org/ From what I hear people saying I'd guess its main usage is being a general purpose language that creates those shitty YAML based formats as outputs. Anyway, I'm in complete agreement with the article. CI pipelines and infrastructure as code should both be defined in code, real code. Configuration files are the place you set the values for your variables, not the place you do calculations with them.
- ChiefOBrien 4y agoLua
- exyi 4y agoOne of the reasons to like Nix. However, having something majority of people can understand is usually the more important concern...
- blagie 4y agoMy experience is that with both YAML and XML, the key is to have a well-designed schema. The problem isn't YAML or XML, but abuses of them. Most of the criticisms come from badly-designed schemas. HTML 2.0 was very usable, and there are other XML schemas which are nice. There are a few keys: 1. Don't reinvent constructs. If your XML looks like <element name="foo"><attribute key="bar" value="biff" /> </element>, you've got a problem. It should be <foo bar="biff"/> 2. Provide reasonable defaults. Everything should be optional. 3. Break up files into manageable chunks I wish everything in /etc/ used a common format, like YAML, or a small set of common formats. I also wish for good configuration libraries, which handled: - Parsing / validating YAML config files, command line parameters, environment variables, database overrides, defaults, etc. - Printing pretty error messages - Worked in a few languages (Python, JavaScript, and perhaps C) - Guided developers towards sane designs. Also, if you're putting objects in XML, documents in YAML, or strictly tabular data in either, you're probably doing something wrong. - YAML: Configuration files, objects - XML: Documents - CSV/TSV/TSVX: Tabular data
- orev 4y ago> I wish everything in /etc/ used a common format, like YAML, or a small set of common formats. This is what makes the Windows registry such a success. I think too many Linux people hated Windows for too long that they ignored this problem. Microsoft already had the solution in Windows 95. Early iterations maybe weren’t perfect, but they stuck with a standard API and everyone got onboard. It completely removes an entire set of problems that Linux people keep reinventing the wheel for.
- blagie 4y agoThe registry is far worse than Unix. A problem with the registry is discoverability and documentation. It's a bottomless black hole. I am glad it's standardized. I'm not glad it's binary, huge, and opaque.
- yegle 4y agoI like the approach in Google: everything is a protobuf message including a config. A couple benefits: 1) all fields are documented and strongly typed, with backward and forward compatibility (old binary + new config, new binary+ old config works). 2) you can use any language to generate the config, simply specify your proto-generator binary as dependency. 3) if your config is simple enough and willing to lose some backward/forward compatibility, a text proto file is perfectly human readable/writable.
- timcavel 4y ago
- vvladymyrov 4y agoThis problem it is not the format itself. The problem (IMHO) is with how users use it. I see huge issue of users looking at YAML as a plan text document without regard of actual document structure. Here is example of widely used blatant misuse in popular tool - Helm Charts - allows to create Jinja templates for YAML documents totaling ignoring structure of the documents - as a result users have tools to control the number of tabs/spaces but no they to work with document structure (like injecting element under certain path or validating if document structure is correct). This is such a shame and loss to DevOps community that such as widely used tool as helm chart continue teaching new users of bad habits.