12 ms·
YAML: Probably not so great after all (2019)
- eyelidlessness 6y agoNearly every time I have to make any changes to a CI environment (or set one up for a new project), I end up with a stream of increasingly unintelligible and often increasingly grumpy commit notes. It's almost always because YAML, as a format, is confusing and frustrating to write. Structured data with only vague, and often misleading, indication of its structure is awful to work with. Sure, it's "human readable", but if your data is of any real complexity you're almost certainly better off just machine-generating it.
- crispyambulance 6y agoIt makes me grumpy. Even the self-ironic name "Yet ANOTHER mark-up lanuguage" is a set-up for disappointment. My personal belief is that XML was "good-enough" having been engineered for many use-cases and easy to comprehend for what it provides. It's a tragedy that XML was abused so vigorously in the early naughts. People got so sick of it because they were compelled to edit XML manually in a text-editor with shitty to none schema support. It didn't help that the worst monstrosities of that era (soap and ws-* crap) heavily invested in turgid, badly designed XML. I think that if there had been some more effort on tooling and good-practices, XML could have remained popular and we would not have had jason, yaml, and whatever awful thing is next.
- thinkloop 6y ago> XML could have remained popular and we would not have had jason One of the selling points of json is file size, xml started dying when people realized that more than half their bandwidth was going to xml structure rather than actual data
- reallydontask 6y agoThis is hardly an issue for server side config files or CI/CD config stuff.
- deleted 6y ago[deleted]
- Thiez 6y agoIs the difference in size really significant? When the files are small it doesn't matter much, and when they are large you can throw some compression in the mix. I've never heard filesize as an argument for json before.
- thinkloop 6y agoImagine a table of numbers, the xml would be 90% repeated header names and structure. Now multiply that by hundreds of little Ajax requests, it added up.
- rcxdude 6y agoI don't think it's just badly-designed XML that turned people off it. XML itself is just not very good for configuration: it's repetitive, there's no straightforward way of making a set of key-value pairs (just a long set of representations with different tradeoffs), and it's quite complex to parse and process because it has such an all-encompassing scope. I feel like XML is only really the right answer if you have something which resembles a document which you want to mark up, which is a tiny subset of what it is marketed and actually used for.
- montroser 6y agoNot that it's any better -- but for what it's worth, YAML is a recursive acronym for "YAML Ain't Markup Language"
- Lammy 6y agohttps://yaml.org/spec/history/2001-08-01.html https://yaml.org/spec/history/2001-08-01.html "Yet Another Markup Language (YAML) 1.0"
- montroser 6y agoInteresting that it originally had that even worse name. But also, I'm not wrong per the current version of the spec: https://yaml.org/spec/1.2/spec.html https://yaml.org/spec/1.2/spec.html "YAML Ain’t Markup Language (YAML™) Version 1.2"
- hn_throwaway_99 6y agoOne of my favorite bugs of all time, which turns out to be a pretty common yaml bug (the article touches on a similar example), was when we used yaml to configure some localizable data. Everything worked fine until Norway came online. Sweden worked fine, so did Denmark and every other country, but the app crashed when it loaded the config for Norway. Turns out the country code for Norway, "no", is interpreted as boolean false when unquoted in yaml.
- m_ke 6y agoI used a JS object as a map back in the day for image recognition results, for some reason it kept breaking the app for pictures of prototypes...
- hombre_fatal 6y agoAre you saying you were using "__proto__" as a key? Suggesting that you were using a function as a map and "prototype" as the key seems even more made up.
- runamok 6y agoI think I did something similar using "static" 20 years ago.
- azaras 6y agoBut it is because you do not use a yaml serializer. If you use a template you have to put "" for strings.
- stingraycharles 6y agoI have been using yaml for years (mostly as a user, not as a dev) and I’ve never heard anyone mention a yaml serializer. Is this common practice? Wouldnt the yaml serializer have the same locality issues, or is it more strict?
- bmn__ 6y ago
- pmoriarty 6y agoMy biggest gripe with YAML is its meaningful whitespace. Debugging nearly invisible indentation problems can be such a pain.
- meowface 6y agoI'd argue that's the defining feature, though. That's one of the main reasons people use it over JSON.
- karussell 6y agoIt is not so much about indentation vs. brackets. For me the main reason over JSON is that YAML works without quoting the entries and the possibility of comments. And btw: every JSON file should be a valid YAML file in theory.
- dylan604 6y agothis is my problem with python. however, this biggest issue why YAML is my least favorite config format. unquoted strings with colons is another top one.
- save_ferris 6y agoI see a lot of criticism of YAML, and I’ve looked a few other minimal configuration languages like TOML as well. Serious question: why doesn’t this language space have a universally adopted candidate like SQL?
- skybrian 6y agoYou might as well ask why there was nothing as popular as markdown before markdown. It's just historical circumstances. JSON is pretty common, and liveable with a few conservative extensions like comment support. But the extensions haven't gained a lot of traction yet.
- stickfigure 6y agoFor a brief period, it did: XML But everyone thought "that's too complex" and so reinvented something simpler. Then kept adding features, and more features, and now YAML is more complicated than XML. This process will repeat ad infinitum.
- ludamad 6y agoI'm fine with XML complexity, some things pay off. It remains entrenched in networking world. Some things are awkward though, like needing to introduce named fields for everything - e.g. not being able to have an XML file that is purely a list of integers, they each need some arbitrarily named field markup
- pjmlp 6y agoThankfully in Java and .NET land it is still mostly about XML. It is so ironic to see these fads come and go, and then one wonders why we are so cinic regarding newcomers.
- reallydontask 6y ago.net core uses json for config files
- 6y ago
- karussell 6y agoIt is so easy to complain about something, but seriously: what are the alternatives for human readable config files?
- m4r35n357 6y agoFor a scripting language, how about the language itself?
- eyelidlessness 6y agoThis is what I do whenever possible. And bonus if your scripting language is statically typed. My projects use TypeScript configurations, with the `config` import statically typed and each configuration file checked against those types. It's a great way to ensure you don't miss a config change for an environment you're not working on at the time.
- baq 6y agoTuring complete configuration is a no-no.
- williamdclt 6y agoMany reasons to prefer a static declarative config over a dynamic imperative one. Sometimes the latter makes more sense, but there is very much a need for the niche YAML tries to fill
- 6y ago
- _bxg1 6y agoI've never understood the love for significant whitespace. I see where the idea came from - "No more missing semicolon errors! Woohoo!" - but it should've been clear after trying out the idea for five minutes that it was not at all worth it. It constantly causes trouble and all just to save a couple of keystrokes. Though: now that I think about it, most of the problems happen at the block level, not the line level. So maybe significant newlines are fine but not indentation.
- ludamad 6y agoMaybe after trying it out for five minutes you make up your mind - but mine was decidedly for it
- thelazydogsback 6y agoYeah, I'd much rather look at Python or F# than Java code all day.
- cannam 6y agoF# uses significant whitespace? I didn't know that. What does it use it for? SML and (I think) Ocaml don't, and I had assumed F# was like Ocaml, since its syntax is largely the same. In the tiny bit of F# I tried writing once I don't remember having had to pay any attention to whitespace. Significant whitespace is one of the things I didn't enjoy about Haskell and Idris. Life's too short to indent code by hand.
- _bxg1 6y agoI didn't think Haskell had significant whitespace; it doesn't have blocks really, and nested expressions can be wrapped in parentheses, right?
- jeremyjh 6y agoWhitespace is significant in Haskell. In do notation and other forms, you have blocks. You can actually use braces and semi-colons instead but I've never seen that except in some very short one-liners. https://en.wikibooks.org/wiki/Haskell/Indentation https://en.wikibooks.org/wiki/Haskell/Indentation
- topkai22 6y agoI find YAML distressingly hard to work with. Just one example- because it’s white space delimited, copying and pasting a block of code will often blow up between docs based on nesting levels. This is intensely frustrating when the setting is tied to a CI pipeline that takes 5 minutes to get to the error. The human unreadability if JSON is greatly exaggerated. While you can get horrible looking JSON, you can also then pretty print it into something much better. If we could just all agree to allow comments into a spec it’d be fine.
- morelisp 6y ago> This is intensely frustrating when the setting is tied to a CI pipeline that takes 5 minutes to get to the error. I also detest this, but it's not really about YAML - it's ridiculous that so many of our tools no longer allow any validation / linting of files before attempting to use them; some that do require the "full service" running to provide sufficient context; even those that don't don't always give good (i.e. consistent, structured) errors. This problem would remain if the format was JSON, TOML, or even XML when working without schemas for every namespace. The worst is the tools that layer Jinja on top of the YAML (Salt/Ansible/Puppet) which is basically impossible to validate statically. At least with GitLab, Docker Compose, and Kubernetes I have some hope - but integration into other tools is awful.
- williamdclt 6y agoCircleCI has a CLI, you can `circleci config validate` (or something like that) and it will check that what you wrote is valid YAML and that it's a valid CircleCI config (so syntaxical and semantic validation). Very useful, one of the many thing that make me prefer Circle over the other solutions I tried (Travis, Jenkins and a couple other I forget)
- mxscho 6y agoEarlier discussions: https://news.ycombinator.com/item?id=17358103 https://news.ycombinator.com/item?id=17358103 https://news.ycombinator.com/item?id=20731160 https://news.ycombinator.com/item?id=20731160
- azaras 6y agoYAML in kubernetes works very well. If yaml is not working for you have to search a serializer format that fit in your project. There is not silver bullet.
- morelisp 6y ago> If yaml is not working for you have to search a serializer format that fit in your project. If I need a serializer format, it's not human-readable/writable, and I might as well have something like XML or s-exp that are easier for many tools to work with, more easily composable, easier to generate automatically, etc. > There is not silver bullet. This phrase needs to be retired. It's true, but invoked far too often to excuse shit bullets.
- j0057 6y agoMoreover, a silver bullet is meant to be especially effective in very specific scenarios, ie. shooting werewolves or vampires, whereas in colloquial usage 'silver bullet' refers to a thing that works well under all circumstances.
- orthoxerox 6y agoThat's because no one writes k8s resource definitions by hand. Everyone just copies their previous Deployment and changes some settings.
- M2Ys4U 6y ago> YAML in kubernetes works very well. I beg to differ.
- teknopaul 6y agoHeartily agree with this post. One of the things I most dislike about yaml is that they persuaded JSON to remove comments from the spec for some psuedo compatability nonsense. Without comments, json is less useful for config files and working/documented examples. I have written a pre-parser to permit comments in JSON and for nodejs apps I use javascript as config. Naturally where security is not a concern. Yaml always seemed like a mess to me. Liking Toml, decent compromise. And linux style: name space value, hash and semi for comments. Unless I really need a heirachy.
- williamdclt 6y ago> I have written a pre-parser to permit comments in JSON and for nodejs apps I use javascript as config. Naturally where security is not a concern. JSON5 allows comments and a few other niceties. I'd trust JSON5 parsers more than my own hand-rolled one
- jrochkind1 6y agoSince he opens by referring to a similar argument against JSON for human-editable configuration files, I wondered what format he did like for that. Answer at the end appears to be TOML, if you must have one at all. > Don’t get me wrong, it’s not like YAML is absolutely terrible – it’s probably better than using JSON – but it’s not exactly great either. There are some drawbacks and surprises that are not at all obvious at first, and there are a number of better alternatives such as TOML and other more specialized formats. > One good alternative might be to just use commandline flags. > If you must use YAML then I recommend you use StrictYAML, which removes some (though not all) of the more hairy parts. (I do agree YAML is in retrospect a mistake. The reasons why remind me in some ways of the problems with Markdown). (Oddly, I can't seem to find an up to date TOML parser in ruby that supports the 1.0 spec...)
- cachestash 6y agoThe opening sentence is incorrect. yaml.load is now a wrapper around yaml.safe_load that negates the risks he highlights
- moron4hire 6y agoWait, what? It's built in to the format that it can execute arbitrary shell commands? "Who approved this?!" I can't imagine the cluster fuck of ideas in a person's head to lead to thinking that this was an ok design for a configuration file. The person who designed this and I just don't live on the same planet. And here I thought the reason I didn't use YAML was because the syntax looked stupid.
- ezrast 6y agoNo, it's built into the format that implementations can extend it with their own types. Then library authors decided it would be neat if any object in their language could be serialized as yaml, and so extended it with types that happen to be able to execute arbitrary code, because that's what objects do in dynamic languages. A more conservative implementation wouldn't have that particular vulnerability.
- crehn 6y agoFun fact: YAML is a superset of JSON. That is, any JSON is also valid YAML. YAML is a complex and unintuitive mess that allows doing every thing in a million ways. I’m surprised it ever got so much traction. TOML is a breath of fresh air next to it.
- deleted 6y ago[deleted]
- chme 6y agoWhat is even greater: with version 1.0 TOML supports different value types in arrays. That means it is compatible with JSON data structures and allows easy conversion of existing applications to it!
- 3pt14159 6y agoMy only gripe with TOML is that numbers can be variable names. I get what they're trying to do (be consistent) but it just looks wrong to have 3.14159 = "pi" to be: {"3":{"14159":"pi"}} Otherwise I think TOML is great.
- BiteCode_dev 6y agoI like toml but the syntaxe for an array of objects is terrible.
- bmn__ 6y ago> Fun fact: YAML is a superset of JSON. That is, any JSON is also valid YAML. That's false. https://metacpan.org/pod/JSON::XS#JSON-and-YAML https://metacpan.org/pod/JSON::XS#JSON-and-YAML
- rudolph9 6y agohttps://cuelang.org https://cuelang.org is a very nice alternative that has nice import/export support for yaml among others
- deleted 6y ago[deleted]
- mrbonner 6y agoI have not seen anything that could come close to the robustness and flexibility XML has to offer. When being used for configuration XML seems to be the superior choice. People tend to give XML a bad name in RPC usage for being verbose. But, for config it’s perfect for me. There are tons of IDEs supporting XML (syntax highlight & collapsing brackets, etc...). XML schema is also great to ensure the config is conforming. I keep thinking we are trying to reinvent a worse wheel here.
- Animats 6y agoYAML allows escapes to executable code? !!python/object/apply:os.system args: ['ls /'] Who put that backdoor in?
- bmn__ 6y agoThat's a leading question. Authors of some libraries did not consider the security implications of the object serialisation part of the spec. I assign the blame to them for not looking beyond their own limited horizon, metaphorically speaking. When they made their libraries, the reference implementation was already available and it was secure by default.
- Animats 6y agoA "call arbitrary external program" feature does not get in there by accident.
- bmn__ 6y agoIt's not a "call arbitrary external program" feature, but an "object serialisation" feature that has security implications which some implementers did not handle correctly because of their language parochialism and lack of experience. That does make it closer to an unfortunate (but entirely avoidable) accident rather than what you believe, that evil people maliciously and intentionally added a backdoor. You can keep believing it, but that does not make it any more true.
- nojvek 6y agoWhen I first saw json, compared to XML, it was so simple. As a fairly new programmer I could say to myself “hey I can write a parser and dumper for this quite easily”. When I first saw YAML, it was nice but it felt a bit too complicated. All I really want is Indented JSON. New lines instead of commas. That’s it. Json is fast to parse. See simdjson doing it at > 2.5GB/s. One can’t do this with YAML where no could mean many things. And god I hate k8s for their bajillion yaml configs. Thank god for jsonnet that can dump to yaml. Jsonnet is truly nice and makes working with json like configs a pleasure.
- Mikhail_Edoshin 6y agoThe specification of YAML is about three times as long as XML 1.0, but while XML includes a whole grammar-based validator with such niceties as referential integrity of IDs and default values for omitted parameters, YAML spends all this on syntactic sugar.
- mixmastamyk 6y agoUse strict yaml, solvable problems solved.
- rs23296008n1 6y agoYAML isnt my idea of clarity but thats fine. I tend to go with using a python script to generate a json file. This supports comments, if-else decision making, modules and all sorts of other smarts. Tell users not to edit the resulting json file - its readonly. Python gives you all the power of scripting including validation and json gives you the easily readable format for both humans and machines. Great for knowing exactly what the settings evaluated as. And when you don't need the power of full expressive python script, you can just json.dump() a python dictionary and be done with it. I've also used sqlite as a config file format and that is very polite and easily read from anything I use.
- beefbroccoli 6y agoI just recently replaced a bunch of YAML in a project with JSON. At face value the YAML still looks easier to grok, but I kept needing to periodically think about YAML<->JSON. At one point in the middle of yet again running JSON through a YAML conversion process to see how I would write something, it dawned on me that YAML wasn't saving making my life easier it was making it harder.