8 ms·

YAML and Configuration Files

71 points by signa11 5y ago

Vitamin_Sushi 5y ago
The solution for defining a more complicated config is to write your own config file format? I would think a custom format wouldn't necessarily be easier for others to read and write unless it came with a guide/readme, but then that's just one more thing to learn. Admittedly, I've never had to write a super complicated config file, but can anyone tell me why I shouldn't continue to use something like JSON for all of my config files?
- beached_whale 5y ago
  Let’s not forget that almost every JSON parser optionally supports comments and trailing commma’s when parsing.
  - simonw 5y ago
    Which ones do that? I've not encountered many myself.
    - Zamicol 5y ago
      And JSON5. https://github.com/json5/json5-go https://github.com/json5/json5-go
      - simonw 5y ago
        What a delightfully readable example of a hand-rolled lexer and parser that is.
        jimmygrapes 5y ago
        I thought you were being sarcastic, but I think I just finally learned how lexers and parsers work by reading the code, and I don't even know Go.
        simonw 5y ago
        Hah yeah no sarcasm intended at all! It's really neat code.
    - thayne 5y ago
      Ruby's supports comments. But that's the only one I know of
    - beached_whale 5y ago
      * C# - builtin via JsonCommentHandling, JsonSerializerOptions.AllowTrailingCommas - Newtonsoft via CommentHandling, trailing out of the box * C++ - Boost.JSON/RapidJSON/DAW JSONLink have options for both, nlohmann has an option to allow comments * Python, I only know doing it via jsmin first * Javascript, use jsmin also These are the ones I have used or seen. It comes up so often that parser writers get issues over and over until they support them. nlohmann did not for years. But people want them for config files and such. Them being an extension is nice in that they are not officially supported and how they are handled isn't specified.
- u801e 5y ago
  Compared to yaml or ini formats, JSON is arguably harder to read and definitely harder to write due to having to keep track of brackets, commas, and/or quotes. Ideally, a configuration file should be easy to read and write/modify by a user of the application. A lot of applications just stick with ini like formats because it meets both requirements.
  - secondcoming 5y ago
    Maybe it's because I've written C++ for years, but give me brackets over significant whitespace any day. If anything, YAML is harder to read because the indentation is important, yet deliberately invisible.
    - Aloha 5y ago
      Agree. Whitespace with inconsistent rules is awful.
  - benjarrell 5y ago
    For me, the brackets, commas, and quotes are what make json readable (and writable). I really struggle to write yaml. I end up writing it as json and converting it. I agree about ini though.
    - HelloNurse 5y ago
      JSON is unpleasant (for me, more unpleasant than XML), but at least it doesn't attempt to be clever like TOML and YAML.
    - dragonwriter 5y ago
      > I really struggle to write yaml. I end up writing it as json and converting it. Since valid JSON is also valid YAML with the same semantics, it is impossible for it to be harder to write YAML than JSON, and no conversion is necessary.
  - bwship 5y ago
    Not to mention being able to have comments in YAML. That helps make the file much clearer as well.
- thayne 5y ago
  Two big reasons you shouldn't use json for config: comments and multi-line strings. json supports neither, although there are many other simple formats that do.
simonw 5y ago
After resisting it for many years I've finally settled on YAML as my default configuration format. I need a way of expressing the core data types of JSON (key/value mappings aka "objects", arrays, strings, integers, floating points and booleans) - plus comments, and multi-line strings that insulate me from complex escaping rules. YAML does that. It's not perfect, but it's good enough. And in Python I can use yaml.safe_load() to avoid some of the more troublesome corners of the spec.
- meowface 5y ago
  YAML seems to be one of the most polarizing things in the software industry. I'm personally a big fan of it, though I agree with the author that it becomes a pain once you start trying to use it like a DSL or templating language. I use it in pretty much every project, but I've worked with people who've said they hate it with a burning passion. And it's kind of a trope to see anti-YAML manifestos on HN and /r/programming fairly regularly, both as article submissions and comments. As you mentioned, one of the most useful features for me is the ability to write raw strings with single quotes (or by omitting quotes). I've been in many situations where and I and others have had to maintain regular expressions in config files, and it's so much more of a hassle with JSON due to the escaping requirements. The opening of the article states "Hot take: YAML isn't a configuration language or a configuration language format, it's a serialization format."; I have the exact same opinion, but with "YAML" swapped for "JSON". (And maybe the author would agree with that, too.)
  - amirkdv 5y ago
    > it becomes a pain once you start trying to use it like a DSL or templating language. I like YAML and use it a lot. But I'd argue the fact that it even allows/encourages you to do such things with it is a design flaw that ultimately has kept it from Do One Thing Well.
    - meowface 5y ago
      I do think some of the more "advanced" features are definitely a mistake. StrictYAML (https://github.com/crdoconnor/strictyaml https://github.com/crdoconnor/strictyaml) is a limited, much saner subset of YAML that I wish people would use more: https://hitchdev.com/strictyaml/features-removed/ https://hitchdev.com/strictyaml/features-removed/ No coercion of y/n/yes/no/on/off to booleans (these were also removed in the official YAML 1.2 spec, thankfully), no direct object representations, no anchors or references, etc.
derefr 5y ago
I think I get what the article is saying. I don't think they expressed it very well. Here's a concrete example of what I think they're complaining about. From a Kubernetes Deployment manifest: - name: DATABASE_URL valueFrom: secretKeyRef: name: db-conn key: uri That valueFrom.secretKeyRef.{name, key} structure is an "exploded AST" — something that in any programming language, you'd be chastised for writing out as a structural-initialization literal, because there's so much stuff there that it's easy to screw it up and forget something. When attempting to express such a literal in most-any programming language, you'd expect to either have: • the Java "literal factory function" approach — some static function to call, that takes either other literals, or a single literal string encoding an expression in a DSL, as arguments, and then produces an initialized value-object; or • the Elixir "custom sigil" approach (there's probably a more popular language with this feature, but Elixir is what I know) — a macro or operator that takes in a raw string or lexemes encoding an expression in a DSL, and then codegens out to the appropriate structural-initialization literal. (Macros are better here, as incorrect DSL syntax can be caught at parse/compile time, just like incorrect syntax of the surrounding language.) ----- To be honest, I don't think I would want YAML to support either of these. YAML shouldn't be arbitrarily extendable; a YAML document should be a YAML document, parsed by any compliant parser. (There could be some Avro-like "embedded schema" format for YAML that enables this, but that format would be better as a layer on top of YAML, rather than being part of YAML itself.) What I would like in YAML, is support for built-in literal sugar for certain specific YAML structures. This is different in concept from YAML having support for a type, where that type must have some 1:1 native type on the host language side for it to encode/decode to. Instead, a "sugared structure" would involve 1. a YAML parser having a sub-parser for certain specific DSLs, where the output of this parser is a small, standardized-in-shape container structure, made out of regular YAML arrays, dicts, and scalars; 2. a YAML generator offering a configuration option to pattern-match on these standardized-in-shape structures, where recognized subtrees are swapped out in the emitted YAML for the appropriate sugared-literal representation. Examples of where this would be helpful include: DateTimes, URIs, UUIDs, Intervals... and that's basically it. There really aren't too many of these. The short list above represents basically the entirety of the set of types I've ever had YAML fall down on in the 10 years I've been generating/parsing YAML documents. For everything else, it works fine.
dangoor 5y ago
I think Cue[1] is a much more powerful and useful config format that also has the ability to generate validated JSON or YAML if needed. [1]: https://cuelang.org https://cuelang.org
- fanf2 5y ago
  CUE was recently discussed at https://news.ycombinator.com/item?id=28127951 https://news.ycombinator.com/item?id=28127951
- contravariant 5y ago
  Not sure if 'much more powerful' is a desirable feature for a configuration file language.
  - Spivak 5y ago
    The ideal configuration format is really powerful and expressive for the author but resolves to a dead simple data structure for the consumer. If you need to template your config files for anything other than supplying external data it’s not expressive enough. I think YAML is closer than many of its contemporaries to this.
tetha 5y ago
Hm. I can see where the article is coming from. Some configuration files grow beyond classical configuration and end up being more like programming. With configuration being "Put the right connection / path strings into the program, enable some subsystems/feature toggles" and programming being along the lines of, e.g., arbitrary metric transformations in a metric collector, or programmable ACLs. With some systems, I very much end up wondering why I couldn't just load lua into the system and do the transformations with that. However, configs in json/toml/yaml come with a lot of tooling. With a custom configuration language, you suddenly need to roll your own syntax highlighting in many editors, maybe linters, pretty-printers, ... Your config can't just be computed by a configuration management system and deployed via `computedConfig | toJSON`, instead it'll be necessary to wrangle templating. And users generally know those languages. And writing parsers is a pain. And generating parsers is also a pain. And using bad parsers is also a pain. I've worked too much on parsers. Those are some pretty big hurdles to overcome and if you don't, the user experience just starts off worse.
- kzrdude 5y ago
  yaml (and json) gives configuration files much-needed hierarchy and structure, which is unbounded (within reason), not limited syntactically or inherently like simpler configuration formats including toml. yaml adds to that comments and a pleasant readable layout, even if it requires a little bit more from the editor to make it pretty. With that, I don't think yaml has much competition.
  - tetha 5y ago
    Hm, again. I don't think I agree it's that clearcut. I've found that if you mostly want a tree of simple key-value assignments, such as database.postgres.user or database.mysql.timezone, TOML is actually very hard to beat. It's extremely simple to teach to less technical coworkers. TOML mostly becomes somewhat strange if you need lists, or maps with free-form keys. Also, YAML is horrible if you have to generate it. Pretty much everyone automating stuff I've met will prefer JSON over YAML once you need to automatically generate the configuration. Pushing some data structure through an as-yaml pretty printer usually works, until it breaks in really weird incompatibilities between YAML parsers. And that happens way too much to be a fun distraction.
thangalin 5y ago
My text editor, KeenWrite[0], integrates YAML using a GUI, which hides the underlying data format[1]. This makes the format irrelevant from the end user's perspective. Hardly anybody edits .odf files or .png files directly, yet configuration files are often updated in a plain text editor rather than a GUI. Structured data formats are amenable to autogenerating GUIs (see [2] and [3]). [0]: https://github.com/DaveJarvis/keenwrite/blob/master/docs/screenshots.md https://github.com/DaveJarvis/keenwrite/blob/master/docs/scr... [1]: https://youtu.be/u_dFd6UhdV8?t=160 https://youtu.be/u_dFd6UhdV8?t=160 [2]: https://www.jeremydorn.com/json-editor https://www.jeremydorn.com/json-editor [3]: http://mb21.github.io/JSONedit/ http://mb21.github.io/JSONedit/
a-b 5y ago
This is why you should consider https://carvel.dev/ytt/ https://carvel.dev/ytt/
dub 5y ago
There are a variety of generic DSLs for encoding configuration concisely with composability, functions, custom validation, etc. Some examples would be jsonnet, dhall, and cue. If the configuration needs to be transformed into a more computer-friendly format like json or yaml for later loading by a binary, that's easy enough to do at build time in a modern-ish build system like bazel
- jedimastert 5y ago
  If you're gonna be generating at build like that, you might as well be using a binary encoding like protos (this is for devs, btw)
louib 5y ago
Currently, my main concern with YAML is that, by the spec, comments are not attached to a particular node (see https://yaml.org/spec/1.2/spec.html#id2767100 https://yaml.org/spec/1.2/spec.html#id2767100). As a result, a lot of YAML parsers (like https://github.com/yaml/libyaml https://github.com/yaml/libyaml and https://github.com/chyh1990/yaml-rust https://github.com/chyh1990/yaml-rust) only filter out the comments during the parsing phase. This makes it less than ideal for a use-case where the configuration file is expected to be modified by both programs and humans. TOML makes it more trivial to associate comments with a node. This is mainly because the language is simpler though, as the spec is not explicit about that (https://toml.io/en/v1.0.0#comment https://toml.io/en/v1.0.0#comment).
mixmastamyk 5y ago
Look into strict yaml, a simpler, safer subset: https://hitchdev.com/strictyaml/ https://hitchdev.com/strictyaml/
seized 5y ago
A while back it took myself, a data center engineer, and two engineers from the appliance vendor a solid 20 minutes to figure out the problems with a ~12 line yaml file that needed two or three lines added (network config file). Between syntax, indentation, etc it required a few tries. Plus there was a copy and paste in there via SSH. I don't know what a better alternative is but yaml can be incredibly frustrating.
- oweiler 5y ago
  The problems you describe wouldn't exist when using XML with a schema.
  - thayne 5y ago
    Maybe, but XML has a whole host of other problems.
toiletaccount 5y ago
when i see yaml, i use other software
thayne 5y ago
> YAML isn't a configuration language or a configuration language format, it's a serialization format I think YAML is even worse as a serialization format than a configuration format. It has too many ambiguities, implementations are too inconsistent, and many implementations are insecure by default with untrusted input. > Using a good custom designed configuration file format instead of trying to shove things through the narrow pipe of YAML means that you have one integrated syntax that can be designed to be more readable, more expressive, and much easier to write. I won't say a custom configuration is never the right thing to do, but I do think it rarely is. Writing a good custom configuration language is very difficult to get right, not to mention you also need syntax highlighting and other editor support for the editors your users use, libraries for popular languages so that other programs can read and write the configuration, etc. If you need something this complicated, I'd reach for using an embedded scripting language like lua (or possibly JavaScript or python) before writing a new format from scratch.
- amirkdv 5y ago
  > I think YAML is even worse as a serialization format than a configuration format. This. I find YAML to be the least offensive option for configuration and one of the worst for serialization. I might be misinformed, but I find it absurd that in 2021 we still don't have a default, universally available tool that supports the basic table stakes without headache: 1. core data types (number, string, etc) 2. lists, maps, and arbitrary nesting 3. comments 4. multiline strings 5. is acceptably readable 6. Just Works everywhere YAML does get 1-5 right (specifically 3 and 4 that JSON doesn't and IMO better in 5). But then it adds a ton of complexity that has left us without a standard, safe, and sane parser implementation: anchors and references (& and *) , casting (via !!), custom data types (via !), loads of other things I don't understand.
  - brundolf 5y ago
    It's a real shame JSON doesn't have 3 and 4, because it would be so easy and it's otherwise pretty much perfect imo. No ambiguity, every value type can be identified by its first character, clean and reasonably minimal syntax.
Aloha 5y ago
My primary gripe with YAML is white space sensitivity, it makes editing it in a hurry in a CLI environment needlessly user unfriendly. Virtually any other option is better for the person who actually has to modify these configurations later. If your configuration is expected to be hand modified directly by a user, it shouldn't use YAML.
Fannon 5y ago
YAML (and JSON of course) has the big advantage, that you can use JSON Schema to validate it and to provide realtime code intelligence in many Editors (VScode, IntelliJ, Monaco Editor in the web, many Web IDEs. Allthough you might need a plugin for YAML, where JSON works out of the box). So it's easy to hate on the feature overload of YAML or on the lack of features of JSON, but it's hard to throw out the rich ecosystem along with them they already have established. One format I personally like is JSON5 (https://json5.org/ https://json5.org/) as it's very much just JSON, but with some more modern JavaScript (ES5) syntax allowed, including comments. Looks like a parser / serializer for it is also still rather consise. Although, I'm always wondering which features of YAML to best not use / touch. My personal approach would have been to leave some advanced features to optional pre-processors or real programming / template languages.
janjones 5y ago
Complex configurations should simply use configuration programming languages like Lua. They can start as simple as JSON/YAML but stay flexible as the complexity grows thanks to functions, references, etc. Parsing them should be also easy (configuration languages are meant to be embedded in others, just like JSON/YAML).
mrloba 5y ago
How about not using configuration files in the first place? Instead expose a library in some programming language (preferably typed) to assist with writing config, then serialize the output in whatever format is easier to parse.
kumarvvr 5y ago
I always wondered, why XML is not used by developers. I use it for my programs. The combination of attributes and data within nodes, to me, has been very useful.
- bschwindHN 5y ago
  It's a slog to write, harder to parse, and you have to make more decisions (do I put this in the node's attribute, or as a child node?). Maven had pom.xml files for declaring dependencies. Very few developers enjoyed editing those, and you don't see many XML-based config files these days for a reason.
  - Bilal_io 5y ago
    I've seen devs that came from the XML era treat json the same way, and I hate it. Exanple: { "key": "the key", "Value": "the value" }
BFLpL0QNek 5y ago
I find YAML an incredibly difficult configuration format to write by hand for anything but the simplest of files. Agree it’s a sterilisation format and not a great one. Likewise for JSON. No one can say setting up a webpack config is simple which uses JSON. It’s a recent trend. I hate working with the Kubernetes ecosystem as it’s lines of YAML config. At the same time I love working with OpenBSD / Linux. OpenBSD especially, the config files are simple and concise. TOML is my preference if I need a config file, it’s easy to read/write and reasonably expressive.
- nanis 5y ago
  Specifically, neither YAML nor JSON are configuration formats (despite everyone's desire to use them as such). TOML is a very good generalization of a simple, straightforward configuration file format which was the "best" until TOML came around. If a service you are using needs JSON/YAML, you can generate it. But, stick with a configuration file format that is suitable for humans for actual configuration. Also, PSA, neither JSON nor YAML files are good places to embed logic. Also, another PSA, the fact that Pipfile is just a TOML file is just silly. Another point in Ruby's favor.
- kangalioo 5y ago
  I think JSON is an easier configuration format than YAML. YAML is so complicated when you include all the edge cases. For example I still have no idea when you need a leading `- ` and when it's not needed. I'm not saying YAML can't be learned, but why learn all the quirks of a new format when we can just use JSON which everyone already knows? It does the same job, especially when you use JSON5 or HJSON. It's still intuitive, also supports comments and multiline strings etc, and you don't have to deal with the mess that is YAML syntax. If only...
no_wizard 5y ago
I know that this won’t likely be a popular opinion but I really like the MSBuild configuration way with xml and interpolation. I have found I can even reason about fairly complex configurations without too much hassle. You can also debug them which is great. It’s not that it doesn’t have its warts it’s that it really does a good job for what it’s mean to: semi-programmable configuration. It’s also extensible. However, it is of course extremely proprietary
mbrodersen 5y ago
Here we go. The never ending story of inventing new syntax to configure software. Because that somehow brings value to what you are trying to accomplish with the software? Really? But why stop there? Why not invent a whole new programming languages to make writing your “To Do” single user web application easier? That makes as much sense to me as spending time thinking about your configuration file syntax.
specialist 5y ago
What about tabular data? All these goofy syntaxes encode groves, meaning trees annotated with key/value pairs. XML, JSON, YAML, etc. Pretty good for serializing object graphs and so forth. What syntax (sugar) is good for tabular data too? My persona grove syntax is (secretly) awesome. Think JSON superset, like HOCON, HJSON, and SCON, but with more scalar data types, tweaked ergonomics, and conveniences. But for the life of me, I still don't have an obvious syntax extension for tables. Like inlining CSV. Or how markdown does tables. (Nested arrays for representing a matrix sucks.) The One True Syntax to unify groves and tables could really benefit all these big data (NumPy) and note keeping (Notion, Roam) projects.

71 points by signa11 5y ago

The solution for defining a more complicated config is to write your own config file format? I would think a custom format wouldn't necessarily be easier for others to read and write unless it came with a guide/readme, but then that's just one more thing to learn. Admittedly, I've never had to write a super complicated config file, but can anyone tell me why I shouldn't continue to use something like JSON for all of my config files?

beached_whale 5y ago
Let’s not forget that almost every JSON parser optionally supports comments and trailing commma’s when parsing.
- simonw 5y ago
  Which ones do that? I've not encountered many myself.
  - Zamicol 5y ago
    And JSON5. https://github.com/json5/json5-go https://github.com/json5/json5-go
    - simonw 5y ago
      What a delightfully readable example of a hand-rolled lexer and parser that is.
      - jimmygrapes 5y ago
        I thought you were being sarcastic, but I think I just finally learned how lexers and parsers work by reading the code, and I don't even know Go.
        simonw 5y ago
        Hah yeah no sarcasm intended at all! It's really neat code.
  - thayne 5y ago
    Ruby's supports comments. But that's the only one I know of
  - beached_whale 5y ago
    * C# - builtin via JsonCommentHandling, JsonSerializerOptions.AllowTrailingCommas - Newtonsoft via CommentHandling, trailing out of the box * C++ - Boost.JSON/RapidJSON/DAW JSONLink have options for both, nlohmann has an option to allow comments * Python, I only know doing it via jsmin first * Javascript, use jsmin also These are the ones I have used or seen. It comes up so often that parser writers get issues over and over until they support them. nlohmann did not for years. But people want them for config files and such. Them being an extension is nice in that they are not officially supported and how they are handled isn't specified.
u801e 5y ago
Compared to yaml or ini formats, JSON is arguably harder to read and definitely harder to write due to having to keep track of brackets, commas, and/or quotes. Ideally, a configuration file should be easy to read and write/modify by a user of the application. A lot of applications just stick with ini like formats because it meets both requirements.
- secondcoming 5y ago
  Maybe it's because I've written C++ for years, but give me brackets over significant whitespace any day. If anything, YAML is harder to read because the indentation is important, yet deliberately invisible.
  - Aloha 5y ago
    Agree. Whitespace with inconsistent rules is awful.
- benjarrell 5y ago
  For me, the brackets, commas, and quotes are what make json readable (and writable). I really struggle to write yaml. I end up writing it as json and converting it. I agree about ini though.
  - HelloNurse 5y ago
    JSON is unpleasant (for me, more unpleasant than XML), but at least it doesn't attempt to be clever like TOML and YAML.
  - dragonwriter 5y ago
    > I really struggle to write yaml. I end up writing it as json and converting it. Since valid JSON is also valid YAML with the same semantics, it is impossible for it to be harder to write YAML than JSON, and no conversion is necessary.
- bwship 5y ago
  Not to mention being able to have comments in YAML. That helps make the file much clearer as well.
thayne 5y ago
Two big reasons you shouldn't use json for config: comments and multi-line strings. json supports neither, although there are many other simple formats that do.

simonw 5y ago

After resisting it for many years I've finally settled on YAML as my default configuration format. I need a way of expressing the core data types of JSON (key/value mappings aka "objects", arrays, strings, integers, floating points and booleans) - plus comments, and multi-line strings that insulate me from complex escaping rules. YAML does that. It's not perfect, but it's good enough. And in Python I can use yaml.safe_load() to avoid some of the more troublesome corners of the spec.

meowface 5y ago
YAML seems to be one of the most polarizing things in the software industry. I'm personally a big fan of it, though I agree with the author that it becomes a pain once you start trying to use it like a DSL or templating language. I use it in pretty much every project, but I've worked with people who've said they hate it with a burning passion. And it's kind of a trope to see anti-YAML manifestos on HN and /r/programming fairly regularly, both as article submissions and comments. As you mentioned, one of the most useful features for me is the ability to write raw strings with single quotes (or by omitting quotes). I've been in many situations where and I and others have had to maintain regular expressions in config files, and it's so much more of a hassle with JSON due to the escaping requirements. The opening of the article states "Hot take: YAML isn't a configuration language or a configuration language format, it's a serialization format."; I have the exact same opinion, but with "YAML" swapped for "JSON". (And maybe the author would agree with that, too.)
- amirkdv 5y ago
  > it becomes a pain once you start trying to use it like a DSL or templating language. I like YAML and use it a lot. But I'd argue the fact that it even allows/encourages you to do such things with it is a design flaw that ultimately has kept it from Do One Thing Well.
  - meowface 5y ago
    I do think some of the more "advanced" features are definitely a mistake. StrictYAML (https://github.com/crdoconnor/strictyaml https://github.com/crdoconnor/strictyaml) is a limited, much saner subset of YAML that I wish people would use more: https://hitchdev.com/strictyaml/features-removed/ https://hitchdev.com/strictyaml/features-removed/ No coercion of y/n/yes/no/on/off to booleans (these were also removed in the official YAML 1.2 spec, thankfully), no direct object representations, no anchors or references, etc.

derefr 5y ago

I think I get what the article is saying. I don't think they expressed it very well. Here's a concrete example of what I think they're complaining about. From a Kubernetes Deployment manifest: - name: DATABASE_URL valueFrom: secretKeyRef: name: db-conn key: uri That valueFrom.secretKeyRef.{name, key} structure is an "exploded AST" — something that in any programming language, you'd be chastised for writing out as a structural-initialization literal, because there's so much stuff there that it's easy to screw it up and forget something. When attempting to express such a literal in most-any programming language, you'd expect to either have: • the Java "literal factory function" approach — some static function to call, that takes either other literals, or a single literal string encoding an expression in a DSL, as arguments, and then produces an initialized value-object; or • the Elixir "custom sigil" approach (there's probably a more popular language with this feature, but Elixir is what I know) — a macro or operator that takes in a raw string or lexemes encoding an expression in a DSL, and then codegens out to the appropriate structural-initialization literal. (Macros are better here, as incorrect DSL syntax can be caught at parse/compile time, just like incorrect syntax of the surrounding language.) ----- To be honest, I don't think I would want YAML to support either of these. YAML shouldn't be arbitrarily extendable; a YAML document should be a YAML document, parsed by any compliant parser. (There could be some Avro-like "embedded schema" format for YAML that enables this, but that format would be better as a layer on top of YAML, rather than being part of YAML itself.) What I would like in YAML, is support for built-in literal sugar for certain specific YAML structures. This is different in concept from YAML having support for a type, where that type must have some 1:1 native type on the host language side for it to encode/decode to. Instead, a "sugared structure" would involve 1. a YAML parser having a sub-parser for certain specific DSLs, where the output of this parser is a small, standardized-in-shape container structure, made out of regular YAML arrays, dicts, and scalars; 2. a YAML generator offering a configuration option to pattern-match on these standardized-in-shape structures, where recognized subtrees are swapped out in the emitted YAML for the appropriate sugared-literal representation. Examples of where this would be helpful include: DateTimes, URIs, UUIDs, Intervals... and that's basically it. There really aren't too many of these. The short list above represents basically the entirety of the set of types I've ever had YAML fall down on in the 10 years I've been generating/parsing YAML documents. For everything else, it works fine.

dangoor 5y ago

I think Cue[1] is a much more powerful and useful config format that also has the ability to generate validated JSON or YAML if needed. [1]: https://cuelang.org https://cuelang.org

fanf2 5y ago
CUE was recently discussed at https://news.ycombinator.com/item?id=28127951 https://news.ycombinator.com/item?id=28127951
contravariant 5y ago
Not sure if 'much more powerful' is a desirable feature for a configuration file language.
- Spivak 5y ago
  The ideal configuration format is really powerful and expressive for the author but resolves to a dead simple data structure for the consumer. If you need to template your config files for anything other than supplying external data it’s not expressive enough. I think YAML is closer than many of its contemporaries to this.

tetha 5y ago

Hm. I can see where the article is coming from. Some configuration files grow beyond classical configuration and end up being more like programming. With configuration being "Put the right connection / path strings into the program, enable some subsystems/feature toggles" and programming being along the lines of, e.g., arbitrary metric transformations in a metric collector, or programmable ACLs. With some systems, I very much end up wondering why I couldn't just load lua into the system and do the transformations with that. However, configs in json/toml/yaml come with a lot of tooling. With a custom configuration language, you suddenly need to roll your own syntax highlighting in many editors, maybe linters, pretty-printers, ... Your config can't just be computed by a configuration management system and deployed via `computedConfig | toJSON`, instead it'll be necessary to wrangle templating. And users generally know those languages. And writing parsers is a pain. And generating parsers is also a pain. And using bad parsers is also a pain. I've worked too much on parsers. Those are some pretty big hurdles to overcome and if you don't, the user experience just starts off worse.

kzrdude 5y ago
yaml (and json) gives configuration files much-needed hierarchy and structure, which is unbounded (within reason), not limited syntactically or inherently like simpler configuration formats including toml. yaml adds to that comments and a pleasant readable layout, even if it requires a little bit more from the editor to make it pretty. With that, I don't think yaml has much competition.
- tetha 5y ago
  Hm, again. I don't think I agree it's that clearcut. I've found that if you mostly want a tree of simple key-value assignments, such as database.postgres.user or database.mysql.timezone, TOML is actually very hard to beat. It's extremely simple to teach to less technical coworkers. TOML mostly becomes somewhat strange if you need lists, or maps with free-form keys. Also, YAML is horrible if you have to generate it. Pretty much everyone automating stuff I've met will prefer JSON over YAML once you need to automatically generate the configuration. Pushing some data structure through an as-yaml pretty printer usually works, until it breaks in really weird incompatibilities between YAML parsers. And that happens way too much to be a fun distraction.

thangalin 5y ago

My text editor, KeenWrite[0], integrates YAML using a GUI, which hides the underlying data format[1]. This makes the format irrelevant from the end user's perspective. Hardly anybody edits .odf files or .png files directly, yet configuration files are often updated in a plain text editor rather than a GUI. Structured data formats are amenable to autogenerating GUIs (see [2] and [3]). [0]: https://github.com/DaveJarvis/keenwrite/blob/master/docs/screenshots.md https://github.com/DaveJarvis/keenwrite/blob/master/docs/scr... [1]: https://youtu.be/u_dFd6UhdV8?t=160 https://youtu.be/u_dFd6UhdV8?t=160 [2]: https://www.jeremydorn.com/json-editor https://www.jeremydorn.com/json-editor [3]: http://mb21.github.io/JSONedit/ http://mb21.github.io/JSONedit/

a-b 5y ago

This is why you should consider https://carvel.dev/ytt/ https://carvel.dev/ytt/

dub 5y ago

There are a variety of generic DSLs for encoding configuration concisely with composability, functions, custom validation, etc. Some examples would be jsonnet, dhall, and cue. If the configuration needs to be transformed into a more computer-friendly format like json or yaml for later loading by a binary, that's easy enough to do at build time in a modern-ish build system like bazel

jedimastert 5y ago
If you're gonna be generating at build like that, you might as well be using a binary encoding like protos (this is for devs, btw)

louib 5y ago

Currently, my main concern with YAML is that, by the spec, comments are not attached to a particular node (see https://yaml.org/spec/1.2/spec.html#id2767100 https://yaml.org/spec/1.2/spec.html#id2767100). As a result, a lot of YAML parsers (like https://github.com/yaml/libyaml https://github.com/yaml/libyaml and https://github.com/chyh1990/yaml-rust https://github.com/chyh1990/yaml-rust) only filter out the comments during the parsing phase. This makes it less than ideal for a use-case where the configuration file is expected to be modified by both programs and humans. TOML makes it more trivial to associate comments with a node. This is mainly because the language is simpler though, as the spec is not explicit about that (https://toml.io/en/v1.0.0#comment https://toml.io/en/v1.0.0#comment).

mixmastamyk 5y ago

Look into strict yaml, a simpler, safer subset: https://hitchdev.com/strictyaml/ https://hitchdev.com/strictyaml/

seized 5y ago

A while back it took myself, a data center engineer, and two engineers from the appliance vendor a solid 20 minutes to figure out the problems with a ~12 line yaml file that needed two or three lines added (network config file). Between syntax, indentation, etc it required a few tries. Plus there was a copy and paste in there via SSH. I don't know what a better alternative is but yaml can be incredibly frustrating.

oweiler 5y ago
The problems you describe wouldn't exist when using XML with a schema.
- thayne 5y ago
  Maybe, but XML has a whole host of other problems.

toiletaccount 5y ago

when i see yaml, i use other software

thayne 5y ago

> YAML isn't a configuration language or a configuration language format, it's a serialization format I think YAML is even worse as a serialization format than a configuration format. It has too many ambiguities, implementations are too inconsistent, and many implementations are insecure by default with untrusted input. > Using a good custom designed configuration file format instead of trying to shove things through the narrow pipe of YAML means that you have one integrated syntax that can be designed to be more readable, more expressive, and much easier to write. I won't say a custom configuration is never the right thing to do, but I do think it rarely is. Writing a good custom configuration language is very difficult to get right, not to mention you also need syntax highlighting and other editor support for the editors your users use, libraries for popular languages so that other programs can read and write the configuration, etc. If you need something this complicated, I'd reach for using an embedded scripting language like lua (or possibly JavaScript or python) before writing a new format from scratch.

amirkdv 5y ago
> I think YAML is even worse as a serialization format than a configuration format. This. I find YAML to be the least offensive option for configuration and one of the worst for serialization. I might be misinformed, but I find it absurd that in 2021 we still don't have a default, universally available tool that supports the basic table stakes without headache: 1. core data types (number, string, etc) 2. lists, maps, and arbitrary nesting 3. comments 4. multiline strings 5. is acceptably readable 6. Just Works everywhere YAML does get 1-5 right (specifically 3 and 4 that JSON doesn't and IMO better in 5). But then it adds a ton of complexity that has left us without a standard, safe, and sane parser implementation: anchors and references (& and *) , casting (via !!), custom data types (via !), loads of other things I don't understand.
- brundolf 5y ago
  It's a real shame JSON doesn't have 3 and 4, because it would be so easy and it's otherwise pretty much perfect imo. No ambiguity, every value type can be identified by its first character, clean and reasonably minimal syntax.

Aloha 5y ago

My primary gripe with YAML is white space sensitivity, it makes editing it in a hurry in a CLI environment needlessly user unfriendly. Virtually any other option is better for the person who actually has to modify these configurations later. If your configuration is expected to be hand modified directly by a user, it shouldn't use YAML.

Fannon 5y ago

YAML (and JSON of course) has the big advantage, that you can use JSON Schema to validate it and to provide realtime code intelligence in many Editors (VScode, IntelliJ, Monaco Editor in the web, many Web IDEs. Allthough you might need a plugin for YAML, where JSON works out of the box). So it's easy to hate on the feature overload of YAML or on the lack of features of JSON, but it's hard to throw out the rich ecosystem along with them they already have established. One format I personally like is JSON5 (https://json5.org/ https://json5.org/) as it's very much just JSON, but with some more modern JavaScript (ES5) syntax allowed, including comments. Looks like a parser / serializer for it is also still rather consise. Although, I'm always wondering which features of YAML to best not use / touch. My personal approach would have been to leave some advanced features to optional pre-processors or real programming / template languages.

janjones 5y ago

Complex configurations should simply use configuration programming languages like Lua. They can start as simple as JSON/YAML but stay flexible as the complexity grows thanks to functions, references, etc. Parsing them should be also easy (configuration languages are meant to be embedded in others, just like JSON/YAML).

mrloba 5y ago

How about not using configuration files in the first place? Instead expose a library in some programming language (preferably typed) to assist with writing config, then serialize the output in whatever format is easier to parse.

kumarvvr 5y ago

I always wondered, why XML is not used by developers. I use it for my programs. The combination of attributes and data within nodes, to me, has been very useful.

bschwindHN 5y ago
It's a slog to write, harder to parse, and you have to make more decisions (do I put this in the node's attribute, or as a child node?). Maven had pom.xml files for declaring dependencies. Very few developers enjoyed editing those, and you don't see many XML-based config files these days for a reason.
- Bilal_io 5y ago
  I've seen devs that came from the XML era treat json the same way, and I hate it. Exanple: { "key": "the key", "Value": "the value" }

BFLpL0QNek 5y ago

I find YAML an incredibly difficult configuration format to write by hand for anything but the simplest of files. Agree it’s a sterilisation format and not a great one. Likewise for JSON. No one can say setting up a webpack config is simple which uses JSON. It’s a recent trend. I hate working with the Kubernetes ecosystem as it’s lines of YAML config. At the same time I love working with OpenBSD / Linux. OpenBSD especially, the config files are simple and concise. TOML is my preference if I need a config file, it’s easy to read/write and reasonably expressive.

nanis 5y ago
Specifically, neither YAML nor JSON are configuration formats (despite everyone's desire to use them as such). TOML is a very good generalization of a simple, straightforward configuration file format which was the "best" until TOML came around. If a service you are using needs JSON/YAML, you can generate it. But, stick with a configuration file format that is suitable for humans for actual configuration. Also, PSA, neither JSON nor YAML files are good places to embed logic. Also, another PSA, the fact that Pipfile is just a TOML file is just silly. Another point in Ruby's favor.
kangalioo 5y ago
I think JSON is an easier configuration format than YAML. YAML is so complicated when you include all the edge cases. For example I still have no idea when you need a leading `- ` and when it's not needed. I'm not saying YAML can't be learned, but why learn all the quirks of a new format when we can just use JSON which everyone already knows? It does the same job, especially when you use JSON5 or HJSON. It's still intuitive, also supports comments and multiline strings etc, and you don't have to deal with the mess that is YAML syntax. If only...

no_wizard 5y ago

I know that this won’t likely be a popular opinion but I really like the MSBuild configuration way with xml and interpolation. I have found I can even reason about fairly complex configurations without too much hassle. You can also debug them which is great. It’s not that it doesn’t have its warts it’s that it really does a good job for what it’s mean to: semi-programmable configuration. It’s also extensible. However, it is of course extremely proprietary

mbrodersen 5y ago

Here we go. The never ending story of inventing new syntax to configure software. Because that somehow brings value to what you are trying to accomplish with the software? Really? But why stop there? Why not invent a whole new programming languages to make writing your “To Do” single user web application easier? That makes as much sense to me as spending time thinking about your configuration file syntax.

specialist 5y ago

What about tabular data? All these goofy syntaxes encode groves, meaning trees annotated with key/value pairs. XML, JSON, YAML, etc. Pretty good for serializing object graphs and so forth. What syntax (sugar) is good for tabular data too? My persona grove syntax is (secretly) awesome. Think JSON superset, like HOCON, HJSON, and SCON, but with more scalar data types, tweaked ergonomics, and conveniences. But for the life of me, I still don't have an obvious syntax extension for tables. Like inlining CSV. Or how markdown does tables. (Nested arrays for representing a matrix sucks.) The One True Syntax to unify groves and tables could really benefit all these big data (NumPy) and note keeping (Notion, Roam) projects.