23 ms·

YAML: Probably not so great after all

• Vosporos 7y ago
  Just go with Dhall and be done with that
• jbaudanza 7y ago
  These are all valid points. But I still find YAML to be the best format for storing my strings for localizations. I find it much easier on my eyes than JSON. I’m open to other suggestions though.
  • rc_mob 7y ago
    Per the article consider Toml
• QuinnyPig 7y ago
  If the JSON and YAML folks can’t get along, I swear I’ll turn this car around and make you all use XML.
  • CameronNemo 7y ago
    inb4 YAML adds "Yeah, nah." and "Nah, yeah." as boolean values. Interpretation is locale dependent.
    • crdoconnor 7y ago
      That's why StrictYAML always interprets as string unless there's a schema. Cuts out the surprise type conversions.
    • Master_Odin 7y ago
      YAML 1.2 fixed this making booleans just true/false and it's a real shame that a lot of things still use 1.1
      • wrs 7y ago
        I’m sure Christopher True and Robert False really appreciate that “fix”.
        • reneberlin 7y ago
          So true. Your comment made my day.
      • erik_seaberg 7y ago
        I don't like NO as a boolean literal (mostly because it's also an ISO country code), but there's only one right way to parse it in a YAML 1.1 document. It was a mistake to apply the 1.2 rules to 1.1 documents and issue a "warning" about incorrect output, rather than require rejecting the document if the 1.1 rules aren't implemented.
    • EdwardDiego 7y ago
      I'd also like "Sweet" to be true and "Stink" to be false.
  • sipos 7y ago
    To be fair, the article begins by linking to a previous article by the same author about the problems with JSON. He is an equal opportunities opponent :)
  • jackfraser 7y ago
    Aren't we just reinventing the wheel, though? Got your structured data format, now you need parsers (tons available for XML, incl SAX, DOM parsers, SimpleXML, Nokogiri...) a schema and validation tools (XSD), a templating mechanism (XSLT), a query language (XPath), ... JSON was a reaction to the verbosity of XML, but a better reaction would have been to work harder on our text editors so that working with XML would be just as easy as working with JSON in terms of the numbers of keystrokes needed. Better parser interfaces that help you treat the dataformat more like it's part of the language would also help (i.e. SAX and DOMDocument suck to work with, but SimpleXML is almost idiomatic).
    • saghm 7y ago
      > JSON was a reaction to the verbosity of XML, but a better reaction would have been to work harder on our text editors so that working with XML would be just as easy as working with JSON in terms of the numbers of keystrokes needed. Isn't that only solving half the problem? XML is also pretty difficult to read
      • EGreg 7y ago
        Look. I am just a web guy. But why is XML so freaking great? We can’t even tell if whitespace is significant or not. If a schema says it’s insignificant then that’s that! https://www.oracle.com/technetwork/articles/wang-whitespace-092897.html https://www.oracle.com/technetwork/articles/wang-whitespace-... That alone is TERRIBLE! (Same problem with YML.) Why should I bother with that? JSON can encode strings, hashes, arrays etc. in a way that’s instantly interoperable with JS and is far far more unambiguous. What exactly is so great about XML that you can’t do with JSON in a better way? Schemas can be stored in JSON. XPATH can specified for JSON. Seriously I never got the appeal of XML except that it was first.
        • tannhaeuser 7y ago
          XML wasn't meant as replacement for JSON, but for HTML without vocabulary-specific parsing rules (eg. SGML DTDs).
        • 7y ago
  • quotemstr 7y ago
    You say that like it'd be a bad thing. I like XML.
    • _eht 7y ago
      You say that like you’ve never really used XML... (Mostly /s. Come at me:))
      • quotemstr 7y ago
        I've used it quite a bit. I even like the namespacing bits. I find that XML composes elegantly in a way that the JSON and friends don't. My one request would be to bring back to SGML-like closing tag abbreviation: That is, instead of <foo><bar>qux</bar></foo> we should be able to write <foo><bar>qux</></> I think this one change would make XML more "palatable" for the JSON/YAML/TOML crowd.
        • _eht 7y ago
          Interestingly I find SGML-esque to be quite unpalatable. I get the feeling doing code reviews would be nightmarish.
          • quotemstr 7y ago
            Why? It's no worse than S-expressions.
            • WillPostForFood 7y ago
              The verbosity makes it harder to parse. It is subjective, but I find ")))" is a lot easier to instantly parse as 3 than "</></></>"
            • _eht 7y ago
              It’s significantly worse than single char s expression for my eye, anyway.
        • deleted 7y ago
  • yakubin 7y ago
    I'd prefer XML because of the stability of the tools available for it. Recently I was writing a custom static site generator for my website. I started with python and yaml using the pyyaml lib. After two months (don't laugh, I wasn't writing this generator all this time; i had a break) I tested if everything I wrote previously was warking. Pyyaml came at me screaming that they deprecated something and I shouldn't use it, otherwise the feds will get me. Let's go several months earlier still. I was learning Python, using Debian Stretch which has Python 3.5 installed. The book I was learning from used Python 3.7. When I got to a point it became clear that 3.5 lacks a few things which are needed to continue learning Python according to the book. So I compiled Python 3.7, set up virtualenv with it and... The code I had previously written stopped working. With a version bump from 3.5 -> 3.7? That's a minor version change. And now 3.5 was expecting at one point to have a string path passed as an argument, and 3.7 was expecting a pathlib path. That was trivial to fix in case of a small example, but I would dread using such a thing for anything big and then having to debug what exactly broke between different (minor!) versions of Python or its libraries. These new hip tools seem to have a backwards-compatibility issue. I eventually settled on using XSLT and a couple really short shell scripts (which all fit on my screen at the same time) and I don't expect them to break in the next two decades. However, XML is still a pain and I would prefer just using S-expressions and Lisp[1]. It's just that for now my only experience with them is writing things for Emacs and I would like to learn Scheme/CommonLisp to do anything outside of Emacs with Lisp. [1] https://sites.google.com/site/steveyegge2/the-emacs-problem https://sites.google.com/site/steveyegge2/the-emacs-problem
    • pietroglyph 7y ago
      This sounds more like a Python and pyyaml problem than a YAML or JSON problem. Someone could come along and totally refactor XSLT even if XML stayed the same, and then you would be in the same boat.
      • ColanR 7y ago
        Parent addressed your comment in his first sentence. > I'd prefer XML because of the stability of the tools available for it.
  • rhizome 7y ago
    I have to wonder if these stories are written by people who simply hate to type, or aren't good at it. Beyond that I can only chalk it up to academic curiosity.
• leoc 7y ago
  XML is as pleasant to look at or touch as a nettle rash, but it seems it can join ALGOL 60 among the ranks of technologies which were a great improvement on their successors.
  • gweinberg 7y ago
    Umm, no. You can find cases where JSON sucks, but you have to look for them. You can find cases where XML doesn't suck, but you have to look for them.
    • michaelmrose 7y ago
      Other than looking ugly and being a pain to type does xml actually suck?
      • jsjohnst 7y ago
        Depends on the XML. When you start mixing in namespaces (like trying to parse Maven pom.xml files in Python), it quickly becomes a mess.
        • jamougha 7y ago
          Namespaces are horrible in Python because the Python XML libraries are deficient. They're generally fine in e.g. Java. (Unless you're trying to represent QName typed data, but that's very niche.)
          • jsjohnst 7y ago
            I used Python as an example, but handling namespaces in most languages besides Java (and even arguably in Java depending on your point of view) is rather painful.
      • tomc1985 7y ago
        I'd say the same about JSON!
• rc_mob 7y ago
  Heh, yaml haters unite! I wish yaml would go away.
  • yjftsjthsd-h 7y ago
    For what use cases, and what would you prefer?
  • jeltz 7y ago
    I hate yaml but I have yet to find a better option for deeply nested confog files. Toml is the closest thing I have seen. Toml support is also not that great. Yaml despite its flaws works pretty well for Ansible playbooks and for storing localizations.
    • zamadatix 7y ago
      Would you say actual YAML does a better job in Ansible over just writing JSON and letting it be parsed as YAML?
      • crdoconnor 7y ago
        Have you tried writing JSON by hand or diffing it in a pull request?
        • geerlingguy 7y ago
          Or commenting JSON?
        • zamadatix 7y ago
          Err, yes? Is quoting your keys, delimitation members with a comma instead of whitespace, and putting brackets/braces around collections really that confusing that people struggle to edit it by hand or read it in a diff? I think the syntax is actually what makes it more human readable, it's still 95% text/numbers just annotated with information that makes it clear what things actually are instead of hiding them behind confusing computer parsing rules nobody is going to think about while reading human-friendly text.
• ozten 7y ago
  Another one: Parsing partial YAML files doesn’t detect an error with loading the complete file. We’ve had a production outage, because of large yaml files getting cutoff and not all settings getting loaded into our server. JSON or XML typically will not parse.
  • CameronNemo 7y ago
    YAML files should have an explicit start point and end point, respectively `---` and `...`. yamllint will enforce this, although I have never used these for integrity checks. https://yamllint.readthedocs.io/en/stable/rules.html#module-yamllint.rules.document_end https://yamllint.readthedocs.io/en/stable/rules.html#module-...
  • ailideex 7y ago
    Is this not an issue with a parser rather than with YAML?
    • afiori 7y ago
      Unreliable parsers are an issue of yaml.
      • ailideex 7y ago
        Why?
• fabian2k 7y ago
  I've used YAML as the format for a config file, and I certainly regret that choice. Trying to explain to someone that doesn't know YAML how to edit it without setting them up for failure is quite annoying. There are too many non-obvious ways to screw up, like forgetting the space after the colon or of course bad indentation.
  • ljm 7y ago
    I’m not keen on how so many tools and services opt for YAML by default, either. Both JSON and YAML are a nightmare to handle once you’ve got 3000 line files and several layers of nesting. CI would be a lot nicer to use if it didn’t rely on a single YAML file to work. And if you want to switch, suddenly you had a build step to convert back to YAML.
    • js2 7y ago
      I keep my YAML CI files as minimal as possible by putting the logic into a Makefile and/or shell scripts and just have the YAML invoke that.
  • booleandilemma 7y ago
    Giving meaning to whitespace causes so many headaches and yet people still embrace Python, for some reason. I don’t understand it.
    • whatshisface 7y ago
      Your editor makes a world of difference here. Since you shouldn't be writing brace-language code without indents anyways, the biggest issue remaining is mixing tabs and spaces. Gedit makes this a big pain with it's default config (it doesn't even auto-indent) but Atom and IDLE handle it well.
      • kevin_thibedeau 7y ago
        Python 3 rejects mixed whitespace so the problem will be caught quickly.
• vlozko 7y ago
  I think the author’s conclusion is in line with my own thought: If JSON is the problem, YAML isn’t the solution. I recall the first time I saw YAML and all I could think to myself was that I have to learn yet another syntax. I find it far less readable than JSON or XML and made me pine for the latter.
• abakus 7y ago
  YAML is horrible. Toml is much better. Even Json is not that messy.
• airencracken 7y ago
  https://noyaml.com/ https://noyaml.com/ YAML is bad. Every YAML parser is a custom YAML parser. https://matrix.yaml.io/valid.html https://matrix.yaml.io/valid.html
  • traderjane 7y ago
    Oh Puppet, why did you use your own executable YAML.
  • takeda 7y ago
    The problem is with parsers, how they are implemented or used. YAML actually has a way to specify type of the data, alternatively the application supposed to suggest desired type. What's this take is showing is what types are assumed when they are not specified.
• crazygringo 7y ago
  So what's the HN consensus on the best format for config files? Is it TOML as the author seems to prefer at the end?
  • philwelch 7y ago
    My vote is yes. Most configuration doesn’t need anything more sophisticated than key-value pairs, perhaps with namespaces. INI can manage that and TOML is basically a better-specified INI.
    • hardwaresofton 7y ago
      I can't tell if I've spent too much time on HN or if I came to this conclusion on my own, but TOML is my language of choice for configuration now. It's flexible in the right ways and sectioning of config is so important.
    • h1d 7y ago
      No reason to use INI over TOML. INI doesn't even have a standard specification.
      • philwelch 7y ago
        That’s my point, yeah. INI is the right idea but TOML is the same thing but actually specified so use it.
  • bmurphy1976 7y ago
    There's no white Knight here, they all suck in some way. Personally I've had decent success with yaml as simple configuration, but I would never use it as an interchange format. If you know it's caveats and you're targeting one language so you can become familiar with the parser it's serviceable.
  • majewsky 7y ago
    If possible, prefer what tools in your vicinity use. My team uses Kubernetes and Concourse extensively, which both use YAML, so I tend to stick with YAML since people are already familiar with it. (More recently, I've come around to prefer plain environment variables for configuration, but that only works nicely when the amount of configuration is fairly limited, say 20 values instead of 1000 values.) For my own use, I do prefer TOML.
• breck 7y ago
  Disclosure: I work on Tree Notation. It’s the future of file formats, IMO. The idea is to have 2 levels: a simple, minimal syntax/notation (think binary) called Tree Notation, and then have higher level grammars on top of that, called tree languages. It works for encoding data and also for programming languages, regardless of paradigm. https://github.com/treenotation/jtree https://github.com/treenotation/jtree
  • philwelch 7y ago
    This sounds a lot like s-expressions.
    • breck 7y ago
      Thinking of it as s-expressions without parens is a very good analogy. That ends up making a huge difference.
  • atombender 7y ago
    Your project looks interesting, but I looked through the Github project and site, but I couldn't find a language specification, reference manual, BNF-like grammar, or anything to indicate what the syntax is, beyond a very trivial example in the Github. To be blunt, I think you need to start with a spec to get any traction. That allows people to understand your data model and particular text encoding of it. If they like it, they might use your tool and perhaps port the system to other languages.
    • breck 7y ago
      Good feedback, thank you. You may have seen the spec, it’s just quite minimal (here’s a more elaborate one: https://github.com/treenotation/jtree/blob/master/spec.txt https://github.com/treenotation/jtree/blob/master/spec.txt) Here’s a BNf: https://github.com/treenotation/jtree/issues/1 https://github.com/treenotation/jtree/issues/1 There’s a FAQ as well. Docs needs work, in particular I’m hoping people will create their own explanations of the ideas in external places, as that might be a better way to understand it. Happy to provide help to anyone that is interested in that.
• cmauniada 7y ago
  I find yaml to be the best for making cloudformation templates. On its own it isn’t much good but if you use the right plugins it really is better than json.
• botto 7y ago
  What about HCL, would it make sense to use this as a config language?
• beders 7y ago
  If you still aren't convinced YAML is terrible, try copying and pasting YAML fragments with a regular text editor. You might end up with valid YAML, but you won't know until the YAML consumer barfs. BTW, all of a sudden XML with DTDs are looking sane again :)
  • KaoruAoiShiho 7y ago
    Use the right tool for the job. I use yaml extensively but never in a situation where someone would want to edit it with a regular text processor.
    • beders 7y ago
      do you deliver a YAML editor with your software? Because people will use notepad or nano to edit that stuff.
      • KaoruAoiShiho 7y ago
        Sort of, I deliver a GUI that exports into YAML for pretty much only reading, portability, and version control. People are expected to do the editing in the GUI, only using YAML for editing when doing complex regex operations that my GUI doesn't support.
        • twic 7y ago
          If people aren't editing it by hand, why does the format matter? Why not just use JSON? Tools for too-complex-for-the-GUI manipulations are at least as good for JSON as they are for YAML, and the editing is less error-prone.
          • KaoruAoiShiho 7y ago
            Internally it is JSON. It exports as YAML for readability when sharing on discord.
• znpy 7y ago
  I've been working on a software that's eavily based on XML and in a number of occasions I've been glad XML is strict and verbose. you can quickly tell if an xml document is malformed (good parsers will tipically point you to the un-closed tag). Yaml on the other hand would probably load anyway, with the application receiving garbage data, potentially misdirecting the application behavior...
• goatinaboat 7y ago
  The superior replacement for XML, JSON and YAML is the SQLite .db file. Easy to “parse”, easy to manipulate programmatically, what more could you want?
  • potatochup 7y ago
    Annoying for diffing probably?
  • akhilcacharya 7y ago
    Not human readable...
    • theothermkn 7y ago
      > Not human readable... This refrain just cheeses me right off every time. Nothing is human readable! Everything requires a program to read it, because no human being can read states of charge or states of magnetic polarization directly. What makes something 'human readable' or not is a software tool. Underlying that tool is a data format that the tool can accept and display. What everyone means when they try to sound smart by saying 'human readable' is just 'plain text.' In other words, they know where to find the dumbest possible reader/editor for it. Text editors are the dumbest possible editors because they cannot constrain edits to conform with the grammar of the interface language; they allow bugs at a point in the development process where it is trivial to disallow bugs, especially considering that interface languages should probably be, at most, regular languages. I'll step off my soapbox, now.
      • derriz 7y ago
        I've a certain sympathy for your position but I interpret "human readable" as meaning readable and somewhat comprehensible without using specialized tools.
        • edejong 7y ago
          But when is a tool specialized? Would a dedicated XML / JSON / YAML / ASN.1+DER / Avro / Protobuf / Parquet editor be specialized? And what if that hierarchical standard would be the de facto industry standard? Is a binary file editor specialized? What about a structured assembler? Personally, I think most editors are rather specialized. They deduce the character sets, often add syntax highlighting and provides paging for very large files. High level strongly typed languages, such as modern C#, Java, Scala are designed to be used in an IDE. You could view and edit it, but it provides a difficult situation (not unlike editing XML by hand). "Human readable" is very subjective. It depends on the person, the task at hand, the intended recipients, etc. etc.
• johnisgood 7y ago
  S-Expression or TOML! I personally use TOML for all my projects' configuration file.
  • boobePhuu7iet7i 7y ago
    TOML is so much easier to read IMO
• al_form2000 7y ago
  As an ansible user, I hate YAML and its broken parsers with a passion, but the security objection does not make much sense. It does apply verbatim to any parser of anything if the implementation decides that a given label means "eval this content right away". I fail to see how this can be a fault of the DDL rather than the parser's.
  • JelteF 7y ago
    The reason this is a fault of the DDL and not the parser is that the DDL spec decides that it has label that evaluates a command. The parser then has two options, either implement it or not conform to the spec (and essentially implementing a different DDL). For programming languages it makes sense to have an eval label/command. For configuration/serialization DDLs I think it's a terrible choice.
    • al_form2000 7y ago
      And terrible it is indeed, but I cannot find it specified - the strings eval, exec, command, statement do not even occur in the official specs (shallow doc perusal, I know)
      • SignalsFromBob 7y ago
        That's because there's nothing in the spec stating anything about execution. The parent is simply incorrect. That's why they haven't responded.
    • SignalsFromBob 7y ago
      > DDL spec decides that it has label that evaluates a command This is simply wrong. There is nothing in the spec stating that.
  • apple4ever 7y ago
    Funny, as an Ansible user I love YAML. It works so well for me.
  • SignalsFromBob 7y ago
• vinay_ys 7y ago
  I strongly recommend doing away with config files completely for sake of ease of use, maintainability and security. Instead just declare all config variables within code itself in a separate config class/module file, along with initialization to default values and provides dynamic getter/setter interface over a debug API (which can be enabled/disabled via a command line flag). If you want, you can also provide a friendly cli tool to interact with the debug api. This tool could output help messages, show current config values - differentiate between default vs overwritten etc. Of course, this can be written once as a utility library and cli and used consistently across all your programs.
  • k2xl 7y ago
    This breaks when you have multiple services in potentially different programming languages that need to read the same config values.
  • wwweston 7y ago
    I follow this plan in many (maybe most) app config situations, but it has at least two noteworthy drawbacks: 1) it presents an obstacle to sharing config files in a multi-language environment. 2) writing user prefs in the host app’s native language is fraught with security issues The first concern is admittedly niche, and both concerns are addressable with some care and thought, but they’re good to consider.
  • al_form2000 7y ago
    This is not going to make you many friends in the server community (think apache, mysql...)
  • johnchristopher 7y ago
    > If you want, you can also provide a friendly cli tool to interact with the debug api. This tool could output help messages, show current config values - differentiate between default vs overwritten etc. And we load the config by curling a JSON payload :) ?
  • twic 7y ago
    For the love of god, if anyone reads this, please don't do this! Config files are fantastic. Trivial to read, write, copy, track in version control, diff, grep, generate with scripts, etc. API-driven configuration has none of these properties. Some Java application servers take this approach of API-driven configuration. It's an improvement over UI-driven configuration, which is what they had before. But it's still significantly worse than simple file-driven configuration. If you want to provide a 'friendly' CLI tool, by all means do so - but provide a tool to interpret and generate config files, not something which replaces config files.
• felixfbecker 7y ago
  I'll say it: I think YAML is great and a joy to use for configuration files. I can write it even with the dumbest editor, I can write comments, multi-line strings, I can get autocompletion and validation with JSON schema, I can share and reference other values. It allows tools to have config schemas that read like a natural domain specific language, but you already know the syntax. I haven't had problems with it at all.
  • martinpw 7y ago
    This was me too - until yesterday, when I made a minor change to one of our YAML config files and everything broke. On investigation it turned out that all of our YAML files had longstanding errors but those errors happened to be valid syntax and also did not cause any bad side effects, so we had been getting away with it by pure luck until I made a change that happened to expose the problem. So now no longer a YAML fan...
    • dragonwriter 7y ago
      That would make me not a fan of the particular parsers/validators I've been using, rather than not a fan of YAML. The big strike against YAML I see there is that it needs a good conformance test suite and implementations need to be tested against it. But that's not a problem with the format but a fairly easy to fix ecosystem problem.
      • deleted 7y ago
        [deleted]
      • Izkata 7y ago
        > of the particular parsers/validators But the syntax was valid, the parsers/validators would've been correct to accept it.
  • morpheuskafka 7y ago
    I've got to say it is the most frustrating config file ever to wrote. The only time I have to use it is for Docker Compose and I am constantly fighting vim on indentation and trying to make sense of confusing errors about "unexpected block start." Do you have any suggested vimrc for YAML?
• ridiculous_fish 7y ago
  fish shell is looking for a new text serialization format for its history file (currently it uses an ad-hoc broken psuedo-YAML). Boxes to check: 1. Self describing format 2. SAX-style parser available to C++ 3. Easy for users to understand and ad-hoc parse using command-line tools 4. No document closing necessary, so appending is trivial YAML looks pretty good: - cmd: git checkout file.txt when: 1565133286 pwd: /home/me/dir/ paths: - file.txt protobuf is also an option: entry { cmd: "git checkout file.txt" when: 1565133286 paths: "file.txt" } though I am unsure of how well its text serialization is supported. Any suggestions?
  • akx 7y ago
    How about JSONL (JSON Lines)? http://jsonlines.org/ http://jsonlines.org/ Ps. Thanks for (all the) fish, it's my daily driver shell and keeps me that much more sane c.f. the alternatives.
    • j0057 7y ago
      That's really close to [RFC 7464](https://tools.ietf.org/html/rfc7464 https://tools.ietf.org/html/rfc7464), JSON Text Sequences. It uses U+001E RECORD SEPARATOR. The `jq` tool supports those if you pass a flag.
      • edraferi 7y ago
        So just add a record separator at the end of each line?
      • meowface 7y ago
        jq can also handle newline-separated JSON objects, with the -s flag.
  • mjevans 7y ago
    (suggestion) Drop the 4th requirement. Having to close contexts is a VERY good 'sanity check' to see if something is malformed or not. If appending is necessary make the parser handle multiple copies of the namespace and merge them upon output. Unknown keys and sections should also always be copied from input to output (this is how you embed comments).
• mixmastamyk 7y ago
  Yaml is great at the core. It just has too many features, the first things I disable. A simplified subset, .syml would be a good idea.
  • tjalfi 7y ago
    StrictYAML[0] is a YAML subset that removes some of the problematic features. The implementation is in Python. [0] https://github.com/crdoconnor/strictyaml https://github.com/crdoconnor/strictyaml
    • mixmastamyk 7y ago
      This should be the post then, a solution rather than griping.
• olliej 7y ago
  The security problem is common to many serialisation formats and similarly terrible bugs have happened in a large number of formats. For instance, the recent iMessage bugs that project zero announced were because NSCodable serialization tells the deserializer what class should instantiated. Followed by remote code execution (woo!) Similar problems have occurred with java serialization over the years, the python serialisation thing (that silly name I can’t recall). I was recently learning swift and was getting frustrated by the verbosity/work for deserialisaing abstract classes when I realized the clunkiness was due to a design that made the deserialise attacker specified objects basically impossible. Obviously you could engineer a solution that would be exploitable but there’s only so much a platform can do to stop developer mistakes.
• uponcoffee 7y ago
  I learned some of the intricacies of yaml the other day when refactoring a docker-compose project. At first glance, it's brilliant... Until I started running into limitations, edge cases, and issues. I like the _idea_ of yaml, but: - it's overly complicated in the wrong ways - common/simple use cases aren't supported and require post processing (i.e. Merging block maps/arrays, string interpolation, etc)
• cryptica 7y ago
  I never understood how YAML is more human-readable than JSON. I find JSON much easier to read. What annoys me the most about YAML is that it's easy to misinterpret the indentation. You need a special IDE to know whether a property belongs to a specific object or to its parent.
  • thayne 7y ago
    > I never understood how YAML is more human-readable than JSON Two things: comments and multi-line strings.
    • throwaway_391 7y ago
      My personal JSON Pet hate is: ``` x = [ "Foo", "Foo2", ] ``` Is not valid, but the following is: ``` x = [ "Foo", "Foo2" ] ``` Makes dealing with packer configs feel like punching yourself in the face. I still prefer it over YAMLs awkward initial learning curve.
      • zamadatix 7y ago
        At first I found it really annoying but then the more I thought about it the more I came to value the "," semantics as proper validation for a "forgot to put the last element in the list" error which would otherwise be silently hidden via the parser.
        • throwaway_391 7y ago
          So your comment is vaild. Having strict and not-strict validation would be a nice compromise though (:
  • meowface 7y ago
    What? No you don't. You don't need a special IDE any more than you need a special IDE to know if a Python statement is in an "if" block or a parent "if" block.
• borntyping 7y ago
  The first example (i.e. `yaml.load()` in Python) doesn't work with the current version of PyYAML. Function application was disabled some time ago, and `yaml.load()` logs a noisy deprecation warning telling users to use `yaml.safe_load()` instead [1]. [1]: https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-...
• Waterluvian 7y ago
  I just want json with comments. Is that too much to ask?
  • worble 7y ago
    https://json5.org/ https://json5.org/
  • zeroimpl 7y ago
    I just want mainstream/built-in JSON parsers to be able to ignore comments and trailing commas. What ever happened to lax parsing?
  • roryokane 7y ago
    Someone else already mentioned JSON5 (https://json5.org/ https://json5.org/), which is JSON with a few ergonomic improvements, including comments. Hjson (https://hjson.org/ https://hjson.org/) is a similar, slightly more complex format with a few extra features such as unquoted strings for object values.
• DannyBee 7y ago
  1. General purpose serialization format is released. 2. Format is declared to have x and y problem, new format is invented that is "simpler and better" Time passes 3. People slowly discover format in #2 has the same issues that led to creating #1. Repeat. (Just like attempting to super-generalize anything else)
• TazeTSchnitzel 7y ago
  I wish the INI file format was standardised. It's easy for computers and humans to read and write, and it has nice features like comments and non-destructive editing!
  • timmytokyo 7y ago
    TOML is basically a standardized version of INI.
    • TazeTSchnitzel 7y ago
      TOML is similar to INI, but it's not the file format I know and love.
• mckinney 7y ago
  Regardless of the reasoning laid out in the OP, it's difficult to argue in YAML's favor comparing it with JSON. I'm not an ardent fan of JSON either -- both YAML and JSON have issues wrt inconsistencies: - what draft of JSON Schema are you using 4? 7? Neither? - what version of Swagger or OpenAPI are you using? - etc. Sure, it's great to see ongoing development of schemas, but with each new development we have yet another dialect to consider/support. In my view, perhaps an even greater problem with structured data formats in general is the void that separates them from programming languages esp. static languages such as Java where static type information is otherwise leveraged. The industry standard solution, code generation, is awful in almost every respect. The Manifold framework looks promising in this regard (http://manifold.systems/ http://manifold.systems/).
  • roryokane 7y ago
    JSON Schema is not affiliated with JSON and should not be confused with it. JSON is a data format, like YAML, and there is only one version of it: the spec at http://json.org/ http://json.org/.
    • bmn__ 7y ago
      > there is only one version of [JSON] Sadly, this is utterly wrong. https://tools.ietf.org/html/rfc8259 https://tools.ietf.org/html/rfc8259 https://tools.ietf.org/html/rfc7159 https://tools.ietf.org/html/rfc7159 https://tools.ietf.org/html/rfc7158 https://tools.ietf.org/html/rfc7158 https://tools.ietf.org/html/rfc4627 https://tools.ietf.org/html/rfc4627 http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf http://www.ecma-international.org/publications/files/ECMA-ST... https://www.iso.org/standard/71616.html https://www.iso.org/standard/71616.html
• techntoke 7y ago
  Configuration files are usually meant to be single purpose. Docker, Kubernetes and Helm all use YAML exceptionally well.
• tptacek 7y ago
  The first argument, about YAML security, isn't valid; YAML is hardly the only format whose parsers have admitted deserialization vulnerabilities (they're endemic in Java; Rails had this problem with XML, and even before ROP-style deserialization was a thing, XML was getting applications owned up through external entity definitions). Format aside, no matter which you choose, you have to pick library interfaces that don't deserialize to arbitrary, constructed objects.
  • afiori 7y ago
    It is a valid criticism when comparing to Json or TOML
• amelius 7y ago
  The problem with most configuration file formats is: you can't put functions in them realistically. The best configuration file is simply source code that initializes whatever you want to run, and then runs it. That way, you can install hooks in the form of closures and make the program behave exactly like you want without the constraints that a simple "value-only" configuration file format has.
  • gjstein 7y ago
    This may be application-specific, but I might worry about security if my configuration files support arbitrary closures.
    • rco8786 7y ago
      Any more so than the rest of your code?
      • dragonwriter 7y ago
        Config files are typically written updated by non-developers and often go through a less rigorous release process, so having a less-complex and dangerous, even if less-capable, language can be desirable.
        • rco8786 7y ago
          Ah, that's not been my experience - but I can understand that if non-developers are the ones making the changes
    • amelius 7y ago
      If an attacker has write access to your configuration files, isn't all lost already?
  • corysama 7y ago
    Lua was originally designed as a configuration file format that supported functions. Description: https://www.netbsd.org/~mbalmer/lua/lua_config.pdf https://www.netbsd.org/~mbalmer/lua/lua_config.pdf Relative to other languages, it is very easy to embed Lua and it is also easy to cut out Lua’s entire standard library to restrict what the scripts are capable of doing within your program.
• simonrepp 7y ago
  tl;dr: Another alternative to YAML (among many great others), this one designed and developed by me: https://eno-lang.org/ https://eno-lang.org/ I've been doing a lot of research and development on language design for file-based content (e.g. for static site generators). I've found that YAML - although established as the go-to format for statically generated blogs, etc. - was never designed for these things as it by its nature does not support simple, essential features for this usecase like for instance unindented blocks of verbatim text (for which YAML frontmatter was invented as a very limited hack). The result of all this R&D is a language called "eno notation" which is designed especially for file-based content usecases, and around which I've also built an entire ecosystem for many languages and editors - if you're working in that field, it might be worth taking a look!
  • roryokane 7y ago
    I find it surprising that your format doesn’t distinguish strings and numbers, or other types of scalar values in general. For example, in your demo “eno's javascript benchmark suite data” on https://eno-lang.org/eno/demos/ https://eno-lang.org/eno/demos/, both of these lines: iterations: 100000 evaluated: Fri Jul 06 2018 09:46:48 GMT+0200 (Central European Summer Time) are tagged below as just a “Field”. Do client programs that read an Eno file need to run `int()`/`float()` or `.to_i`/`.to_f` on the field values they know should be numbers? That seems unergonomic.
    • simonrepp 7y ago
      You are correct! The thinking behind this is that for the majority of file-based configuration and content usecases the expected types are fixed and known beforehand already - ergo it makes more sense that a developer has to specify once which type a field is (gaining in return 100% type safety, validation, localized validation messages, ...) than all users later having to e.g. explicitly write quotes a million times when writing configuration/content, just to tell the application something about the type it already knows anyway (and wouldn't expect/accept any other way too). I think this is really more ergonomic, even in the short run.
• dang 7y ago
  Discussed last year: https://news.ycombinator.com/item?id=17358103 https://news.ycombinator.com/item?id=17358103
• reilly3000 7y ago
  Kubernetes supports JSON but overwhelmingly leans towards YAML. I've had to spend some time really grokking it to do basic dev ops, and now have my IDE pretty dialed to support it. That said, its not my favorite by a long shot. Can Jsonette save us?
  • PaulHoule 7y ago
    JSON is valid YAML.
  • chupasaurus 7y ago
    Kubernetes API accepts only JSON with a single exception of Server-Side Apply which is an alpha feature.
  • reilly3000 7y ago
    I should clarify that kubectl converts YAML to JSON, so most examples I see are in YAML in a repo, then applied by a CI system via kubectl where it is transformed into JSON.
• jchw 7y ago
  The issue is, I think most people (myself included) enter YAML into their lives as basically a JSON alternative with lighter syntax. Without really realizing, or perhaps without internalizing, the rather ridiculous number of different ways to represent the same thing, the painful subtle syntax differences that lead to entirely different representations, the sometimes difficult to believe number of features that the language has that are seldom used.. It's not just alternate skin for JSON, and yet that's what most people use it for. Some users also want things like map keys that aren't strings, which is actually pretty useful. I recall there being CoffeeScript Object Notation as well... perhaps that would've been better for many use cases, all things said.
  • crdoconnor 7y ago
    JSON alternative with lighter syntax and comments is basically what I tried to make StrictYAML. I made it largely because I saw a disconnect with what YAML was, and what people - including me - thought it was (which is what it should be). Don't agree with non-string map keys though... they're a complication I never saw a use for.
    • zenexer 7y ago
      They’re fairly useful in applications that use numeric IDs. For example, if I’m using SQL, and I have a table with an AUTOINCREMENT primary key, I’m going to have a lot of numeric IDs. If I want to reference these in a config file of some kind, I don’t want to have to read them as strings and handle the parsing on my end. Even if you’re of the opinion that IDs shouldn’t be numeric, there are a lot of cases where you’re stuck with integers—on Linux, user IDs, group IDs, and inodes are just a few examples.
      • crdoconnor 7y ago
        Ah I see, yes that makes perfect sense. I've used integer keys too. Sorry, I thought by non-string you meant non-scalar - i.e. the idea of using lists as keys (allowed in YAML).
• jokoon 7y ago
  I liked the indented style of YAML, asked how to properly parse an indented file, and wrote my own small "parser". What's nice about YAML is the choice to use indentation, for the rest, I have a hard time following the language's choices.
• ivan4th 7y ago
  From my experience, while YAML itself is something one can learn to live with, the true horror starts when people start using text template engines to generate YAML. Like it's done in Helm charts, for example, https://github.com/helm/charts/blob/master/stable/grafana/templates/deployment.yaml https://github.com/helm/charts/blob/master/stable/grafana/te... Aren't these "indent" filters beautiful?
  • regecks 7y ago
    Why would they have chosen to use template/text to generate YAML? That seems insane. Surely using an encoder on an object/structure hierarchy (like people do with encoding/json) is the way to go? On the other hand, the quality of the yaml libraries in Go wasn't great, last time I had to choose a configuration file format.
    • dharmab 7y ago
      A lot of people working with YAML have an ops background and aren't familiar with basic data structures.
      • dev_dull 7y ago
        personally I’d prefer a templatized yaml file over an over-engineered, snowflake DSL created by a “real programmer” and not an ops person.
        • celim307 7y ago
          Ok, those aren't the only two choices though.
      • kevml 7y ago
        That’s disingenuous. Most “ops” folks I work with despise templating files but it’s the easiest way to parameterize things, especially when providing ways for “devs” to do “ops”. Yes, we may not have a cleaner way of deploying k8s Deployment configs to different clusters but the desire to templatize YAML is easy for everyone to understand. The decision to abstract or templatize is one rooted in time and cost, not ability to understand data structures.
• Legogris 7y ago
  Despite spending time writing and reading YAML on a daily basis for years, it still trips me up once things get non-trivial. It's definitely my least-favorite non-propritery config file format. XML might be overly verbose, but there are no surprises (unless you go bananas with schemas).
  • krapht 7y ago
    Yeah. I never got the hate for XML. I feel like it was always mismatched expectations: some people wanted something the Markdown of configuration files, and other people wanted something extensible enough to encode any possible data structures.
    • Zelphyr 7y ago
      Not to mention; while pretty wordy, XSLT was incredibly powerful.
    • marcosdumay 7y ago
      It's verbose, illegible, redundant, and shares mos of the problems YAML has. Not to talk about the attribute/content duality and all the ill-defined parsers it leads to.
  • cuillevel3 7y ago
    XML is great for documents too: <t>Some <b>text</b> is here </t> And I found some things weird, like entities and (external?) DTD references. But it's great for building your own formats. For data interchange it has it's problems, like no types. Everything is kind of a string. With encoding problems and XML in XML everything ends up in CDATA... I think that's why XML makes for heavy parsers. I remember it was better in Java, because of the strong libraries.
  • zelly 7y ago
    XML solved this problem a long time ago, but everyone hates it now because it's too enterprise and Javalike.
• cuillevel3 7y ago
  YAML is really so much more than JSON. * YAML can have several 'documents' in the same file,separated by --- * there are anchors and references * easy to read multi line texts * it's also a superset of JSON I can see, how choosing YAML when you just wanted readable JSON might give you more headaches than expected. And like someone else said, putting another template engine (or two) on top of YAML is when the real problems start.
  • c3534l 7y ago
    It's really to maintain JSON without being allowed comments. JSON is fine for transmitting data across the web, but if you need complex configuration data in version control that many people work on, you need comments so new teammates can be easily onboarded onto a project. There's definitely a need for something that is programming-language-like, but entirely about structuring and formatting data for configuration. YAML is the closest thing to that. It's not perfect, but it works better for that purpose than JSON or XML.
• acd 7y ago
  ThoughtWorks has templating in yaml on their hold list. https://www.thoughtworks.com/radar/techniques/templating-in-yaml https://www.thoughtworks.com/radar/techniques/templating-in-... How does one write Kubernetes specs and Ansible without yaml?
  • q3k 7y ago
    As YAML is a superset of JSON, use $favourite_language to generate it. I like JSONnet for that.
• ngold 7y ago
  YAML (/ˈjæməl/, rhymes with camel[2]) was first proposed by Clark Evans in 2001,[10] who designed it together with Ingy döt Net[11] and Oren Ben-Kiki.[11] Originally YAML was said to mean Yet Another Markup Language,[12] referencing its purpose as a markup language with the yet another construct, but it was then repurposed as YAML Ain't Markup Language, a recursive acronym, to distinguish its purpose as data-oriented, rather than document markup.
• devnulloverflow 7y ago
  As much as I am an old-school Unix zealot, I think it is time to move towards a well standardised binary config format with non-trivial types (i.e a schema). There still has to be a standard text format, but only for the source from which the live configs have to be built. Done right, this has several advantages: 1. Built-time validation (or at least type checking). 2. Built configs can be easy to parse but (potentially) rich enough to avoid confusing templating. 3. Separation of concerns between storing/maintaining configs and applying them. E.g. scoop text configs off a source repo, but send out binary configs over the network. All this is a fantasy in my head. Right now the closest mainstream thing is protobufs. But they make trade-offs for non-config use cases, and thus don't really cut it in the "... rich enough to avoid confusing templating" department.
  • becauseiam 7y ago
    Property Lists already do tick some of those boxes.
  • zrail 7y ago
    SQLite databases might fit the bill. Fairly lightweight. Can talk to them in basically any language. Instead of templates you copy the database file and issue some UPDATEs.
    • zbentley 7y ago
      They don't type check though; most constraints arent enforced, and the underlying reality (slinging around mostly strings) leaks out often.
    • raymondh 7y ago
      I agree this a pretty choice.
    • shandor 7y ago
      But that, and the parent's idea of binary formats in general, throws away the absolute golden property of text format configuration files: you can put those in git, and see with an accuracy of a single character what has changed. My impression was always that this was a huge reason for plain text files in the first place. Someone mentioned protobufs, maybe with something like those one could have both?
• schpaencoder 7y ago
  EDN
• patsplat 7y ago
  It's best to consider YAML in it's appropriate context as a better XML fragment. The ideas in YAML evolved into JSON which is preferable today. However at the time, a tree data format that deserialized into native types was quite useful. The alternative was writing event based SAX parsers, or incredibly verbose XML object apis.
• daveisfera 7y ago
  There's two types of formats: 1) those people complain about, and 2) those no one use.
  • meowface 7y ago
    TOML seems widely used but I've never seen complaints about it. I'm sure there are some, but the only time I see it mentioned is when someone is recommending someone else switch to TOML. Out of curiosity, is there anyone here who doesn't like TOML for configuration?
    • jillesvangurp 7y ago
      I've encountered toml a couple of times but I wouldn't call it wide spread. It's alright for small configuration files. However, if you keep things simple, json and yaml are also not so bad and even properties files or good old ini files will work. Doing e.g. cloudformation stuff in toml is not a thing though and it supports both yaml and json. If your data is simple, use a simple format. I've always liked properties files with simple name value pairs separated by =. Still very common in the Java world though yaml has replaced a lot of that. BTW. I've handled all of those formats using jackson on Java & Kotlin. It has a flexible parser framework originally intended for json. But it has lots of plugins for different tree like configuration files. Look for jackson-dataformat-yaml and jackson-dataformat-toml on github. There are loads more formats that you can support with jackson. Nice if you need to translate from one to the other or need to support multiple formats. IMHO Json with some tweaks would be really nice. E.g. just supporting comments and multi line strings would make it a lot nicer. A lot of json becomes unreadable due to the need to escape strings. I've come across Hocon a couple of times (jackson-dataformat-hocon) and it's a strict superset of json, which means that if you accept hocon as input, you implicitly also accept json.
• ale22 7y ago
  I hate YAML with a passion and reach for XML filetypes to prototype a custom object that I'm trying to design. Don't know why people hate XML though.
• methou 7y ago
  I have a completely unrelated question with the topic but derived from the font-face used in the article. https://arp242.net/yaml-config.html#can-be-hard-to-edit-especially-for-large-files https://arp242.net/yaml-config.html#can-be-hard-to-edit-espe... In the heading, how was sp ligatures in the `espcially` written, is there a name for this? How do you connect the beginning of a `s` to the beginning of a `p`?
  • TheDong 7y ago
    These are discretionary ligatures [0]. They're turned on in html with the following two css lines (though on firefox, either one is enough to have them happen): font-variant-ligatures: common-ligatures discretionary-ligatures; font-feature-settings: 'liga' on, 'dlig' on; [0]: https://www.fonts.com/content/learning/fontology/level-3/signs-and-symbols/ligatures-2 https://www.fonts.com/content/learning/fontology/level-3/sig...
    • Carpetsmoker 7y ago
      Note it won't work for every font, or some fonts may have a different flag to enable it (I think there's an historical-ligatures, as well).
  • deleted 7y ago
    [deleted]
• jaten 7y ago
  S-expressions rule. I use them for configuration everywhere. I wrote a lovely library for parsing them in Go (along with a full lisp interpreter if you like) https://github.com/glycerine/zygomys https://github.com/glycerine/zygomys Provides comments, multiline strings, and automatic translation into Go structs using reflection.
• markpapadakis 7y ago
  I dislike whitespace sensitive languages or definition formats with a passion. Especially when tabs and spaces are not treated equally. I don’t mind python as much nowadays but YAML is borderline insulting to me. I hope we all move on to something more sane soon.
• namelosw 7y ago
  YAML is bad. It's like markdown, there are too many parsers behave differently. Unlike markdown just for reading, it is used in configurations for critical systems. JSON is much better, it IS readable and writable, people using package.json all the time without problems. Templating YAML is even worse. Templating is an ad-hoc abstraction, and very easy to run into issues. A minimal JavaScript runtime with JSON would be much better, JSON is JavaScript Object Notation after all.
  • ailideex 7y ago
    Markdown's problem is no single standard. This is not the case with YAML, so no, it is not like markdown. And you can technically write assembler also.
    • namelosw 7y ago
      You are right, but if we shrink the scope to CommonMark the problems still exists. And what is assembler, may I ask? Is it for YAML or Markdown?
• deepsun 7y ago
  I think author confuses YAML problems with his favorite languages problems. I bet those problems (at least most of them) are non existent in Java, for example, only because Java programmers usually more responsible. Same for Haskell or Rust I think. But in other languages with notoriously irresponsible coders (JS, PHP) I bet to see even more of these problems. (I coded in all of them)
  • xenator 7y ago
    Exact after-taste I felt after article. Why not just move the focus to point that author is not like Ruby and other Ruby frameworks anymore. During my working life using Python I got few meh-moments with YAML. And this is all. Never lost real joy of using it.
• perlgeek 7y ago
  I've recently started using https://jsonnet.org/ https://jsonnet.org/ to generate more complex config. It's easier to write than JSON (no need to quote keys, allows trailing commas), has reusability through functions and objects, and can output JSON which is much easier to parse than YAML. Downside: you need another build step for the config.
• pjmlp 7y ago
  I still keep using XML as my favourite format. Get to use parsers out of the box, validation tooling, support comments, IDE code completion and they are super easy to transform. In a couple of years some trendy SV unicorn will make XML the best format of the world, as these cycles happen to be.
  • DonHopkins 7y ago
    The irony is that the best format that takes over the world will be an XML representation of JSON, enabling comments and trailing commas.
• jwilk 7y ago
  FYI, PyYAML 5.1 partially fixes the security issues: https://github.com/yaml/pyyaml/issues/265 https://github.com/yaml/pyyaml/issues/265
• nrvn 7y ago
  The points in the article are pretty solid. But here is a big question. Imagine you can influence the switch of the configuration formats for projects like Ansible, Kubernetes, Docker Compose, AWS CloudFormation, Google Cloud Deployment Manager, et al. You can take any project with huge user base and all of those project will have one thing in common: JSON-based configuration with an option to write this configuration in YAML. Since basically anyone talking YAML in the context of JSON is talking about a JSON superset. So here's the task: propose a JSON-compatible alternative to YAML. Things to keep in mind: - backwards compatibility - easy migration from YAML to a new format - full JSON compatibility - relatively cheap to get supported by the project of interest.
  • theknarf 7y ago
    Just parse the YAML and spit out JSON5 (https://json5.org/ https://json5.org/) and then keep that version.
• jaten 7y ago
  Google uses a subset of python called Starlark for build configuring (available in Go and I think Java). Nice if you want to be able to compute things during config. https://github.com/google/starlark-go https://github.com/google/starlark-go
• shermozle 7y ago
  WTF is that weird ligature between s and p on the heading "Can be hard to edit, especially for large files"? That is an odd font.
• linkerzx 7y ago
  Not a fan of YAML, but it has its advantages over JSON, such as the ability to comment specific portions easily. Haven't used TOML yet, but it seems promising given that for most use cases you would only use a portion of the YAML language.