9 ms·

Drupal 8 uses YAML* as its configuration language because JSON doesn't support comments. That simple. Thank you for YAML, it does deliver for us: it's human rea

by chx 8y ago

Drupal 8 uses YAML* as its configuration language because JSON doesn't support comments. That simple. Thank you for YAML, it does deliver for us: it's human readable and it's easy to parse (see below). * I mean, it uses an ill defined subset of YAML. The definition is "whatever the Symfony YAML parser supports".

jiveturkey 8y ago
> JSON doesn't support comments eh? { "firstName": "John", "lastName": "Smith", "comment": "foo", } I know it isn't the same as #comments, but who cares really.
- hexane360 8y ago
  It seems the main difference is your comments are all parsed and loaded into memory with the file, while official comments aren't.
- trynewideas 8y ago
  > who cares really the person who came up with HOCON, probably
- Smaug123 8y ago
  The trouble there is that your comments come in-band. What if you're trying to serialise something and you don't have the power to insist that it's not a dictionary with "comment" as a key?
- dharmab 8y ago
  How do I do something like: { # comment with a note about the value of foo "foo": "bar", # comment with a note about the value of baz "baz": "qux" } Without driving myself and future readers insane with fooComments and bazComments? What if I need a multiline comment explaining a yak-shaving story for why a key is set to a certain value? What if the object in question is a set of keyword arguments, and adding new fields changes the behavior of whaever is parsing the document?
  - mottosso 8y ago
    Ok, I'll bite. { "#": "A foo variable", "foo": true, "#": "A bar variable", "bar": false } Alternatively. { "# A foo variable": "", "foo": true, "# A multiline..": "", "# .. bar variable": "", "bar": false } Presto!
    - chx 8y ago
      Sigh. All I wanted to do is to say thanks for the YAML standard -- comments are important but not the only problem with JSON. And truly I can't be expected to remember all of this discussion from like six plus years ago. One thing I remember though, it the trailing comma problem -- we upstreamed a grammar change to Doctrine annotation so "foo, bar," is OK because PHP arrays accept that and it's bonkers trying to code a mostly PHP system without trailing comma support. Also, JSON is no fun to write , you need to have [] {} all correct where YAML is much easier. The less sigils the better and most of Drupal YAMLs only use the dash, the colon and the quote. This is the grave mistake Doctrine committed as well, instead of simple arithmetic (>=1.0) they used mysterious sigils in version specification (~1.0). Drupal is in the business of constantly accepting new contributors and (~R∊R∘.×R)/R←1↓ιR is not newbie friendly, no matter how you slice and dice it. There are certainly advantages of sigil heavy languages like APL and Perl but the scare factor is too high.
      - jiveturkey 8y ago
        fair enough, but then you probably shouldn't have led off your earlier comment with "that simple".
    - goatlover 8y ago
      That's just ugly and you're mixing your comments with the data structure, which is potentially confusing. Also, Jason requires a lot more typing. I don't want to have to manually add in all the brackets, quotes and commas when editing config a file.
    - deleted 8y ago
      [deleted]
merb 8y ago
well there is also toml and hocon (json supersets) which are "yaml like"
- chx 8y ago
  TOM: Initial release 23 February 2013; 5 years ago Drupal 8 file format discussion was in 2011, predating it by two years. https://groups.drupal.org/node/159044 https://groups.drupal.org/node/159044
- benatkin 8y ago
  TOML is more like ini than YAML. I don't like it because it uses the = symbol which seems imperative rather than declarative. (Same with HCL, it might be a nitpick but these are languages I'm going to be using all the time.) HOCON is interesting but at first glance it seems it might be too ambiguous for my tastes, because like YAML, because it supports both js-style ("//") and shell-style ("#") comments. JSON plus comments is beautiful because it adds minimally to an unambiguous language which lends itself to automatic formatting (stringificiation).
  - colatkinson 8y ago
    I'd argue that = only feels imperative if you're used to imperative languages. Prolog and Haskell, both of which focus on being declarative, also both use the equals sign.
    - benatkin 8y ago
      Fair enough. It still seems overkill to me though. A ":" seems much more unassuming than an "=".
- user5994461 8y ago
  Don't forget HJSON if you want a json clone with comments. https://hjson.org/ https://hjson.org/
skybrian 8y ago
VS Code uses JSON with comments for config files. [1] Technically, this is not JSON. You won't be able to use a standard JSON parser without stripping comments first. But you can use a simple, JSON-like language with comments for config. [1] https://code.visualstudio.com/docs/languages/json#_json-with-comments https://code.visualstudio.com/docs/languages/json#_json-with...
- EamonnMR 8y ago
  > you can use a simple, JSON-like language with comments for config. YAML can be employed as a simple JSON-like language with comments.
  - crooked-v 8y ago
    > simple YAML is much, much more complicated than JSON.
    - hk__2 8y ago
      > simple YAML is much, much more complicated than JSON. Quoting a single word from the parent’s sentence is misleading. The sentence "YAML can be employed as a simple JSON-like language with comments." is true because JSON is YAML, so you can parse a JSON file with #-comments using a YAML parser.
      - skybrian 8y ago
        The parser is not simple, though, and that's what counts.
        bdhess 8y ago
        Most YAML users don't need to look at the source for a YAML parser. I appreciate elegant simplicity, but I don't think parser complexity is the most important metric by which to judge a data interchange format.
wwweston 8y ago
You know what else is human readable, easy to parse if you're using PHP, and supports comments? PHP. I understand why some languages rely on common configuration file formats. I don't understand why the popular dynamic script-y languages don't more commonly use the natively-expressable associative/list data structures that they're famous for making convenient.
- krapp 8y ago
  >I don't understand why the popular dynamic script-y languages don't more commonly use the natively-expressable associative/list data structures that they're famous for making convenient. You picked the wrong language... PHP comes with its own JSON parser. And INI and XML and even CSV. But, the reason is that, generally, you want config files to describe data or state only. Yes, you could just make your config native code, but then the temptation to add functions and methods and logic to that becomes irresistible and soon your config is an application that needs its own config. Config formats need to be simple, and preferably not Turing complete.
  - user5994461 8y ago
    Any configuration will eventually become a programming language. See The Configuration Complexity Clock. https://mikehadlow.blogspot.com/2012/05/configuration-complexity-clock.html https://mikehadlow.blogspot.com/2012/05/configuration-comple...
    - krapp 8y ago
      Just because it's common doesn't mean it should be encouraged by writing your config in native code to begin with. INI is still simple, and JSON doesn't support logic, so the madness can be held at bay at least for a time. XML and s-expressions are lost causes, though.
- akavel 8y ago
  Interestingly, the Lua programming language actually evolved from configuration files: https://www.lua.org/history.html https://www.lua.org/history.html (and is still officially deemed useful for writing them)
DonHopkins 8y ago
I totally agree that in the ideal world, JSON should support comments. I yearn for them, and none of the in-band work-arounds or post-processing tools are acceptable substitutes. But to play the devil's advocate, how would JSON be able to support round-tripping comments like XML can, since  are part of the DOM model that you can read and write, while JSON // and /* comments */ are invisible to JavaScript programs. There's nowhere to store the comments in the JSON object model, which you would need to be able to write them back out later! On important feature of JSON is being able to read and write JSON files with full fidelity and not lose any information like comments. XML can do that, but JSON can't. To fix that you'd have to go back and redesign (and vastly complicate) fundamental JavaScript objects and arrays and values, to be as complex and byzantine as the DOM API. The less-than-ideal situation we're in isn't JSON's fault or JavaScript's fault, because JSON is just a post-hoc formalization of something that was designed for a different purpose. But JSON is rightly more popular than XML, because it's extremely simple, and nicely impedance matched with many popular languages. YAML suffers from the same problem as JSON that it can't round-trip comments like XML can, but it fails to be as simple as JSON, is almost as complex as XML, and doesn't even map directly to many popular languages (as the article points out, you can't use a list as a dict key in Python, PHP, JavaScript, or Go, etc). You can sidestep some of JSON's problems by representing JSON as outlines and tables in spreadsheets, without any need for syntax and sigils like brackets, braces, commas, no commas, quoting, escaping, tabs, spaces, etc, but in a way that supports rich formatted comments and content (you can even paste pictures and live charts into most spreadsheets if you like), and even dynamic transformations with spreadsheet expressions and JavaScript. See my comments about that in this and another article: https://news.ycombinator.com/item?id=17360071 https://news.ycombinator.com/item?id=17360071 https://news.ycombinator.com/item?id=17309132 https://news.ycombinator.com/item?id=17309132
- rhacker 8y ago
  I'm kinda sad that JSON has been struggling for like 15 years to get comments. Is there like some kind of gestapo that's saying no or something? All it takes is for the maintainers of probably 15 popular libraries to start handling comments. At the end of the day I'm sure the reason we don't have JSON comments is somewhere listed in this page: xkcd.com/927/
pacala 8y ago
Try https://jsonnet.org/ https://jsonnet.org/. Supports comments, plus a handful of additional useful features.
- nojvek 8y ago
  Jsonnet is awesome. We use it to generate our yaml files for kubernetes. YAML isn’t easy to parse, nor is it very flexible as a templating language. It gets cumbersome very quickly. Jsonnet is a relief. Kubernetes should have been a dumb json config from the get go. JSON is ridiculously simple to parse and emit. It has huge interoperability as well with lots of programming languages.
  - pests 8y ago
    Kubenetes never intended to get stuck on YAML. The CNCF is backing Ksonnet which is Jsonnet for k8s if you haven't seen it before.
- alessioalex 8y ago
  Oh man, you just made my day. Comments, imports and mixins, I just had an 'evrika' moment. P.S. You Romanian by any chance?
kpil 8y ago
Is there a yaml parser that preserves comments and a writer that manages to write them back though?
- TimZehta 8y ago
  The parser linked from the article does: https://github.com/crdoconnor/strictyaml https://github.com/crdoconnor/strictyaml

by chx 8y ago

jiveturkey 8y ago

> JSON doesn't support comments eh? { "firstName": "John", "lastName": "Smith", "comment": "foo", } I know it isn't the same as #comments, but who cares really.

hexane360 8y ago
It seems the main difference is your comments are all parsed and loaded into memory with the file, while official comments aren't.
trynewideas 8y ago
> who cares really the person who came up with HOCON, probably
Smaug123 8y ago
The trouble there is that your comments come in-band. What if you're trying to serialise something and you don't have the power to insist that it's not a dictionary with "comment" as a key?
dharmab 8y ago
How do I do something like: { # comment with a note about the value of foo "foo": "bar", # comment with a note about the value of baz "baz": "qux" } Without driving myself and future readers insane with fooComments and bazComments? What if I need a multiline comment explaining a yak-shaving story for why a key is set to a certain value? What if the object in question is a set of keyword arguments, and adding new fields changes the behavior of whaever is parsing the document?
- mottosso 8y ago
  Ok, I'll bite. { "#": "A foo variable", "foo": true, "#": "A bar variable", "bar": false } Alternatively. { "# A foo variable": "", "foo": true, "# A multiline..": "", "# .. bar variable": "", "bar": false } Presto!
  - chx 8y ago
    Sigh. All I wanted to do is to say thanks for the YAML standard -- comments are important but not the only problem with JSON. And truly I can't be expected to remember all of this discussion from like six plus years ago. One thing I remember though, it the trailing comma problem -- we upstreamed a grammar change to Doctrine annotation so "foo, bar," is OK because PHP arrays accept that and it's bonkers trying to code a mostly PHP system without trailing comma support. Also, JSON is no fun to write , you need to have [] {} all correct where YAML is much easier. The less sigils the better and most of Drupal YAMLs only use the dash, the colon and the quote. This is the grave mistake Doctrine committed as well, instead of simple arithmetic (>=1.0) they used mysterious sigils in version specification (~1.0). Drupal is in the business of constantly accepting new contributors and (~R∊R∘.×R)/R←1↓ιR is not newbie friendly, no matter how you slice and dice it. There are certainly advantages of sigil heavy languages like APL and Perl but the scare factor is too high.
    - jiveturkey 8y ago
      fair enough, but then you probably shouldn't have led off your earlier comment with "that simple".
  - goatlover 8y ago
    That's just ugly and you're mixing your comments with the data structure, which is potentially confusing. Also, Jason requires a lot more typing. I don't want to have to manually add in all the brackets, quotes and commas when editing config a file.
  - deleted 8y ago
    [deleted]

merb 8y ago

well there is also toml and hocon (json supersets) which are "yaml like"

chx 8y ago
TOM: Initial release 23 February 2013; 5 years ago Drupal 8 file format discussion was in 2011, predating it by two years. https://groups.drupal.org/node/159044 https://groups.drupal.org/node/159044
benatkin 8y ago
TOML is more like ini than YAML. I don't like it because it uses the = symbol which seems imperative rather than declarative. (Same with HCL, it might be a nitpick but these are languages I'm going to be using all the time.) HOCON is interesting but at first glance it seems it might be too ambiguous for my tastes, because like YAML, because it supports both js-style ("//") and shell-style ("#") comments. JSON plus comments is beautiful because it adds minimally to an unambiguous language which lends itself to automatic formatting (stringificiation).
- colatkinson 8y ago
  I'd argue that = only feels imperative if you're used to imperative languages. Prolog and Haskell, both of which focus on being declarative, also both use the equals sign.
  - benatkin 8y ago
    Fair enough. It still seems overkill to me though. A ":" seems much more unassuming than an "=".
user5994461 8y ago
Don't forget HJSON if you want a json clone with comments. https://hjson.org/ https://hjson.org/

skybrian 8y ago

VS Code uses JSON with comments for config files. [1] Technically, this is not JSON. You won't be able to use a standard JSON parser without stripping comments first. But you can use a simple, JSON-like language with comments for config. [1] https://code.visualstudio.com/docs/languages/json#_json-with-comments https://code.visualstudio.com/docs/languages/json#_json-with...

EamonnMR 8y ago
> you can use a simple, JSON-like language with comments for config. YAML can be employed as a simple JSON-like language with comments.
- crooked-v 8y ago
  > simple YAML is much, much more complicated than JSON.
  - hk__2 8y ago
    > simple YAML is much, much more complicated than JSON. Quoting a single word from the parent’s sentence is misleading. The sentence "YAML can be employed as a simple JSON-like language with comments." is true because JSON is YAML, so you can parse a JSON file with #-comments using a YAML parser.
    - skybrian 8y ago
      The parser is not simple, though, and that's what counts.
      - bdhess 8y ago
        Most YAML users don't need to look at the source for a YAML parser. I appreciate elegant simplicity, but I don't think parser complexity is the most important metric by which to judge a data interchange format.

wwweston 8y ago

You know what else is human readable, easy to parse if you're using PHP, and supports comments? PHP. I understand why some languages rely on common configuration file formats. I don't understand why the popular dynamic script-y languages don't more commonly use the natively-expressable associative/list data structures that they're famous for making convenient.

krapp 8y ago
>I don't understand why the popular dynamic script-y languages don't more commonly use the natively-expressable associative/list data structures that they're famous for making convenient. You picked the wrong language... PHP comes with its own JSON parser. And INI and XML and even CSV. But, the reason is that, generally, you want config files to describe data or state only. Yes, you could just make your config native code, but then the temptation to add functions and methods and logic to that becomes irresistible and soon your config is an application that needs its own config. Config formats need to be simple, and preferably not Turing complete.
- user5994461 8y ago
  Any configuration will eventually become a programming language. See The Configuration Complexity Clock. https://mikehadlow.blogspot.com/2012/05/configuration-complexity-clock.html https://mikehadlow.blogspot.com/2012/05/configuration-comple...
  - krapp 8y ago
    Just because it's common doesn't mean it should be encouraged by writing your config in native code to begin with. INI is still simple, and JSON doesn't support logic, so the madness can be held at bay at least for a time. XML and s-expressions are lost causes, though.
akavel 8y ago
Interestingly, the Lua programming language actually evolved from configuration files: https://www.lua.org/history.html https://www.lua.org/history.html (and is still officially deemed useful for writing them)

DonHopkins 8y ago

I totally agree that in the ideal world, JSON should support comments. I yearn for them, and none of the in-band work-arounds or post-processing tools are acceptable substitutes. But to play the devil's advocate, how would JSON be able to support round-tripping comments like XML can, since  are part of the DOM model that you can read and write, while JSON // and /* comments */ are invisible to JavaScript programs. There's nowhere to store the comments in the JSON object model, which you would need to be able to write them back out later! On important feature of JSON is being able to read and write JSON files with full fidelity and not lose any information like comments. XML can do that, but JSON can't. To fix that you'd have to go back and redesign (and vastly complicate) fundamental JavaScript objects and arrays and values, to be as complex and byzantine as the DOM API. The less-than-ideal situation we're in isn't JSON's fault or JavaScript's fault, because JSON is just a post-hoc formalization of something that was designed for a different purpose. But JSON is rightly more popular than XML, because it's extremely simple, and nicely impedance matched with many popular languages. YAML suffers from the same problem as JSON that it can't round-trip comments like XML can, but it fails to be as simple as JSON, is almost as complex as XML, and doesn't even map directly to many popular languages (as the article points out, you can't use a list as a dict key in Python, PHP, JavaScript, or Go, etc). You can sidestep some of JSON's problems by representing JSON as outlines and tables in spreadsheets, without any need for syntax and sigils like brackets, braces, commas, no commas, quoting, escaping, tabs, spaces, etc, but in a way that supports rich formatted comments and content (you can even paste pictures and live charts into most spreadsheets if you like), and even dynamic transformations with spreadsheet expressions and JavaScript. See my comments about that in this and another article: https://news.ycombinator.com/item?id=17360071 https://news.ycombinator.com/item?id=17360071 https://news.ycombinator.com/item?id=17309132 https://news.ycombinator.com/item?id=17309132

rhacker 8y ago
I'm kinda sad that JSON has been struggling for like 15 years to get comments. Is there like some kind of gestapo that's saying no or something? All it takes is for the maintainers of probably 15 popular libraries to start handling comments. At the end of the day I'm sure the reason we don't have JSON comments is somewhere listed in this page: xkcd.com/927/

pacala 8y ago

Try https://jsonnet.org/ https://jsonnet.org/. Supports comments, plus a handful of additional useful features.

nojvek 8y ago
Jsonnet is awesome. We use it to generate our yaml files for kubernetes. YAML isn’t easy to parse, nor is it very flexible as a templating language. It gets cumbersome very quickly. Jsonnet is a relief. Kubernetes should have been a dumb json config from the get go. JSON is ridiculously simple to parse and emit. It has huge interoperability as well with lots of programming languages.
- pests 8y ago
  Kubenetes never intended to get stuck on YAML. The CNCF is backing Ksonnet which is Jsonnet for k8s if you haven't seen it before.
alessioalex 8y ago
Oh man, you just made my day. Comments, imports and mixins, I just had an 'evrika' moment. P.S. You Romanian by any chance?

kpil 8y ago

Is there a yaml parser that preserves comments and a writer that manages to write them back though?

TimZehta 8y ago
The parser linked from the article does: https://github.com/crdoconnor/strictyaml https://github.com/crdoconnor/strictyaml