5 ms·

> YAML isn't a configuration language or a configuration language format, it's a serialization format I think YAML is even worse as a serialization format than

by thayne 5y ago

> YAML isn't a configuration language or a configuration language format, it's a serialization format I think YAML is even worse as a serialization format than a configuration format. It has too many ambiguities, implementations are too inconsistent, and many implementations are insecure by default with untrusted input. > Using a good custom designed configuration file format instead of trying to shove things through the narrow pipe of YAML means that you have one integrated syntax that can be designed to be more readable, more expressive, and much easier to write. I won't say a custom configuration is never the right thing to do, but I do think it rarely is. Writing a good custom configuration language is very difficult to get right, not to mention you also need syntax highlighting and other editor support for the editors your users use, libraries for popular languages so that other programs can read and write the configuration, etc. If you need something this complicated, I'd reach for using an embedded scripting language like lua (or possibly JavaScript or python) before writing a new format from scratch.

amirkdv 5y ago
> I think YAML is even worse as a serialization format than a configuration format. This. I find YAML to be the least offensive option for configuration and one of the worst for serialization. I might be misinformed, but I find it absurd that in 2021 we still don't have a default, universally available tool that supports the basic table stakes without headache: 1. core data types (number, string, etc) 2. lists, maps, and arbitrary nesting 3. comments 4. multiline strings 5. is acceptably readable 6. Just Works everywhere YAML does get 1-5 right (specifically 3 and 4 that JSON doesn't and IMO better in 5). But then it adds a ton of complexity that has left us without a standard, safe, and sane parser implementation: anchors and references (& and *) , casting (via !!), custom data types (via !), loads of other things I don't understand.
- brundolf 5y ago
  It's a real shame JSON doesn't have 3 and 4, because it would be so easy and it's otherwise pretty much perfect imo. No ambiguity, every value type can be identified by its first character, clean and reasonably minimal syntax.
  - plusmax1 5y ago
    well there are json alternatives which fit this bill, such as HJSON. https://hjson.github.io/ https://hjson.github.io/ might not be as "common" but it has good implementations for many languages.
  - MichaelMoser123 5y ago
    you could allow python like comments and strip them with a regular expression substitution before parsing. Something like this. (it strips all the lines that start with whitespace, followed by #, followed by anything until the end of the line) cat cfg.json | sed -e 's/\s*#.*$//g' | jq . Allowing multiple lines is more complicated, can't do that with regex alone.
    - brundolf 5y ago
      There are lots of ways to implement it. The problem is that one of JSON's strengths is its ubiquity: every language under the sun has half a dozen different battle-tested parsers for it. Clients and servers and everything in-between have first-class support out of the box. You can even paste it directly into JavaScript as valid code. If anybody short of a standards body tries to expand the spec, you lose out on most of that.
      - MichaelMoser123 5y ago
        i think you can possibly define how you want to use json for a configuration file; json by itself is not much more than javascript objects/maps, defined as a data format. I frankly don't think that you need to be too pious about standard compliance if dealing with a cofiguration format for your application.
        brundolf 5y ago
        Editors, for one, will be an issue
        SigmundA 5y ago
        >json by itself is not much more than javascript objects/maps And without comments or optional quoted keys single quotes or trailing commas, which makes editing by hand more work and IMO less readable.
- sofixa 5y ago
  HCL does all of those. TOML is pretty close as well.
  - scrollaway 5y ago
    HCL is neither acceptably readable nor does it just work everywhere. TOML is getting there though yeah.
- nanoscopic 5y ago
  My JSON parser ( github.com/nanoscopic/ujsonin ) has these things: 0. Looks essentially the same as JSON 1. Core data types, and customizable data types can be added easily. 2. Arrays, Objects, and arbitrary nesting. 3. Comments ( both /* */ and // format ) 4. Multiline strings ( by default; carriage returns are no problem within strings ) 5. It is JSON with relaxed restrictions and slight addition for actual named types. 6. I've written C, Perl, and Golang implementations so far.
  - jasfi 5y ago
    I like the idea of what you're doing. May I suggest a Nim port? Nim outputs C anyway, but is safe, so you'd worry less about bugs.
    - nanoscopic 5y ago
      Thank you. Glancing through the Nim documentation it doesn't appear to support goto. Searching online reveals one can hack something into the underlying instruction set, but it looks very unclean. To port my parser I would need to alter the core of the processor a bunch to make it effectively be a large switch statement inside of a loop to build the state machine. Right now all three implementation are essentially following the same logic with just the language syntax altered between them to make maintenance of all implementations simultaneously easier. I have spent some time generating ragel grammars for this sort of thing, and ragel can output a number of different languages itself. The code generated by ragel, is, though, quite messy itself and can have random problems that require workarounds, which is why I haven't decided fully upon going down the ragel path. May I ask what the point is of having a Nim port? You've listed safety; which can be better addressed by building a state machine generator and a state machine DSL. Also, the code is quite small, so I don't think "safety" is the primary concern for the codebase. Nim compiles to C, C++, and Javascript. I can already effectively run the C parser in JS by compiling to WASM ( and I've done so with my XML parser ) C++ code can call the C code without issue. A wrapper to auto-call object deletion in C++ would be trivial to write. Rewriting in Nim would not, so far as I can see, offer any benefits to me that are worth the effort.
- MichaelMoser123 5y ago
  >loads of other things I don't understand. also you can have multiple instances of yaml trees in one file, each one separated with -- . I think this makes it very confusing if used as a configuration language (they like to use yaml for configuration in kubernetes)
  - Spivak 5y ago
    I mean sure but I’ve never seen any software actually use that feature in the wild except for “you’re allowed to concat your YAML files instead of separating them if you want.” Like I’ve never seen software require a certain number of documents in a file with different schemas.
    - MichaelMoser123 5y ago
      i saw it being used in some CI system; it was very confusing.
- dheera 5y ago
  Besides the above (especially comments), YAML also has one huge advantage for configuration files and that is clean diffs. JSON's lack of support for trailing commas messes up diffs.
- MaxGabriel 5y ago
  If Dhall was more popular, would it meet your criteria?
  - deleted 5y ago
    [deleted]
- inkeddeveloper 5y ago
  If you come up with a format, call it IJW. It just works.
- petre 5y ago
  I used to think that before I had to edit the ejabberd.yml config file. Now I think it's only remotely useful as a subset to use in Jelkyll headers. Just use TOML for configuration instead.
  - account42 5y ago
    > I used to think that before I had to edit the ejabberd.yml config file. Hey, it's still better than having to write the configuration in Erlang like you had to before.
- alephu5 5y ago
  Use dhall and serialise to JSON/YAML!
mh0pe 5y ago
My stance is that YAML is a good format for configuration management and generation -- it's wonderful at filling gaps as your deployment model increases in complexity to provide a mechanism to "render" your configuration -- much like Skylark [1] does (derived from Google's internal GCL). YAML ends up being a powerfully declarative model [2] for the state of a data structure, rather than a straight representation, ironically often enough being used in turn for an imperative model like in Ansible [3]. Definitely friendlier than JSON. But personally, I really like YAML because it lets me compose using a traits/mixins-like model using & and *, which allows for verbose, structured configuration inputs but concise configuration files. docker-compose YAML files extension fields [4], imo, are a great example of this type of model in action. When you leave this much pre-deserialization flexibility in your configuration representation, it makes building cool stuff like docker-compose ECS support x-aws-* extension keys [5] and other plugin system-type capabilities much more straightforward than, for example, adding a new language feature to HCL. [1]: https://github.com/google/skylark https://github.com/google/skylark [2]: https://en.wikipedia.org/wiki/YAML#Advanced_components https://en.wikipedia.org/wiki/YAML#Advanced_components [3]: https://docs.ansible.com/ansible/latest/user_guide/playbooks_advanced_syntax.html https://docs.ansible.com/ansible/latest/user_guide/playbooks... [4]: https://docs.docker.com/compose/compose-file/compose-file-v3/#extension-fields https://docs.docker.com/compose/compose-file/compose-file-v3... [5]: https://docs.docker.com/cloud/ecs-integration/#rolling-update https://docs.docker.com/cloud/ecs-integration/#rolling-updat...

by thayne 5y ago

amirkdv 5y ago

> I think YAML is even worse as a serialization format than a configuration format. This. I find YAML to be the least offensive option for configuration and one of the worst for serialization. I might be misinformed, but I find it absurd that in 2021 we still don't have a default, universally available tool that supports the basic table stakes without headache: 1. core data types (number, string, etc) 2. lists, maps, and arbitrary nesting 3. comments 4. multiline strings 5. is acceptably readable 6. Just Works everywhere YAML does get 1-5 right (specifically 3 and 4 that JSON doesn't and IMO better in 5). But then it adds a ton of complexity that has left us without a standard, safe, and sane parser implementation: anchors and references (& and *) , casting (via !!), custom data types (via !), loads of other things I don't understand.

brundolf 5y ago
It's a real shame JSON doesn't have 3 and 4, because it would be so easy and it's otherwise pretty much perfect imo. No ambiguity, every value type can be identified by its first character, clean and reasonably minimal syntax.
- plusmax1 5y ago
  well there are json alternatives which fit this bill, such as HJSON. https://hjson.github.io/ https://hjson.github.io/ might not be as "common" but it has good implementations for many languages.
- MichaelMoser123 5y ago
  you could allow python like comments and strip them with a regular expression substitution before parsing. Something like this. (it strips all the lines that start with whitespace, followed by #, followed by anything until the end of the line) cat cfg.json | sed -e 's/\s*#.*$//g' | jq . Allowing multiple lines is more complicated, can't do that with regex alone.
  - brundolf 5y ago
    There are lots of ways to implement it. The problem is that one of JSON's strengths is its ubiquity: every language under the sun has half a dozen different battle-tested parsers for it. Clients and servers and everything in-between have first-class support out of the box. You can even paste it directly into JavaScript as valid code. If anybody short of a standards body tries to expand the spec, you lose out on most of that.
    - MichaelMoser123 5y ago
      i think you can possibly define how you want to use json for a configuration file; json by itself is not much more than javascript objects/maps, defined as a data format. I frankly don't think that you need to be too pious about standard compliance if dealing with a cofiguration format for your application.
      - brundolf 5y ago
        Editors, for one, will be an issue
      - SigmundA 5y ago
        >json by itself is not much more than javascript objects/maps And without comments or optional quoted keys single quotes or trailing commas, which makes editing by hand more work and IMO less readable.
sofixa 5y ago
HCL does all of those. TOML is pretty close as well.
- scrollaway 5y ago
  HCL is neither acceptably readable nor does it just work everywhere. TOML is getting there though yeah.
nanoscopic 5y ago
My JSON parser ( github.com/nanoscopic/ujsonin ) has these things: 0. Looks essentially the same as JSON 1. Core data types, and customizable data types can be added easily. 2. Arrays, Objects, and arbitrary nesting. 3. Comments ( both /* */ and // format ) 4. Multiline strings ( by default; carriage returns are no problem within strings ) 5. It is JSON with relaxed restrictions and slight addition for actual named types. 6. I've written C, Perl, and Golang implementations so far.
- jasfi 5y ago
  I like the idea of what you're doing. May I suggest a Nim port? Nim outputs C anyway, but is safe, so you'd worry less about bugs.
  - nanoscopic 5y ago
    Thank you. Glancing through the Nim documentation it doesn't appear to support goto. Searching online reveals one can hack something into the underlying instruction set, but it looks very unclean. To port my parser I would need to alter the core of the processor a bunch to make it effectively be a large switch statement inside of a loop to build the state machine. Right now all three implementation are essentially following the same logic with just the language syntax altered between them to make maintenance of all implementations simultaneously easier. I have spent some time generating ragel grammars for this sort of thing, and ragel can output a number of different languages itself. The code generated by ragel, is, though, quite messy itself and can have random problems that require workarounds, which is why I haven't decided fully upon going down the ragel path. May I ask what the point is of having a Nim port? You've listed safety; which can be better addressed by building a state machine generator and a state machine DSL. Also, the code is quite small, so I don't think "safety" is the primary concern for the codebase. Nim compiles to C, C++, and Javascript. I can already effectively run the C parser in JS by compiling to WASM ( and I've done so with my XML parser ) C++ code can call the C code without issue. A wrapper to auto-call object deletion in C++ would be trivial to write. Rewriting in Nim would not, so far as I can see, offer any benefits to me that are worth the effort.
MichaelMoser123 5y ago
>loads of other things I don't understand. also you can have multiple instances of yaml trees in one file, each one separated with -- . I think this makes it very confusing if used as a configuration language (they like to use yaml for configuration in kubernetes)
- Spivak 5y ago
  I mean sure but I’ve never seen any software actually use that feature in the wild except for “you’re allowed to concat your YAML files instead of separating them if you want.” Like I’ve never seen software require a certain number of documents in a file with different schemas.
  - MichaelMoser123 5y ago
    i saw it being used in some CI system; it was very confusing.
dheera 5y ago
Besides the above (especially comments), YAML also has one huge advantage for configuration files and that is clean diffs. JSON's lack of support for trailing commas messes up diffs.
MaxGabriel 5y ago
If Dhall was more popular, would it meet your criteria?
- deleted 5y ago
  [deleted]
inkeddeveloper 5y ago
If you come up with a format, call it IJW. It just works.
petre 5y ago
I used to think that before I had to edit the ejabberd.yml config file. Now I think it's only remotely useful as a subset to use in Jelkyll headers. Just use TOML for configuration instead.
- account42 5y ago
  > I used to think that before I had to edit the ejabberd.yml config file. Hey, it's still better than having to write the configuration in Erlang like you had to before.
alephu5 5y ago
Use dhall and serialise to JSON/YAML!

mh0pe 5y ago

My stance is that YAML is a good format for configuration management and generation -- it's wonderful at filling gaps as your deployment model increases in complexity to provide a mechanism to "render" your configuration -- much like Skylark [1] does (derived from Google's internal GCL). YAML ends up being a powerfully declarative model [2] for the state of a data structure, rather than a straight representation, ironically often enough being used in turn for an imperative model like in Ansible [3]. Definitely friendlier than JSON. But personally, I really like YAML because it lets me compose using a traits/mixins-like model using & and *, which allows for verbose, structured configuration inputs but concise configuration files. docker-compose YAML files extension fields [4], imo, are a great example of this type of model in action. When you leave this much pre-deserialization flexibility in your configuration representation, it makes building cool stuff like docker-compose ECS support x-aws-* extension keys [5] and other plugin system-type capabilities much more straightforward than, for example, adding a new language feature to HCL. [1]: https://github.com/google/skylark https://github.com/google/skylark [2]: https://en.wikipedia.org/wiki/YAML#Advanced_components https://en.wikipedia.org/wiki/YAML#Advanced_components [3]: https://docs.ansible.com/ansible/latest/user_guide/playbooks_advanced_syntax.html https://docs.ansible.com/ansible/latest/user_guide/playbooks... [4]: https://docs.docker.com/compose/compose-file/compose-file-v3/#extension-fields https://docs.docker.com/compose/compose-file/compose-file-v3... [5]: https://docs.docker.com/cloud/ecs-integration/#rolling-update https://docs.docker.com/cloud/ecs-integration/#rolling-updat...