3 ms·

A practical issue with YAML: a schema is not documentation

10 points by zdw 4y ago

nickm12 4y ago
This is a good critique, but there is nothing specific to YAML in it. For any data serialization format (XML, JSON, Avro, protobufs), a schema in a schema IDL is a good idea and often required. However, the schema alone cannot tell you everything a user should know about the data. The thing I frequently see is systems where there is a source file in a schema IDL and one comment per field as the "documentation". This is something, but for most non-trivial data there are things that should be documented at a higher level than the field.
sergius 4y ago
Do not confuse the data markup with a schema DSL. In the latter the meaning of each field should be specified and available for a documentation generating tool. Bottom line: You need a better schema definition and validation tool/framework. It is not YAML's fault.
rad_gruchalski 4y ago
I have a practical problem with this post. What’s the problem, is yaml the problem and I should use something else (what), or is the problem that the documentation (the schema) is not detailed enough.
AtlasBarfed 4y ago
But yaml + comments (+ commented out optional settings) is basically the best semi self documenting config format in existence. Jason doesn't allow comments, Xml is too verbose and lacks a zero-depth later (aka a properties file), properties files can't do arrays or hierarchies or complex inline stuff, and aren't well standardized, and TOML is ugly. I'm not some yaml zealot, it's not a perfect format, but ... I've seen nothing better.
- AndyKluger 4y ago
  In case you're interested and haven't seen it, I've become a big fan of NestedText, which is similar to YAML but without the complicated parts, and without types (just strings, lists, and dicts). The idea is that any meaningful validation and coercion belongs in code anyway. An extra cool part is that nothing ever needs to be escaped, so the content is super clean and unambiguous. If you want to play around with it, I made NestedTextTo (nt2 on PyPI), for CLI conversion between NestedText and YAML, TOML, or JSON, with a pretty cool (IMO) way to cast value types along the way. https://nestedtext.org/en/stable/ https://nestedtext.org/en/stable/ (not my project) https://github.com/AndydeCleyre/nestedtextto https://github.com/AndydeCleyre/nestedtextto (my project)
sweetgiorni 4y ago
Isn't "schema" sort of meaningless if you don't specify the metaschema? Assuming this post is referring to the ever popular JSON Schema, there is no shortage of fields devoted to documenting the schema objects. So yes, a schema most certainly is documentation - as long as you actually write the documentation. https://json-schema.org/understanding-json-schema/reference/generic.html#annotations https://json-schema.org/understanding-json-schema/reference/...

10 points by zdw 4y ago

This is a good critique, but there is nothing specific to YAML in it. For any data serialization format (XML, JSON, Avro, protobufs), a schema in a schema IDL is a good idea and often required. However, the schema alone cannot tell you everything a user should know about the data. The thing I frequently see is systems where there is a source file in a schema IDL and one comment per field as the "documentation". This is something, but for most non-trivial data there are things that should be documented at a higher level than the field.

sergius 4y ago

Do not confuse the data markup with a schema DSL. In the latter the meaning of each field should be specified and available for a documentation generating tool. Bottom line: You need a better schema definition and validation tool/framework. It is not YAML's fault.

rad_gruchalski 4y ago

I have a practical problem with this post. What’s the problem, is yaml the problem and I should use something else (what), or is the problem that the documentation (the schema) is not detailed enough.

AtlasBarfed 4y ago

But yaml + comments (+ commented out optional settings) is basically the best semi self documenting config format in existence. Jason doesn't allow comments, Xml is too verbose and lacks a zero-depth later (aka a properties file), properties files can't do arrays or hierarchies or complex inline stuff, and aren't well standardized, and TOML is ugly. I'm not some yaml zealot, it's not a perfect format, but ... I've seen nothing better.

AndyKluger 4y ago
In case you're interested and haven't seen it, I've become a big fan of NestedText, which is similar to YAML but without the complicated parts, and without types (just strings, lists, and dicts). The idea is that any meaningful validation and coercion belongs in code anyway. An extra cool part is that nothing ever needs to be escaped, so the content is super clean and unambiguous. If you want to play around with it, I made NestedTextTo (nt2 on PyPI), for CLI conversion between NestedText and YAML, TOML, or JSON, with a pretty cool (IMO) way to cast value types along the way. https://nestedtext.org/en/stable/ https://nestedtext.org/en/stable/ (not my project) https://github.com/AndydeCleyre/nestedtextto https://github.com/AndydeCleyre/nestedtextto (my project)

sweetgiorni 4y ago

Isn't "schema" sort of meaningless if you don't specify the metaschema? Assuming this post is referring to the ever popular JSON Schema, there is no shortage of fields devoted to documenting the schema objects. So yes, a schema most certainly is documentation - as long as you actually write the documentation. https://json-schema.org/understanding-json-schema/reference/generic.html#annotations https://json-schema.org/understanding-json-schema/reference/...