10 ms·

> data type does not change based on seemingly insignificant details (2, 2.0, 2.0.0, “2”) …only because NestedText does not support numeric types at all. That

by comex 6y ago

> data type does not change based on seemingly insignificant details (2, 2.0, 2.0.0, “2”) …only because NestedText does not support numeric types at all. That seems like throwing out the baby with the bathwater.

clarkevans 6y ago
In the design of YAML, Ingy made the case that we shouldn't have types for scalar values, that they should all just be strings. His argument was that each application knows what it needs and should have the ability to direct how those scalars are processed. If the format needs to be standardized across applications, one can use a schema system layered on top. In retrospect, I think he was right and I was wrong. For example, Fast Healthcare Interoperability Resources (FHIR) serialized as JSON treat numeric values as decimals, e.g. `2.3` is not a floating point value. Moreover, since parsing numbers is slow, it should be deferred till you actually need to interpret the numeric value.
crazygringo 6y ago
Indeed. It's less of a problem if you're using it for configuration files, where a program knows what key's values need to be cast to an integer or float. But it seems disastrous if you wanted to use it for storing or transmitting data, above all between applications. You're immediately throwing out the possibility of being able to serialize and then deserialize data in basically any programming language. I shudder at the idea of an API that accepted NestedText, where I'd need to worry about whether my floating-point output was compatible with its floating-point string parser. Yikes. I want the serialization format to handle that. Isn't a major criticism of JSON that it doens't have a built-in datetime representation?
- Mindless2112 6y ago
  You wouldn't use this for arbitrary object serialization. Lists can only contain strings, not other lists or dictionaries. As for stringification, JSON's data types are mismatched with pretty much everything that isn't JavaScript to some degree. If you need to serialize a 64-bit integer to JSON, you serialize it as a string because the parser on the other end is probably going to try to parse it as a double-precision floating point number. Once you've started serializing numbers as strings anyway, it's not too far to "serialize every scalar as a string".
  - crazygringo 6y ago
    Ah ha, got it. The page never really makes it clear that this is for configuration files only, not serialization. Indeed, not being able to have lists of dictionaries, or lists of lists, is very restrictive. Seems to be for very simple configurations only. E.g. a set of preferences, but not a set of monitor calibrations. (Inventing arbitrary dictionary keys seems pretty hacky.)
  - recursive 6y ago
    Well, there's nothing about the JSON spec that says you can't serialize 64-bit integers to JSON. The grammar allows unlimited precision. Implementations may vary.
  - thewakalix 6y ago
    > Lists can only contain strings, not other lists or dictionaries. I don’t think this is true. None of the examples have lists containing non-string objects, but the documentation doesn’t seem to draw a distinction between lists and dictionaries wrt what can be placed in them. (Both lists and dictionaries are initially described as only containing strings, and later this description is expanded to include nesting; this counterintuitive arrangement may explain the confusion.)
- xg15 6y ago
  To be fair, this seems to be designed specifically for config files. Just as JSON should be used for data transmission but not for config, I could imagine, this should be used for config but not for data transmission.
  - remote_phone 6y ago
    What if someone inputs 2.2.5 when the code is expecting an int? Or “abc”? It seems like it’s pushing all the config validation into user code which sucks.
    - geofft 6y ago
      The code already has that problem. Certainly, if you're using YAML, a user could type 'version: 2.2.5' where you're expecting a major version number (an int), and all of a sudden your code is passed a string instead (or a string-flavored variant). You can imagine the same sort of problem in JSON too, usually from someone leaving off a quote where you're expecting a string. NestedText's philosophy seems to be, you're going to need your code to handle this anyway, we'll just pass you a string in all cases and it's up to you to convert it to an int on your own and validate it. Frankly, in most languages, this is better because you don't have the types of objects randomly change based on user input. (In a few languages, with a few libraries, you can specify the type of the document to the parser and have it fail to parse the entire document if it can't deserialize to the right type, in which case this is a little weaker. But you can still do that with NestedText, just one step after the parser - have your own function that takes a ComplexStructure<..., String, ...> and returns a ComplexStructure<..., int, ...> or throws an error.)
      - Netch 6y ago
        > The code already has that problem. Only in untyped or dynamically-typed languages. But even in JavaScript one may write +obj.version instead of obj.version to make it numeric. Evading this is a straight way to hell. In a statically typed one, conversion is typically generated from description and type checking applies just at reading. The problem with vaguely specified format is in more simplex cases. Shall we accept 45x as number (and what it value will be, 45 or 0)? 045? 045x? What date is 1/2/3, 1-2-3? And so on.
dan-robertson 6y ago
I think avoiding numeric types is a good decision. It tends to eventually cause problems when one implementation converts numbers to doubles, another to either doubles or longs depending on whether they have a . or e, and another which converts them to bignums (or passes them as strings to the caller). One should remember that any sane application will be parsing the config file into internal data structures and validating it anyway so it gets little benefit from the numbers being already “parsed”. There are also issues when something looks numeric but doesn’t parse (eg 1.2.3, 3/2, 12in, 4h30m2s, 2:30, 2020-02-29, etc). One way to deal with these is a tokenisation rule like in Common Lisp: if it is a valid number syntax then treat it as a number, otherwise it’s a symbol, but this can lead to issues (eg you would need to know that when your number needs more than float precision or otherwise doesn’t follow the rules, it should be in quotes. It seems crazy to pass that detail on to the poor sod who has to write the config file).
- kanox 6y ago
  > One should remember that any sane application will be parsing the config file into internal data structures and validating it anyway so it gets little benefit from the numbers being already “parsed”. The benefit of standard numeric and boolean types is that different tools can exchange data in a well-understood way. Getting rid of yaml's 30 ways to write "true" and "false" by making everything is a string just means that you now have 30 tool-specific ways to write "true" and "false". The "everything is a string" approach already exists in shell scripts and TCL and it's not really that great.
  - pdonis 6y ago
    > The benefit of standard numeric and boolean types is that different tools can exchange data in a well-understood way. I don't think that's what this tool is for. This tool is for humans to read and edit data. That's a different use case from automated programs exchanging data, for which I agree you should be using standardized numeric and boolean data types and not making everything a string. But how that standardized data gets determined from data that humans enter should be up to the individual application.

by comex 6y ago

clarkevans 6y ago

In the design of YAML, Ingy made the case that we shouldn't have types for scalar values, that they should all just be strings. His argument was that each application knows what it needs and should have the ability to direct how those scalars are processed. If the format needs to be standardized across applications, one can use a schema system layered on top. In retrospect, I think he was right and I was wrong. For example, Fast Healthcare Interoperability Resources (FHIR) serialized as JSON treat numeric values as decimals, e.g. `2.3` is not a floating point value. Moreover, since parsing numbers is slow, it should be deferred till you actually need to interpret the numeric value.

crazygringo 6y ago

Indeed. It's less of a problem if you're using it for configuration files, where a program knows what key's values need to be cast to an integer or float. But it seems disastrous if you wanted to use it for storing or transmitting data, above all between applications. You're immediately throwing out the possibility of being able to serialize and then deserialize data in basically any programming language. I shudder at the idea of an API that accepted NestedText, where I'd need to worry about whether my floating-point output was compatible with its floating-point string parser. Yikes. I want the serialization format to handle that. Isn't a major criticism of JSON that it doens't have a built-in datetime representation?

Mindless2112 6y ago
You wouldn't use this for arbitrary object serialization. Lists can only contain strings, not other lists or dictionaries. As for stringification, JSON's data types are mismatched with pretty much everything that isn't JavaScript to some degree. If you need to serialize a 64-bit integer to JSON, you serialize it as a string because the parser on the other end is probably going to try to parse it as a double-precision floating point number. Once you've started serializing numbers as strings anyway, it's not too far to "serialize every scalar as a string".
- crazygringo 6y ago
  Ah ha, got it. The page never really makes it clear that this is for configuration files only, not serialization. Indeed, not being able to have lists of dictionaries, or lists of lists, is very restrictive. Seems to be for very simple configurations only. E.g. a set of preferences, but not a set of monitor calibrations. (Inventing arbitrary dictionary keys seems pretty hacky.)
- recursive 6y ago
  Well, there's nothing about the JSON spec that says you can't serialize 64-bit integers to JSON. The grammar allows unlimited precision. Implementations may vary.
- thewakalix 6y ago
  > Lists can only contain strings, not other lists or dictionaries. I don’t think this is true. None of the examples have lists containing non-string objects, but the documentation doesn’t seem to draw a distinction between lists and dictionaries wrt what can be placed in them. (Both lists and dictionaries are initially described as only containing strings, and later this description is expanded to include nesting; this counterintuitive arrangement may explain the confusion.)
xg15 6y ago
To be fair, this seems to be designed specifically for config files. Just as JSON should be used for data transmission but not for config, I could imagine, this should be used for config but not for data transmission.
- remote_phone 6y ago
  What if someone inputs 2.2.5 when the code is expecting an int? Or “abc”? It seems like it’s pushing all the config validation into user code which sucks.
  - geofft 6y ago
    The code already has that problem. Certainly, if you're using YAML, a user could type 'version: 2.2.5' where you're expecting a major version number (an int), and all of a sudden your code is passed a string instead (or a string-flavored variant). You can imagine the same sort of problem in JSON too, usually from someone leaving off a quote where you're expecting a string. NestedText's philosophy seems to be, you're going to need your code to handle this anyway, we'll just pass you a string in all cases and it's up to you to convert it to an int on your own and validate it. Frankly, in most languages, this is better because you don't have the types of objects randomly change based on user input. (In a few languages, with a few libraries, you can specify the type of the document to the parser and have it fail to parse the entire document if it can't deserialize to the right type, in which case this is a little weaker. But you can still do that with NestedText, just one step after the parser - have your own function that takes a ComplexStructure<..., String, ...> and returns a ComplexStructure<..., int, ...> or throws an error.)
    - Netch 6y ago
      > The code already has that problem. Only in untyped or dynamically-typed languages. But even in JavaScript one may write +obj.version instead of obj.version to make it numeric. Evading this is a straight way to hell. In a statically typed one, conversion is typically generated from description and type checking applies just at reading. The problem with vaguely specified format is in more simplex cases. Shall we accept 45x as number (and what it value will be, 45 or 0)? 045? 045x? What date is 1/2/3, 1-2-3? And so on.

dan-robertson 6y ago

I think avoiding numeric types is a good decision. It tends to eventually cause problems when one implementation converts numbers to doubles, another to either doubles or longs depending on whether they have a . or e, and another which converts them to bignums (or passes them as strings to the caller). One should remember that any sane application will be parsing the config file into internal data structures and validating it anyway so it gets little benefit from the numbers being already “parsed”. There are also issues when something looks numeric but doesn’t parse (eg 1.2.3, 3/2, 12in, 4h30m2s, 2:30, 2020-02-29, etc). One way to deal with these is a tokenisation rule like in Common Lisp: if it is a valid number syntax then treat it as a number, otherwise it’s a symbol, but this can lead to issues (eg you would need to know that when your number needs more than float precision or otherwise doesn’t follow the rules, it should be in quotes. It seems crazy to pass that detail on to the poor sod who has to write the config file).

kanox 6y ago
> One should remember that any sane application will be parsing the config file into internal data structures and validating it anyway so it gets little benefit from the numbers being already “parsed”. The benefit of standard numeric and boolean types is that different tools can exchange data in a well-understood way. Getting rid of yaml's 30 ways to write "true" and "false" by making everything is a string just means that you now have 30 tool-specific ways to write "true" and "false". The "everything is a string" approach already exists in shell scripts and TCL and it's not really that great.
- pdonis 6y ago
  > The benefit of standard numeric and boolean types is that different tools can exchange data in a well-understood way. I don't think that's what this tool is for. This tool is for humans to read and edit data. That's a different use case from automated programs exchanging data, for which I agree you should be using standardized numeric and boolean data types and not making everything a string. But how that standardized data gets determined from data that humans enter should be up to the individual application.