5 ms·

All of the keys in JSON must be strings, so they should not need tags for themselves. Instead why not put the tag of the value assigned to the key in the key:

by bruth 10y ago

All of the keys in JSON must be strings, so they should not need tags for themselves. Instead why not put the tag of the value assigned to the key in the key: { "s:string":"Hello, world!", "b64:binary":"SGVsbG8sIHdvcmxk", "i:integer":42, "f:float":42.0, "t:timestamp":"2016-11-02T02:07:30Z" } This prevents having to mess with the values in general and integers don't need to be encoded as strings. EDIT: I see this constraint: Member names in TJSON must be distinct. The use of the same member name more than once in the same object is an error. which is still satisfied, however you could have `i:foo` and `s:foo` which would result in redundant keys in the resulting JSON document. This constraint could be clarified that, untagged key names must be unique. Another question, is a mimetype planned for this? `application/tjson`?

nemothekid 10y ago
I think you may still want to encode integers as strings anyways if you are encoding/decoding in Javascript.
bascule 10y ago
That is not the case. Binary data is also allowed as the keys of objects (see https://www.tjson.org https://www.tjson.org or the spec). As noted in the "Content-Aware Hashing" section, an intended future feature is to support redaction, so tags on keys are needed to support this feature. Finally, if you were to do it that way I think it would make more sense to place the type tags on the values, not the keys, both visually and semantically.
- bruth 10y ago
  > That is not the case. Binary data is also allowed as the keys of objects What is the value of binary key? A key is just the name for a value, it should not contain any data itself. > I think it would make more sense to place the type tags on the values, not the keys, both visually and semantically. Tags on keys are like types for columns or any other schema. I would rather not have to pre-process the values. To be pedantic, this would require copying all string-based values just to add a prefix.
  - bascule 10y ago
    Binary keys are useful anywhere data is named/identified by a cryptographic key or hash, such as content-addressable systems: https://en.wikipedia.org/wiki/Content-addressable_storage https://en.wikipedia.org/wiki/Content-addressable_storage A keyring where the object members are named by public keys is another example of where binary keys are useful. Tags on keys are like types for columns or any other schema. I would rather not have to pre-process the values. Tags on keys do not work for arrays, at least as the format is presently specified. They could potentially work if arrays always consist of homogenous types, and objects were the only allowed nonterminal allowed by the root symbol. See: https://github.com/tjson/tjson-spec/issues/23 https://github.com/tjson/tjson-spec/issues/23
    - heavenlyhash 10y ago
      I'd like to register a weak vote of dissent on this. And I'm pretty well down the conversion funnel on "Content-addressable is the way, the truth, and the light". Binary keys are very dubious. I'd rather a format without them. It's just incredibly annoying to work with non-string keys in almost every language. To pick an example, just for the sake of being concrete: in golang, `map[string]interface{}` is manageable; `map[interface{}]interface{}` is utterly disgusting to work with. We have to print keys, almost invariably. Values we can sometimes shrug and say "...elided [binary content]...", but doing it on the keys is typically nonviable. Keeping the data in binary and choosing ways to stringify it to print at runtime has historically been a disaster: keys, key fingerprints, which base-$N they're going to use, and so forth, has been an unmitigated trainwreck in openssl and its ilk. I have a cheatsheet of pgp and ssl commands to print key fingerprints in various formats and I hate that cheatsheet with the cold weeping fire of a disintegrating neutron. Let's not do that again, for anything, ever, please. Picking a format composed of printable characters once and using it consistently in an application is the far better road. Non-string keys are something that if permitted, almost no one will ever use; and yet every client library will have a massively more complicated interface in order to handle. At the same time, if my prior experience with people using e.g. yaml parsers that return wildcard types for keys is any indication, every caller will so aggressively disregard the feature that whether libraries support it will be moot: callers writing in any strongly typed language will write code that rejects non-string keys out of hand anyway in order to simply the rest of their program. I can't imagine the battle to be worth fighting.
snarf21 10y ago
I agree, why define a new format that is more verbose when you can just make it by convention at first and let parsers evolve. I probably wouldn't use : even in quotes to prevent confusion. Something like this seems safe and doesn't break anything: { "string$s":"Hello, world!", "binary$b64:":"SGVsbG8sIHdvcmxk", "integer$i":42, "float$f":42.0, "timestamp$t":"2016-11-02T02:07:30Z" } This makes it easy for the parser to determine if they should perform type checking. If you run this JSON through a non typed parser, you could easily strip out the $type yourself (until they evolve as well). Surely not perfect but gives you self describing data and ability to perform type checking if desired. $0.02
- bascule 10y ago
  Putting type sigils on object keys does not solve the problem of typing arrays, unless the types of array array elements are always homogenous, disallowed as the root symbol (they are presently allowed), and are always typed by their membership in an object (and therefore by the key referring to them). This also does not solve the problem of how to type multidimensional arrays. The question of homogenous types for non-scalars is still an open issue, and is probably the best place to further discuss this: https://github.com/tjson/tjson-spec/issues/23 https://github.com/tjson/tjson-spec/issues/23 As an aesthetic note: I personally find "$" visually noisy as a sigil, and think it has generally lost favor as a sigil for commonly used expressions in programming languages, but is probably familiar to users of Perl, PHP, bash, and BASIC
  - snarf21 10y ago
    Array is a more complex issue. The biggest issue is whether it can break existing parsers or not. Additionally, the extra typing for things like heterogeneous or nested arrays will require application code that understands the typing instead of leaving that up to the parser. I think the simplest rule for now would be to only allow homogeneous arrays. This is quite an interesting problem. (Other suggestions to send along a JSONSchema seem unrealistic and the beauty of JSON is its simplicity and brevity, nobody wants another XML) { homog1$ai : [2, 3, 4] } { homog2$as : ["a", "b", "c"] } I don't love $ but _ is so much more likely to be used in a key name for clarity like first_name. I also doubt that many people end their keys with $type so there is unlikely to be a conflict. If they do, it is probably a code standard they are using internally anyway for a similar purpose. Personally, I think that things like jQuery, etc. have trained people to see $ as a marker for "identifier" that I think it feels pretty natural, at least at this point. Again, just my $0.02 and you're mileage may vary....
bascule 10y ago
For anyone interested in further discussing encoding type information about object members in the names instead of the values, there's an open issue on the GitHub repo for the spec: https://github.com/tjson/tjson-spec/issues/28 https://github.com/tjson/tjson-spec/issues/28 Regarding MIME types, since the format is JSON-compatible I would prefer it remain "application/json" however, "application/json+tjson" might make sense.
agumonkey 10y ago
A human readable bencode.

by bruth 10y ago

nemothekid 10y ago

I think you may still want to encode integers as strings anyways if you are encoding/decoding in Javascript.

bascule 10y ago

That is not the case. Binary data is also allowed as the keys of objects (see https://www.tjson.org https://www.tjson.org or the spec). As noted in the "Content-Aware Hashing" section, an intended future feature is to support redaction, so tags on keys are needed to support this feature. Finally, if you were to do it that way I think it would make more sense to place the type tags on the values, not the keys, both visually and semantically.

bruth 10y ago
> That is not the case. Binary data is also allowed as the keys of objects What is the value of binary key? A key is just the name for a value, it should not contain any data itself. > I think it would make more sense to place the type tags on the values, not the keys, both visually and semantically. Tags on keys are like types for columns or any other schema. I would rather not have to pre-process the values. To be pedantic, this would require copying all string-based values just to add a prefix.
- bascule 10y ago
  Binary keys are useful anywhere data is named/identified by a cryptographic key or hash, such as content-addressable systems: https://en.wikipedia.org/wiki/Content-addressable_storage https://en.wikipedia.org/wiki/Content-addressable_storage A keyring where the object members are named by public keys is another example of where binary keys are useful. Tags on keys are like types for columns or any other schema. I would rather not have to pre-process the values. Tags on keys do not work for arrays, at least as the format is presently specified. They could potentially work if arrays always consist of homogenous types, and objects were the only allowed nonterminal allowed by the root symbol. See: https://github.com/tjson/tjson-spec/issues/23 https://github.com/tjson/tjson-spec/issues/23
  - heavenlyhash 10y ago
    I'd like to register a weak vote of dissent on this. And I'm pretty well down the conversion funnel on "Content-addressable is the way, the truth, and the light". Binary keys are very dubious. I'd rather a format without them. It's just incredibly annoying to work with non-string keys in almost every language. To pick an example, just for the sake of being concrete: in golang, `map[string]interface{}` is manageable; `map[interface{}]interface{}` is utterly disgusting to work with. We have to print keys, almost invariably. Values we can sometimes shrug and say "...elided [binary content]...", but doing it on the keys is typically nonviable. Keeping the data in binary and choosing ways to stringify it to print at runtime has historically been a disaster: keys, key fingerprints, which base-$N they're going to use, and so forth, has been an unmitigated trainwreck in openssl and its ilk. I have a cheatsheet of pgp and ssl commands to print key fingerprints in various formats and I hate that cheatsheet with the cold weeping fire of a disintegrating neutron. Let's not do that again, for anything, ever, please. Picking a format composed of printable characters once and using it consistently in an application is the far better road. Non-string keys are something that if permitted, almost no one will ever use; and yet every client library will have a massively more complicated interface in order to handle. At the same time, if my prior experience with people using e.g. yaml parsers that return wildcard types for keys is any indication, every caller will so aggressively disregard the feature that whether libraries support it will be moot: callers writing in any strongly typed language will write code that rejects non-string keys out of hand anyway in order to simply the rest of their program. I can't imagine the battle to be worth fighting.

snarf21 10y ago

I agree, why define a new format that is more verbose when you can just make it by convention at first and let parsers evolve. I probably wouldn't use : even in quotes to prevent confusion. Something like this seems safe and doesn't break anything: { "string$s":"Hello, world!", "binary$b64:":"SGVsbG8sIHdvcmxk", "integer$i":42, "float$f":42.0, "timestamp$t":"2016-11-02T02:07:30Z" } This makes it easy for the parser to determine if they should perform type checking. If you run this JSON through a non typed parser, you could easily strip out the $type yourself (until they evolve as well). Surely not perfect but gives you self describing data and ability to perform type checking if desired. $0.02

bascule 10y ago
Putting type sigils on object keys does not solve the problem of typing arrays, unless the types of array array elements are always homogenous, disallowed as the root symbol (they are presently allowed), and are always typed by their membership in an object (and therefore by the key referring to them). This also does not solve the problem of how to type multidimensional arrays. The question of homogenous types for non-scalars is still an open issue, and is probably the best place to further discuss this: https://github.com/tjson/tjson-spec/issues/23 https://github.com/tjson/tjson-spec/issues/23 As an aesthetic note: I personally find "$" visually noisy as a sigil, and think it has generally lost favor as a sigil for commonly used expressions in programming languages, but is probably familiar to users of Perl, PHP, bash, and BASIC
- snarf21 10y ago
  Array is a more complex issue. The biggest issue is whether it can break existing parsers or not. Additionally, the extra typing for things like heterogeneous or nested arrays will require application code that understands the typing instead of leaving that up to the parser. I think the simplest rule for now would be to only allow homogeneous arrays. This is quite an interesting problem. (Other suggestions to send along a JSONSchema seem unrealistic and the beauty of JSON is its simplicity and brevity, nobody wants another XML) { homog1$ai : [2, 3, 4] } { homog2$as : ["a", "b", "c"] } I don't love $ but _ is so much more likely to be used in a key name for clarity like first_name. I also doubt that many people end their keys with $type so there is unlikely to be a conflict. If they do, it is probably a code standard they are using internally anyway for a similar purpose. Personally, I think that things like jQuery, etc. have trained people to see $ as a marker for "identifier" that I think it feels pretty natural, at least at this point. Again, just my $0.02 and you're mileage may vary....

bascule 10y ago

For anyone interested in further discussing encoding type information about object members in the names instead of the values, there's an open issue on the GitHub repo for the spec: https://github.com/tjson/tjson-spec/issues/28 https://github.com/tjson/tjson-spec/issues/28 Regarding MIME types, since the format is JSON-compatible I would prefer it remain "application/json" however, "application/json+tjson" might make sense.

agumonkey 10y ago

A human readable bencode.