7 ms·

Introducing TJSON: Tagged JSON with Rich Types

• laurent123456 9y ago
  I'm curious what would be the use case for this? JSON is a human readable/writable format, however this kind of syntax is not anymore: "{"nested-array:A<A<s>>": [["Nested"], ["Array!"]]}" So it feels more like a machine format, but in that case why not use a more efficient one, like a binary format?
  • adambrod 9y ago
    It's very readable if you've used a typed language before. The <> brackets are like generics.
    • dolmen 9y ago
      JSON is a subset of JavaScript. <> brackets are not in JavaScript.
      • alayne 9y ago
        That's not technically true (JSON is a subset of JavaScript) due to some extra characters allowed in JSON http://timelessrepo.com/json-isnt-a-javascript-subset http://timelessrepo.com/json-isnt-a-javascript-subset.
  • bascule 9y ago
    Hello, I created TJSON. The answer to your question can be found in the second sentence on the page: > TJSON documents are amenable to "content-aware hashing" where different encodings of the same data (including both TJSON and binary formats like Protocol Buffers, MessagePack, BSON, etc) can share the same content hash and therefore the same cryptographic signature. TJSON is designed to facilitate documents that retain the same content hash when transcoded to/from binary formats.
    • laurent123456 9y ago
      If hashing is the main concern, wouldn't a "strict" spec for JSON do the job? eg. "all keys must be sorted", "all dates must be ISO-xxx", etc.?
      • bascule 9y ago
        You're describing canonicalization, which incorporates elements of the encoding format into the final hash, and therefore does not facilitate retaining the same content hash when transcoding to different formats. Also, canonicalization is a bit of a mess. There are several incompatible canonicalization schemes for JSON, and even within a single one of those people have a difficult time implementing them correctly. See e.g. https://github.com/theupdateframework/tuf/issues/362 https://github.com/theupdateframework/tuf/issues/362
        • dolmen 9y ago
          So you are just creating another non-interoperable canonicalization format that takes out from JSON everything that makes it great: terseness. https://xkcd.com/927/ https://xkcd.com/927/
          • bascule 9y ago
            You seem to have missed the point: it's not a canonicalization format. Content-aware hashing is a more powerful alternative to canonicalization which avoids many of the problems involved in designing a canonicalization scheme.
          • jerf 9y ago
            I'm not sure what you mean by "terseness". Do you mean the size of the spec? Because as serialization formats go, it is on the short side. However, typical JSON data is anything but terse; it is almost the most verbose serialization format in use, beaten out only by XML. Which I suppose could be where you're getting the idea that it is terse from, but in that case, it is "terse" only in the sense that it is the second worse of all the couple dozen common formats. It's kind of like how I've said a couple of times on HN that new compiled language designers should be grateful to C++ for setting the bar for compilation speed so low; it makes it very easy to put "compiles more quickly that C++!" into the initial elevator pitch. However, you are not "fast" merely by beating the slowest, nor are you "terse" merely by being slightly more efficient than the worst.
  • zmanian 9y ago
    The point of this format to push data over the wire in a format that is both semantically richer and authenticatable using techniques like object-hash. This gets us to one true unambiguous representation of the data which you need for redactable signatures and rich credentials.
• zwerdlds 9y ago
  Anyone care to give a first blush comparison vs protobuffs/json schema?
  • dolmen 9y ago
    JSON schema is a much more complete language for validation. And every types in the limited TJSON can be described with JSON Schema. Besides that, in JSON Schema the schema is not bundled with the data. This is a feature for input validation: the receiver must know what it allows, not just what is received. This is a feature for readability (which is a great feature of JSON) as the data is not uncumbered with the schema. A receiver is free to use a schema or not. While TJSON imposes a receiver to recognize its dirty format. So TJSON brings nothing new, except interoperability problems.
• pfooti 9y ago
  literally the only use case I see here is dates. Like everything else I can infer the type of the field based on its contents. "boolean": false, no kidding. "event_ts": 1223349483, is that an index number or milliseconds since epoch or what? Well, probably ms since epoch, but my one gripe about json is that there's no good way to push dates without domain knowledge (anything whose property name ends in _at or _ts gets converted? all numbers in a certain range get converted?)
  • geezerjay 9y ago
    > literally the only use case I see here is dates. { "date": "1937-01-01T12:00:27.87+00:20" } As you can see, JSON doesn't stop anyone from using RFC3339 to encode dates.
    • pfooti 9y ago
      Sure, and then on the other end of the connection you need to say: newThing.date = new Date(newThing.date), or else it'll deserialize as a string. What I'm getting as is that a date gets serialized into JSON as either a string or a number, depending on who wrote the toJSON method, and that the consumer of that JSON needs knowledge about the schema of the data in order to properly deserialize it.
      • dolmen 9y ago
        With TJSON the receiver will have to know about TJSON deserialization. So how is it an improvement?
      • geezerjay 9y ago
        > Sure, and then on the other end of the connection you need to say: newThing.date = new Date(newThing.date), or else it'll deserialize as a string. Only if the software running on the other end does not support your data format. Meanwhile, HTML uses string attributes to declare languages, and no one ever complained that browsers may interpret the lang tag as a string.
        • 9y ago
• escherize 9y ago
  Edn seems like a better solution here. Not o ly is the tagging more straightforward (wow not embedded in a string?), But you can write your own tags for custom types.
• tobltobs 9y ago
  Dear JS Hipsters, even if you all suffer from NIHS, could you please take look at XML before you invent another format. I am sure you will get used to those square brackets.
  • ungzd 9y ago
    This thing is designed for representing common data structures (array, set, date) in JSON. XML does not have that too. It's not validator such as XML Schema (JSON has JSON schema which is quite popular), it does not check anything.
    • zepolen 9y ago
      XML provisions an extensible way to markup your data though - TJSON is just a hack upon JSON...and it's full of potential problems that XML has solved years ago.
    • keithwhor 9y ago
      If it's not a validator, then what's the value? Why would I use this? I assume the TJSON libraries throw errors if invalid types or formats are provided --- which is good, but that makes this a validator. Developers have been representing non-standard formats in JSON for years. Google's response to JSON's limitations was the Protocol Buffer [1], and as I understand it, it's used internally relatively extensively, but there hasn't been much adoption outside of Google. JSON is just the right mix of simple + robust for the majority of use cases. [1] https://developers.google.com/protocol-buffers/ https://developers.google.com/protocol-buffers/
  • seagreen 9y ago
    First: XML is a markup language for encoding documents, JSON is a data-interchange language. Each can be twisted to do the job of the other, but they don't naturally do the same job. Second, XML is extraordinarily complicated. Flipping around the XML 1.0 spec (https://www.w3.org/TR/xml/ https://www.w3.org/TR/xml/) isn't really encouraging me that all of this is there for a reason. I'd love to be proved wrong though! In contrast, RFC 7159 is incredibly short and readable: https://tools.ietf.org/html/rfc7159 https://tools.ietf.org/html/rfc7159. The TJSON spec isn't bad either: https://www.tjson.org/spec/ https://www.tjson.org/spec/. Even combining both the result is still far shorter and more clear than XML.
• ungzd 9y ago
  This is literally hungarian notation for JSON.
• dolmen 9y ago
  As any normal JSON document is not a valid TJSON document (and worse, some JSON document may be valid TJSON documents but TJSON imposes a different interpretation) using the "JSON" suffix is just misleading.
• user5994461 9y ago
  If you want JSON with type checking, use a json schema. http://json-schema.org/examples.html http://json-schema.org/examples.html Been there for almost a decade. Already supported by all the major json libraries in all the major languages.
• keithwhor 9y ago
  Biggest complaint: Unreadable format, as mentioned in this thread. {"key:A<A<s>>":[["values"],["here"]]} This doesn't mean anything to me as a developer, unless I've seen the spec. It's kludgy. It's not reverse-compatible if you don't install a TJSON parser. Two solutions immediately strike me as better, one has been mentioned here. (1) Not optimal, but actually spell out words in key names. There's no reason "A" has to mean Array. That doesn't mean anything to me. If I'm seeing it for the first time and have no idea what TJSON is, the very next value could be "key2:B<B<t>>". (2) Far more optimal: as an example has been provided with "date", just nest objects as values for any extended types. Then this spec is completely reverse compatible and compliant, and as a developer I don't have to worry about parsing key names. e.g. { "some_nested_array": { "type": "array.array.string", "value": [ ["values"], ["here"] ] } } Extremely easy to implement and not reliant on a governing body.
  • kafkaesq 9y ago
    And that was the beauty of JSON: There was no "format", per se. And you certainly didn't need to think or care about some friggen "spec". All you had to do was sort of take a look at it, say "OK", and get moving with the business of writing your application.
• hajile 9y ago
  If you're making it unreadable with types, you might as well switch to a statically typed binary JSON format like bson or ubjson instead. You get smaller files, faster parsing, partial parsing (skip what you don't need), and (in some implementations) streaming of large files. http://ubjson.org/ http://ubjson.org/ http://bsonspec.org/ http://bsonspec.org/
• dep_b 9y ago
  Backward compatibility seems terrible to me. A regular JSON parser will produce garbage from this since variable names are changed while an XML parser parsing without any context how to parse specific fields will still provide correct data. Your dates might remain strings for example but the string is still correct.
• panic 9y ago
  Some previous discussion: https://news.ycombinator.com/item?id=12856968 https://news.ycombinator.com/item?id=12856968
• portlander12345 9y ago
  Surprised no one here has mentioned Transit. It's an extensible typed data format with both JSON and binary representations. In other words, you can configure custom types, such as Immutable data structures, and they'll be automatically serialized and restored for you. Good intro: http://cognitect.github.io/transit-tour/ http://cognitect.github.io/transit-tour/ GitHub: https://github.com/cognitect/transit-js https://github.com/cognitect/transit-js Note that in the introduction they provide a simple benchmark where Transit is both more compact and faster to parse than JSON with custom hydration.
• keredson 9y ago
  Why would you define the type of something in the PARENT object?
  • cartercole 9y ago
    so you can be cute and have a reason to enforce no top level arrays with the added benefit of ajax security... but really why would you dirty your key that allows fast lookup with type information and keep yourself from using : in keys
• Entangled 9y ago
  I prefer YAML no matter how much lipstick you put on JSON.
  • dolmen 9y ago
    A major problem with YAML is that the spec is so complex that no existing parser implements it fully. And each implementation supports a different subset. YAML is just not interoperable.
• TazeTSchnitzel 9y ago
  I question the value of the tags where they state the obvious. Does this {"foo:O":{}} really tell you more than {"foo":{}} ? The ability to encode sets, integers, binary data and time stamps is useful. But why tag things which are what they look like? It's a waste of space.
  • bascule 9y ago
    Domain separation. Unless everything is tagged, an attacker can trick the parser into misinterpreting the type of an object. Or, a more mundane explanation: the parser will silently clobber the name because it contains a ":" Leaving any names untagged is ambiguous.