9 ms·

To think about the difference between serialization formats, here's an analogy I hope will help. Protocol Buffers (and I think Thrift, and maybe Avro) are sort

To think about the difference between serialization formats, here's an analogy I hope will help. Protocol Buffers (and I think Thrift, and maybe Avro) are sort of like C or C++: you declare your types ahead of time, and then you take some binary payload and "cast" it (parse it actually) into your predefined type. If those bytes weren't actually serialized as that type, you'll get garbage. On the plus side, the fact that you declared your types statically means that you get lots of useful compile-time checking and everything is really efficient. It's also nice because you can use the schema file (ie. .proto files) to declare your schema formally and document everything. JSON and Ion are more like a Python/Javascript object/dict. Objects are just attribute-value bags. If you say it has field fooBar at runtime, now it does! When you parse, you don't have to know what message type you are expecting, because the key names are all encoded on the wire. On the downside, if you misspell a key name, nothing is going to warn you about it. And things aren't quite as efficient because the general representation has to be a hash map where every value is dynamically typed. On the plus side, you never have to worry about losing your schema file. I think this is a case where "strongly typed" isn't the clearest way to think about it. It's "statically typed" vs. "dynamically typed" that is the useful distinction.

sound_of_basker 11y ago
!!!! My understanding went up several orders of magnitude! Thank you!!
jcrites 11y ago
That's a great analogy! However, I do think strongly typed vs. weakly typed has a role in thinking about this, just a different dimension than the one you're describing. Let's say we come across a JSON structure that looks like this: {"start": "2007-03-01"} Is that a timestamp? Maybe! Does it support a time within the day? Perhaps I can write "2007-03-01T13:00:00" in ISO 8601 format if we're lucky. Can I supply a time zone? Who knows for sure? It's weakly typed data. The actual specification of that type of that field lives in a layer on top of JSON, if it's even specified at all. It might be "specified" only in terms of what the applications that handle it can parse and generate. I could drop that value into Excel and treat it as all sorts of different things. Ion by comparison has a specific data type for timestamps defined in the spec [1]. The timestamp has a canonical representation in both text and binary form. For this reason, I know that "2007-02-23T20:14:33.Z" and "2007-02-23T12:14:33.079-08:00" are valid Ion timestamp text values. In this instance I would describe Ion as strongly typed and JSON as weakly typed. Or, as the Ion documentation puts it, "richly typed". To make an analogy, weakly typed is the Excel cell that can store whatever value you put in it, or the PHP integer 1 which is considered equal to "1" (loose equality). Strongly typed is the relational database row with a column described precisely by the table schema. Weakly typed is the CSV file; strongly typed is the Ion document. [1] http://amznlabs.github.io/ion-docs/spec.html http://amznlabs.github.io/ion-docs/spec.html
- haberman 11y ago
  Ion has more data types than JSON, it's true. Ion has a timestamp type and JSON does not, so you could say it's "richer" if you want, but that just means "it has more types." However I don't think it's accurate to say that the typing of Ion is any "stronger." Both Ion and JSON are fully dynamically typed, which means that types are attached to every value on the wire. It's just that without an actual timestamp type in JSON, you have to encode timestamp data into a more generic type.
  - jcrites 11y ago
    The notions of "strong" and "weak" typing have never been particularly well-defined, but I think my usage is in line with their usual meaning: https://en.wikipedia.org/wiki/Strong_and_weak_typing https://en.wikipedia.org/wiki/Strong_and_weak_typing > Some programming languages make it easy to use a value of one type as if it were a value of another type. This is sometimes described as "weak typing". Strong typing makes it difficult to use a value of one type as if it were another. In PHP, you can compare the integer value 1 to the string value "1" and the equality test returns boolean true. Conflating integer 1 and string "1" is weak typing. A data format that expresses the concept of the timestamp 1999-12-31T23:14:33.079-08:00 using the same fundamental type as the string "Party like it's 1999!" is what I would call weakly typed. Ion does not make it easy to use a string as if it were a timestamp or vice versa. It has types like arbitrary precision decimals, or binary blobs, that can't easily be represented in a strongly-typed way in JSON. You can certainly invent a representation, like specifying strings as ISO 8601 for timestamps, or an array of numbers for binary -- actually, wait, how about a base64-encoded string instead? Where there's choice there's ambiguity. These concepts of "type" live in the application layer in JSON, instead of in the data layer like they do in Ion. Note as well that stronger is my term. The Ion documentation says "richly-typed". Certainly Ion does not include every type in the world. Perhaps a future serialization framework might capture "length" with a unit of "meters", or provide a currency type with unit "dollars", and if that existed I'd call it stronger-(ly?)-typed or more richly typed than Ion. In that case, the data layer would prevent you from accidentally converting "3 inches" to "3 centimeters" by accident, since those would be different types. That would be stronger typing than an example where you simply have the integer 3, and it's the application's job to track which integers represent inches, and which represent centimeters. So perhaps "strong" and "weak" are not the best terms, so much as "stronger" and "weaker".
dietrichepp 11y ago
That's a good description, but I'd say that we have a strongly <-> weakly typed axis and a statically <-> dynamically typed axis here. Or I might actually prefer to name the first axis poorly <-> richly typed. poorly typed <-------------> richly typed dynamic CSV, INI JSON YAML, Ion static Bencode, ASN.1 Protobuf What I mean by "richly typed" is that you would never read a timestamp off the wire and not know that it's a timestamp. By comparison, with CSV or INI files, you just have strings everywhere. Formats on the richly typed side have separate and explicit types for binary blobs and text, for example.
- haberman 11y ago
  Sure, I think your "poorly typed" vs. "richly typed" axis just refers to how many built-in types it has. It's true that CSV and INI only have one type (string). And it's true that when more types are built in, you have fewer cases where you have to just stuff your data into a specially-formatted string.
  - dietrichepp 11y ago
    Yes, that's exactly what I said.
_pmf_ 11y ago
> It's "statically typed" vs. "dynamically typed" that is the useful distinction. I officially propose to use the term "accidentally typed" or "eventually typed".

sound_of_basker 11y ago

!!!! My understanding went up several orders of magnitude! Thank you!!

jcrites 11y ago

That's a great analogy! However, I do think strongly typed vs. weakly typed has a role in thinking about this, just a different dimension than the one you're describing. Let's say we come across a JSON structure that looks like this: {"start": "2007-03-01"} Is that a timestamp? Maybe! Does it support a time within the day? Perhaps I can write "2007-03-01T13:00:00" in ISO 8601 format if we're lucky. Can I supply a time zone? Who knows for sure? It's weakly typed data. The actual specification of that type of that field lives in a layer on top of JSON, if it's even specified at all. It might be "specified" only in terms of what the applications that handle it can parse and generate. I could drop that value into Excel and treat it as all sorts of different things. Ion by comparison has a specific data type for timestamps defined in the spec [1]. The timestamp has a canonical representation in both text and binary form. For this reason, I know that "2007-02-23T20:14:33.Z" and "2007-02-23T12:14:33.079-08:00" are valid Ion timestamp text values. In this instance I would describe Ion as strongly typed and JSON as weakly typed. Or, as the Ion documentation puts it, "richly typed". To make an analogy, weakly typed is the Excel cell that can store whatever value you put in it, or the PHP integer 1 which is considered equal to "1" (loose equality). Strongly typed is the relational database row with a column described precisely by the table schema. Weakly typed is the CSV file; strongly typed is the Ion document. [1] http://amznlabs.github.io/ion-docs/spec.html http://amznlabs.github.io/ion-docs/spec.html

haberman 11y ago
Ion has more data types than JSON, it's true. Ion has a timestamp type and JSON does not, so you could say it's "richer" if you want, but that just means "it has more types." However I don't think it's accurate to say that the typing of Ion is any "stronger." Both Ion and JSON are fully dynamically typed, which means that types are attached to every value on the wire. It's just that without an actual timestamp type in JSON, you have to encode timestamp data into a more generic type.
- jcrites 11y ago
  The notions of "strong" and "weak" typing have never been particularly well-defined, but I think my usage is in line with their usual meaning: https://en.wikipedia.org/wiki/Strong_and_weak_typing https://en.wikipedia.org/wiki/Strong_and_weak_typing > Some programming languages make it easy to use a value of one type as if it were a value of another type. This is sometimes described as "weak typing". Strong typing makes it difficult to use a value of one type as if it were another. In PHP, you can compare the integer value 1 to the string value "1" and the equality test returns boolean true. Conflating integer 1 and string "1" is weak typing. A data format that expresses the concept of the timestamp 1999-12-31T23:14:33.079-08:00 using the same fundamental type as the string "Party like it's 1999!" is what I would call weakly typed. Ion does not make it easy to use a string as if it were a timestamp or vice versa. It has types like arbitrary precision decimals, or binary blobs, that can't easily be represented in a strongly-typed way in JSON. You can certainly invent a representation, like specifying strings as ISO 8601 for timestamps, or an array of numbers for binary -- actually, wait, how about a base64-encoded string instead? Where there's choice there's ambiguity. These concepts of "type" live in the application layer in JSON, instead of in the data layer like they do in Ion. Note as well that stronger is my term. The Ion documentation says "richly-typed". Certainly Ion does not include every type in the world. Perhaps a future serialization framework might capture "length" with a unit of "meters", or provide a currency type with unit "dollars", and if that existed I'd call it stronger-(ly?)-typed or more richly typed than Ion. In that case, the data layer would prevent you from accidentally converting "3 inches" to "3 centimeters" by accident, since those would be different types. That would be stronger typing than an example where you simply have the integer 3, and it's the application's job to track which integers represent inches, and which represent centimeters. So perhaps "strong" and "weak" are not the best terms, so much as "stronger" and "weaker".

dietrichepp 11y ago

That's a good description, but I'd say that we have a strongly <-> weakly typed axis and a statically <-> dynamically typed axis here. Or I might actually prefer to name the first axis poorly <-> richly typed. poorly typed <-------------> richly typed dynamic CSV, INI JSON YAML, Ion static Bencode, ASN.1 Protobuf What I mean by "richly typed" is that you would never read a timestamp off the wire and not know that it's a timestamp. By comparison, with CSV or INI files, you just have strings everywhere. Formats on the richly typed side have separate and explicit types for binary blobs and text, for example.

haberman 11y ago
Sure, I think your "poorly typed" vs. "richly typed" axis just refers to how many built-in types it has. It's true that CSV and INI only have one type (string). And it's true that when more types are built in, you have fewer cases where you have to just stuff your data into a specially-formatted string.
- dietrichepp 11y ago
  Yes, that's exactly what I said.

_pmf_ 11y ago

> It's "statically typed" vs. "dynamically typed" that is the useful distinction. I officially propose to use the term "accidentally typed" or "eventually typed".