15 ms·

For the life of me I do not understand the JSON obsession. When I was young and naive, JSON indeed looked cool because it is both human and computer readable. U

For the life of me I do not understand the JSON obsession. When I was young and naive, JSON indeed looked cool because it is both human and computer readable. Until you run into some of its limitations. For example, AFAIK there is still no agreement on how to handle binary data, e.g. what do you do with: >>> json.dumps(["\xb8\xc3\xb6\xbb"]) Traceback (most recent call last): ... UnicodeDecodeError: 'utf8' codec can't decode byte 0xb8 in position 0: invalid start byte You can base64-encode it or do a number of other things, but then that's outside the JSON standard, and suddenly it isn't the flexible encoding standard it seemed at first...

maxxxxx 10y ago
Totally agree. XML is probably over engineered but JSON is under engineered. No date type, no binaries, no comments, no easy append to existing files. We jumped from one bad standard format to another bad standard format. Other than maybe being a little easier on the eye I don't see how JSON is an improvement over other formats.
- onli 10y ago
  Why would you need any of that? A date type is bloat, store an epoch number. Comments have no place in a data format. Append is exactly the same as with XML: You take the part you want and add it at the right place in a list of the other file. I'd argue the lack of namespaces makes it easier than with XML. For the binary data: JSON is not a binary format, one should not try to mix those. To encode it and store it as a string is exactly right. Otherwise a database as blob store is the better place for that.
  - maxxxxx 10y ago
    Comments are important ina format that's being used for configuration as JSON is. Commenting out things makes for easier revert if something doesn't work. Having a Line of comment in front of each setting is also a good thing. This is just basic practicality. You can't easily append something to an already existing JSON file. You have to decode the whole thing first and then write it back whole. XML has the same problem. That's why I said we replaced one bad format with another. JSON is not an improvement over XML. It replaced one set of weaknesses with another.
    - IgorPartola 10y ago
      Maybe we shouldn't use JSON as a configuration format. There are numerous better alternatives such as INI files and YAML.
      - maxxxxx 10y ago
        I personally use INI files quit a bit but the cool kids insist on JSON.
        dvirsky 10y ago
        TOML seems like a nice middle ground, it's gaining some popularity in the Go world.
        merb 10y ago
        or HOCON which is often used in Scala ;)
      - vram22 10y ago
        INI files are cool, IMO [1]. Though they may have originated in Windows or DOS, other tech uses them too nowadays (and from long, of course). Some languages must have support for INI files in their standard or 3rd-party libraries. I know Python does, in the ConfigParser module. In fact, I'm currently using it in a project I'm working on for a client. [1] I think they are cool because of their simplicity - it is just a two-level hierarchy - sections and options within sections. And options are key-value pairs. For many applications, this is enough, and any format that supports more may be overkill (though would still work). Edited to add the [1] footnote.
    - davexunit 10y ago
      JSON is not for people to write, despite what people try to do. It's for machines.
      - pg314 10y ago
  - azeirah 10y ago
    Comments do have a place in a data format if the format is intended to be human readable.
    - natch 10y ago
      I think JSON is intended to strike the right balance between readability by machines (primary goal) and readability by humans (secondary). Comments don't make sense to the machine and are just overhead. And they tend to be abused or extended in hacky ways. So I'm glad they're not there.
    - Klathmon 10y ago
      To expand on what the other commenter said. The creator of JSON was explicitly against comments for a few reasons. The biggest one was that comments are often used to "extend" a format. Look at doc-block, annotations, etc... In order for the format to work, everyone had to use the exact same set of rules, and that meant that if people started adding their own "crap" to the format in a "backwards compatible" way you'd lose what makes it special. So he didn't add comments. And I personally think it worked out.
      - azeirah 10y ago
        Ah ok, I hadn't considered that, that's quite clever really.
      - dv_dt 10y ago
        So now lots of JSON parsers have extensions to ignore comment extensions to the format...
      - legutierr 10y ago
        This argument against the inclusion of comments is a good argument in favor of the inclusion of a date data type and a binary data type. Right now, there are a number of different ways to encode both types of data, and you see them all in the wild. And extra custom serialization and deserialization steps are always required to handle them.
  - jimbokun 10y ago
    "Otherwise a database as blob store is the better place for that." Actually, I was thinking something more like Protocol Buffers or Avro if you are looking for a binary format for transmitting structured data.
    - onli 10y ago
      Sure, both seem better for that. At that moment I was thinking only of storage, not transmitting.
    - vram22 10y ago
      Coincidentally I was looking at MessagePack recently. It seems to have some interesting features. Not done a proper study of it vs. other competing or comparable formats, though.
  - derefr 10y ago
    > A date type is bloat, store an epoch number. Unix timestamps are lossy. 1. No time zones: you can't recover the sender's time-zone from it. (Not the end of the world, obviously.) 2. No defined calendar: if you convert an arbitrary datetime into a timestamp, you don't actually have enough information to convert it back into a datetime, because the same instant of continuous time has different representations in different calendars. 3. The unix epoch is very recent, and there is no such thing as a "negative timestamp": you can't convert historical datetimes to timestamps. If you're trying to pass around e.g. medical records tagged with their dates, what do you do with the ones about things that happened before 1970? (And without problem 3, problem 2 would be even worse: you'd have to care about calendars beyond the Julian and Gregorian ones, to represent those historical dates. Some calendars don't map monotonically to continuous time! Some instants have multiple representations in the same calendar! Some instants have no representation in a given calendar!) But the worst problem of all, that affects you even if you don't care about encoding weird historical dates: 4. Timestamps don't "care about" leap-seconds [i.e. the standard doesn't force timestamps to either include or exclude them.] Therefore, every time there's a leap second, Unix time becomes less precise by one second (the integer representing a timestamp after a leap second could map to dt n, or dt n-1, depending on if [and exactly when!] the system that generated it adjusted its clock for the leap-second.) That means that right now, if an unknown system handed you a timestamp that purportedly represents the current time, you'd not actually be able to know that with less than a ±27 second error-bar. That number will only keep climbing. > to encode it and store it as a string is exactly right. That's not actually the problem. It'd be fine if JSON took binary data and then represented it in some human-readable encoding like Base64. The problem is that JSON doesn't specify the representation of binary data, or the mapping between binary data and such a representation. So you can't take a binary buffer, drop it into a JSON serializer, and expect it to pop out the other side on any random JSON-speaking system as a binary buffer. Instead, both sides have to have known characteristics (an agreement to represent binaries in JSON a certain way.) The whole point of a serialization format is to bundle up those guaranteed characteristics, so that once you know "this system speaks JSON", you don't have to ask any more questions. JSON fails at being a serialization format because you still have to ask the other side how it "expects to see" binaries represented, or dates represented.
    - 10y ago
- kedean 10y ago
  I'd argue that all of those are either problems in XML, or considered features. Datatypes and comments would give room for extensibility, which is against what JSON is intended to be. JSON should be instantly readable by any system, it shouldn't need a schema describing what all the datatypes and comment annotations mean first (because that's inevitably where comments would lead). Appending is a problem in any structural format, XML is not immune. Binary data is a problem, I will agree. The real benefit to JSON is that it's so lean and easy to parse. I can get a hashmap or a plain object out of some JSON text in a few lines of code in almost any language, the same can't really be said of XML, especially not the kind of XML usually used as a data interchange. Additionally, that lean-ness means it has significantly less data to transfer, I've seen JSON versions of data be 1/10 the size of the XML equivalent, which can matter at the far end of the spectrum. The real strength of JSON, though, is arrays. List structures have always been a major weak point of XML, as there's no way to do it without a million tags. God help you if your list has anything other than a primitive in it.
- vram22 10y ago
  >We jumped from one bad standard format to another bad standard format. So, what do or would [1] you consider a good format. Asking because I'm interested in data formats. They are inputs for my xtopdf toolkit, plus I'm generally interested in data munging and have done it a good amount. [1] "do" for existing ones, "would" for non-existent ones.
- vithlani 10y ago
  I do not think there was an 'engineering' involved -- Crockford merely stumbled across it. https://vimeo.com/8692019 https://vimeo.com/8692019
dvirsky 10y ago
I feel the same about JS in general, yet it rules the world.
jerf 10y ago
Worse is Better: http://www.dreamsongs.com/RiseOfWorseIsBetter.html http://www.dreamsongs.com/RiseOfWorseIsBetter.html Note that I think many people skim that too quickly and miss the point. It is more an explanation of why the "worse" things so often seem to beat out the "better" things than simply a cynical statement or something.
- fatbird 10y ago
  He does actually say "The lesson to be learned from this is that it is often undesirable to go for the right thing first. It is better to get half of the right thing available so that it spreads like a virus. Once people are hooked on it, take the time to improve it to 90% of the right thing." He's explicitly blessing the worse approach up front to get something working that then gets accepted and improved. It's ultimately a bird-in-the-hand argument.
- jchrisa 10y ago
  See also http://urbanhonking.com/ideasfordozens/2012/06/24/designing-for-and-against-the-manufactured-normalcy-field/ http://urbanhonking.com/ideasfordozens/2012/06/24/designing-... about how design's goal is to keep people from noticing big changes. Makes "bird in hand" a good strategy for people trying to make a dent in the universe.
pg314 10y ago
Even something as simple as integers is problematic, because they are underspecified. Since javascript uses 64 bit floating point numbers for all numeric types, 64 bit integers can not accurately be represented. If you need 64 bit integers, your best bet is to use strings...
- EpicEng 10y ago
  How is that a JSON issue? It's inability to specify the type of a data element?
  - pg314 10y ago
    A better format would specify the precision of integers so you can depend on it. Now you need to know the limitations of all implementations if you want to use it as an interchange format. It doesn't even mention the issue in the spec, even going as far as saying, "A number is very much like a C or Java number"[1], which is misleading to say the least. Your API written in C might work with a Java client, but if someone writes a JavaScript client, it will break. I'm sure there is a ton of code out there that doesn't take this precision issue into account. It seems like Twitter ran into that issue at some point, because in their APIs they have both id and id_str (see [2]). [1] http://www.json.org/ http://www.json.org/ [2] https://dev.twitter.com/overview/api/twitter-ids-json-and-snowflake https://dev.twitter.com/overview/api/twitter-ids-json-and-sn...
arenaninja 10y ago
I think if you have control over your data you can take into consideration whether this is a likely scenario for you. If it's not, then just run with it. A lot of applications can get by with zero errors sticking to JSON. The ones that can't should just use something else.
dsp1234 10y ago
For example, AFAIK there is still no agreement on how to handle binary data, e.g. what do you do with: >>> json.dumps(["\xb8\xc3\xb6\xbb"]) Strings in javascript/json are UTF-8, UTF-16, or UTF-32 encoded[0]. So providing a non-Unicode string is going to fail. Instead, use an array of numbers limited to the byte range (as you would for any other binary data). console.log(JSON.stringify({buffer: [0xb8, 0xc3, 0xb6, 0xbb]})) //{"buffer":[184,195,182,187]} [0] - http://rfc7159.net/rfc7159#rfc.section.8.1 http://rfc7159.net/rfc7159#rfc.section.8.1
- MichaelGG 10y ago
  It seems like Base64 or beyond would be a vastly better way.
  - dsp1234 10y ago
    I would personally use Base64 or Hex encoded binary data too if I controlled both sides of the communication. However, the original commenter was worried about stuff "that's outside the JSON standard,". Storing byte-sized numbers in an array is well within the JSON standard, and any system that could consume json would be able to consume it without issue, and without additional dependencies.
    - anorwell 10y ago
      It seems to me that an array of ints depends on agreed convention (beyond the JSON standard) just as much as a base64 string does. In general, interpreting the meaning of a JSON primitive (int, string, boolean) is something that is being agreed upon by the users of the format.
- tracker1 10y ago
  JSON is UTF8
phailhaus 10y ago
"I don't understand JSON, what do you do if JSON doesn't cover all your use cases?" Uh, then don't use it. Nobody said JSON was the answer to everything.
- derefr 10y ago
  Often you're forced to use it, because the infrastructure you're working with only speaks JSON (e.g. databases whose wire protocol is JSON-based, like CouchDB.) And even if you get to pick your own infrastructure, a lot of development effort is "stolen" by JSON-focused tooling and infrastructure, such that there's very little development effort given to protocols that aren't as limited. Consider: how many languages that offer a "batteries-included" JSON parser+generator, also offer a "batteries-included" parser+generator for e.g. ASN.1?
  - dickbasedregex 10y ago
    Clearly almost no one is asking for alternatives.
    - ajamesm 10y ago
      Off the top of my head -- protobuf, thrift, avro, XML, lua tables, properties files, yaml.
  - wolf550e 10y ago
    ASN.1 is an abomination, nobody would choose to use it if they could use JSON instead. Protobuf, Thrift, Cap’n Proto, Msgpack, XML, etc. have tooling for many languages.
  - tracker1 10y ago
    There are a few things that must be parsed to/from strings, but that is usually safer as a string, since implementation details may vary by environment anyway... short of using a binary representation (there are a few out there), which may or may not be faster in any given language. Also, much like shoving large blobs in most databases, maybe you shouldn't be shoving large blobs into JSON?
my_ghola 10y ago
All I want is a standard date format. I had it with .Net's `/Date(xyz)/` format.
- ihsw2 10y ago
  What's wrong with Unix timestamps (ie: integer or decimal) and UTC offset (string)?
  - SigmundA 10y ago
    Nothing, pick either and make part of the standard so that when it gets deserialized it becomes a actual date instead of a string or number. Wouldn't it be annoying if numbers and booleans didn't exist in JSON, you just used strings? Same thing, JSON has a few data types it just missing a very common one.
    - xelxebar 10y ago
      That would severely complicate implementations, not to mention encourage bad programming practices, all for dubious convenience. Somewhat surprisingly, human readable dates are an absolute mess of historical convention and complicated geophysics. To write a human readable date, at the most basic level, requires us to account for both calendar for and time zone. But even that isn't enough. To get these display dates to match up with everything else in the world we also have to worry about a whole smorgasboard of national holidays, differing leap year conventions, leap seconds, historical mishaps, typographic conventions, and host of even more obscure minutia. All this complexity obscures from the programmer that human readable dates, for the above reasons, are almost useless at reliably determining time sequences. Instead, we should think of dates as UI details on the same level as a color scheme. Usually what we want is better thought of as a point in a time sequence which is best represented as ITA timestamps, e.g. Unix Time.
dickbasedregex 10y ago
Beats the hell out of XML. The reason JSON works so well is b/c it's dead simple and no one can implement their own stupid version. I have seen so many takes on xml and soap over the years. Every one of them made me sneer in disgust.
- Clubber 10y ago
  Bingo. It's a way to arrange three dimensional data into a string, just like XML is. It's more pithy and saves bandwidth money.
dec0dedab0de 10y ago
The reason people like JSON is because it maps nicely to the basic data structures in most languages. And if the data doesn't already map to those structures, then it forces the API developer to come up with a serialization strategy.
mvindahl 10y ago
Pros: - it's a lot like XML but with less red tape. Looks nicer to humans. Also allows expressing the semantic difference between an (ordered) array and a map. - it maps extremely well to Javascript, and a lot of APIs are consumed by web clients written in Javascript these days I think the latter is the real explanation. That being said, I agree that JSON is not a panacea. My personal pain points: 1) Missing an elegant way to express an unordered set. IMHO the three basic collection data types are array, set, and map. I think JSON would be stronger if it could express a set like you can do in ES these days: { "a", "b " } 2) Not really DRY if you encode a large array of objects and sned them over the wire. So yeah, your API should support more compact formats as well. 3) Allowing comments would be really nice when we use JSON for e.g. package.json. I don't think it would make the parsers any slower so why not? XML has it.

maxxxxx 10y ago

Totally agree. XML is probably over engineered but JSON is under engineered. No date type, no binaries, no comments, no easy append to existing files. We jumped from one bad standard format to another bad standard format. Other than maybe being a little easier on the eye I don't see how JSON is an improvement over other formats.

onli 10y ago
Why would you need any of that? A date type is bloat, store an epoch number. Comments have no place in a data format. Append is exactly the same as with XML: You take the part you want and add it at the right place in a list of the other file. I'd argue the lack of namespaces makes it easier than with XML. For the binary data: JSON is not a binary format, one should not try to mix those. To encode it and store it as a string is exactly right. Otherwise a database as blob store is the better place for that.
- maxxxxx 10y ago
  Comments are important ina format that's being used for configuration as JSON is. Commenting out things makes for easier revert if something doesn't work. Having a Line of comment in front of each setting is also a good thing. This is just basic practicality. You can't easily append something to an already existing JSON file. You have to decode the whole thing first and then write it back whole. XML has the same problem. That's why I said we replaced one bad format with another. JSON is not an improvement over XML. It replaced one set of weaknesses with another.
  - IgorPartola 10y ago
    Maybe we shouldn't use JSON as a configuration format. There are numerous better alternatives such as INI files and YAML.
    - maxxxxx 10y ago
      I personally use INI files quit a bit but the cool kids insist on JSON.
      - dvirsky 10y ago
        TOML seems like a nice middle ground, it's gaining some popularity in the Go world.
        merb 10y ago
        or HOCON which is often used in Scala ;)
    - vram22 10y ago
      INI files are cool, IMO [1]. Though they may have originated in Windows or DOS, other tech uses them too nowadays (and from long, of course). Some languages must have support for INI files in their standard or 3rd-party libraries. I know Python does, in the ConfigParser module. In fact, I'm currently using it in a project I'm working on for a client. [1] I think they are cool because of their simplicity - it is just a two-level hierarchy - sections and options within sections. And options are key-value pairs. For many applications, this is enough, and any format that supports more may be overkill (though would still work). Edited to add the [1] footnote.
  - davexunit 10y ago
    JSON is not for people to write, despite what people try to do. It's for machines.
    - pg314 10y ago
- azeirah 10y ago
  Comments do have a place in a data format if the format is intended to be human readable.
  - natch 10y ago
    I think JSON is intended to strike the right balance between readability by machines (primary goal) and readability by humans (secondary). Comments don't make sense to the machine and are just overhead. And they tend to be abused or extended in hacky ways. So I'm glad they're not there.
  - Klathmon 10y ago
    To expand on what the other commenter said. The creator of JSON was explicitly against comments for a few reasons. The biggest one was that comments are often used to "extend" a format. Look at doc-block, annotations, etc... In order for the format to work, everyone had to use the exact same set of rules, and that meant that if people started adding their own "crap" to the format in a "backwards compatible" way you'd lose what makes it special. So he didn't add comments. And I personally think it worked out.
    - azeirah 10y ago
      Ah ok, I hadn't considered that, that's quite clever really.
    - dv_dt 10y ago
      So now lots of JSON parsers have extensions to ignore comment extensions to the format...
    - legutierr 10y ago
      This argument against the inclusion of comments is a good argument in favor of the inclusion of a date data type and a binary data type. Right now, there are a number of different ways to encode both types of data, and you see them all in the wild. And extra custom serialization and deserialization steps are always required to handle them.
- jimbokun 10y ago
  "Otherwise a database as blob store is the better place for that." Actually, I was thinking something more like Protocol Buffers or Avro if you are looking for a binary format for transmitting structured data.
  - onli 10y ago
    Sure, both seem better for that. At that moment I was thinking only of storage, not transmitting.
  - vram22 10y ago
    Coincidentally I was looking at MessagePack recently. It seems to have some interesting features. Not done a proper study of it vs. other competing or comparable formats, though.
- derefr 10y ago
  > A date type is bloat, store an epoch number. Unix timestamps are lossy. 1. No time zones: you can't recover the sender's time-zone from it. (Not the end of the world, obviously.) 2. No defined calendar: if you convert an arbitrary datetime into a timestamp, you don't actually have enough information to convert it back into a datetime, because the same instant of continuous time has different representations in different calendars. 3. The unix epoch is very recent, and there is no such thing as a "negative timestamp": you can't convert historical datetimes to timestamps. If you're trying to pass around e.g. medical records tagged with their dates, what do you do with the ones about things that happened before 1970? (And without problem 3, problem 2 would be even worse: you'd have to care about calendars beyond the Julian and Gregorian ones, to represent those historical dates. Some calendars don't map monotonically to continuous time! Some instants have multiple representations in the same calendar! Some instants have no representation in a given calendar!) But the worst problem of all, that affects you even if you don't care about encoding weird historical dates: 4. Timestamps don't "care about" leap-seconds [i.e. the standard doesn't force timestamps to either include or exclude them.] Therefore, every time there's a leap second, Unix time becomes less precise by one second (the integer representing a timestamp after a leap second could map to dt n, or dt n-1, depending on if [and exactly when!] the system that generated it adjusted its clock for the leap-second.) That means that right now, if an unknown system handed you a timestamp that purportedly represents the current time, you'd not actually be able to know that with less than a ±27 second error-bar. That number will only keep climbing. > to encode it and store it as a string is exactly right. That's not actually the problem. It'd be fine if JSON took binary data and then represented it in some human-readable encoding like Base64. The problem is that JSON doesn't specify the representation of binary data, or the mapping between binary data and such a representation. So you can't take a binary buffer, drop it into a JSON serializer, and expect it to pop out the other side on any random JSON-speaking system as a binary buffer. Instead, both sides have to have known characteristics (an agreement to represent binaries in JSON a certain way.) The whole point of a serialization format is to bundle up those guaranteed characteristics, so that once you know "this system speaks JSON", you don't have to ask any more questions. JSON fails at being a serialization format because you still have to ask the other side how it "expects to see" binaries represented, or dates represented.
  - 10y ago
kedean 10y ago
I'd argue that all of those are either problems in XML, or considered features. Datatypes and comments would give room for extensibility, which is against what JSON is intended to be. JSON should be instantly readable by any system, it shouldn't need a schema describing what all the datatypes and comment annotations mean first (because that's inevitably where comments would lead). Appending is a problem in any structural format, XML is not immune. Binary data is a problem, I will agree. The real benefit to JSON is that it's so lean and easy to parse. I can get a hashmap or a plain object out of some JSON text in a few lines of code in almost any language, the same can't really be said of XML, especially not the kind of XML usually used as a data interchange. Additionally, that lean-ness means it has significantly less data to transfer, I've seen JSON versions of data be 1/10 the size of the XML equivalent, which can matter at the far end of the spectrum. The real strength of JSON, though, is arrays. List structures have always been a major weak point of XML, as there's no way to do it without a million tags. God help you if your list has anything other than a primitive in it.
vram22 10y ago
>We jumped from one bad standard format to another bad standard format. So, what do or would [1] you consider a good format. Asking because I'm interested in data formats. They are inputs for my xtopdf toolkit, plus I'm generally interested in data munging and have done it a good amount. [1] "do" for existing ones, "would" for non-existent ones.
vithlani 10y ago
I do not think there was an 'engineering' involved -- Crockford merely stumbled across it. https://vimeo.com/8692019 https://vimeo.com/8692019

dvirsky 10y ago

I feel the same about JS in general, yet it rules the world.

jerf 10y ago

Worse is Better: http://www.dreamsongs.com/RiseOfWorseIsBetter.html http://www.dreamsongs.com/RiseOfWorseIsBetter.html Note that I think many people skim that too quickly and miss the point. It is more an explanation of why the "worse" things so often seem to beat out the "better" things than simply a cynical statement or something.

fatbird 10y ago
He does actually say "The lesson to be learned from this is that it is often undesirable to go for the right thing first. It is better to get half of the right thing available so that it spreads like a virus. Once people are hooked on it, take the time to improve it to 90% of the right thing." He's explicitly blessing the worse approach up front to get something working that then gets accepted and improved. It's ultimately a bird-in-the-hand argument.
jchrisa 10y ago
See also http://urbanhonking.com/ideasfordozens/2012/06/24/designing-for-and-against-the-manufactured-normalcy-field/ http://urbanhonking.com/ideasfordozens/2012/06/24/designing-... about how design's goal is to keep people from noticing big changes. Makes "bird in hand" a good strategy for people trying to make a dent in the universe.

pg314 10y ago

Even something as simple as integers is problematic, because they are underspecified. Since javascript uses 64 bit floating point numbers for all numeric types, 64 bit integers can not accurately be represented. If you need 64 bit integers, your best bet is to use strings...

EpicEng 10y ago
How is that a JSON issue? It's inability to specify the type of a data element?
- pg314 10y ago
  A better format would specify the precision of integers so you can depend on it. Now you need to know the limitations of all implementations if you want to use it as an interchange format. It doesn't even mention the issue in the spec, even going as far as saying, "A number is very much like a C or Java number"[1], which is misleading to say the least. Your API written in C might work with a Java client, but if someone writes a JavaScript client, it will break. I'm sure there is a ton of code out there that doesn't take this precision issue into account. It seems like Twitter ran into that issue at some point, because in their APIs they have both id and id_str (see [2]). [1] http://www.json.org/ http://www.json.org/ [2] https://dev.twitter.com/overview/api/twitter-ids-json-and-snowflake https://dev.twitter.com/overview/api/twitter-ids-json-and-sn...

arenaninja 10y ago

I think if you have control over your data you can take into consideration whether this is a likely scenario for you. If it's not, then just run with it. A lot of applications can get by with zero errors sticking to JSON. The ones that can't should just use something else.

dsp1234 10y ago

For example, AFAIK there is still no agreement on how to handle binary data, e.g. what do you do with: >>> json.dumps(["\xb8\xc3\xb6\xbb"]) Strings in javascript/json are UTF-8, UTF-16, or UTF-32 encoded[0]. So providing a non-Unicode string is going to fail. Instead, use an array of numbers limited to the byte range (as you would for any other binary data). console.log(JSON.stringify({buffer: [0xb8, 0xc3, 0xb6, 0xbb]})) //{"buffer":[184,195,182,187]} [0] - http://rfc7159.net/rfc7159#rfc.section.8.1 http://rfc7159.net/rfc7159#rfc.section.8.1

MichaelGG 10y ago
It seems like Base64 or beyond would be a vastly better way.
- dsp1234 10y ago
  I would personally use Base64 or Hex encoded binary data too if I controlled both sides of the communication. However, the original commenter was worried about stuff "that's outside the JSON standard,". Storing byte-sized numbers in an array is well within the JSON standard, and any system that could consume json would be able to consume it without issue, and without additional dependencies.
  - anorwell 10y ago
    It seems to me that an array of ints depends on agreed convention (beyond the JSON standard) just as much as a base64 string does. In general, interpreting the meaning of a JSON primitive (int, string, boolean) is something that is being agreed upon by the users of the format.
tracker1 10y ago
JSON is UTF8

phailhaus 10y ago

"I don't understand JSON, what do you do if JSON doesn't cover all your use cases?" Uh, then don't use it. Nobody said JSON was the answer to everything.

derefr 10y ago
Often you're forced to use it, because the infrastructure you're working with only speaks JSON (e.g. databases whose wire protocol is JSON-based, like CouchDB.) And even if you get to pick your own infrastructure, a lot of development effort is "stolen" by JSON-focused tooling and infrastructure, such that there's very little development effort given to protocols that aren't as limited. Consider: how many languages that offer a "batteries-included" JSON parser+generator, also offer a "batteries-included" parser+generator for e.g. ASN.1?
- dickbasedregex 10y ago
  Clearly almost no one is asking for alternatives.
  - ajamesm 10y ago
    Off the top of my head -- protobuf, thrift, avro, XML, lua tables, properties files, yaml.
- wolf550e 10y ago
  ASN.1 is an abomination, nobody would choose to use it if they could use JSON instead. Protobuf, Thrift, Cap’n Proto, Msgpack, XML, etc. have tooling for many languages.
- tracker1 10y ago
  There are a few things that must be parsed to/from strings, but that is usually safer as a string, since implementation details may vary by environment anyway... short of using a binary representation (there are a few out there), which may or may not be faster in any given language. Also, much like shoving large blobs in most databases, maybe you shouldn't be shoving large blobs into JSON?

my_ghola 10y ago

All I want is a standard date format. I had it with .Net's `/Date(xyz)/` format.

ihsw2 10y ago
What's wrong with Unix timestamps (ie: integer or decimal) and UTC offset (string)?
- SigmundA 10y ago
  Nothing, pick either and make part of the standard so that when it gets deserialized it becomes a actual date instead of a string or number. Wouldn't it be annoying if numbers and booleans didn't exist in JSON, you just used strings? Same thing, JSON has a few data types it just missing a very common one.
  - xelxebar 10y ago
    That would severely complicate implementations, not to mention encourage bad programming practices, all for dubious convenience. Somewhat surprisingly, human readable dates are an absolute mess of historical convention and complicated geophysics. To write a human readable date, at the most basic level, requires us to account for both calendar for and time zone. But even that isn't enough. To get these display dates to match up with everything else in the world we also have to worry about a whole smorgasboard of national holidays, differing leap year conventions, leap seconds, historical mishaps, typographic conventions, and host of even more obscure minutia. All this complexity obscures from the programmer that human readable dates, for the above reasons, are almost useless at reliably determining time sequences. Instead, we should think of dates as UI details on the same level as a color scheme. Usually what we want is better thought of as a point in a time sequence which is best represented as ITA timestamps, e.g. Unix Time.

dickbasedregex 10y ago

Beats the hell out of XML. The reason JSON works so well is b/c it's dead simple and no one can implement their own stupid version. I have seen so many takes on xml and soap over the years. Every one of them made me sneer in disgust.

Clubber 10y ago
Bingo. It's a way to arrange three dimensional data into a string, just like XML is. It's more pithy and saves bandwidth money.

dec0dedab0de 10y ago

The reason people like JSON is because it maps nicely to the basic data structures in most languages. And if the data doesn't already map to those structures, then it forces the API developer to come up with a serialization strategy.

mvindahl 10y ago

Pros: - it's a lot like XML but with less red tape. Looks nicer to humans. Also allows expressing the semantic difference between an (ordered) array and a map. - it maps extremely well to Javascript, and a lot of APIs are consumed by web clients written in Javascript these days I think the latter is the real explanation. That being said, I agree that JSON is not a panacea. My personal pain points: 1) Missing an elegant way to express an unordered set. IMHO the three basic collection data types are array, set, and map. I think JSON would be stronger if it could express a set like you can do in ES these days: { "a", "b " } 2) Not really DRY if you encode a large array of objects and sned them over the wire. So yeah, your API should support more compact formats as well. 3) Allowing comments would be really nice when we use JSON for e.g. package.json. I don't think it would make the parsers any slower so why not? XML has it.