8 ms·

Show HN: JSON5 — modern JSON

64 points by aseemk 14y ago

aseemk 14y ago
Author here. This idea stemmed from long-running frustration every time I hand-wrote JSON (particularly common for Node/npm packages and test cases). After stewing on it for over a year, I bit the bullet today and made it a reality. Feedback greatly welcome!
- thwarted 14y ago
  Why are you hand writing JSON? Pass a JavaScript object, which doesn't have these limitations, to the JSON serialerizer. The support for evaling in the various engines doesn't matter because you should not be evaling JSON strings for security purposes. The serialization format is strict for a reason: look at where less strict HTML parsing has gotten us.
  - aseemk 14y ago
    Writing, or at least editing, JSON by hand seems unavoidable if you develop on Node, as modules' "manifests" are JSON files. I also love to make tests data-driven where possible. JSON is the natural format for data on Node, and again, test cases are usually written by hand (before code exists to generate or validate them). I'm not sure I understand your point about eval'ing. This parser doesn't use eval(), and it works in every modern browser. Finally, I'm not advocating for arbitrary looseness. These additions conform directly to ES5's additions.
    - aurelianito 14y ago
      I believe you should do something like this: json.stringify({unquoted:'Some value', trailing_comma:true,}) instead of hand-writting JSON data in tests.
    - ZenPsycho 14y ago
      I have implemented a javascript eval into my text editor. When I select text, and call the command, it passes it off to a node.JS engine and replaces the selection with the return result. In this way I can write a JSON.stringify([anyvalid javascript],null, [indentation]); and instantly get perfect JSON. This is much less troublesome than trying to make a new slightly incompatible format.
- stock_toaster 14y ago
  * Objects and arrays can have trailing commas.* If I had a penny for every time a trailing comma in a json config file bit me, I would have...many pennies! I am indifferent about single vs double quotes. The backslash newline escape seems 'messy'. Not a fan of keys not being quoted. That stikes me as wrongish for some reason that I can't put my finger on.
- benatkin 14y ago
  Please don't call it JSON. Actually I don't care much, as your project will be shunned. The first rule of JSON is that JSON doesn't change.
- gurkendoktor 14y ago
  I absolutely hate hand-writing JSON, but that's why I use YAML. I think you should push for a properly documented subset of YAML, or a full YAML parser in JavaScript instead. Just a Ruby programmer's 2 Eurocent :)
  - aseemk 14y ago
    Thanks for the feedback! I never considered YAML precisely because there doesn't seem to be much support for it. I'll definitely look into it. =)
- brentashley 14y ago
  Douglas built the JSON spec 1) to be language agnostic. Following ES5's syntax directions is fine for ES5 focused development, possibly not for others. 2) to be final - i.e. there is no version number. If you want to extend or build something else, I am fairly certain Douglas would prefer that you call it something else. See http://inkdroid.org/journal/2012/04/30/lessons-of-json/ http://inkdroid.org/journal/2012/04/30/lessons-of-json/ at 5:30 in the video. Don't take these points as criticisms, I too would like to be able to use single quotes and trailing commas and everything else here. I think though that in order to work with JSON as a static spec, an alternative approach could be to have your project be a layer like HAML, SASS or CoffeeScript. Each of those tools compiles or transforms to conforming input for their target specs rather than replacing them.
  - aseemk 14y ago
    JSON5 is also meant to be language-agnostic. It derives its syntax from ES5 in the same way that JSON derived its syntax from ES3. I'm aware that JSON is final — it's my hope that a new format that's easier to write (whether this or something else) picks up steam — but thanks for the feedback on the name! I consciously used a different file extension (.json5) to avoid conflicts; hopefully that's a good start. I already use (and love) CoffeeScript, but my motivation for building this was to have tools use it natively, so that sibling files in a different format wouldn't have to be maintained alongside the needed JSON.
- melloclello 14y ago
  Horrible. There's a reason JSON is homoiconic with valid javascript. You should never have done this.
- wvenable 14y ago
  JSON is an amazing serialization format but hand-written JSON is completely wrong. I think this fixes most of the problems of using it in hand-written situations. Just the lack of comments in JSON makes it inappropriate for configuration files and tests. But this shouldn't be called JSON. JSON is strict and doesn't include comments because it's designed for interoperability between different systems. Just because it's been abused to be used for configuration files doesn't mean it needs fixing. One just needs to use the right tool for the job. Your project might be the right tool but then it's not JSON.
cheeaun 14y ago
JSON 3 here: https://github.com/bestiejs/json3 https://github.com/bestiejs/json3
- nfriedly 14y ago
  I believe Facebook recently switched to JSON 3 for their JS SDK.
simanyay 14y ago
Three things: * Comments in JSON files are not a great idea because people might (will?) abuse them to add special instructions for their own serializers. As soon as this happens we'll have a format that is not compatible with other serializers. * JSON today is a universal format. So, what are special characters? Should '1hello' be quoted? If no, then its not a subset of JavaScript anymore. If you're writing a new specification you have to define all those things. Same with multiline strings: space after backslash? * Why? I know that hand-writing JSON can be annoying but is this really the biggest problem we have?
- aseemk 14y ago
  Comments: yes, I read Douglas Crockford's post on the matter. I guess I'm not genuinely worried about that, though, and my hope is that shipping a JS parser that works on both the client and the server will help standardize implementations from the start. Special characters: sorry I didn't write a formal spec =), but the intent is to be a pure subset of ES5 just like regular JSON is a pure subset of ES3. Unquoted object keys in ES5 can contain only letters/numbers/_/$, and only begin with letters/_/$. So yes, '1hello' would need to be quoted, just like regular ES5. I'm not sure how to respond to your why other than what I've already written. I never claimed this was the biggest problem we have. =)
  - Too 14y ago
    JSON is not exactly tied to javascript. For example the number-type is not specified in much detail. You could imagine this to be a 64bit int but if you load it into javascript it will be converted to a double and you will loose precision if your number is big.
  - simanyay 14y ago
    Crock made a post about JSON comments? Anyway, I wouldn't focus on JavaScript engines because JSON is now a universal format so for languages other than JavaScript you will have to define a formal specification.
zacharyvoase 14y ago
Can I ask why YAML wasn't good enough?
- chc 14y ago
  My thoughts exactly. YAML is a superset of JSON that already has all this stuff and more.
  - aseemk 14y ago
    I actually had no idea that YAML is a superset of JSON. Thanks! I'll look into it.
- erikpukinskis 14y ago
  Partly because JSON has the lovely feature of being able to just eval() it into an object in browsers.
  - deleted 14y ago
    [deleted]
  - alcuadrado 14y ago
    That's not the right way to parse JSON, it's really insecure
    - Too 14y ago
      And slow, JSON.parse is magnitudes faster than using user-javascript to parse it yourself.
  - chc 14y ago
    That's not really a lovely feature in this day and age. If anything it's an unfortunate artifact, as the right way to deserialize JSON is JSON.parse().
- silentbicycle 14y ago
  For the same reason sexps weren't good enough, apparently.
Mizza 14y ago
Please, don't call it JSON. There is only one JSON, and you didn't make it. It's cool that you made this, and that you made it open source, just please call it something else.
enneff 14y ago
JSON is wonderful because it is simple and unambiguous. It just works, and serialized data will always look the same. This "JSON5" thing is an abomination. Use whatever quotes you like! More string escaping rules! Adding these things would make JSON worse, not better. Here's another idea: why not write a tool that tidies up your hand-written JSON files? You could even re-use your "JSON5" parser to do it. That way you can be as lazy as you like when writing files, but not expect the rest of the world to learn your new dialect.
- aseemk 14y ago
  Thanks for the feedback. It certainly would be great if there existed tools to let me write and update JSON with nicer syntax like this; any ideas how that might be possible? My motivation for this was actually hoping that this gets adopted in some environments that use JSON. Node relies on package.json files, for example, which frequently need to be hand-edited. Maintaining a sibling file in a different format feels equally wrong to me as it seems JSON5 seems to you. =)
  - benatkin 14y ago
    That's npm, not node itself, that relies on package.json. When its creator chose it, he knew exactly what JSON is. He's maintained on HN and elsewhere that he made a good decision. So even if you stop calling this JSON, he's not buying it. The package.json files, by and large, don't have a multiline string problem. For trailing and missing commas, I think you could make something that automatically fixes them, if that's the only problem, and run it before npm is run, perhaps using a shell alias.
    - aseemk 14y ago
      You seem to be implying that making an effort to seek change in the format that npm uses is a bad idea. I'm not sure I understand why that is. Surely the status quo isn't always perfect? In the software world in general, and in Node/npm land especially, code talks. I thought it'd be more productive to build and share a working ES5-style JSON parser than to simply ask Isaac if npm would support ES5-style JSON. =) Yes, you're right about trailing commas. One motivation that such an approach couldn't address is documentation: I've frequently wanted comments in my package.json files, e.g. explaining a particular dependency version, or a script command.
greghinch 14y ago
(young) people seem to always want to find ways to skirt the "rules" that Crockford sets up (see also, jslint). Just stop. The rules he lays out are there to make your _long term_ programing life easier. Yes you may have to remember to use semicolons and quote things, but when you have to live with an application you've been working on for years, hopefully you'll learn to appreciate the fact that putting in a little extra time now saves you tons of headaches down the road.
- aseemk 14y ago
  I'm actually a big fan of Douglas Crockford. And don't worry, my code uses semicolons. ;) I'm not sure why you lump quoting keys with semicolons, though; his regular JS style guide doesn't discourage quoting object key (neither does JSLint).
  - ZenPsycho 14y ago
    there's a reason he requires it for JSON. it's that javascript has problems when you use a javascript reserved word as a key- for instance, if you were making an RPC api, and you had a json object {do:"methodname"} the javascript eval method would throw a fit. So rather than include the full list of javascript reserved words in a spec intended to be language agnostic, he just made quoting the key a requirement, which makes things vastly simpler.
    - aseemk 14y ago
      This was indeed a problem in the IE6 days. Every browser today (and ES5) supports reserved keywords as unquoted object keys. There are thus no reserved keywords for object keys in ES5, and by extension, JSON5.
      - ZenPsycho 14y ago
        No, but there ARE a ton of JSON parsers that expect and require quoted keys now. So it is, in fact, still a requirement. And I don't know about you, you may be lucky, but I still have to make my stuff work on IE6.
ZenPsycho 14y ago
This entirely misses the point of JSON. if you want these features it's perfectly acceptable to use YAML instead of JSON.
kwamenum86 14y ago
I don't think any of these suggestions make JSON easier to read, generate, or consume. What problem are you setting out to solve? Opening paragraph: "JSON is strict. Keys need to be quoted; strings can only be double-quoted; objects and arrays can't have trailing commas; and comments aren't allowed." If this represents a comprehensive list of the problems JSON5 solves I think you'll have a hard time rallying people around your cause.
- aseemk 14y ago
  Yes, I realize I didn't explain it as well in the README as I had in conversations and emails. I find JSON tedious and error-prone to generate by hand. I also frequently wish I could document the data with comments. Others have written about these same ideas, e.g. [1]; after feeling the frustration for over a year, I decided it might be productive to actually build a JS parser to get the conversation started. My main use cases are Node/npm package.json files, and test case data. [1] http://bolinfest.com/essays/json.html http://bolinfest.com/essays/json.html
  - ninjin 14y ago
    > I find JSON tedious and error-prone to generate by hand. I also frequently wish I could document the data with comments. I whole-heartedly agree with you on the tediousness, but JSON has this dubious double-nature that it is sometimes written by hand but mostly used as a protocol. As a protocol we want (and I would even argue need) this strictness since it makes day-to-day operations easier and parsers generally more reliable (how many times have we seen someone implement their own JSON parser? Now, what about XML? The first is a nice evening the latter is, well, not...). As for package.json, I would argue the problem is their choice of serialisation rather than with JSON itself. Right tool for the wrong job. YAML would probably be a better pick.
  - arnarbi 14y ago
    > I find JSON tedious and error-prone to generate by hand. Why do you need to generate it by hand? Create object structures in whatever language you are working in and then simply serialize them to JSON.
drawkbox 14y ago
JSON is great because it is simple basic cs types (string, numeric, boolean, object, list, date), a perfect middle ground between messaging and readability without going all XML/SOAP. And the unchanging simplicity is that way because it is so broadly accepted from Javascript to web services and beyond. It fits nicely into the system because of it's limitations but that also gives it standardized power. Any changes to JSON should be fought til death. Use YAML or make a new format if you need more.
- aseemk 14y ago
  Certainly. I'm not advocating for a change to JSON -- this is explicitly a new format. (The test cases use a .json5 extension to illustrate this.) FWIW, I'm not adding any new data types or functionality, just making the syntax more human-friendly. It also continues to be a strict subset of ECMAScript, but v5 now.
  - drawkbox 14y ago
    Yes I was just talking in a general sense that JSON should not go all XML and mutate. In some cases I have added layers like this for instance in a C++ parser for game engines it allowed comments in the json but were stripped out at runtime on load and on send/receive from the services. But conforming to JSON that current browsers, libraries etc can use. I see where you were going with it, ES5 support as a baseline is nice, also comments are nice but also bulky to a streamlined messaging format in most cases (config is usually yaml or some other format for this reason maybe). I guess the naming is what maybe might cause some confusion and a good engineer makes things more simple not more complex, the simplicity of JSON is hard to achieve with such success but it is also its killer feature so that should be recognized. As long as we don't have to support 50 different 'modern JSON' formats then we are good. There are some good ones like BSON for binary JSON. I guess JSON5 works for your ES5 support. I think here you have good reasons for your work but it goes back to the age old be 'conservative in what you send, liberal in what you accept'. That standards agreement makes everything work and to me being an engineer is taking complexity and simplifying always, bloat is bad. Here it isn't that but XML started out simple as well and became a verbose, bloated hell so people get touchy with JSON hehe.
genericdave 14y ago
Seems like you're getting a lot of criticism for this, a lot of which I have to agree with. Writing valid JSON is extremely simple and unfortunately it's hard to see how this project would do anything besides promote bad practices for a near-universal format. Suggestion: turn these criticisms to your favor. Make this into something that promotes good form. Perhaps by acting as a compiler that transforms your forgiving JSON into valid JSON. Make a command line utility. Emphasize how the output passes JSONlint or something. In other words, I'm suggesting that you remake this into a sort of coffeescript for JSON. In any case, the path you are on isn't gonna garner much support. Change up your stride a bit and you could be on to something rather nice, though.
YmMot 14y ago
I won't begrudge someone a fun little personal project, however (almost) no-one will ever adopt this. It removes too many benefits of JSON/introduces drawbacks for very little pay off. Yes, JSON is a pain to edit/write by hand at times, but that's an editing problem and therefore should be solved in the editor. In any halfway decent editor it should be trivial to extend it with helpers for editing JSON. Perhaps have it convert .json files to YAML(etc) on opening, and back to .json on save etc. This is a problem that plagues many technical people (present company included). You had a good instinct, but then took a left turn somewhere. "JSON is hard to hand edit...let's undo all the speed/compatibility progress we've made with JSON in the last ~8 years and introduce flaws in the protocol with a new non-standardized format...all so it's superficially easier to edit by hand."
- aseemk 14y ago
  I'd love to understand what benefits this removes and what drawbacks this introduces, aside from the chicken-and-egg problem of having any new format. Apologies if I've missed that in the discussion so far. You're definitely right that some of the hand-editing problems could in theory be solved by tools, but it unfortunately doesn't address documentation/comments. I've heard the chorus on YAML though, and I'll definitely be looking into it. Thanks for the feedback!
  - YmMot 14y ago
    JSON is supported natively (i.e. fast and correctly) in all modern JS environments. Yes, at one point we used Crock's parser but that's the past. Likewise, it currently is only used as a stop-gap measure to begrudgingly support older browsers. If we're going to undo years of progress we should do it for a very good reason. JSON has wide support in many languages. Some of them have fast low level parsers. All that work has to be duplicated. Again, this just doesn't seem sufficient reason. Right now if I want to throw together a Python script to crawl a bunch of NPM packages and read their package.json (let's say to put together a chart or collect statistics), I can easily. If NPM switches to JSON5, we lose compatibility without a lot of duplicated effort. Again, there's just not enough benefits. I consider comments a flaw, because they add little benefit. JSON is a data serialization/interchange format and there shouldn't be much need to comment on it, it should be largely self explanatory. If you really need to tag things with info, you can in JSON. Adding comments is to tempting for people to abuse. However, if you really need lots of comments, there are many existing formats which might be better suited for your task. JSON is ultimately for machines. It has a benefit that humans can read/edit it easily...but if you have data that is intended more for humans and needs lots of meta information like a complicated config file, you should either create a front end for it or use a format better suited to that purpose. When you consider that writing a plugin for your editor, or just a stand-alone translator would solve all of the problems this does, there's no reason to put up with all the drawbacks. What it comes down to IMHO, is this isn't suitable to replace JSON and for other uses there are already existing formats that are better.
mitchellh 14y ago
While we're at it, I felt it necessary to launch HTML7: https://github.com/mitchellh/html7 https://github.com/mitchellh/html7
- chris24 14y ago
  The license was a nice touch.
- aseemk 14y ago
  Haha, I actually found that hilarious. Nice work.
tferris 14y ago
Please rename your project or take it down. People here gave you enough reasons.
- veemjeem 14y ago
  don't worry, next week I'll release JSON6 and nobody will remember JSON5 anymore.
augustflanagan 14y ago
I think that it's really a shame to see people slamming aseemk here. He had an idea, worked on it, and released it. That's fantastic. There are a few constructive comments here explaining why his idea might not actually be that great, but there are also quite few comments that seem out of place for HN.
jQueryIsAwesome 14y ago
I can't believe people are taking this seriously... Lets write a python parser who accepts semicolons instead of indentation, just to fix when I forget that i am not writing Javascript; oh, and and make it accept the "function" keyword instead of def.
iamcarbon 14y ago
If only it were this easy to extended JavaScript with a new serialization format! If the serialized text can't be assigned to a JavaScript variable, it's not JavaScript notion. I also tried something similar a few years back when I was sick of dealing with XML and coming to terms with the required quotes. http://xjson.org/ http://xjson.org/ While it's certainly prettier to have native literals and optional quotes, it's simply not worth giving up the interoperability with JavaScript.
DavidAbrams 14y ago
Seems like quoted keys are a pain in the ass. That's the only useful change I see here.
pygorex 14y ago
I have to agree with most of the criticism in this thread. The lack of comments, strict quoting, no trailing slashes - these aren't problems to fix they are features. As others have noted this is not JSON and is not necessarily an improvement to JSON - so calling it JSON5 is presumptuous. Embrace the criticism. Instead of JSON5 call it NotJSON - a standard that breaks the JSON rules for a few narrow use cases - namely when you want a hand-written, commented JSON-like syntax.
MarkR42 14y ago
Congratulations, you've made something which looks superficially like JSON, but most documents won't parse in 99.9% of JSON parsers. Interoperability is lost, nothing is gained!
jwr 14y ago
I don't think making JSON more human-writable and readable is a goal worth pursuing. JSON has its flaws, but is a surprisingly robust and elegant solution to many data interchange problems. But it is not (and I don't think it was ever intended to be) a language for human-edited configuration and data.
jahewson 14y ago
This reminds me of I guy I met who was trying to build a clone of Twitter, but with messages of unlimited length. In his view this made it objectively "better". He understood the what of the software, but not the why. If you really want ES5 for writing package.json files, then you could save the time and effort by just using ES5: var _ = { some: 'object literal', // a comment num: 3, } fs.writeFileSync(path.basename(__filename, '.js') + '.json', JSON.stringify(_))
timrobinson 14y ago
> [IDEA: Allow octal and hexadecimal numbers.] Who actually uses octal, except by accident?
- sneeu 14y ago
  The only time I ever use it is with `chmod`.
jimrandomh 14y ago
You know that something has gone very deeply wrong somewhere when people oppose your project because they are ideologically opposed to comments. In my own experience, finding out that many JSON parsers reject comments was a real WTF moment, and a deal breaker for my config-file application, so I ended up using JSON-plus-comments for it instead of JSON. At the same time, lack of support for trailing commas and unquoted member names have been a minor but persistent thorn in my side for no good reason. The justification for not having comments in JSON is that in the great disaster that was XML, some projects would parse the comments and take them as semantically significant. However, the real problem there was that parser libraries would expose the comments, and that some generators would put important information only in comments. But I think that these mistakes are unlikely to be repeated, and that the proposed alternative - moving all comments into the markup, or eliminating them entirely - is just obviously worse. One of the things that ruined the XML ecosystem was a persistent belief that XML was to be read and written by machines, combined with a reality in which it was mostly used for human-written config files, leading to a tolerance for awful syntax (like prefixing every single attribute with a namespace, and the ridiculous CDATA notation). I'm seeing the same thing with JSON: A significant fraction of its use is for human-written and human-read config files (which often desperately need comments) and people are pretending it's strictly a data interchange format that shouldn't be used for that. It is sometimes said that JSON was discovered, rather than invented - that the syntax was already out there. So it is with JSON5: There is nothing new in this, it is simply return to JavaScript Object Notation and bringing in the rest of what Javascript has. So, all you pooh-poohing ideologists: please seriously rethink whether disallowing comments, trailing commas, and quoteless member names is actually a good idea. Consider this in light of the fact that JSON is widely used, today, as a config-file language, and that {"--":"This is a comment"} is ridiculous enough that no one does it in practice, can't be inserted on any line, and invites consumers of your data to parse the comments.
- aseemk 14y ago
  Thanks for stating so eloquently what I've been struggling to communicate. The heart of the matter to me really does boil down to the fact that JSON is meant for humans, too, not just machines. You nailed this sentiment. (As a friend pointed out, "Do you not comment your code just because it's meant for machines?")

64 points by aseemk 14y ago

Author here. This idea stemmed from long-running frustration every time I hand-wrote JSON (particularly common for Node/npm packages and test cases). After stewing on it for over a year, I bit the bullet today and made it a reality. Feedback greatly welcome!

thwarted 14y ago
Why are you hand writing JSON? Pass a JavaScript object, which doesn't have these limitations, to the JSON serialerizer. The support for evaling in the various engines doesn't matter because you should not be evaling JSON strings for security purposes. The serialization format is strict for a reason: look at where less strict HTML parsing has gotten us.
- aseemk 14y ago
  Writing, or at least editing, JSON by hand seems unavoidable if you develop on Node, as modules' "manifests" are JSON files. I also love to make tests data-driven where possible. JSON is the natural format for data on Node, and again, test cases are usually written by hand (before code exists to generate or validate them). I'm not sure I understand your point about eval'ing. This parser doesn't use eval(), and it works in every modern browser. Finally, I'm not advocating for arbitrary looseness. These additions conform directly to ES5's additions.
  - aurelianito 14y ago
    I believe you should do something like this: json.stringify({unquoted:'Some value', trailing_comma:true,}) instead of hand-writting JSON data in tests.
  - ZenPsycho 14y ago
    I have implemented a javascript eval into my text editor. When I select text, and call the command, it passes it off to a node.JS engine and replaces the selection with the return result. In this way I can write a JSON.stringify([anyvalid javascript],null, [indentation]); and instantly get perfect JSON. This is much less troublesome than trying to make a new slightly incompatible format.
stock_toaster 14y ago
* Objects and arrays can have trailing commas.* If I had a penny for every time a trailing comma in a json config file bit me, I would have...many pennies! I am indifferent about single vs double quotes. The backslash newline escape seems 'messy'. Not a fan of keys not being quoted. That stikes me as wrongish for some reason that I can't put my finger on.
benatkin 14y ago
Please don't call it JSON. Actually I don't care much, as your project will be shunned. The first rule of JSON is that JSON doesn't change.
gurkendoktor 14y ago
I absolutely hate hand-writing JSON, but that's why I use YAML. I think you should push for a properly documented subset of YAML, or a full YAML parser in JavaScript instead. Just a Ruby programmer's 2 Eurocent :)
- aseemk 14y ago
  Thanks for the feedback! I never considered YAML precisely because there doesn't seem to be much support for it. I'll definitely look into it. =)
brentashley 14y ago
Douglas built the JSON spec 1) to be language agnostic. Following ES5's syntax directions is fine for ES5 focused development, possibly not for others. 2) to be final - i.e. there is no version number. If you want to extend or build something else, I am fairly certain Douglas would prefer that you call it something else. See http://inkdroid.org/journal/2012/04/30/lessons-of-json/ http://inkdroid.org/journal/2012/04/30/lessons-of-json/ at 5:30 in the video. Don't take these points as criticisms, I too would like to be able to use single quotes and trailing commas and everything else here. I think though that in order to work with JSON as a static spec, an alternative approach could be to have your project be a layer like HAML, SASS or CoffeeScript. Each of those tools compiles or transforms to conforming input for their target specs rather than replacing them.
- aseemk 14y ago
  JSON5 is also meant to be language-agnostic. It derives its syntax from ES5 in the same way that JSON derived its syntax from ES3. I'm aware that JSON is final — it's my hope that a new format that's easier to write (whether this or something else) picks up steam — but thanks for the feedback on the name! I consciously used a different file extension (.json5) to avoid conflicts; hopefully that's a good start. I already use (and love) CoffeeScript, but my motivation for building this was to have tools use it natively, so that sibling files in a different format wouldn't have to be maintained alongside the needed JSON.
melloclello 14y ago
Horrible. There's a reason JSON is homoiconic with valid javascript. You should never have done this.
wvenable 14y ago
JSON is an amazing serialization format but hand-written JSON is completely wrong. I think this fixes most of the problems of using it in hand-written situations. Just the lack of comments in JSON makes it inappropriate for configuration files and tests. But this shouldn't be called JSON. JSON is strict and doesn't include comments because it's designed for interoperability between different systems. Just because it's been abused to be used for configuration files doesn't mean it needs fixing. One just needs to use the right tool for the job. Your project might be the right tool but then it's not JSON.

cheeaun 14y ago

JSON 3 here: https://github.com/bestiejs/json3 https://github.com/bestiejs/json3

nfriedly 14y ago
I believe Facebook recently switched to JSON 3 for their JS SDK.

simanyay 14y ago

Three things: * Comments in JSON files are not a great idea because people might (will?) abuse them to add special instructions for their own serializers. As soon as this happens we'll have a format that is not compatible with other serializers. * JSON today is a universal format. So, what are special characters? Should '1hello' be quoted? If no, then its not a subset of JavaScript anymore. If you're writing a new specification you have to define all those things. Same with multiline strings: space after backslash? * Why? I know that hand-writing JSON can be annoying but is this really the biggest problem we have?

aseemk 14y ago
Comments: yes, I read Douglas Crockford's post on the matter. I guess I'm not genuinely worried about that, though, and my hope is that shipping a JS parser that works on both the client and the server will help standardize implementations from the start. Special characters: sorry I didn't write a formal spec =), but the intent is to be a pure subset of ES5 just like regular JSON is a pure subset of ES3. Unquoted object keys in ES5 can contain only letters/numbers/_/$, and only begin with letters/_/$. So yes, '1hello' would need to be quoted, just like regular ES5. I'm not sure how to respond to your why other than what I've already written. I never claimed this was the biggest problem we have. =)
- Too 14y ago
  JSON is not exactly tied to javascript. For example the number-type is not specified in much detail. You could imagine this to be a 64bit int but if you load it into javascript it will be converted to a double and you will loose precision if your number is big.
- simanyay 14y ago
  Crock made a post about JSON comments? Anyway, I wouldn't focus on JavaScript engines because JSON is now a universal format so for languages other than JavaScript you will have to define a formal specification.

zacharyvoase 14y ago

Can I ask why YAML wasn't good enough?

chc 14y ago
My thoughts exactly. YAML is a superset of JSON that already has all this stuff and more.
- aseemk 14y ago
  I actually had no idea that YAML is a superset of JSON. Thanks! I'll look into it.
erikpukinskis 14y ago
Partly because JSON has the lovely feature of being able to just eval() it into an object in browsers.
- deleted 14y ago
  [deleted]
- alcuadrado 14y ago
  That's not the right way to parse JSON, it's really insecure
  - Too 14y ago
    And slow, JSON.parse is magnitudes faster than using user-javascript to parse it yourself.
- chc 14y ago
  That's not really a lovely feature in this day and age. If anything it's an unfortunate artifact, as the right way to deserialize JSON is JSON.parse().
silentbicycle 14y ago
For the same reason sexps weren't good enough, apparently.

Mizza 14y ago

Please, don't call it JSON. There is only one JSON, and you didn't make it. It's cool that you made this, and that you made it open source, just please call it something else.

enneff 14y ago

JSON is wonderful because it is simple and unambiguous. It just works, and serialized data will always look the same. This "JSON5" thing is an abomination. Use whatever quotes you like! More string escaping rules! Adding these things would make JSON worse, not better. Here's another idea: why not write a tool that tidies up your hand-written JSON files? You could even re-use your "JSON5" parser to do it. That way you can be as lazy as you like when writing files, but not expect the rest of the world to learn your new dialect.

aseemk 14y ago
Thanks for the feedback. It certainly would be great if there existed tools to let me write and update JSON with nicer syntax like this; any ideas how that might be possible? My motivation for this was actually hoping that this gets adopted in some environments that use JSON. Node relies on package.json files, for example, which frequently need to be hand-edited. Maintaining a sibling file in a different format feels equally wrong to me as it seems JSON5 seems to you. =)
- benatkin 14y ago
  That's npm, not node itself, that relies on package.json. When its creator chose it, he knew exactly what JSON is. He's maintained on HN and elsewhere that he made a good decision. So even if you stop calling this JSON, he's not buying it. The package.json files, by and large, don't have a multiline string problem. For trailing and missing commas, I think you could make something that automatically fixes them, if that's the only problem, and run it before npm is run, perhaps using a shell alias.
  - aseemk 14y ago
    You seem to be implying that making an effort to seek change in the format that npm uses is a bad idea. I'm not sure I understand why that is. Surely the status quo isn't always perfect? In the software world in general, and in Node/npm land especially, code talks. I thought it'd be more productive to build and share a working ES5-style JSON parser than to simply ask Isaac if npm would support ES5-style JSON. =) Yes, you're right about trailing commas. One motivation that such an approach couldn't address is documentation: I've frequently wanted comments in my package.json files, e.g. explaining a particular dependency version, or a script command.

greghinch 14y ago

(young) people seem to always want to find ways to skirt the "rules" that Crockford sets up (see also, jslint). Just stop. The rules he lays out are there to make your _long term_ programing life easier. Yes you may have to remember to use semicolons and quote things, but when you have to live with an application you've been working on for years, hopefully you'll learn to appreciate the fact that putting in a little extra time now saves you tons of headaches down the road.

aseemk 14y ago
I'm actually a big fan of Douglas Crockford. And don't worry, my code uses semicolons. ;) I'm not sure why you lump quoting keys with semicolons, though; his regular JS style guide doesn't discourage quoting object key (neither does JSLint).
- ZenPsycho 14y ago
  there's a reason he requires it for JSON. it's that javascript has problems when you use a javascript reserved word as a key- for instance, if you were making an RPC api, and you had a json object {do:"methodname"} the javascript eval method would throw a fit. So rather than include the full list of javascript reserved words in a spec intended to be language agnostic, he just made quoting the key a requirement, which makes things vastly simpler.
  - aseemk 14y ago
    This was indeed a problem in the IE6 days. Every browser today (and ES5) supports reserved keywords as unquoted object keys. There are thus no reserved keywords for object keys in ES5, and by extension, JSON5.
    - ZenPsycho 14y ago
      No, but there ARE a ton of JSON parsers that expect and require quoted keys now. So it is, in fact, still a requirement. And I don't know about you, you may be lucky, but I still have to make my stuff work on IE6.

ZenPsycho 14y ago

This entirely misses the point of JSON. if you want these features it's perfectly acceptable to use YAML instead of JSON.

kwamenum86 14y ago

I don't think any of these suggestions make JSON easier to read, generate, or consume. What problem are you setting out to solve? Opening paragraph: "JSON is strict. Keys need to be quoted; strings can only be double-quoted; objects and arrays can't have trailing commas; and comments aren't allowed." If this represents a comprehensive list of the problems JSON5 solves I think you'll have a hard time rallying people around your cause.

aseemk 14y ago
Yes, I realize I didn't explain it as well in the README as I had in conversations and emails. I find JSON tedious and error-prone to generate by hand. I also frequently wish I could document the data with comments. Others have written about these same ideas, e.g. [1]; after feeling the frustration for over a year, I decided it might be productive to actually build a JS parser to get the conversation started. My main use cases are Node/npm package.json files, and test case data. [1] http://bolinfest.com/essays/json.html http://bolinfest.com/essays/json.html
- ninjin 14y ago
  > I find JSON tedious and error-prone to generate by hand. I also frequently wish I could document the data with comments. I whole-heartedly agree with you on the tediousness, but JSON has this dubious double-nature that it is sometimes written by hand but mostly used as a protocol. As a protocol we want (and I would even argue need) this strictness since it makes day-to-day operations easier and parsers generally more reliable (how many times have we seen someone implement their own JSON parser? Now, what about XML? The first is a nice evening the latter is, well, not...). As for package.json, I would argue the problem is their choice of serialisation rather than with JSON itself. Right tool for the wrong job. YAML would probably be a better pick.
- arnarbi 14y ago
  > I find JSON tedious and error-prone to generate by hand. Why do you need to generate it by hand? Create object structures in whatever language you are working in and then simply serialize them to JSON.

drawkbox 14y ago

JSON is great because it is simple basic cs types (string, numeric, boolean, object, list, date), a perfect middle ground between messaging and readability without going all XML/SOAP. And the unchanging simplicity is that way because it is so broadly accepted from Javascript to web services and beyond. It fits nicely into the system because of it's limitations but that also gives it standardized power. Any changes to JSON should be fought til death. Use YAML or make a new format if you need more.

aseemk 14y ago
Certainly. I'm not advocating for a change to JSON -- this is explicitly a new format. (The test cases use a .json5 extension to illustrate this.) FWIW, I'm not adding any new data types or functionality, just making the syntax more human-friendly. It also continues to be a strict subset of ECMAScript, but v5 now.
- drawkbox 14y ago
  Yes I was just talking in a general sense that JSON should not go all XML and mutate. In some cases I have added layers like this for instance in a C++ parser for game engines it allowed comments in the json but were stripped out at runtime on load and on send/receive from the services. But conforming to JSON that current browsers, libraries etc can use. I see where you were going with it, ES5 support as a baseline is nice, also comments are nice but also bulky to a streamlined messaging format in most cases (config is usually yaml or some other format for this reason maybe). I guess the naming is what maybe might cause some confusion and a good engineer makes things more simple not more complex, the simplicity of JSON is hard to achieve with such success but it is also its killer feature so that should be recognized. As long as we don't have to support 50 different 'modern JSON' formats then we are good. There are some good ones like BSON for binary JSON. I guess JSON5 works for your ES5 support. I think here you have good reasons for your work but it goes back to the age old be 'conservative in what you send, liberal in what you accept'. That standards agreement makes everything work and to me being an engineer is taking complexity and simplifying always, bloat is bad. Here it isn't that but XML started out simple as well and became a verbose, bloated hell so people get touchy with JSON hehe.

genericdave 14y ago

Seems like you're getting a lot of criticism for this, a lot of which I have to agree with. Writing valid JSON is extremely simple and unfortunately it's hard to see how this project would do anything besides promote bad practices for a near-universal format. Suggestion: turn these criticisms to your favor. Make this into something that promotes good form. Perhaps by acting as a compiler that transforms your forgiving JSON into valid JSON. Make a command line utility. Emphasize how the output passes JSONlint or something. In other words, I'm suggesting that you remake this into a sort of coffeescript for JSON. In any case, the path you are on isn't gonna garner much support. Change up your stride a bit and you could be on to something rather nice, though.

YmMot 14y ago

I won't begrudge someone a fun little personal project, however (almost) no-one will ever adopt this. It removes too many benefits of JSON/introduces drawbacks for very little pay off. Yes, JSON is a pain to edit/write by hand at times, but that's an editing problem and therefore should be solved in the editor. In any halfway decent editor it should be trivial to extend it with helpers for editing JSON. Perhaps have it convert .json files to YAML(etc) on opening, and back to .json on save etc. This is a problem that plagues many technical people (present company included). You had a good instinct, but then took a left turn somewhere. "JSON is hard to hand edit...let's undo all the speed/compatibility progress we've made with JSON in the last ~8 years and introduce flaws in the protocol with a new non-standardized format...all so it's superficially easier to edit by hand."

aseemk 14y ago
I'd love to understand what benefits this removes and what drawbacks this introduces, aside from the chicken-and-egg problem of having any new format. Apologies if I've missed that in the discussion so far. You're definitely right that some of the hand-editing problems could in theory be solved by tools, but it unfortunately doesn't address documentation/comments. I've heard the chorus on YAML though, and I'll definitely be looking into it. Thanks for the feedback!
- YmMot 14y ago
  JSON is supported natively (i.e. fast and correctly) in all modern JS environments. Yes, at one point we used Crock's parser but that's the past. Likewise, it currently is only used as a stop-gap measure to begrudgingly support older browsers. If we're going to undo years of progress we should do it for a very good reason. JSON has wide support in many languages. Some of them have fast low level parsers. All that work has to be duplicated. Again, this just doesn't seem sufficient reason. Right now if I want to throw together a Python script to crawl a bunch of NPM packages and read their package.json (let's say to put together a chart or collect statistics), I can easily. If NPM switches to JSON5, we lose compatibility without a lot of duplicated effort. Again, there's just not enough benefits. I consider comments a flaw, because they add little benefit. JSON is a data serialization/interchange format and there shouldn't be much need to comment on it, it should be largely self explanatory. If you really need to tag things with info, you can in JSON. Adding comments is to tempting for people to abuse. However, if you really need lots of comments, there are many existing formats which might be better suited for your task. JSON is ultimately for machines. It has a benefit that humans can read/edit it easily...but if you have data that is intended more for humans and needs lots of meta information like a complicated config file, you should either create a front end for it or use a format better suited to that purpose. When you consider that writing a plugin for your editor, or just a stand-alone translator would solve all of the problems this does, there's no reason to put up with all the drawbacks. What it comes down to IMHO, is this isn't suitable to replace JSON and for other uses there are already existing formats that are better.

mitchellh 14y ago

While we're at it, I felt it necessary to launch HTML7: https://github.com/mitchellh/html7 https://github.com/mitchellh/html7

chris24 14y ago
The license was a nice touch.
aseemk 14y ago
Haha, I actually found that hilarious. Nice work.

tferris 14y ago

Please rename your project or take it down. People here gave you enough reasons.

veemjeem 14y ago
don't worry, next week I'll release JSON6 and nobody will remember JSON5 anymore.

augustflanagan 14y ago

I think that it's really a shame to see people slamming aseemk here. He had an idea, worked on it, and released it. That's fantastic. There are a few constructive comments here explaining why his idea might not actually be that great, but there are also quite few comments that seem out of place for HN.

jQueryIsAwesome 14y ago

I can't believe people are taking this seriously... Lets write a python parser who accepts semicolons instead of indentation, just to fix when I forget that i am not writing Javascript; oh, and and make it accept the "function" keyword instead of def.

iamcarbon 14y ago

If only it were this easy to extended JavaScript with a new serialization format! If the serialized text can't be assigned to a JavaScript variable, it's not JavaScript notion. I also tried something similar a few years back when I was sick of dealing with XML and coming to terms with the required quotes. http://xjson.org/ http://xjson.org/ While it's certainly prettier to have native literals and optional quotes, it's simply not worth giving up the interoperability with JavaScript.

DavidAbrams 14y ago

Seems like quoted keys are a pain in the ass. That's the only useful change I see here.

pygorex 14y ago

I have to agree with most of the criticism in this thread. The lack of comments, strict quoting, no trailing slashes - these aren't problems to fix they are features. As others have noted this is not JSON and is not necessarily an improvement to JSON - so calling it JSON5 is presumptuous. Embrace the criticism. Instead of JSON5 call it NotJSON - a standard that breaks the JSON rules for a few narrow use cases - namely when you want a hand-written, commented JSON-like syntax.

MarkR42 14y ago

Congratulations, you've made something which looks superficially like JSON, but most documents won't parse in 99.9% of JSON parsers. Interoperability is lost, nothing is gained!

jwr 14y ago

I don't think making JSON more human-writable and readable is a goal worth pursuing. JSON has its flaws, but is a surprisingly robust and elegant solution to many data interchange problems. But it is not (and I don't think it was ever intended to be) a language for human-edited configuration and data.

jahewson 14y ago

This reminds me of I guy I met who was trying to build a clone of Twitter, but with messages of unlimited length. In his view this made it objectively "better". He understood the what of the software, but not the why. If you really want ES5 for writing package.json files, then you could save the time and effort by just using ES5: var _ = { some: 'object literal', // a comment num: 3, } fs.writeFileSync(path.basename(__filename, '.js') + '.json', JSON.stringify(_))

timrobinson 14y ago

> [IDEA: Allow octal and hexadecimal numbers.] Who actually uses octal, except by accident?

sneeu 14y ago
The only time I ever use it is with `chmod`.

jimrandomh 14y ago

You know that something has gone very deeply wrong somewhere when people oppose your project because they are ideologically opposed to comments. In my own experience, finding out that many JSON parsers reject comments was a real WTF moment, and a deal breaker for my config-file application, so I ended up using JSON-plus-comments for it instead of JSON. At the same time, lack of support for trailing commas and unquoted member names have been a minor but persistent thorn in my side for no good reason. The justification for not having comments in JSON is that in the great disaster that was XML, some projects would parse the comments and take them as semantically significant. However, the real problem there was that parser libraries would expose the comments, and that some generators would put important information only in comments. But I think that these mistakes are unlikely to be repeated, and that the proposed alternative - moving all comments into the markup, or eliminating them entirely - is just obviously worse. One of the things that ruined the XML ecosystem was a persistent belief that XML was to be read and written by machines, combined with a reality in which it was mostly used for human-written config files, leading to a tolerance for awful syntax (like prefixing every single attribute with a namespace, and the ridiculous CDATA notation). I'm seeing the same thing with JSON: A significant fraction of its use is for human-written and human-read config files (which often desperately need comments) and people are pretending it's strictly a data interchange format that shouldn't be used for that. It is sometimes said that JSON was discovered, rather than invented - that the syntax was already out there. So it is with JSON5: There is nothing new in this, it is simply return to JavaScript Object Notation and bringing in the rest of what Javascript has. So, all you pooh-poohing ideologists: please seriously rethink whether disallowing comments, trailing commas, and quoteless member names is actually a good idea. Consider this in light of the fact that JSON is widely used, today, as a config-file language, and that {"--":"This is a comment"} is ridiculous enough that no one does it in practice, can't be inserted on any line, and invites consumers of your data to parse the comments.

aseemk 14y ago
Thanks for stating so eloquently what I've been struggling to communicate. The heart of the matter to me really does boil down to the fact that JSON is meant for humans, too, not just machines. You nailed this sentiment. (As a friend pointed out, "Do you not comment your code just because it's meant for machines?")