6 ms·

It's not just JSON without [ and ], it's a bunch of JSONs represented as a table. The first line sets the property's key and every other line are the values. It

It's not just JSON without [ and ], it's a bunch of JSONs represented as a table. The first line sets the property's key and every other line are the values. It is a representation that needs a parser to be read but that's way faster for a row based database to generate it than to output full json. Plus it's smaller than minified json.

tlrobinson 10y ago
You can represent a table using JSON arrays, and I doubt CJS is "way faster" or smaller than gzipped JSON. There is nothing inherently preventing you from writing an almost identically performing streaming JSON parser or serializer.
- proteusguy 10y ago
  This is likely content coming down from an HTTP API so likely already gzipped. Since you can gzip both CSJ and JSON I think that's not really relevant. With this proposed format you only need to parse the object definition once and you also get the ordering fixed. From then on you need only validate each field in each line as being legal and you can build your object quite fast. This format saves probably half the network bandwidth, half or more in memory requirements, and more than half the cpu complexity in parsing and getting your final objects instantiated. That's pretty good. Remember the goal isn't to replace JSON. It's to replace CSV. JSON and CSV have different goals. This is simply a JSON structure built to meet the same goals as CSV. It seems to do that quite nicely and can use existing JSON parsers internally to do it. Pretty nifty.
dalke 10y ago
The spec doesn't specify what the first line contains. It only says "Semantically a CSJ file is an array of JSON objects which share a common set of keys." You used your knowledge of how these representations usually work to infer that the first line contains a list of keys for the remaining lines. Some obvious questions for a table spec include: 1) must those keys be strings? 2) Are duplicate keys allowed? 3) Can different rows have a different number of columns? Or must the writer generate null fields? FWIW, I've used JSON Lines for my own work. It's very easy to read a line and (in Python), json.loads(line). How should I parse this "JSON without [ and ]"? There's not a built-in function for it. Either I have to write my own JSON tokenizer, or I have to terminate the string with '[' and ']' and parse that. Both are much more complicated than using JSON Lines? You'll notice they only gave output generation timing numbers, not input parsing. If a JSON-based table format is important, why not layer those same table semantics on top of JSON Lines, rather than make a format which is harder to read?
- simlevesque 10y ago
  The format for the keys is the same as a javascript object. They are unique strings or numbers. It's already in the Ecmascript standard for objects.
  - tlrobinson 10y ago
    That doesn't mean it's as easy to parse as plain JSON or JSON Lines. In JS, non-streaming: JSON.parse(blob) Or blob.split("\n").map(line => JSON.parse(line)) respectively.
  - dalke 10y ago
    Could you help me understand your response? I find it hard to match your statement to the JSON spec(s). Neither RFC 7159 nor ECMA 404 use the term "key". The standard for objects requires a string for the 'name' of an object member, which is certainly what you refer to. However, JSON does not allow a number. I tried looking at the ECMAScript spec. In http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf http://www.ecma-international.org/publications/files/ECMA-ST... : > An Object is logically a collection of properties. ... Properties are identified using key values . A property key value is either an ECMAScript String value or a Symbol value . That 'Symbol' has nothing to do with numbers, so only strings make sense for an external representation. Perhaps you mean to include the integer from the Array object? The spec says "An Array object is an exotic object that gives special treatment to array index property key ... A String property name P is an array index if and only if ToString(ToUint32(P)) is equal to P and ToUint32(P) is not equal to 2^32-1." Is that limited range of integers what you mean when you saying that numbers are allowed as keys? Regarding uniqueness, the CSJ spec says nothing about uniqueness, so how do you know the keys must be unique? It doesn't inherit that from JSON. RFC 7159 says that in an object "The names within an object SHOULD be unique."; uniqueness is not required. It then adds "When the names within an object are not unique, the behavior of software that receives such an object is unpredictable." Therefore, I don't see how you drew the conclusion you did.
- KayEss 10y ago
  > rather than make a format which is harder to read Do you really believe that this is harder to read than JSON lines? We use both, and I've never heard a user complain that CSJ is hard to read, but the JSON lines has a lot of clutter. {"age": 45, "name": "Kirit Sælensminde", "job": "Minister Without Portfolio"} {"job": null, "age": 5, "name": "Freyja Sælensminde"} That seems fine for a low number of columns, but gets much harder as the column count goes up. Especially so as you lose control of the key order in the line.

tlrobinson 10y ago

You can represent a table using JSON arrays, and I doubt CJS is "way faster" or smaller than gzipped JSON. There is nothing inherently preventing you from writing an almost identically performing streaming JSON parser or serializer.

proteusguy 10y ago
This is likely content coming down from an HTTP API so likely already gzipped. Since you can gzip both CSJ and JSON I think that's not really relevant. With this proposed format you only need to parse the object definition once and you also get the ordering fixed. From then on you need only validate each field in each line as being legal and you can build your object quite fast. This format saves probably half the network bandwidth, half or more in memory requirements, and more than half the cpu complexity in parsing and getting your final objects instantiated. That's pretty good. Remember the goal isn't to replace JSON. It's to replace CSV. JSON and CSV have different goals. This is simply a JSON structure built to meet the same goals as CSV. It seems to do that quite nicely and can use existing JSON parsers internally to do it. Pretty nifty.

dalke 10y ago

The spec doesn't specify what the first line contains. It only says "Semantically a CSJ file is an array of JSON objects which share a common set of keys." You used your knowledge of how these representations usually work to infer that the first line contains a list of keys for the remaining lines. Some obvious questions for a table spec include: 1) must those keys be strings? 2) Are duplicate keys allowed? 3) Can different rows have a different number of columns? Or must the writer generate null fields? FWIW, I've used JSON Lines for my own work. It's very easy to read a line and (in Python), json.loads(line). How should I parse this "JSON without [ and ]"? There's not a built-in function for it. Either I have to write my own JSON tokenizer, or I have to terminate the string with '[' and ']' and parse that. Both are much more complicated than using JSON Lines? You'll notice they only gave output generation timing numbers, not input parsing. If a JSON-based table format is important, why not layer those same table semantics on top of JSON Lines, rather than make a format which is harder to read?

simlevesque 10y ago
The format for the keys is the same as a javascript object. They are unique strings or numbers. It's already in the Ecmascript standard for objects.
- tlrobinson 10y ago
  That doesn't mean it's as easy to parse as plain JSON or JSON Lines. In JS, non-streaming: JSON.parse(blob) Or blob.split("\n").map(line => JSON.parse(line)) respectively.
- dalke 10y ago
  Could you help me understand your response? I find it hard to match your statement to the JSON spec(s). Neither RFC 7159 nor ECMA 404 use the term "key". The standard for objects requires a string for the 'name' of an object member, which is certainly what you refer to. However, JSON does not allow a number. I tried looking at the ECMAScript spec. In http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf http://www.ecma-international.org/publications/files/ECMA-ST... : > An Object is logically a collection of properties. ... Properties are identified using key values . A property key value is either an ECMAScript String value or a Symbol value . That 'Symbol' has nothing to do with numbers, so only strings make sense for an external representation. Perhaps you mean to include the integer from the Array object? The spec says "An Array object is an exotic object that gives special treatment to array index property key ... A String property name P is an array index if and only if ToString(ToUint32(P)) is equal to P and ToUint32(P) is not equal to 2^32-1." Is that limited range of integers what you mean when you saying that numbers are allowed as keys? Regarding uniqueness, the CSJ spec says nothing about uniqueness, so how do you know the keys must be unique? It doesn't inherit that from JSON. RFC 7159 says that in an object "The names within an object SHOULD be unique."; uniqueness is not required. It then adds "When the names within an object are not unique, the behavior of software that receives such an object is unpredictable." Therefore, I don't see how you drew the conclusion you did.
KayEss 10y ago
> rather than make a format which is harder to read Do you really believe that this is harder to read than JSON lines? We use both, and I've never heard a user complain that CSJ is hard to read, but the JSON lines has a lot of clutter. {"age": 45, "name": "Kirit Sælensminde", "job": "Minister Without Portfolio"} {"job": null, "age": 5, "name": "Freyja Sælensminde"} That seems fine for a low number of columns, but gets much harder as the column count goes up. Especially so as you lose control of the key order in the line.