3 ms·

JCOF is not usable in many of the use cases where JSON is. Here is why: * JSON is really schemaless so I don't have to assume all objects are shaped the same o

JCOF is not usable in many of the use cases where JSON is. Here is why: * JSON is really schemaless so I don't have to assume all objects are shaped the same or that they are even the same kind of object. This allows for streaming serialization, and does not require the data structure to be known or to introspect data to create the heading lines. * Nested objects look to be difficult, especially if the schema is not known. * JSON is very human friendly, JCOF is not. I do think the format is better than CSV, TSV and other delimited flat record formats. JCOF appears to be able to handle objects that are not tabular, which CSV just can't really do. A replacement for JSON this is not, and storage efficiency isn't really relevant because of the difference in purpose.

nine_k 4y ago
JCOF is just not radical enough. It should stop pretending to remain human-readable, go binary, and become a variant of protocol buffers, thrift, etc. BTW being schemaless is a boon during initial hacking things together, and an impediment when operating and developing the system further down the line. It's the data equivalent of dynamic typing vs static typing, interpreted vs compiled in code.
- indymike 4y ago
  > It should stop pretending to remain human-readable, go binary, and become a variant of protocol buffers, thrift, etc. There are lots of ways standard ways to do serialization to binary formats. https://en.wikipedia.org/wiki/Comparison_of_data-serialization_formats#Comparison_of_binary_formats https://en.wikipedia.org/wiki/Comparison_of_data-serializati... > BTW being schemaless is a boon during initial hacking things together, and an impediment when operating and developing the system further down the line. It's the data equivalent of dynamic typing vs static typing, interpreted vs compiled in code. Sounds like XML, which is a whole other discussion :-)
mort96 4y ago
Hey, JCOF author here. I didn't notice this had made it to HN before now. * JCOF allows for streaming serialization if you want, you can just use the key-value object syntax and use inline strings everywhere. You won't get much size improvement over JSON when using a streaming approach, but you can even do a half-and-half approach; maybe you have a set of strings which you know you're using a lot, so you put those strings in the string table, then you do a streaming encode of the rest of your data, with string references to the string table. You could even do the same with object shapes, where you hard-code a couple of object shapes you know you use a lot, and use the JSON-style key-value syntax for other kinds of objects. Basically, you can choose where you want to be on the efficiency vs streaming curve. * Nested objects aren't difficult; an object can go in any place where a value can go, just like in JSON. This object can be either a "shaped" object where the keys are listed in the object shape table, or it can be a "keyed" object where you write out the keys inline (as either literal strings or references to the string table). * JCOF is human-writable; you can write a document which looks a whole lot like JSON. This is a valid document: `;;{"people":[{"name":"Bob",Aage":36},{"name":"Alice","age":40}]}`. But optimal JCOF is mostly unreadable. So it's less human friendly than JSON, more human friendly than binary formats; it's a trade-off. I think JCOF hits enough interesting, unexplored points on enough relevant curves that it has some value. It also ends up being smaller than any of the binary formats I've tried like MessagePack and CBOR (even with the string references extension), while being text-based. It's obviously not going to replace JSON, but it's useful in some cases.
- indymike 4y ago
  Oh, I think you have something, but I don't think JSON and JCOF are comparable. I think it makes a very interesting replacement for formats where there is a header line like CSV, TSV. JCOF allows for some level of consistent nested structures (which would be nice for header/detail records which are really common in the real world) and would be so much nicer than many of the bespoke formats you see out there... and also, sooo much more efficient than XML, and without the issues that come with XML. Incidentally, these kinds of uses are where JSON really doesn't work well because you always have the potential of a mis-shaped object. In short: nice work. Might be fun to roll an implementation or two :-)

nine_k 4y ago

JCOF is just not radical enough. It should stop pretending to remain human-readable, go binary, and become a variant of protocol buffers, thrift, etc. BTW being schemaless is a boon during initial hacking things together, and an impediment when operating and developing the system further down the line. It's the data equivalent of dynamic typing vs static typing, interpreted vs compiled in code.

indymike 4y ago
> It should stop pretending to remain human-readable, go binary, and become a variant of protocol buffers, thrift, etc. There are lots of ways standard ways to do serialization to binary formats. https://en.wikipedia.org/wiki/Comparison_of_data-serialization_formats#Comparison_of_binary_formats https://en.wikipedia.org/wiki/Comparison_of_data-serializati... > BTW being schemaless is a boon during initial hacking things together, and an impediment when operating and developing the system further down the line. It's the data equivalent of dynamic typing vs static typing, interpreted vs compiled in code. Sounds like XML, which is a whole other discussion :-)

mort96 4y ago

Hey, JCOF author here. I didn't notice this had made it to HN before now. * JCOF allows for streaming serialization if you want, you can just use the key-value object syntax and use inline strings everywhere. You won't get much size improvement over JSON when using a streaming approach, but you can even do a half-and-half approach; maybe you have a set of strings which you know you're using a lot, so you put those strings in the string table, then you do a streaming encode of the rest of your data, with string references to the string table. You could even do the same with object shapes, where you hard-code a couple of object shapes you know you use a lot, and use the JSON-style key-value syntax for other kinds of objects. Basically, you can choose where you want to be on the efficiency vs streaming curve. * Nested objects aren't difficult; an object can go in any place where a value can go, just like in JSON. This object can be either a "shaped" object where the keys are listed in the object shape table, or it can be a "keyed" object where you write out the keys inline (as either literal strings or references to the string table). * JCOF is human-writable; you can write a document which looks a whole lot like JSON. This is a valid document: `;;{"people":[{"name":"Bob",Aage":36},{"name":"Alice","age":40}]}`. But optimal JCOF is mostly unreadable. So it's less human friendly than JSON, more human friendly than binary formats; it's a trade-off. I think JCOF hits enough interesting, unexplored points on enough relevant curves that it has some value. It also ends up being smaller than any of the binary formats I've tried like MessagePack and CBOR (even with the string references extension), while being text-based. It's obviously not going to replace JSON, but it's useful in some cases.

indymike 4y ago
Oh, I think you have something, but I don't think JSON and JCOF are comparable. I think it makes a very interesting replacement for formats where there is a header line like CSV, TSV. JCOF allows for some level of consistent nested structures (which would be nice for header/detail records which are really common in the real world) and would be so much nicer than many of the bespoke formats you see out there... and also, sooo much more efficient than XML, and without the issues that come with XML. Incidentally, these kinds of uses are where JSON really doesn't work well because you always have the potential of a mis-shaped object. In short: nice work. Might be fun to roll an implementation or two :-)