3 ms·

Binary format makes me believe it's not human-readable. How doesn't this compare in size to gzipped JSON? JSON overhead is fairly small (some quotes, colons, br

by dplgk 11y ago

Binary format makes me believe it's not human-readable. How doesn't this compare in size to gzipped JSON? JSON overhead is fairly small (some quotes, colons, brackets and keys) - it's no XML.

Cyph0n 11y ago
I think it can make quite a difference in long messages, as well as when serving a high number of requests.
VStack 11y ago
Yes, binary formats are not easily readable in a text editor. But, it is actually possible to convert ION to an XML format and back again without loss of information (we have not implemented this yet). This should make it easier to read messages during debug - especially because you don't need to know the schema for the given message to conver it to XML. Regarding GZipped JSON, it is true that GZiped JSON is small. But, due to the CRIME and BREACH attacks it is not recommended to compress data sent over encrypted connections (TLS). If you look at our performance benchmarks page you can see a list of serialized length comparisons. As you can see, as soon as you send a few objects in an ION table, the difference is big. More than what you normally can gain with GZip (except perhaps for String).http://tutorials.jenkov.com/iap/ion-performance-benchmarks.html#serialized-length http://tutorials.jenkov.com/iap/ion-performance-benchmarks.h... Furthermore, GZip only helps with transfer time, and actually slows down parsing time. If you look at our performance benchmarks you will see that ION parsing time is a lot faster than JSON. Additionally, if you really, really want high speed you do not parse ION (or JSON) into Java objects. You process the data directly in its binary form. If you look at our read-and-use benchmark you can see just how big a speed difference that gives. ION is designed for being processed directly. JSON isn't as good for that purpose. Finally, ION is designed for fast arbitrary hierarchical navigation. JSON is not.
efaref 11y ago
One thing that I like about CBOR is that with very little knowledge it's surprisingly readable in a hex-dump. Low value positive integers are the value of the byte itself, strings all have the form "0x6L [string]" or "0x7X [len] [string]". Arrays and maps are similarly obvious. Of course, anything more than a simple construct you're better off using a decoder (e.g. a Wireshark one). Also, the fact that it's compatible with JSON means that you can use JSON in your development, and then switch to CBOR at the end for the reduction in packet size. In python it's as simple as changing: import json as encoder to import cbor as encoder

by dplgk 11y ago

Binary format makes me believe it's not human-readable. How doesn't this compare in size to gzipped JSON? JSON overhead is fairly small (some quotes, colons, brackets and keys) - it's no XML.

Cyph0n 11y ago

I think it can make quite a difference in long messages, as well as when serving a high number of requests.

VStack 11y ago

Yes, binary formats are not easily readable in a text editor. But, it is actually possible to convert ION to an XML format and back again without loss of information (we have not implemented this yet). This should make it easier to read messages during debug - especially because you don't need to know the schema for the given message to conver it to XML. Regarding GZipped JSON, it is true that GZiped JSON is small. But, due to the CRIME and BREACH attacks it is not recommended to compress data sent over encrypted connections (TLS). If you look at our performance benchmarks page you can see a list of serialized length comparisons. As you can see, as soon as you send a few objects in an ION table, the difference is big. More than what you normally can gain with GZip (except perhaps for String).http://tutorials.jenkov.com/iap/ion-performance-benchmarks.html#serialized-length http://tutorials.jenkov.com/iap/ion-performance-benchmarks.h... Furthermore, GZip only helps with transfer time, and actually slows down parsing time. If you look at our performance benchmarks you will see that ION parsing time is a lot faster than JSON. Additionally, if you really, really want high speed you do not parse ION (or JSON) into Java objects. You process the data directly in its binary form. If you look at our read-and-use benchmark you can see just how big a speed difference that gives. ION is designed for being processed directly. JSON isn't as good for that purpose. Finally, ION is designed for fast arbitrary hierarchical navigation. JSON is not.

efaref 11y ago

One thing that I like about CBOR is that with very little knowledge it's surprisingly readable in a hex-dump. Low value positive integers are the value of the byte itself, strings all have the form "0x6L [string]" or "0x7X [len] [string]". Arrays and maps are similarly obvious. Of course, anything more than a simple construct you're better off using a decoder (e.g. a Wireshark one). Also, the fact that it's compatible with JSON means that you can use JSON in your development, and then switch to CBOR at the end for the reduction in packet size. In python it's as simple as changing: import json as encoder to import cbor as encoder