3 ms·

This is a super flawed argument. Clearly flat buffers and even protocol buffers are faster to serialize and deserialize than json, regardless of what you benchm

This is a super flawed argument. Clearly flat buffers and even protocol buffers are faster to serialize and deserialize than json, regardless of what you benchmark in python.

rapsey 8y ago
And for the amount of messages that are being sent, the speed difference is irrelevant. This is the same conclusion sqlite developers came to. They tested turning JSON column types to binary and the speed difference was not large enough to warrant maintaining that code so they kept the data in JSON.
- IshKebab 8y ago
  If the speed different is irrelevant, why are they struggling with it?
  - rapsey 8y ago
    Because most implementations are reasonably efficient. Swift default one is apparently not.
lifthrasiir 8y ago
So you claim that decoding flat buffers and protobuf is faster than decoding with `struct`? I'm pretty much aware of various flaws and even stated some, but I barely buy that claim without a separate benchmark (which I really welcome by the way). At least I fully understand what the `struct` module actually does under the hood---it sorta compiles to a list of fields and "interprets" the dead simple VM in C. Oh, of course I've used the precompiled `struct.Struct` for that reason (but it was only 20% faster). Anyway, this arrangement is typical for most schematic serialization formats in any language: a bunch of function calls for gluing the desired format, plus a set of well-optimized core functions (not necessarily written in C :-). Henceforth my justification that this is close to the "bare-bones" serialization format.
paulfurtado 8y ago
Python might be the one language that isn't true for. In my Python experience, the Google protobuf library is frustratingly slower than the built-in json module for any data structures I've cared about, which is why things like pyrobuf exist to solve that performance problem: https://github.com/appnexus/pyrobuf https://github.com/appnexus/pyrobuf

This is a super flawed argument. Clearly flat buffers and even protocol buffers are faster to serialize and deserialize than json, regardless of what you benchmark in python.

rapsey 8y ago

And for the amount of messages that are being sent, the speed difference is irrelevant. This is the same conclusion sqlite developers came to. They tested turning JSON column types to binary and the speed difference was not large enough to warrant maintaining that code so they kept the data in JSON.

IshKebab 8y ago
If the speed different is irrelevant, why are they struggling with it?
- rapsey 8y ago
  Because most implementations are reasonably efficient. Swift default one is apparently not.

lifthrasiir 8y ago

So you claim that decoding flat buffers and protobuf is faster than decoding with `struct`? I'm pretty much aware of various flaws and even stated some, but I barely buy that claim without a separate benchmark (which I really welcome by the way). At least I fully understand what the `struct` module actually does under the hood---it sorta compiles to a list of fields and "interprets" the dead simple VM in C. Oh, of course I've used the precompiled `struct.Struct` for that reason (but it was only 20% faster). Anyway, this arrangement is typical for most schematic serialization formats in any language: a bunch of function calls for gluing the desired format, plus a set of well-optimized core functions (not necessarily written in C :-). Henceforth my justification that this is close to the "bare-bones" serialization format.

paulfurtado 8y ago

Python might be the one language that isn't true for. In my Python experience, the Google protobuf library is frustratingly slower than the built-in json module for any data structures I've cared about, which is why things like pyrobuf exist to solve that performance problem: https://github.com/appnexus/pyrobuf https://github.com/appnexus/pyrobuf