(2018) Original thread here: https://news.ycombinator.com/item?id=18188519 https://news.ycombinator.com/item?id=18188519 Protobufs are the worst (de)serializat

(2018) Original thread here: https://news.ycombinator.com/item?id=18188519 https://news.ycombinator.com/item?id=18188519 Protobufs are the worst (de)serialization format, except for all the others. My chief complaints are: - Protoc is very obtuse and tricky to use for anything where you want packaging, especially with python - The gRPC compiler plugin is even more frustrating in this regard - It's very optimized for compactness on the wire, at the expense of serving as a useful structure within programs (I can't find the source, I think it's somewhere in the protobuf dev docs, but I've had multiple coworkers tell me this) - The gRPC server python implementation does weird things with multiprocessing under the hood that I do not understand, which interferes with other modules trying to use multiprocessing. - I still have not found an ideal way to organize files to work well with importing and still compile correctly with protoc/grpc plugin, and generate python files with correct import syntax. If anyone knows the "correct" way to do this that doesn't require too much setup.py hackery, please let me know. External schema '.proto' files is a feature, not a bug. The complaints in the article about the type system are pretty silly to me. I mean, they are great features, but they are not really in the sphere of the engineering goals when Google set out to make pb/gRPC. Here's what I love about it though: - Support, in particular gRPC's support across so many languages. - Language agnostic data structure contracts - Shallow learning curve to get a smoke test hello world put together - I found it a lot easier than Thrift to get up and start playing To quote hardwaresofon: > in combining three solutions (schema enforcement, binary representation and convenient client/server code generation), they've created something that's just easier to use Specific comparisons: Cap'n proto looks great on paper but at the time (about a year ago) it had some issues with python2.7 and 3.6, which made it a nonstarter for the application at the time. MsgPack-RPC might work well but I'm a bit dissuaded by the unhealthy looking repo of the python/go/cpp implementations. Anything over HTTP - you have the binary-to-text issue. Which, if there are better solutions for this nowadays, let me know. I believe that XML is dead/dying as a ser/de format (outside of the markup domains it has already demonstrated to be very proficient at). Similar lack of binary support. That leaves Thrift and Avro, which have juuust enough of a barrier to entry, with my lack of time to dig into alternatives, that I have not been able to research thoroughly yet.