2 ms·

Enough people have given reasons why Protobufs (and other schema-based interchange formats) are different from the unstructured JSON/BSON/TSON/whatever. But if

Enough people have given reasons why Protobufs (and other schema-based interchange formats) are different from the unstructured JSON/BSON/TSON/whatever. But if you are using a schema there are now far better alternatives to Protobufs. If you're primarily sending data over the network I've found Google's Flatbuffers to be great. For writing to disk Capn'Proto is similar and equally good. Receiving market data where every nanosecond matters? Simple Binary Encoding (SBE). All of these formats employ some form of code generation to extract values from what are essentially cleverly packed structs. All data is sent little endian and follow machine word sizes. Not truly cross-language but Flatbuffers has native support for C/C++, Python, Java, Go, C# and 3rd party support for Rust.

bascule 10y ago
The main reason to prefer protobufs is GRPC: there is now a pseudo-standard HTTP/2-based RPC format with many robust language implementations. That's mostly to say: I think I've generally observed GRPC being embraced. My intended deployment profile is having a single HTTP(/2) server listening on a single TCP port which can speak JSON over HTTP as well as protos over GRPC, both of which can be authenticated by the same objecthash/signature. You can store whatever format you want on disk, but if you primarily intend to serve proto-consuming clients, you might as well store protos on disk so what you serve to the network is an opaque blob of bytes with no transcoding. Don't get me wrong, I really love capnp, particularly the CapTP-like features, but I feel like many of the novelties of its IDL/serialization format (possibly ones involving kentonv's original work before he left Google) have actually shipped in proto3. I really love capnp, but there's this handwavy "this is the way the wind is blowing" argument to be made for GRPC, I think.

bascule 10y ago

The main reason to prefer protobufs is GRPC: there is now a pseudo-standard HTTP/2-based RPC format with many robust language implementations. That's mostly to say: I think I've generally observed GRPC being embraced. My intended deployment profile is having a single HTTP(/2) server listening on a single TCP port which can speak JSON over HTTP as well as protos over GRPC, both of which can be authenticated by the same objecthash/signature. You can store whatever format you want on disk, but if you primarily intend to serve proto-consuming clients, you might as well store protos on disk so what you serve to the network is an opaque blob of bytes with no transcoding. Don't get me wrong, I really love capnp, particularly the CapTP-like features, but I feel like many of the novelties of its IDL/serialization format (possibly ones involving kentonv's original work before he left Google) have actually shipped in proto3. I really love capnp, but there's this handwavy "this is the way the wind is blowing" argument to be made for GRPC, I think.