4 ms·

What benefits do I get from ProtoBuf, apart from the standard binary wire format? JSON is just more popular as a serialization format. It doesn't matter what w

What benefits do I get from ProtoBuf, apart from the standard binary wire format? JSON is just more popular as a serialization format. It doesn't matter what what programming language or OS I am on, there is almost always a built-in library that de/serialize JSONs at reasonable speed. To send the JSON objects around from one service to another, I can just gzip the string if it's big, or just plain UTF-8 string if it's not. ProtoBuf has to provide more values for people like me to switch. I would rather try out Apache Avro first as a replacement for what I am doing right now.

haberman 11y ago
In my opinion, the biggest benefit from using protobuf is that the schema exists in a .proto file. This can be used to provide all sorts of conveniences. With a plain JSON-based API, you copy and paste field names out of sample code or the documentation. If you spell a field name wrong, there will be no error on the client. If you're lucky, the server might error out because it didn't recognize the property name, but it also might not. If you send an integer when the server was expecting a string, the server might automatically convert or it might not. With protobuf, the schema is explicit in a .proto file. That means that the client library can tell you, at the precise moment that you say msg.misspledFieldName, that the field name doesn't exist. Or if you try to put an integer in there instead of a string, it can tell you about that too. Basically it makes for a tighter feedback loop, which is almost always better. In statically-typed languages like C++ or Java, the schema can be used to generate static types too, so it's actually a compile-time error when you misspell a field name. > It doesn't matter what what programming language or OS I am on, there is almost always a built-in library that de/serialize JSONs at reasonable speed. Yep, that's one reason that proto3 will support JSON as a first-class citizen: https://developers.google.com/protocol-buffers/docs/proto3#json https://developers.google.com/protocol-buffers/docs/proto3#j...
- gcb0 11y ago
  summary: xml annoyances for json :-)
  - haberman 11y ago
    XML isn't painful because it has a schema, XML is painful because it wasn't really designed for RPC, so getting to feature parity with something like Protocol Buffers takes a whole stack of XML technologies and a huge mess of complexity. Protocol Buffers were designed from the ground up for RPC, and as a result are far simpler and more convenient to use than XML. Seriously, nobody who uses Protocol Buffers compares them to XML, because it's not even a comparison. https://developers.google.com/protocol-buffers/docs/overview#whynotxml https://developers.google.com/protocol-buffers/docs/overview...
nostrademons 11y ago
The encoding for protobufs is significantly more compact than JSON. If you're logging or persistently storing your data on the server, this can cut down your storage and bandwidth costs significantly, particularly if you're operating at Google scale. Haberman also mentioned the schema benefits. All that said, I'm using JSON for my current startup. I view them as optimizing for different parts of the product's lifecycle: JSON lets you quickly adapt the protocol and switch out different languages for different services when you're figuring out what product to build, while Protobuf saves you money when you're trying to scale it. I'm also pretty intrigued by Cap'n Proto as a high-performance serialization format, since it fixes a lot of the problems we faced using protobufs at scale at Google, but its language support just isn't up to protobuf/JSON yet, and the protocol is quite complicated.
- mhahn 11y ago
  I use Protobufs for my startup and it has saved us an incredible amount of time building out iOS, Android, and Web clients. With a small team, any time we can shave by not having to re-write the modeling layer in all of these languages is a big win. As the writer of the APIs, I publish the new Protobuf models/services and then can switch over and instantly start working with real objects in Swift or Java. Coming from a larger startup, I've also experienced the pains of trying to maintain JSON objects between different services. Protobufs have some quirks, but I think its a great solution to get behind at any stage.

haberman 11y ago

In my opinion, the biggest benefit from using protobuf is that the schema exists in a .proto file. This can be used to provide all sorts of conveniences. With a plain JSON-based API, you copy and paste field names out of sample code or the documentation. If you spell a field name wrong, there will be no error on the client. If you're lucky, the server might error out because it didn't recognize the property name, but it also might not. If you send an integer when the server was expecting a string, the server might automatically convert or it might not. With protobuf, the schema is explicit in a .proto file. That means that the client library can tell you, at the precise moment that you say msg.misspledFieldName, that the field name doesn't exist. Or if you try to put an integer in there instead of a string, it can tell you about that too. Basically it makes for a tighter feedback loop, which is almost always better. In statically-typed languages like C++ or Java, the schema can be used to generate static types too, so it's actually a compile-time error when you misspell a field name. > It doesn't matter what what programming language or OS I am on, there is almost always a built-in library that de/serialize JSONs at reasonable speed. Yep, that's one reason that proto3 will support JSON as a first-class citizen: https://developers.google.com/protocol-buffers/docs/proto3#json https://developers.google.com/protocol-buffers/docs/proto3#j...

gcb0 11y ago
summary: xml annoyances for json :-)
- haberman 11y ago
  XML isn't painful because it has a schema, XML is painful because it wasn't really designed for RPC, so getting to feature parity with something like Protocol Buffers takes a whole stack of XML technologies and a huge mess of complexity. Protocol Buffers were designed from the ground up for RPC, and as a result are far simpler and more convenient to use than XML. Seriously, nobody who uses Protocol Buffers compares them to XML, because it's not even a comparison. https://developers.google.com/protocol-buffers/docs/overview#whynotxml https://developers.google.com/protocol-buffers/docs/overview...

nostrademons 11y ago

The encoding for protobufs is significantly more compact than JSON. If you're logging or persistently storing your data on the server, this can cut down your storage and bandwidth costs significantly, particularly if you're operating at Google scale. Haberman also mentioned the schema benefits. All that said, I'm using JSON for my current startup. I view them as optimizing for different parts of the product's lifecycle: JSON lets you quickly adapt the protocol and switch out different languages for different services when you're figuring out what product to build, while Protobuf saves you money when you're trying to scale it. I'm also pretty intrigued by Cap'n Proto as a high-performance serialization format, since it fixes a lot of the problems we faced using protobufs at scale at Google, but its language support just isn't up to protobuf/JSON yet, and the protocol is quite complicated.

mhahn 11y ago
I use Protobufs for my startup and it has saved us an incredible amount of time building out iOS, Android, and Web clients. With a small team, any time we can shave by not having to re-write the modeling layer in all of these languages is a big win. As the writer of the APIs, I publish the new Protobuf models/services and then can switch over and instantly start working with real objects in Swift or Java. Coming from a larger startup, I've also experienced the pains of trying to maintain JSON objects between different services. Protobufs have some quirks, but I think its a great solution to get behind at any stage.