10 ms·
Protobuf is the worst serialization format and schema IDL, except for all the others. The truth is, a lot of these critiques are valid. You could redesign prot
by jchw 4y ago
Protobuf is the worst serialization format and schema IDL, except for all the others.
The truth is, a lot of these critiques are valid. You could redesign protobuf from the ground up and improve it greatly. However, I think this article is bad. Firstly, it repeatedly implies that protobuf was designed by amateurs because of its design pitfalls, ignoring the fact that it's also clear that protobuf grew organically into what it is today rather than deliberately. Secondly, I do not feel it is giving enough credit to the idea that protobuf simply was not designed to be elegant in the first place. It's not a piece of modern art. It is not a Ferrari. It is a Ford F150.
What protobuf does have right is the stuff that's boring; it has a ton of language bindings that are mostly pretty efficient and complete. I'm not going to say it's all perfect, because it's not. But protobuf has an ecosystem that, say, Cap'n Proto does not. This matters more than the fact that protobuf is inelegant. Protobuf may be inelegant, but it does in fact get the job done for many of the use cases.
In my opinion, protobuf is actually good enough. It's not that ugly. Most programming languages you would use it from are more inelegant and incongruent than it is. I don't feel like modelling data or APIs in it is overly limited by this. Etc, etc. That said, if you want to make the most of protobuf, you'll want to make use of more than just protoc, and that's the benefit of having a rich ecosystem: there are plenty of good tools and libraries built around it. I'll admit that outside of Google, it's a bit harder to get going, but my experiences on side projects have made me feel that protobuf outside of Google is still a great idea. It is, in my opinion, significantly less of a pain in the ass than using OpenAPI based solutions or GraphQL, and much more flexible than language-specific tools like tRPC.
For a selection of what the ecosystem has to offer, I think this list is a good start.
https://github.com/grpc-ecosystem/awesome-grpc#protocol-buffers https://github.com/grpc-ecosystem/awesome-grpc#protocol-buff...
- hot_gril 4y ago> Protobuf is the worst serialization format and schema IDL, except for all the others. I don't even feel this way. I can complain for days about most technical things, including things Google has made (e.g. Golang), but not protobufs. They're just good. They don't totally replace JSON of course, but for the intended use case of internal distributed systems, they're solid.
- elteto 4y agoOr JSON backed by a protobuf schema for validation. Best of both worlds.
- hot_gril 4y agoGets a little annoying cause of the translation between the two, but it's valid.
- jchw 4y agoIt's also possible to use JSON directly with gRPC/DRPC/Connect, so you can take advantage of a Protobuf IDL and codegen with a bit less of the logistical complexity.
- GauntletWizard 4y agoJSON is good for dumps of your data, for humans reading it, and for interoperability without the schema - On your backends, you should always serialized to proto because it's darn more efficient.
- elteto 4y agoAlways is such a strong word, always ;) Human readable is very important to me. If performance is not an issue I'd rather grep through it.
- hot_gril 4y agoIf it's a startup or a personal project with no special performance requirements, chance of me using protobufs for anything is very slim. Human readability aside, just bothering to set up the serdes and other boilerplate is too low on my list of priorities. Makes more sense at medium scale.
- lmm 4y agoSlapping it all into gRPC is less boilerplate than whatever else you'd be doing (unless you're doing GraphQL or something). I'd always default to doing that (or thrift, honestly, but same difference) unless there were a particular reason to use something like a manual API.
- zemnmez 4y agoI think this kind of gets to the point of what protobuf is, and what a lot of tech is like at Google in general. I spent a lot of my career believing elegance and expressiveness was so important to strive for, even if we as engineers often fall short. But tech at Google tends to be pragmatic in the specific way that protobuf is. It's not perfect, it doesn't fit neatly into a grand ideology, but it is (at least within Google itself) simple enough, easy enough to understand, portable and fit for purpose. In a similar way to bazel, it's full of components worthy of criticism, but those fixes get made when they become the ecosystem's most pressing issues, and not before.
- eek2121 4y agoNote that a solution does not (and should not) solve every problem. That does not make it bad. Very few (read: NO) solutions can solve every problem. I've known this forever, but when my company started doing outreach to customers, the point was really driven home. Find a problem. Solve it. That is it. If the solution is popular enough it will pick up momentum. There will always be naysayers. It might not be the best solution, or even the right one, but it IS a solution, which is better than what others are doing. Think something is bad? Fix it. Submit a PR if it is open source. Don't complain for the sake of complaining unless you are willing to show how it should be done. Note that I'm not one of those open source jerk holes, however I do recognize that unless I can show I can do better, my opinions are meaningless.