5 ms·

I still don't understand why protobuf has to be in so many Go libraries. It's impossible to avoid unless you roll your own metrics, logging, maybe even mux...

b33j0r 2y ago
Because the same group in google developed both, indeed. They were developed as a microservice high-performance lang, compiler, and serialization protocol in the same meetings.
- jsnell 2y ago
  Where are you getting this nonsense from, and if you're just speculating, why are you stating it with such confidence as a fact? Protobufs predate Go by more than half a decade, and there is no overlap between the authors.
  - akshayshah 2y ago
    Per multiple sources, including the team currently maintaining the Protobuf toolchain within Google, proto3 was largely designed by Rob Pike. Of course the Protobuf wire format is quite a bit older, but some aspects of proto3 and Go's shared semantics (like implicit zero values) do seem to have come from the same mind.
    - dilyevsky 2y ago
      proto1 was designed by Sanjay and Jeff, proto2 by Kenton Varda (who later designed Cap'n'Proto), proto3 I can't remember who but I never heard Rob Pike being credited with this. He did write the very first Go proto binding package though and a separate serialization package called gob with a totally separate rpc library.
    - jsnell 2y ago
      That's a much smaller claim, proto3 schemas are a fairly minor evolution over proto2 schemas, mostly removing features. My impression was always that it was either removing things that were expensive to support in Javascript, impossible to serialize to idiomatic JSON, or that the team thought were misfeatures that nobody should ever use. That's a far cry from the original claim that Go and protobufs were co-designed "in the same meetings". But even that limited claim is kind of hard to believe. Can you link to one of those multiple sources making that claim?
  - smegsicle 2y ago
    "i came up with the term 'protobuf', and go was not what i had in mind" -rob pike
    - fidotron 2y ago
      https://go.dev/blog/gob https://go.dev/blog/gob
- intelVISA 2y ago
  You WILL accept The One Right Way to write code and serialize data! -rob pike Do wake me when they get around to adding the 'high performance' lang part though. Wild how all this ecosystem junk originates from a few early Googlers' fear of learning beautiful C.
  - b33j0r 2y ago
    Just for posterity, I actually agree with you, and was representing the history of our situation. Not my own opinions. “High performance” absolutely means “buzzword.” I can attest that the intentions were good, and it’s another tool in our toolbox. A good one, but not the silver bullet sometimes some said.
TillE 2y ago
It's simple, lightweight, safe serialization. If you need to convert data structures to bytes, you're going to use protobuf or something comparable.
jsheard 2y ago
Even if protobuf isn't your preferred serializer, having any tight binary format be the standard across the ecosystem is miles better than everything slinging JSON around. Lord knows how much energy we are wasting turning binary data into human readable text that no human will ever look at before it's parsed back into binary.
- hot_gril 2y ago
  Performance aside, it's also really nice how a protobuf spec tells you what data structure to expect ahead of time for an API.
  - jon_richards 2y ago
    The number of APIs with a single JSON example as the only documentation is... unfortunate. Especially if you want to know if a field is nullable.
    - mf192 2y ago
      Taking this a bit further, it's surprising how many companies sell API as a service but provide minimal support for that API, publishing like 1 client/sdk (and often not in the language you need, lol). Just mindblowing that companies don't adopt a schema-first approach to appeal to a broader customer base. I've lost count of the number of hnd rolled clients I've had to write over the years. It gets even better if the API itself is backed by some RPC framework. With that, users can simply generate a client and get going. It feels like we can do better as an industry, this cracked me up because it's so true. "a single JSON example as the only documentation is... unfortunate"
    - hot_gril 2y ago
      Funny you should mention nullability. That was a big debate in proto2 vs proto3.
- wbl 2y ago
  XDR was already a core part of NFS. And then there was a DEC alternative also.
beeboobaa3 2y ago
Because it's useful? What do you propose instead?
- xyst 2y ago
  Smoke signals
choppaface 2y ago
golang is a Google technology, so protobuf is going to be there, right? Moreover golang is compiled and protobuf interoperates best with compiled languages. Now just wait for golang packages that have conflicting editions requirements!

b33j0r 2y ago

Because the same group in google developed both, indeed. They were developed as a microservice high-performance lang, compiler, and serialization protocol in the same meetings.

jsnell 2y ago
Where are you getting this nonsense from, and if you're just speculating, why are you stating it with such confidence as a fact? Protobufs predate Go by more than half a decade, and there is no overlap between the authors.
- akshayshah 2y ago
  Per multiple sources, including the team currently maintaining the Protobuf toolchain within Google, proto3 was largely designed by Rob Pike. Of course the Protobuf wire format is quite a bit older, but some aspects of proto3 and Go's shared semantics (like implicit zero values) do seem to have come from the same mind.
  - dilyevsky 2y ago
    proto1 was designed by Sanjay and Jeff, proto2 by Kenton Varda (who later designed Cap'n'Proto), proto3 I can't remember who but I never heard Rob Pike being credited with this. He did write the very first Go proto binding package though and a separate serialization package called gob with a totally separate rpc library.
  - jsnell 2y ago
    That's a much smaller claim, proto3 schemas are a fairly minor evolution over proto2 schemas, mostly removing features. My impression was always that it was either removing things that were expensive to support in Javascript, impossible to serialize to idiomatic JSON, or that the team thought were misfeatures that nobody should ever use. That's a far cry from the original claim that Go and protobufs were co-designed "in the same meetings". But even that limited claim is kind of hard to believe. Can you link to one of those multiple sources making that claim?
- smegsicle 2y ago
  "i came up with the term 'protobuf', and go was not what i had in mind" -rob pike
  - fidotron 2y ago
    https://go.dev/blog/gob https://go.dev/blog/gob
intelVISA 2y ago
You WILL accept The One Right Way to write code and serialize data! -rob pike Do wake me when they get around to adding the 'high performance' lang part though. Wild how all this ecosystem junk originates from a few early Googlers' fear of learning beautiful C.
- b33j0r 2y ago
  Just for posterity, I actually agree with you, and was representing the history of our situation. Not my own opinions. “High performance” absolutely means “buzzword.” I can attest that the intentions were good, and it’s another tool in our toolbox. A good one, but not the silver bullet sometimes some said.

TillE 2y ago

It's simple, lightweight, safe serialization. If you need to convert data structures to bytes, you're going to use protobuf or something comparable.

jsheard 2y ago

Even if protobuf isn't your preferred serializer, having any tight binary format be the standard across the ecosystem is miles better than everything slinging JSON around. Lord knows how much energy we are wasting turning binary data into human readable text that no human will ever look at before it's parsed back into binary.

hot_gril 2y ago
Performance aside, it's also really nice how a protobuf spec tells you what data structure to expect ahead of time for an API.
- jon_richards 2y ago
  The number of APIs with a single JSON example as the only documentation is... unfortunate. Especially if you want to know if a field is nullable.
  - mf192 2y ago
    Taking this a bit further, it's surprising how many companies sell API as a service but provide minimal support for that API, publishing like 1 client/sdk (and often not in the language you need, lol). Just mindblowing that companies don't adopt a schema-first approach to appeal to a broader customer base. I've lost count of the number of hnd rolled clients I've had to write over the years. It gets even better if the API itself is backed by some RPC framework. With that, users can simply generate a client and get going. It feels like we can do better as an industry, this cracked me up because it's so true. "a single JSON example as the only documentation is... unfortunate"
  - hot_gril 2y ago
    Funny you should mention nullability. That was a big debate in proto2 vs proto3.
wbl 2y ago
XDR was already a core part of NFS. And then there was a DEC alternative also.

beeboobaa3 2y ago

Because it's useful? What do you propose instead?

xyst 2y ago
Smoke signals

choppaface 2y ago

golang is a Google technology, so protobuf is going to be there, right? Moreover golang is compiled and protobuf interoperates best with compiled languages. Now just wait for golang packages that have conflicting editions requirements!