4 ms·

Protobuf Editions are here: don't panic

31 points by perezd 2y ago

perezd 2y ago
Google is days away from making Protobuf Editions GA. Never heard of it? Read on.
- orf 2y ago
  What’s the meaning behind this comment?
  - perezd 2y ago
    Many folks don't know about what Protobuf Editions are, and it's about to be generally available. I was encouraging people to read the article :)
    - deleted 2y ago
      [deleted]
  - beeboobaa3 2y ago
    looks like an LLM spambot
    - perezd 2y ago
      lol, well. Sorry man, I'm a real boy. Check my account age. I can solve some math problems or look at what photos are bridges if you want.
      - dilyevsky 2y ago
        Meh, that's too easy now - if you really want to prove it post something politically incorrect.
tb303 2y ago
> It’s unclear what an edition's official support lifetime will be today, although Google has suggested it might be “like 10 years.” ambiguous timeline? sounds right for google.
- VHRanger 2y ago
  The timeline depends on when the project will stop being useful to generate promotions. Thanks Sundar for great org incentive structure!
candiddevmike 2y ago
I still don't understand why protobuf has to be in so many Go libraries. It's impossible to avoid unless you roll your own metrics, logging, maybe even mux...
- b33j0r 2y ago
  Because the same group in google developed both, indeed. They were developed as a microservice high-performance lang, compiler, and serialization protocol in the same meetings.
  - jsnell 2y ago
    Where are you getting this nonsense from, and if you're just speculating, why are you stating it with such confidence as a fact? Protobufs predate Go by more than half a decade, and there is no overlap between the authors.
    - akshayshah 2y ago
      Per multiple sources, including the team currently maintaining the Protobuf toolchain within Google, proto3 was largely designed by Rob Pike. Of course the Protobuf wire format is quite a bit older, but some aspects of proto3 and Go's shared semantics (like implicit zero values) do seem to have come from the same mind.
      - dilyevsky 2y ago
        proto1 was designed by Sanjay and Jeff, proto2 by Kenton Varda (who later designed Cap'n'Proto), proto3 I can't remember who but I never heard Rob Pike being credited with this. He did write the very first Go proto binding package though and a separate serialization package called gob with a totally separate rpc library.
      - jsnell 2y ago
        That's a much smaller claim, proto3 schemas are a fairly minor evolution over proto2 schemas, mostly removing features. My impression was always that it was either removing things that were expensive to support in Javascript, impossible to serialize to idiomatic JSON, or that the team thought were misfeatures that nobody should ever use. That's a far cry from the original claim that Go and protobufs were co-designed "in the same meetings". But even that limited claim is kind of hard to believe. Can you link to one of those multiple sources making that claim?
hot_gril 2y ago
I've always liked the simplicity of protobuf, and this change seems to add unnecessary complexity. And: "What problems are Editions solving? In short, nothing really (for end users). The introduction of Editions is the result of major Google-internal refactoring[...]"
- hehdhdjehehegwv 2y ago
  Design by promo.
  - riwsky 2y ago
    Promobuf
- hot_gril 2y ago
  Thinking about this more, this looks overly dismissive. Seems Google wants to be very explicit about future syntax and allow users to mix in different features. I don't think it's worthwhile, but it's doing something. Is the author just annoyed that Buf now has to put in the work to support editions?
rco8786 2y ago
As if I couldn’t hate protobufs any more. They go and layer on yet more complexity. These sorts of things need to be as simple as humanly possibly and get out of your way so you can actually do some programming. Instead protobufs are the opposite. Full of backwards compat issues, footguns, and needless complexity that make them an absolute horror to work with.
- drunner 2y ago
  Do you have a preferred message protocol?
  - VHRanger 2y ago
    JSON with a good schema if you're doing it across languages?
    - swatcoder 2y ago
      XML, JSON, and YAML belong to a totally diffent class of use than protobuf, flatbuffers, capnproto, etc It's like comparing apples and screwdrivers.
      - hot_gril 2y ago
        They're comparable for a lot of use cases. In fact many protobuf-based APIs will also have a JSON mode.
        swatcoder 2y ago
        Exactly because the consumers of those API's exhibit a strong preference for one or the other. Library/module/API developers often seek to deliver maximimum adaptibility in their interfaces so that consumers of them can satisfy their own more specific/pressing requirements.
        dilyevsky 2y ago
jujube3 2y ago
Oh look, Google is shipping their org chart again. > What problems are Editions solving? > > In short, nothing really (for end users). The introduction of Editions > is the result of major Google-internal refactoring of how protoc and its > plugin architecture implements and observes feature checks when generating > code. This isn’t intended to force breaking changes to existing projects, > nor is it designed to impact any of the existing encodings. > > It should be a boring change that gives plugin maintainers finer-grained > control over how future versions of their Protobuf runtimes behave, > improvements are made, and new features are introduced. Having said that, > it’s impossible to ignore the explosion of verbosity that Editions has > introduced to the project as a side-effect of this level of available control. According to Google themselves, this change "solves no user problems" and introduces "an explosion of verbosity."
- comex 2y ago
  This blog post wasn’t written by Google themselves, but by Buf, a confusingly-named startup that makes their own protobuf implementation.
- zeroxfe 2y ago
  I don't understand why this is a problem. It's not uncommon to see multi-decade old systems refactored for extensibility and maintainability every once in a while. What worked when it was initially designed won't work forever, and it's hard to make users lives better if the maintainers can't make their own lives better. (BTW, this isn't "shipping their org chart.")
  - aeonik 2y ago
    I thought Conway's law was fairly prolific. Though, the closer it is to a true a "law", then the closer GP's comment approaches a tautology. https://wikipedia.org/wiki/Conway%27s_law https://wikipedia.org/wiki/Conway%27s_law
  - tjpnz 2y ago
    Did it ever work? I remember having to write some pretty gnarly hacks to work around the broken Python code protoc would barf out. Working outside of Go and perhaps Java/Kotlin I've little faith that they're not going to break things further.

31 points by perezd 2y ago

perezd 2y ago

Google is days away from making Protobuf Editions GA. Never heard of it? Read on.

orf 2y ago
What’s the meaning behind this comment?
- perezd 2y ago
  Many folks don't know about what Protobuf Editions are, and it's about to be generally available. I was encouraging people to read the article :)
  - deleted 2y ago
    [deleted]
- beeboobaa3 2y ago
  looks like an LLM spambot
  - perezd 2y ago
    lol, well. Sorry man, I'm a real boy. Check my account age. I can solve some math problems or look at what photos are bridges if you want.
    - dilyevsky 2y ago
      Meh, that's too easy now - if you really want to prove it post something politically incorrect.

tb303 2y ago

> It’s unclear what an edition's official support lifetime will be today, although Google has suggested it might be “like 10 years.” ambiguous timeline? sounds right for google.

VHRanger 2y ago
The timeline depends on when the project will stop being useful to generate promotions. Thanks Sundar for great org incentive structure!

candiddevmike 2y ago

I still don't understand why protobuf has to be in so many Go libraries. It's impossible to avoid unless you roll your own metrics, logging, maybe even mux...

b33j0r 2y ago
Because the same group in google developed both, indeed. They were developed as a microservice high-performance lang, compiler, and serialization protocol in the same meetings.
- jsnell 2y ago
  Where are you getting this nonsense from, and if you're just speculating, why are you stating it with such confidence as a fact? Protobufs predate Go by more than half a decade, and there is no overlap between the authors.
  - akshayshah 2y ago
    Per multiple sources, including the team currently maintaining the Protobuf toolchain within Google, proto3 was largely designed by Rob Pike. Of course the Protobuf wire format is quite a bit older, but some aspects of proto3 and Go's shared semantics (like implicit zero values) do seem to have come from the same mind.
    - dilyevsky 2y ago
      proto1 was designed by Sanjay and Jeff, proto2 by Kenton Varda (who later designed Cap'n'Proto), proto3 I can't remember who but I never heard Rob Pike being credited with this. He did write the very first Go proto binding package though and a separate serialization package called gob with a totally separate rpc library.
    - jsnell 2y ago
      That's a much smaller claim, proto3 schemas are a fairly minor evolution over proto2 schemas, mostly removing features. My impression was always that it was either removing things that were expensive to support in Javascript, impossible to serialize to idiomatic JSON, or that the team thought were misfeatures that nobody should ever use. That's a far cry from the original claim that Go and protobufs were co-designed "in the same meetings". But even that limited claim is kind of hard to believe. Can you link to one of those multiple sources making that claim?

hot_gril 2y ago

I've always liked the simplicity of protobuf, and this change seems to add unnecessary complexity. And: "What problems are Editions solving? In short, nothing really (for end users). The introduction of Editions is the result of major Google-internal refactoring[...]"

hehdhdjehehegwv 2y ago
Design by promo.
- riwsky 2y ago
  Promobuf
hot_gril 2y ago
Thinking about this more, this looks overly dismissive. Seems Google wants to be very explicit about future syntax and allow users to mix in different features. I don't think it's worthwhile, but it's doing something. Is the author just annoyed that Buf now has to put in the work to support editions?

rco8786 2y ago

As if I couldn’t hate protobufs any more. They go and layer on yet more complexity. These sorts of things need to be as simple as humanly possibly and get out of your way so you can actually do some programming. Instead protobufs are the opposite. Full of backwards compat issues, footguns, and needless complexity that make them an absolute horror to work with.

drunner 2y ago
Do you have a preferred message protocol?
- VHRanger 2y ago
  JSON with a good schema if you're doing it across languages?
  - swatcoder 2y ago
    XML, JSON, and YAML belong to a totally diffent class of use than protobuf, flatbuffers, capnproto, etc It's like comparing apples and screwdrivers.
    - hot_gril 2y ago
      They're comparable for a lot of use cases. In fact many protobuf-based APIs will also have a JSON mode.
      - swatcoder 2y ago
        Exactly because the consumers of those API's exhibit a strong preference for one or the other. Library/module/API developers often seek to deliver maximimum adaptibility in their interfaces so that consumers of them can satisfy their own more specific/pressing requirements.
        dilyevsky 2y ago

jujube3 2y ago

Oh look, Google is shipping their org chart again. > What problems are Editions solving? > > In short, nothing really (for end users). The introduction of Editions > is the result of major Google-internal refactoring of how protoc and its > plugin architecture implements and observes feature checks when generating > code. This isn’t intended to force breaking changes to existing projects, > nor is it designed to impact any of the existing encodings. > > It should be a boring change that gives plugin maintainers finer-grained > control over how future versions of their Protobuf runtimes behave, > improvements are made, and new features are introduced. Having said that, > it’s impossible to ignore the explosion of verbosity that Editions has > introduced to the project as a side-effect of this level of available control. According to Google themselves, this change "solves no user problems" and introduces "an explosion of verbosity."

comex 2y ago
This blog post wasn’t written by Google themselves, but by Buf, a confusingly-named startup that makes their own protobuf implementation.
zeroxfe 2y ago
I don't understand why this is a problem. It's not uncommon to see multi-decade old systems refactored for extensibility and maintainability every once in a while. What worked when it was initially designed won't work forever, and it's hard to make users lives better if the maintainers can't make their own lives better. (BTW, this isn't "shipping their org chart.")
- aeonik 2y ago
  I thought Conway's law was fairly prolific. Though, the closer it is to a true a "law", then the closer GP's comment approaches a tautology. https://wikipedia.org/wiki/Conway%27s_law https://wikipedia.org/wiki/Conway%27s_law
- tjpnz 2y ago
  Did it ever work? I remember having to write some pretty gnarly hacks to work around the broken Python code protoc would barf out. Working outside of Go and perhaps Java/Kotlin I've little faith that they're not going to break things further.