4 ms·

https://news.ycombinator.com/item?id=18190005 https://news.ycombinator.com/item?id=18190005 Just FYI: an obligatory comment from the protobuf v2 designer. Yea

https://news.ycombinator.com/item?id=18190005 https://news.ycombinator.com/item?id=18190005 Just FYI: an obligatory comment from the protobuf v2 designer. Yeah, protobuf has lots of design mistakes but this article is written by someone who does not understand the problem space. Most of the complexity of serialization comes from implementation compatibility between different timepoints. This significantly limits design space.

thethimble 1y ago
Relatedly, most of the author's concerns are solved by wrapping things in a message. > oneof fields can’t be repeated. Wrap oneof field in message which can be repeated > map fields cannot be repeated. Wrap in message which can contain repeated fields > map values cannot be other maps. Wrap map in message which can be a value Perhaps this is slightly inconvenient/un-ergonomic, but the author is positioning these things as "protos fundamentally can't do this".
- evanmoran 1y ago
  To clarify. Protobuf’s simplest change is adding a field to a message so wrapping maps of maps, maps of fields, oneof fields into a message makes these play to its strengths. It feels like over engineering to turn your Inventory map of items into a Inventory message, but you will be grateful for it when you need a capacity field later.
- deleted 1y ago
  [deleted]
missinglugnut 1y ago
>Most of the complexity of serialization comes from implementation compatibility between different timepoints. The author talks about compatibility a fair bit, specifically the importance of distinguishing a field that wasn't set from one that was intentionally set to a default, and how protobuffs punted on this. What do you think they don't understand?
- summerlight 1y ago
  If you see some statements like below on the serialization topic: > Make all fields in a message required. This makes messages product types. > One possible argument here is that protobuffers will hold onto any information present in a message that they don't understand. In principle this means that it's nondestructive to route a message through an intermediary that doesn't understand this version of its schema. Surely that's a win, isn't it? > Granted, on paper it's a cool feature. But I've never once seen an application that will actually preserve that property. Then it is fair to raise eyebrows on the author's expertise. And please don't ask if I'm attached to protobuf; I can roast the protocol buffer on its wrong designs for hours. It is just that the author makes series of wrong claims presumably due to their bias toward principled type systems and inexperience of working on large scale systems.
  - instig007 1y ago
    > If you see some statements like below on the serialization topic: > Make all fields in a message required. This makes messages product types. > Then it is fair to raise eyebrows on the author's expertise. It's fair to raise eyebrows on your expertise, since required fields don't contribute to b/w incompatibility at all, as every real-world protocol has a mandatory required version number that's tied to a direct parsing strategy with strictly defined algebra, both for shrinking (removing data fragments) and growing (introducing data fragments) payloads. Zero-values and optionality in protobuf is one version of that algebra, it's the most inferior one, subject to lossy protocol upgrades, and is the easiest one for amateurs to design. Then, there's next lavel when the protocol upgrade is defined in terms of bijective functions and other elements of symmetric groups that can tell you whether the newly announced data change can be carried forward (new required field) or dropped (removed field) as long as both the sending and receiving ends are able to derive new compound structures from previously defined pervasive types (the things the protobuf says are oneoffs and messages, for example).
    - xyzzyz 1y ago
      What you describe using many completely unnecessary mathematical terms is not only not found in “every real-world protocol”, but in fact is something virtually absent from overwhelming majority of actually used protocols, with a notable exception of the kind of protocol that gets a four digit numbered RFC document that describes it. Believe it or not, but in the software industry, nobody is defining a new “version number” with “strictly defined algebra” when they want to add a new field to an communication protocol between two internal backend services.

thethimble 1y ago

Relatedly, most of the author's concerns are solved by wrapping things in a message. > oneof fields can’t be repeated. Wrap oneof field in message which can be repeated > map fields cannot be repeated. Wrap in message which can contain repeated fields > map values cannot be other maps. Wrap map in message which can be a value Perhaps this is slightly inconvenient/un-ergonomic, but the author is positioning these things as "protos fundamentally can't do this".

evanmoran 1y ago
To clarify. Protobuf’s simplest change is adding a field to a message so wrapping maps of maps, maps of fields, oneof fields into a message makes these play to its strengths. It feels like over engineering to turn your Inventory map of items into a Inventory message, but you will be grateful for it when you need a capacity field later.
deleted 1y ago
[deleted]

missinglugnut 1y ago

>Most of the complexity of serialization comes from implementation compatibility between different timepoints. The author talks about compatibility a fair bit, specifically the importance of distinguishing a field that wasn't set from one that was intentionally set to a default, and how protobuffs punted on this. What do you think they don't understand?

summerlight 1y ago
If you see some statements like below on the serialization topic: > Make all fields in a message required. This makes messages product types. > One possible argument here is that protobuffers will hold onto any information present in a message that they don't understand. In principle this means that it's nondestructive to route a message through an intermediary that doesn't understand this version of its schema. Surely that's a win, isn't it? > Granted, on paper it's a cool feature. But I've never once seen an application that will actually preserve that property. Then it is fair to raise eyebrows on the author's expertise. And please don't ask if I'm attached to protobuf; I can roast the protocol buffer on its wrong designs for hours. It is just that the author makes series of wrong claims presumably due to their bias toward principled type systems and inexperience of working on large scale systems.
- instig007 1y ago
  > If you see some statements like below on the serialization topic: > Make all fields in a message required. This makes messages product types. > Then it is fair to raise eyebrows on the author's expertise. It's fair to raise eyebrows on your expertise, since required fields don't contribute to b/w incompatibility at all, as every real-world protocol has a mandatory required version number that's tied to a direct parsing strategy with strictly defined algebra, both for shrinking (removing data fragments) and growing (introducing data fragments) payloads. Zero-values and optionality in protobuf is one version of that algebra, it's the most inferior one, subject to lossy protocol upgrades, and is the easiest one for amateurs to design. Then, there's next lavel when the protocol upgrade is defined in terms of bijective functions and other elements of symmetric groups that can tell you whether the newly announced data change can be carried forward (new required field) or dropped (removed field) as long as both the sending and receiving ends are able to derive new compound structures from previously defined pervasive types (the things the protobuf says are oneoffs and messages, for example).
  - xyzzyz 1y ago
    What you describe using many completely unnecessary mathematical terms is not only not found in “every real-world protocol”, but in fact is something virtually absent from overwhelming majority of actually used protocols, with a notable exception of the kind of protocol that gets a four digit numbered RFC document that describes it. Believe it or not, but in the software industry, nobody is defining a new “version number” with “strictly defined algebra” when they want to add a new field to an communication protocol between two internal backend services.