6 ms·

I just love how a zero byte protobuf message deserialises into a valid struct. Very reassuring.

net_ 4y ago
In the other direction, the "all fields at default value serializes as size 0 message that you can't send" behavior has been an unending annoyance since switching to proto3.
jeffbee 4y ago
That's the example I always use for the people who say that protocol buffers are "type-safe" which they emphatically are not. Any empty buffer will successfully decode as any message that has only optional fields!
- dathinab 4y ago
  the idea of protobuf is "init to default value and then incremental merge message fragments". Which is even more crazy then missing parts have default value, when you consider how that behaves in some cases of mismatching schemas (it also can introduce annoying limitations when trying to create idiomatic libraries in some languages, especially such with "type-safety"). And then add in some of the "automatic type conversions it does". And to top if of some edge cases tend to very often diverge in implementations of different languages, no matter what the original intention was. And the result is uh, just uh, very very uh. I understand why google uses them, it fits them. It probably a good idea given google scale problems, company and team structure and "skill level" of each member. But that doesn't mean it's a good fit for everyone.
  - jeffbee 4y ago
    To be fair to protobuf, its own docs make no claims about "safe". That is projected upon it by people who do not understand it.
    - xyzzy_plugh 4y ago
      protobuf can certainly be type safe, but "protobuf" means many things, which is confusing. My understanding is that type safety comes in the form of generated clients/servers. > That is projected upon it by people who do not understand it. This is the the most Googley response possible. If it's so hard to understand, maybe Google should have done a better job of explaining it?
      - jeffbee 4y ago
        One peer has no reason to believe that the other is using compatible generated code. I can encode AreCatsCuteRequest and you can decode NukeRussiaRequest. There is no safety in it, anywhere. Confusion about with whom you are communicating has caused real, notorious outages, even at Google.
        xyzzy_plugh 4y ago
        Isn't this much of the point of gRPC? To generate clients/servers so that sending the wrong request to the wrong endpoint is much more difficult and inconvenient? This property doesn't seem unique to protobuf nor does it seem relevant to type safety. If you're referring to protobuf's binary encoding as being problematic due to field numbers and default values, then sure, I agree that's a weakness, but again that's not what most people refer to when taking about type safety. It's perfectly reasonable to use protobuf as an IDL and not as an encoding. > Confusion about with whom you are communicating has caused real, notorious outages, even at Google. What does this have to do with type safety? This is authentication. mTLS and to a lesser extent TLS practically solve this.
- 0x457 4y ago
  Well, isn't still type-safe then? If every field is optional, then a zero byte message is a totally valid message. The issue is that every field is optional, and there are no required fields at all. I'm very annoyed that all fields are optional. Which means I have to do additional validation on server side and in a language like rust it's double annoying because now you have `T` or `Option<T>` and it's confusing for some types: - `string` will be rust's `String` except it is an empty string by default - All "custom" types are `Option<T>` even if that makes no sense semantically So now you have to check if a string is empty, which I guess is technically a string anyway. However, now you have no idea, it is an empty string as a value or lack of value? You end up wrapping `string` into a `StringValue` and now linter is complaining that you shouldn't use those. Overall parser for protobuf messages got simpler, but now enforcing safety is on server's implementation.
  - jeffbee 4y ago
    Well, I always try to keep in mind the distinction between protobuf the structured encoding and protobuf the code generator. You can use the former without the latter. Considering that the project doesn't even ship a codegen for Rust, you're already doing that. So if there's some more convenient API you'd like, you can just have it. You can also do this with protoc plugins and the official codegen, or you can hand-roll your own protocol parsers which isn't that crazy with such a simple format.

net_ 4y ago

In the other direction, the "all fields at default value serializes as size 0 message that you can't send" behavior has been an unending annoyance since switching to proto3.

jeffbee 4y ago

That's the example I always use for the people who say that protocol buffers are "type-safe" which they emphatically are not. Any empty buffer will successfully decode as any message that has only optional fields!

dathinab 4y ago
the idea of protobuf is "init to default value and then incremental merge message fragments". Which is even more crazy then missing parts have default value, when you consider how that behaves in some cases of mismatching schemas (it also can introduce annoying limitations when trying to create idiomatic libraries in some languages, especially such with "type-safety"). And then add in some of the "automatic type conversions it does". And to top if of some edge cases tend to very often diverge in implementations of different languages, no matter what the original intention was. And the result is uh, just uh, very very uh. I understand why google uses them, it fits them. It probably a good idea given google scale problems, company and team structure and "skill level" of each member. But that doesn't mean it's a good fit for everyone.
- jeffbee 4y ago
  To be fair to protobuf, its own docs make no claims about "safe". That is projected upon it by people who do not understand it.
  - xyzzy_plugh 4y ago
    protobuf can certainly be type safe, but "protobuf" means many things, which is confusing. My understanding is that type safety comes in the form of generated clients/servers. > That is projected upon it by people who do not understand it. This is the the most Googley response possible. If it's so hard to understand, maybe Google should have done a better job of explaining it?
    - jeffbee 4y ago
      One peer has no reason to believe that the other is using compatible generated code. I can encode AreCatsCuteRequest and you can decode NukeRussiaRequest. There is no safety in it, anywhere. Confusion about with whom you are communicating has caused real, notorious outages, even at Google.
      - xyzzy_plugh 4y ago
        Isn't this much of the point of gRPC? To generate clients/servers so that sending the wrong request to the wrong endpoint is much more difficult and inconvenient? This property doesn't seem unique to protobuf nor does it seem relevant to type safety. If you're referring to protobuf's binary encoding as being problematic due to field numbers and default values, then sure, I agree that's a weakness, but again that's not what most people refer to when taking about type safety. It's perfectly reasonable to use protobuf as an IDL and not as an encoding. > Confusion about with whom you are communicating has caused real, notorious outages, even at Google. What does this have to do with type safety? This is authentication. mTLS and to a lesser extent TLS practically solve this.
0x457 4y ago
Well, isn't still type-safe then? If every field is optional, then a zero byte message is a totally valid message. The issue is that every field is optional, and there are no required fields at all. I'm very annoyed that all fields are optional. Which means I have to do additional validation on server side and in a language like rust it's double annoying because now you have `T` or `Option<T>` and it's confusing for some types: - `string` will be rust's `String` except it is an empty string by default - All "custom" types are `Option<T>` even if that makes no sense semantically So now you have to check if a string is empty, which I guess is technically a string anyway. However, now you have no idea, it is an empty string as a value or lack of value? You end up wrapping `string` into a `StringValue` and now linter is complaining that you shouldn't use those. Overall parser for protobuf messages got simpler, but now enforcing safety is on server's implementation.
- jeffbee 4y ago
  Well, I always try to keep in mind the distinction between protobuf the structured encoding and protobuf the code generator. You can use the former without the latter. Considering that the project doesn't even ship a codegen for Rust, you're already doing that. So if there's some more convenient API you'd like, you can just have it. You can also do this with protoc plugins and the official codegen, or you can hand-roll your own protocol parsers which isn't that crazy with such a simple format.