4 ms·

Curious to know why they are choosing to go with protobuf for intracluster communications as opposed to zero copy protocols like capn proto or flatbuffers. No

Curious to know why they are choosing to go with protobuf for intracluster communications as opposed to zero copy protocols like capn proto or flatbuffers. No doubt protobufs are probably much more battle tested in google scale environments, but are there any other clear benefits if the goal is to reduce spending cpu time encoding/decoding messages? Especially in SOA deployments where many small services need to communicate with one another, I would think that the ability to quickly read any field from a message and pass it on (without first having to decode the entire message) would be a very desirable trait.

kentonv 11y ago
Protobuf is the Google standard, used by basically every single server at Google for the last 15 years. They have built an internal ecosystem of tools around the format. For a Google project to use something different would be weird and would face lots of internal push-back, for good reasons. Even though FlatBuffers is technically from Google, it's from a sub-team of Android working on tools aimed at Android games. The idea was that you'd store your assets in this format. IIRC the initial release didn't do bounds checking so was totally vulnerable to malicious input (but it wasn't intended for such use cases anyhow). I doubt it is widely used on Google's servers. Cap'n Proto is not from Google and there's simply no way they'd choose to use it. To be fair, its support for languages other than C++ remains weak, largely because Sandstorm.io doesn't currently have the resources to build it out. FWIW the ability to read a single field from a message is less important in networking situations because sending/receiving the message is already O(n) and the messages are small-ish, so parsing in O(n) is not a huge deal. Random-access parsing really shines when the input is a massive file on disk. (I'm the author of Cap'n Proto and also of Protobuf v2 (the first version Google open sourced).)
jbeda 11y ago
Gory details: https://github.com/kubernetes/kubernetes/issues/8132 https://github.com/kubernetes/kubernetes/issues/8132

kentonv 11y ago

Protobuf is the Google standard, used by basically every single server at Google for the last 15 years. They have built an internal ecosystem of tools around the format. For a Google project to use something different would be weird and would face lots of internal push-back, for good reasons. Even though FlatBuffers is technically from Google, it's from a sub-team of Android working on tools aimed at Android games. The idea was that you'd store your assets in this format. IIRC the initial release didn't do bounds checking so was totally vulnerable to malicious input (but it wasn't intended for such use cases anyhow). I doubt it is widely used on Google's servers. Cap'n Proto is not from Google and there's simply no way they'd choose to use it. To be fair, its support for languages other than C++ remains weak, largely because Sandstorm.io doesn't currently have the resources to build it out. FWIW the ability to read a single field from a message is less important in networking situations because sending/receiving the message is already O(n) and the messages are small-ish, so parsing in O(n) is not a huge deal. Random-access parsing really shines when the input is a massive file on disk. (I'm the author of Cap'n Proto and also of Protobuf v2 (the first version Google open sourced).)

jbeda 11y ago

Gory details: https://github.com/kubernetes/kubernetes/issues/8132 https://github.com/kubernetes/kubernetes/issues/8132