3 ms·

> e.g. Protobufs are very good for C++ <-> C++ communication, but Java and Python users seem to dislike them equally, and even Go users do too. Well, hilarious

> e.g. Protobufs are very good for C++ <-> C++ communication, but Java and Python users seem to dislike them equally, and even Go users do too. Well, hilariously, when I looked at using it on a project, JSON outperformed protobuf in Python. JSON is implemented in C, Protobuf in Python, and the C decoder for a less efficient format won out. (Now, technically, Protobuf has a C implementation, but at the time I was testing, it segfaulted reliably. Which is the problem with C…) I also recall there were some severe problems with protobuf's ability to represent some types, but I don't remember what they are at this point. I thought it was something to do with sum types, but I'm looking at it now and it does have oneof, so IDK.

chubot 5y ago
Yup, it is crazy how much optimization these "narrow waist" formats get, and it goes even further with simdjson and so forth [1]. The Python protobuf implementation has a somewhat checkered history (I used protobuf v1 and v2 for a long time, and reviewed v3 a tiny bit). The type system issue is that protobufs to a large extent "replace" your language's types. It's essentially language-independent type. So that means you are limited to a lowest common denominator, and you have the issues of "winners" and "losers"... I would call Python somewhat of a "loser" in the protobuf world, i.e. it feels more second class and is more of a compromise. This doesn't mean that anybody did a bad job; it's just a fundamental issue with such IDLs. In contrast, JSON/XML/CSV are "data-first" and there are multiple ways of using them and parsing them. You can lazily parse all of them, DOM and SAX, for example, and you have push and pull parsers, etc. Protobufs have grown some of that but it wasn't the primary usage, and many people don't know about it. [1] https://github.com/simdjson/simdjson https://github.com/simdjson/simdjson

chubot 5y ago

Yup, it is crazy how much optimization these "narrow waist" formats get, and it goes even further with simdjson and so forth [1]. The Python protobuf implementation has a somewhat checkered history (I used protobuf v1 and v2 for a long time, and reviewed v3 a tiny bit). The type system issue is that protobufs to a large extent "replace" your language's types. It's essentially language-independent type. So that means you are limited to a lowest common denominator, and you have the issues of "winners" and "losers"... I would call Python somewhat of a "loser" in the protobuf world, i.e. it feels more second class and is more of a compromise. This doesn't mean that anybody did a bad job; it's just a fundamental issue with such IDLs. In contrast, JSON/XML/CSV are "data-first" and there are multiple ways of using them and parsing them. You can lazily parse all of them, DOM and SAX, for example, and you have push and pull parsers, etc. Protobufs have grown some of that but it wasn't the primary usage, and many people don't know about it. [1] https://github.com/simdjson/simdjson https://github.com/simdjson/simdjson