3 ms·
Yup, it is crazy how much optimization these "narrow waist" formats get, and it goes even further with simdjson and so forth [1]. The Python protobuf implement
by chubot 5y ago
Yup, it is crazy how much optimization these "narrow waist" formats get, and it goes even further with simdjson and so forth [1].
The Python protobuf implementation has a somewhat checkered history (I used protobuf v1 and v2 for a long time, and reviewed v3 a tiny bit).
The type system issue is that protobufs to a large extent "replace" your language's types. It's essentially language-independent type. So that means you are limited to a lowest common denominator, and you have the issues of "winners" and "losers"... I would call Python somewhat of a "loser" in the protobuf world, i.e. it feels more second class and is more of a compromise.
This doesn't mean that anybody did a bad job; it's just a fundamental issue with such IDLs. In contrast, JSON/XML/CSV are "data-first" and there are multiple ways of using them and parsing them. You can lazily parse all of them, DOM and SAX, for example, and you have push and pull parsers, etc. Protobufs have grown some of that but it wasn't the primary usage, and many people don't know about it.
[1] https://github.com/simdjson/simdjson https://github.com/simdjson/simdjson