4 ms·
Ah, that's very interesting -- your desire for more "pure" Python comes from a desire for speed. (Some people who wish the generated code was more concrete wan
by haberman 11y ago
Ah, that's very interesting -- your desire for more "pure" Python comes from a desire for speed. (Some people who wish the generated code was more concrete want this so the generated code is more readable).
I am curious what led you to the conclusion that descriptors and metaclasses were making Python protobuf JIT-unfriendly. I ask because most of the "meta" stuff happens at startup, when you first import the module. It uses a metaclass to generate a bunch of Python methods, but after that they are just regular Python methods, and should be as easy for PyPy to optimize as anything else. There is a bit of reflection happening in some code-paths (like the __init__ method does loop over a list of fields to decide how to initialize them), but the metaclass at least is pretty much gone after import.
For this reason, I suspect that even if we ditched all the metaclass stuff, you'd see PyPy performance pretty close to what you're seeing now. The only thing I could see potentially making a more significant difference is if there were generated parsing code that switches on field number, instead of looking up fields by number in a dictionary. But given that Python doesn't have an actual switch statement, this might not be an improvement at all.
- fijal 11y agoHi The python code that's generated is ok, but the problem is that everything is very dynamic. If you want this to work fast on PyPy, it should really generate a bunch of getters and setters that use attribute access and not go through generic __getattr__ that does some dictionary lookups. Additionally a lot of code is written in a way that creates a lot of temporary lists iterating over all fields and double checking stuff - this is bound to be way slower than a simple C/Java stuff that does the very simplest "check type - attribute access" sort of stuff.