5 ms·

I use a lot of Python for web stuff and I haven't been in a situation where Python itself was the performance bottleneck. I always thought that when you run int

I use a lot of Python for web stuff and I haven't been in a situation where Python itself was the performance bottleneck. I always thought that when you run into a situation where Python is the bottleneck, you replace the critical bits with something like C/C++/Rust. Following this approach, you would get the best of both worlds: rapid proof-of-concept/time-to-marked with the option to improve performance critical parts later (which often isn't necessary). Could anybody share some experience with this?

Erwin 9y ago
It requires that your code is architected so performance critical sections of Python can move into C etc. Let's say that your code creates a complex object tree from some configuration settings, and executes Python methods and code from all over it, using heavy OO. That is difficult to move to C++ as your performance is spent on Python bookkeeping -- you are calling methods and thus looking things up in dictionary, you are modifying fields and also looking up more thing in dictionaries, increasing decreasing refcounts etc. If you have a million 32-bit numbers that you current run Python code on, great, you don't have to convert Python objects to C at all.
- Rotareti 9y ago
  Thanks for the insights! > Let's say that your code creates a complex object tree from some configuration settings, and executes Python methods and code from all over it, using heavy OO. Luckily, this does not apply to the codebases I'm working on, which are all quite functional (no classes, no inheritance, pure functions exclusively, immutable data types, etc.) I have the feeling that this will not strike me that hard. If you rely on pure functions you have all the application state that you need for the function on the parameters and you pass all new state back through `return`. I guess all I had to do is convert the types once for the C function call (from Python to C) and once for the `return` (from C to Python)?

Erwin 9y ago

It requires that your code is architected so performance critical sections of Python can move into C etc. Let's say that your code creates a complex object tree from some configuration settings, and executes Python methods and code from all over it, using heavy OO. That is difficult to move to C++ as your performance is spent on Python bookkeeping -- you are calling methods and thus looking things up in dictionary, you are modifying fields and also looking up more thing in dictionaries, increasing decreasing refcounts etc. If you have a million 32-bit numbers that you current run Python code on, great, you don't have to convert Python objects to C at all.

Rotareti 9y ago
Thanks for the insights! > Let's say that your code creates a complex object tree from some configuration settings, and executes Python methods and code from all over it, using heavy OO. Luckily, this does not apply to the codebases I'm working on, which are all quite functional (no classes, no inheritance, pure functions exclusively, immutable data types, etc.) I have the feeling that this will not strike me that hard. If you rely on pure functions you have all the application state that you need for the function on the parameters and you pass all new state back through `return`. I guess all I had to do is convert the types once for the C function call (from Python to C) and once for the `return` (from C to Python)?