4 ms·
Somewhat ironically, Python is used a lot for things that would benefit from raw speed (data processing pipelines) and do not benefit at all from dynamic typing
by _pmf_ 9y ago
Somewhat ironically, Python is used a lot for things that would benefit from raw speed (data processing pipelines) and do not benefit at all from dynamic typing (since the kind of property bags / data frame views over data are easily replicated in statically typed languages). But Python's C extension API is quite a bit easier than p.e. Matlab's MEX API (to me at least); can typical Python IDEs compile and relink extension modules without an external build step?
> Your bottleneck is most likely not CPU or Python itself.
With applications that are dominated by raw data processing, it's very, very easy to be CPU dominated. Hell, I had one quite trivial data converter for logfiles where the "parsing the printf string" part of Java's printf dominated processing and writing a custom formatter halved processing time (while regexes can be compiled, the format string cannot be precompiled and will be interpreted each time); it's one of those things where I would intuitively say "why did this moron write his custom formatter" if I stumbled upon it in a code review. Intuitively, you'd expect this to be a simple case of an IO dominated task (which it is now once the bottleneck has been removed).
If it's fire-and-forget batch jobs, you can get away with it, but if the converter is part of a user facing fat client application that runs on a old office laptop, you don't have that luxury.