3 ms·
Performance in native Python really is bad. Nine out of ten times, you can get away with it by figuring out how to push all the computation into the layer of C
by jakobnissen 5y ago
Performance in native Python really is bad. Nine out of ten times, you can get away with it by figuring out how to push all the computation into the layer of C that Python rests on, but sometimes you just can't, and then Python becomes essentially unusable.
From my own experience, implementing Needleman-Wunch or k-mer counting is literally >100x slower in Python than Julia or C. Meaning a two-minute task turns into a 3 hour task - at least. At this point you can begin coding in Rust, but it feels bitter to learn Python, get into it, and then realize you've learned an entire programming language just to use it as wrapper for your actual business code.
- xf1cf 5y agoAgree, after using python for work on research for one of my papers it was wholly insufficient. I ended up writing more Cython than Python to achieve the performance I needed on a mere 100,000 points of data. My performance metrics were similar but I think it was 200-300x slower for me due to all the extra cruft that gets carried around with data structures. That being said I use Python in my professional life without complaint. It’s performant for 99% of that work. I will just never use Python for novel research again unless there are already libraries I can lean on. You’ll find a lot of the big FAANGs using Python have their own implementation of it. Famously Dropbox hired GVR to implement a special performant LLVM based Python.