3 ms·
I highly doubt that numpy can ever be a bottleneck. In typical python app - there are other things like I/O that consume resources and become bottleneck, before
by slt2021 4y ago
I highly doubt that numpy can ever be a bottleneck. In typical python app - there are other things like I/O that consume resources and become bottleneck, before you run into numpy limits and justify rewrite in C.
- _aavaa_ 4y agoI haven't personally run into IO bottlenecks so I have no idea how you would speed those up in Python. But there's two schools of thoughts I've heard from people regarding how to think about these bottlenecks: 1. IO/network is such a bottleneck so it doesn't matter if the rest is not as fast as possible. 2. IO/network is a bottleneck so you have to work extra hard on everything else to make up for it as much as possible. I tend to fall in the second camp. If you can't work on the data as it's being loaded and have to wait till it's fully loaded, then you need to make sure you process it as quickly as possibly to make up for the time you spend waiting.
- dagw 4y agoIn my typical python apps, it's 0.1-20 seconds of IO and pre-processing, followed by 30 seconds to 10 hours of number crunching, followed by 0.1-20 seconds of post processing and IO.