2 ms·
FFI and allocation overhead, plus not being able to perform global optimizations. Each numpy call is pretty fast, and if that's all you're doing then it's proba
by hansvm 5y ago
FFI and allocation overhead, plus not being able to perform global optimizations. Each numpy call is pretty fast, and if that's all you're doing then it's probably pretty close to optimal (ignoring for the moment that their routines explicitly optimize large inputs at the expense of small -- for small vectors you can beat numpy even in pure python if you don't explicitly rewrite the problem as a function of larger vectors with appropriate shapes), but a composite call graph potentially (and often) might have additional optimizations that can be applied.
For an illustrative example, imagine computing the sum of all elements of some `x-1` where x is some numpy vector. Each individual operation is dispatched to numpy without any global context, so it must explicitly allocate a buffer for the x-1 result before computing the sum. The astute observer might note that you can instead use np.subtract(x, 1, out=x) or np.subtract(x, 1, out=buffer) to reduce the impact of allocation, but the problem is _still_ dominated by memory bandwidth on modern hardware (at one point in the past memory access was about as slow as a computation, and now it's 100x slower at least), and having to iterate over roughly the same elements twice rather than doing a bunch of CPU/GPU operations at each element roughly doubles the total computation time if everything doesn't fit an a very fast cache. Even when using numpy to its fullest (which most people don't), the model of stringing together a bunch of local optimizations can't always and doesn't usually compete with the model of at least attempting to globally optimize the whole computation.