4 ms·
Why would it be slow?
by kingkilr 13y ago
Why would it be slow?
- gcr 13y agoOne possible reason: Numpy arrays are generally strongly typed, which allows Numpy's custom C implementation of sum to work while only dispatching types once instead of for each element.
- onalark 13y agoGold star for you :) The NumPy implementation is basically a for loop. The only cost is the very brief inspection of the dtype to dispatch on.
- kingkilr 13y agoWhy can't normal pythons sum do that?
- Ihmahr 13y agoBecause python lists can contain different types, unlike numpy arrays
- gcr 13y agoExactly. To further elaborate, try the following in python, if you have the memory: >>> x = list(range(100000000)) + ["break"] After some time, you get a list with a hundred million integers with a string at the end. >>> len(x) 100000001 Now take the sum: >>> sum(x) On my machine, I get a traceback, but after about half a second of hesitation. Python's been iterating through each of those elements, testing to see if it's an integer and building a running sum, only to throw an exception because it didn't expect the string at the end. That means that in order to avoid memory corruption (or other terrible fates worse than an exception), Python must double-check each element of the list as it goes; it cannot take shortcuts.
- kingkilr 13y agoCan't Python notice that a list always has the same types and do something special?
- Ihmahr 13y agoThat could have been implemented but it is not. I dont think it will be because it is not that usefull and would be a lot of work/code for the small benefits. If you want faster python use pypy, if you want homogenieus lists you probably could use numpy arrays.
- rguillebert 13y agoBecause CPython has no JIT :)
- freyrs3 13y agoAt least in CPython sum() will go through PyNumber_Add over heap allocated Python objects. With numpy.sum, the inner loop of the ufunc is summing over an contiguous block of machine types.