3 ms·
(1) Even taking a 20% speedup at face value, I'd still describe "single-process Python can only use 1 core" as accurate, with rounding. (2) You're not measurin
by paulddraper 7y ago
(1) Even taking a 20% speedup at face value, I'd still describe "single-process Python can only use 1 core" as accurate, with rounding.
(2) You're not measuring what you think you are measuring.
Take out the Pool altogether:
def run(items, n):
step = len(items) // n
res = [sum(items[i*step : (i+1)*step]) for i in range(n)]
return sum(r for r in res)
and the results are similar:
1 0.170
2 0.170
3 0.147
4 0.145
5 0.142
6 0.142
7 0.138
8 0.137
9 0.135
10 0.138
I have some guesses why it improves, but I'm not certain. It's definitely nothing to do with multithreading.
EDIT: As an aside, you can speed this up 20x by using Numpy:
items = numpy.array([random.random() for _ in range(10 ** 7)])
def run(items, n):
return numpy.sum(items)
1 0.007
Supporting the original point that running code in low-level languages produces amazing speedups vs fancy parallelism methods.