3 ms·

I was surprised to see that allocating with numpy is actually 2x slower than allocating a list: In [4]: length = 10000 In [5]: %%timeit ...: #preal

I was surprised to see that allocating with numpy is actually 2x slower than allocating a list: In [4]: length = 10000 In [5]: %%timeit ...: #preallocating list ...: pre = [None] \* length ...: for i in range(length): ...: pre[i] = i ...: 346 µs ± 10.3 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each) In [6]: %%timeit ...: pre = np.empty(length, dtype=int) ...: for i in range(length): ...: pre[i] = i ...: 688 µs ± 20.3 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

sharikous 5y ago
You are measuring two things together: allocation and an initialization inside a loop. I would try to separate them (I.e. measure [None]*length and np.empty(length)
- knappa 5y ago
  In [1]: length = 10_000 In [2]: %%timeit ...: pre = [None] * length ...: ...: 38.7 µs ± 128 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each) In [3]: %%timeit ...: pre = np.empty(length, dtype=int) ...: ...: 1.32 µs ± 6.59 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)
erdewit 5y ago
It's that the array access of numpy is surprisingly slow compared to a Python list: lst = [0] %timeit lst[0] = 1 33.9 ns ± 0.0415 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) arr = np.empty(1, dtype=int) %timeit arr[0] = 1 73.4 ns ± 0.306 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
w-m 5y ago
Here was a weird one for me, why is numba jitted initialization twice as fast as np.arange, which presumably has a C implementation? In [1]: import numpy as np In [2]: %timeit np.arange(10000, dtype=np.int64) 3.03 µs ± 94.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each) In [3]: import numba In [4]: @numba.jit ...: def alloc_np_jit(length): ...: pre = np.empty(length, dtype=np.int64) ...: for i in range(length): ...: pre[i] = i ...: return pre ...: In [5]: %timeit alloc_np_jit(10000) 1.56 µs ± 12.7 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

sharikous 5y ago

You are measuring two things together: allocation and an initialization inside a loop. I would try to separate them (I.e. measure [None]*length and np.empty(length)

knappa 5y ago
In [1]: length = 10_000 In [2]: %%timeit ...: pre = [None] * length ...: ...: 38.7 µs ± 128 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each) In [3]: %%timeit ...: pre = np.empty(length, dtype=int) ...: ...: 1.32 µs ± 6.59 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)

erdewit 5y ago

It's that the array access of numpy is surprisingly slow compared to a Python list: lst = [0] %timeit lst[0] = 1 33.9 ns ± 0.0415 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) arr = np.empty(1, dtype=int) %timeit arr[0] = 1 73.4 ns ± 0.306 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

w-m 5y ago

Here was a weird one for me, why is numba jitted initialization twice as fast as np.arange, which presumably has a C implementation? In [1]: import numpy as np In [2]: %timeit np.arange(10000, dtype=np.int64) 3.03 µs ± 94.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each) In [3]: import numba In [4]: @numba.jit ...: def alloc_np_jit(length): ...: pre = np.empty(length, dtype=np.int64) ...: for i in range(length): ...: pre[i] = i ...: return pre ...: In [5]: %timeit alloc_np_jit(10000) 1.56 µs ± 12.7 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)