4 ms·

Yep, a problem I repeatedly run into with Numpy is that my data is too big to fit in memory as Python data, but can easily fit as Numpy matrices. It seems incre

• tgb 9y ago
  I think the standard recommendation is to know how much you need to allocate and allocate it in advance. You can call numpy.zeros( size ) to get the appropriate array, then fill it with your actual data, for example. If you don't know how much you need, you can try to over-estimate it and then shrink the array (i.e. allocate one of the right size and copy) after your done.
• tempay 9y ago
  Where are you filling your matrix from?
• inlineint 9y ago
  Under the hood this `push!` function have to do memory reallocation and copying the entire array when the array grows to be larger than some internal buffer, which happens each time the array becomes larger than a^n, where a is usually, but not necessary, selected to be equal to 2. The amortized complexity is θ(1), but individual insertions might take O(n) time. It means that it is not hard to have the same in numpy, with the same performance, the only difference is that the growing have to be done manually and that the push_back function has to be called as something like x, x_len = push_back(x, x_len, 10) with push_back being defined like def push_back(x, x_len, value): if len(x) == x_len: x = np.concatenate((x, np.empty(max(len(x), 1))) x[x_len] = value return x, x_len + 1 so that the x variable is supposed to be updated after each insertion.