4 ms·
I've had this experience when writing a simple algorithm for run length encoding, a problem which I struggled to vectorize. Compare the numpy algo with pure pyt
by tda 7y ago
I've had this experience when writing a simple algorithm for run length encoding, a problem which I struggled to vectorize. Compare the numpy algo with pure python:
def rle_encode_pure_python(sequence):
"""Run length encode array."""
previous = sequence[0]
count = 1
out = []
for element in sequence[1:]:
if element == previous:
count += 1
else:
out.append((count, previous))
previous = element
count = 1
out.append((count, previous))
return out
def rle_encode_numpy(sequence):
diffs = np.concatenate(( np.array((True, )), np.diff(sequence)!=0))
indices = np.concatenate((np.where(diffs)[0],np.array((sequence.size, ))))
counts = np.diff(indices).astype('uint16')
values = sequence[diffs].astype('uint8')
return np.rec.fromarrays((counts, values),names=('count','value'))
The numpy version is faster, but a lot less legible (and makes some assumptions about data types and array length). Luckily the pure python version can be jitted with numba and is faster than numpy.
- jononor 7y agoNice and concise example. Would love a mini-writeup / benchmarks for this (including the numba version).
- tda 7y agoHere is the code in a notebook: https://github.com/Python-Meetup-Rotterdam/meetup1/blob/master/approaches%20to%20speeding%20up%20run%20length%20encoding.ipynb https://github.com/Python-Meetup-Rotterdam/meetup1/blob/mast... Gave a little presentation on my findings on using different approaches
- enriquto 7y agoThis is a great example, I love it, thank you very much! Note: the numpy version is missing the import. Note2: I tend to prefer using numba than numpy. Yet, sometimes numpy is inevitable, especially for linear algebra routines. In that case, I am in a world of pain because numba and numpy do not interact well at all!
- deshpand 7y ago< numba and numpy do not interact well at all Can you please elaborate on this? Not challenging you but trying to understand the nuances. Numba tutorial says otherwise: https://numba.pydata.org/numba-doc/dev/user/5minguide.html https://numba.pydata.org/numba-doc/dev/user/5minguide.html "Numba likes NumPy functions"
- m4r35n357 7y agoReturning a list? Nay, Yield!
- tda 7y agoCompletely agreed, would never write it this way anymore but the example is a bit older