3 ms·
How about an efficient structure for storing "actual" arrays in python. Numpy is my friend.
by jostmey 10y ago
How about an efficient structure for storing "actual" arrays in python. Numpy is my friend.
- Animats 10y agoDoes PyPy have enough type inference smarts to figure out when it can use a homogeneous array? In CPython, everything, including every array entry, is a CObject. (Except strings and now, bytearrays. They're special.) That's why Python arrays of numbers are so slow. NumPy's representation is very close to that of FORTRAN. This problem was mostly solved in the 1950s and 1960s, when FORTRAN DO-loop optimizations for arrays got good.
- ludamad 10y agohttp://doc.pypy.org/en/latest/release-1.8.0.html http://doc.pypy.org/en/latest/release-1.8.0.html It does this since 1.8 it seems (mentioned under 'list specialization'). Probably 'type inference' isn't the right way to describe it. It will optimistically assume it's a homogeneous array and will have to recreate this array if the assumption is broken. This will be useful in the vast majority of cases, as mixed type arrays are not usually used in performance-sensitive code.
- rlayton2 10y agoFurther, there is almost always a better data structure for any mixed-type list. So if you know your code will go through pypy, you can just use a different structure.
- tgb 10y agoWhat kind of DO-loop optimizations were there?
- Animats 10y agoThe same ones we have now. Advancing through an array with pointers instead of computing the subscript each time. Advancing down a higher dimension of an array with a stride. Backus had those working in a FORTRAN compiler in the 1950s. If FORTRAN had been slower than assembly, nobody would have used it; machines were too slow and too expensive.
- DasIch 10y agoYou mean like the array[1] that's in the standard library? [1]: https://docs.python.org/3/library/array.html https://docs.python.org/3/library/array.html