5 ms·
One common misconception (or should I say, overgeneralization) is repeated in the article: threads are always unsuited to CPU intensive work. For instance, mos
by pletnes 9y ago
One common misconception (or should I say, overgeneralization) is repeated in the article: threads are always unsuited to CPU intensive work.
For instance, most numpy operations release the GIL, meaning that you can perform heavy computation on multiple threads simultaneously. Certain other C extensions do the same, including some bits of the standard library. The usual caveats apply about threading bugs, of course.
Another detail is that numpy linked to e.g Intel MKL will multithread some operations by default. Running multiple threads-in-threads is likely to cause slowdown.
- jzwinck 9y agoOnly np.dot() has intrinsic multithreading. No other functions do. Bizzarely np.dot() is the fastest way to do things other than dot product (like copy or multiply) in some cases.
- vladf 9y agowell, also np.linalg routines that call LAPACK may be multithreaded.
- jzwinck 9y agoThat's misleading--you say linalg routines "may be" multithreaded, but the vast majority of them never have been. matmul and einsum, despite being candidates for intrinsic multithreading, are not multithreaded. You can read discussion about that here: https://jackkamm.github.io/blog/a-parallel-einsum/ https://jackkamm.github.io/blog/a-parallel-einsum/
- vladf 9y agoI'm sorry, isn't "objects belonging to class C may have property P" a fair way to say there are some members which have it and some which do not? I don't see how that's misleading. I was correcting your statement about np.dot being the only parallel fn
- jzwinck 9y agoCan you name one function other than dot() and tensordot() which has intrinsic multithreading?
- vladf 9y agoEig inv svd det, at least, in 2010. https://dpinte.wordpress.com/2010/01/15/numpy-performance-improvement-with-the-mkl/ https://dpinte.wordpress.com/2010/01/15/numpy-performance-im...
- uranusjr 9y agoI only see the article mentioning CPU (and GIL) once, but in any case, the generalisation is correct for any pure Python code. You can only release the GIL in extension code (for CPython), and in that point you’re not really dealing with Python threads (although you may use Python’s wrapper API for threading), but its underlying implementation (e.g. pthreads) instead. The framing of the statement is very important, and I wouldn’t call it overgeneralising in the context of this particular article.