3 ms·

> if you call np.dot on the same underlying ndarray from multiple threads, what are the semantics? Usually, this will be safe because np.dot only reads from it

> if you call np.dot on the same underlying ndarray from multiple threads, what are the semantics? Usually, this will be safe because np.dot only reads from its arguments, and produces a fresh new array to hold the result. I just checked the documentation and found there is an out=argument to pass a destination array. (In any case there are other numpy functions that do modify their argument.) If you pass the same array to that parameter in two different threads then I don't know what happens. Maybe numpy locks the GIL to write its output. Maybe it produces C-style undefined behaviour (this would be my guess ... and my preference). > I think most people want a single invocation of np.dot in a single thread to use multiple worker threads Maybe? I'm not convinced that's what I want! But it's pretty irrelevant either way to my point; np.dot was just an example. My real point is that lots of CPU-heavy Python functions are implemented in C and release the GIL, so in practice parallelism with Python is often possible.

dekhn 5y ago
I've worked in this field awhile. 99% of people want this: np.dot and others use all the cores on your system by default. This accelerates the common use case, which is individual user on a multicore machine who wants single result ASAP. Then, there's an env var that lets you configure the underlying matrix library to use only a single (or maybe another number) of cores, and you work with the multithreaded/multiprocess libraries to run many independent calculations (possibly on shared read-only source data); then that scheduler has a tunable set to saturate the cores or IO on the system. This is what most BLASses use and it's how data analysts and supercomputer folks can get along.
- quietbritishjim 5y ago
  To reiterate: None of this contradicts my main point that numpy.dot releases the GIL, which it does even if it calls into a BLAS library that uses multiple cores (although of course that would then have its own global lock - potentially even an interprocess lock - to manage concurrent uses). numpy.dot was only meant to be an example in the first place! I could just as easily have talked about some optimisation routine in scipy or something in the standard library (actually I did mention ZipFile.extract). > I've worked in this field awhile. This statement undermines your point more than it supports it. There is no "this field" for numpy. Just working in some field that uses it is no qualification for knowing whether users as a whole would like it multithreaded automatically. "Data analysts" and "supercomputer folks" no doubt include many users but certainly not all and possibly not even most (for all we know). > 99% of people want this: ... I can believe this might be true. I don't think I ever claimed otherwise. In fact I said "maybe"! I only claimed that I don't want this. I will make one more claim though: I don't think there's any way that you could know this, even if it's really true. But yes, I agree and already knew that it's how BLAS libraries work by default, including OpenBLAS which is what the prebuilt numpy wheels use. Even then, as you're probably already aware by the sounds of it, that's only for sufficiently large matrices. If you're just using small matrices and vectors (e.g. in 3D to represent real-world coordinates) then processing will be done directly in the calling thread (because the overhead of using multiple threads would be greater than the benefit).

dekhn 5y ago

I've worked in this field awhile. 99% of people want this: np.dot and others use all the cores on your system by default. This accelerates the common use case, which is individual user on a multicore machine who wants single result ASAP. Then, there's an env var that lets you configure the underlying matrix library to use only a single (or maybe another number) of cores, and you work with the multithreaded/multiprocess libraries to run many independent calculations (possibly on shared read-only source data); then that scheduler has a tunable set to saturate the cores or IO on the system. This is what most BLASses use and it's how data analysts and supercomputer folks can get along.

quietbritishjim 5y ago
To reiterate: None of this contradicts my main point that numpy.dot releases the GIL, which it does even if it calls into a BLAS library that uses multiple cores (although of course that would then have its own global lock - potentially even an interprocess lock - to manage concurrent uses). numpy.dot was only meant to be an example in the first place! I could just as easily have talked about some optimisation routine in scipy or something in the standard library (actually I did mention ZipFile.extract). > I've worked in this field awhile. This statement undermines your point more than it supports it. There is no "this field" for numpy. Just working in some field that uses it is no qualification for knowing whether users as a whole would like it multithreaded automatically. "Data analysts" and "supercomputer folks" no doubt include many users but certainly not all and possibly not even most (for all we know). > 99% of people want this: ... I can believe this might be true. I don't think I ever claimed otherwise. In fact I said "maybe"! I only claimed that I don't want this. I will make one more claim though: I don't think there's any way that you could know this, even if it's really true. But yes, I agree and already knew that it's how BLAS libraries work by default, including OpenBLAS which is what the prebuilt numpy wheels use. Even then, as you're probably already aware by the sounds of it, that's only for sufficiently large matrices. If you're just using small matrices and vectors (e.g. in 3D to represent real-world coordinates) then processing will be done directly in the calling thread (because the overhead of using multiple threads would be greater than the benefit).