4 ms·

Agree 100%. If you need concurrency at the moment, you have already switched to using multiprocessing, so having a no-GIL multithreading is useless. The only

Agree 100%. If you need concurrency at the moment, you have already switched to using multiprocessing, so having a no-GIL multithreading is useless. The only issue with Python/multiprocessing, is that sometimes you don't want queues, but shared mutable state. And as you said, placing Python objects in shared memory at the moment is convoluted, restrictive, and suboptimal. Fixing _that_ should be the objective. What Python need is better support for placing native instances in shared memory.

Znafon 3y ago
> If you need concurrency at the moment, you have already switched to using multiprocessing, so having a no-GIL multithreading is useless. > The only issue with Python/multiprocessing, is that sometimes you don't want queues, but shared mutable state. And as you said, placing Python objects in shared memory at the moment is convoluted, restrictive, and suboptimal. The PEP goes into the motivation behind this work, and using multiple process does not magically solves all the issues: > Multiprocessing, with communication via shared memory or UNIX sockets, adds much complexity and in effect rules out interacting with CUDA from different workers, severely restricting the design space. > I reimplemented parts of HMMER, a standard method for multiple-sequence alignment. I chose this method because it stresses both single-thread performance (scoring) and multi-threaded performance (searching a database of sequences). The GIL became the bottleneck when using only eight threads. This is a method where the current popular implementations rely on 64 or even 128 threads per process. I tried moving to subprocesses but was blocked by the prohibitive IPC costs. > NumPy does release the GIL in its inner loops (which do the heavy lifting), but that is not nearly enough. NumPy doesn’t offer a solution to utilize all CPU cores of a single machine well, and instead leaves that to Dask and other multiprocessing solutions. Those aren’t very efficient and are also more clumsy to use. That clumsiness comes mainly in the extra abstractions and layers the users need to concern themselves with when using, e.g., dask.array which wraps numpy.ndarray. It also shows up in oversubscription issues that the user must explicitly be aware of and manage via either environment variables or a third package, threadpoolctl. The main reason is that NumPy calls into BLAS for linear algebra - and those calls it has no control over, they do use all cores by default via either pthreads or OpenMP. and it discusses the alternatives at https://peps.python.org/pep-0703/#alternatives https://peps.python.org/pep-0703/#alternatives.
- zarzavat 3y ago
  You don’t need OS processes for multiprocessing. You can use threads in the same OS process. See: Erlang.
  - Znafon 3y ago
    Would the work on sub-interpreters be interested for that then (https://lwn.net/SubscriberLink/941090/8bcb029dbf548f26/ https://lwn.net/SubscriberLink/941090/8bcb029dbf548f26/) ?
    - zarzavat 3y ago
      Yes, this is exactly what I’m taking about. You don’t need to share objects. Have explicitly shared buffers instead. Python is a dynamic language so you can easily build proxy objects that are views into a shared buffer, and this allows you to keep all your single threaded performance because no objects are shared. For example: buf = sharedbytes.alloc(1 * GiB) with buf.lock(lockid): buf[10:20] = 42 message_other_process(buf.id) # other process def recv_buffer(bufid): buf = sharedbytes.get(bufid) with buf.lock(lockid): print(buf[10:20]) Most people don’t even need that and would be satisfied with just virtual processes and copying message passing between them.
usrbinbash 3y ago
> so having a no-GIL multithreading is useless. Wrong. Processes pose hurdles via the limits of IPC and control between processes that Threaded applications don't have to bother with. There are ample examples for this in the PEP. > What Python need is better support for placing native instances in shared memory. If that were the case, then threading wouldn't exist in the first place. Shared memory is still IPC. It still requires context switching into the Kernel. It still poses problems that threads don't have.
- Galanwe 3y ago
  You are talking nonsense... > It still requires context switching into the Kernel. There is no less context switching between two threads than between two processes. > It still poses problems that threads don't have. Hu, no, it's actually the same problem. If you see threads as different processes with heap being mmap'd at the same location, then you're basically 99.9% right, up to some minor process accounting metadata differences.
  - usrbinbash 3y ago
    > There is no less context switching between two threads than between two processes. I am not talking about switching context between the threads of execution. I am talking about a context switch to kernel code simply to access the shared memory. All interactions with SM require syscalls. So no, I am not "talking nonsense", and yes, communication between multiple processes requires more context switches than communication between threads.
    - Galanwe 3y ago
      > I am talking about a context switch to kernel code simply to access the shared memory. All interactions with SM require syscalls. Please enlighten me with the syscall you use to access a shared memory... because there are none. As far as the kernel is concerned, "memory" is just a set of pages mapped to a process at a specific address. These pages can be anonymous or named (meaning other processes can map them through that name). There is no syscall, no context switch, involved to read or write memory, named or anonymous. Hell, that's even the whole purpose of memory mapped IO. As mentioned earlier, there is also no real difference between a thread and a process as far as the kernel is concerned. A thread is just a special case of process which maps its heap to its parent. Even the word "thread" doesn't really exist for the Linux kernel. We just call these "lightweight processes", because these are just processes with a shared heap.

Znafon 3y ago

> If you need concurrency at the moment, you have already switched to using multiprocessing, so having a no-GIL multithreading is useless. > The only issue with Python/multiprocessing, is that sometimes you don't want queues, but shared mutable state. And as you said, placing Python objects in shared memory at the moment is convoluted, restrictive, and suboptimal. The PEP goes into the motivation behind this work, and using multiple process does not magically solves all the issues: > Multiprocessing, with communication via shared memory or UNIX sockets, adds much complexity and in effect rules out interacting with CUDA from different workers, severely restricting the design space. > I reimplemented parts of HMMER, a standard method for multiple-sequence alignment. I chose this method because it stresses both single-thread performance (scoring) and multi-threaded performance (searching a database of sequences). The GIL became the bottleneck when using only eight threads. This is a method where the current popular implementations rely on 64 or even 128 threads per process. I tried moving to subprocesses but was blocked by the prohibitive IPC costs. > NumPy does release the GIL in its inner loops (which do the heavy lifting), but that is not nearly enough. NumPy doesn’t offer a solution to utilize all CPU cores of a single machine well, and instead leaves that to Dask and other multiprocessing solutions. Those aren’t very efficient and are also more clumsy to use. That clumsiness comes mainly in the extra abstractions and layers the users need to concern themselves with when using, e.g., dask.array which wraps numpy.ndarray. It also shows up in oversubscription issues that the user must explicitly be aware of and manage via either environment variables or a third package, threadpoolctl. The main reason is that NumPy calls into BLAS for linear algebra - and those calls it has no control over, they do use all cores by default via either pthreads or OpenMP. and it discusses the alternatives at https://peps.python.org/pep-0703/#alternatives https://peps.python.org/pep-0703/#alternatives.

zarzavat 3y ago
You don’t need OS processes for multiprocessing. You can use threads in the same OS process. See: Erlang.
- Znafon 3y ago
  Would the work on sub-interpreters be interested for that then (https://lwn.net/SubscriberLink/941090/8bcb029dbf548f26/ https://lwn.net/SubscriberLink/941090/8bcb029dbf548f26/) ?
  - zarzavat 3y ago
    Yes, this is exactly what I’m taking about. You don’t need to share objects. Have explicitly shared buffers instead. Python is a dynamic language so you can easily build proxy objects that are views into a shared buffer, and this allows you to keep all your single threaded performance because no objects are shared. For example: buf = sharedbytes.alloc(1 * GiB) with buf.lock(lockid): buf[10:20] = 42 message_other_process(buf.id) # other process def recv_buffer(bufid): buf = sharedbytes.get(bufid) with buf.lock(lockid): print(buf[10:20]) Most people don’t even need that and would be satisfied with just virtual processes and copying message passing between them.

usrbinbash 3y ago

> so having a no-GIL multithreading is useless. Wrong. Processes pose hurdles via the limits of IPC and control between processes that Threaded applications don't have to bother with. There are ample examples for this in the PEP. > What Python need is better support for placing native instances in shared memory. If that were the case, then threading wouldn't exist in the first place. Shared memory is still IPC. It still requires context switching into the Kernel. It still poses problems that threads don't have.

Galanwe 3y ago
You are talking nonsense... > It still requires context switching into the Kernel. There is no less context switching between two threads than between two processes. > It still poses problems that threads don't have. Hu, no, it's actually the same problem. If you see threads as different processes with heap being mmap'd at the same location, then you're basically 99.9% right, up to some minor process accounting metadata differences.
- usrbinbash 3y ago
  > There is no less context switching between two threads than between two processes. I am not talking about switching context between the threads of execution. I am talking about a context switch to kernel code simply to access the shared memory. All interactions with SM require syscalls. So no, I am not "talking nonsense", and yes, communication between multiple processes requires more context switches than communication between threads.
  - Galanwe 3y ago
    > I am talking about a context switch to kernel code simply to access the shared memory. All interactions with SM require syscalls. Please enlighten me with the syscall you use to access a shared memory... because there are none. As far as the kernel is concerned, "memory" is just a set of pages mapped to a process at a specific address. These pages can be anonymous or named (meaning other processes can map them through that name). There is no syscall, no context switch, involved to read or write memory, named or anonymous. Hell, that's even the whole purpose of memory mapped IO. As mentioned earlier, there is also no real difference between a thread and a process as far as the kernel is concerned. A thread is just a special case of process which maps its heap to its parent. Even the word "thread" doesn't really exist for the Linux kernel. We just call these "lightweight processes", because these are just processes with a shared heap.