5 ms·

The comments here are missing a massive use case: shared memory. Shared memory isn't just about programmer convenience. It's about using a machine's memory reso

The comments here are missing a massive use case: shared memory. Shared memory isn't just about programmer convenience. It's about using a machine's memory resources more effectively. Yes, shared memory is available in multi-processing, but it doesn't necessarily interact well with existing codes. I've been working on adding Python support to Legion [1], a task-based runtime system for HPC. Legion wants to manage shared memory so that multiple cores don't necessarily need multiple copies of the data, when the executing tasks don't conflict (all are read-only, or access disjoint data). Legion is C++, so this mostly "just works". Some additional work is required to support GPUs, but it's still not so difficult. But with Python, if we go with multiprocessing, we have to switch to a different mechanism. Worse, Python is an optional dependency for Legion, so we can't depend on Python's multiprocessing support either. If you have a large existing project, and a use case that can take advantage of shared memory, being forced into Python's multiprocessing scheme for parallelism is a pain. We've been investigating using a dlmopen approach as well, based on this proof of concept [2]. Turns out that dlmopen in every available version of libc has a critical bug that prevents it from being practically useful, if you have any desire to make use of native modules. You can build a custom libc with this patch [3] but rolling a custom libc is also a massive pain. In all likelihood we'll end up rolling our own multiprocessing to make this work. If the GIL were truly gone though, we could potentially avoid many of these issues. [1]: http://legion.stanford.edu/ http://legion.stanford.edu/ [2]: https://news.ycombinator.com/item?id=11844268 https://news.ycombinator.com/item?id=11844268 [3]: https://patchwork.ozlabs.org/patch/496559/ https://patchwork.ozlabs.org/patch/496559/

sametmax 9y ago
But then multi-interpreters would allow that, and the article discard it as a valid solution. I find it harsh. It seems much easier to implement, doesn't have the same serialization problem than multiprocessing has and allow to utilize all the CPUs. Yes it's not as good proper threads because you do have more overhead, but it's an order of magnitude better than what we currently have, while being way easier to do that getting rid of the GIL. Too bad the current project is on hold.
- sitkack 9y ago
  The points against sub-interpreters or multi-interpreters is a false dichotomy. There are plenty of scenarios where having multiple interpreters in the same address space would be valuable. Queues could be effectively to communicate between interpreters, no sharing needed. Where sharing is required, those structures can be on their own non-VM heap with the required locks.
detroitcoder 9y ago
^This. It is a very common usecase for applications I work with to create a very large in memory read-only pd dataframe and then put a flask interface to operations on that dataframe using gunicorn and expose as an API. If I use async workers, the dataframe operations are bound by GIL restraints. If I use sync workers, each process needs a copy of the pd dataframe which the server cannot handle (I have never seen pre-fork shared memory work for this problem). I don't want to introduce another technology to solve this problem.
- sambe 9y ago
  I rarely hear people complain about genuine use-cases but this would seem to be one. However, aren't most/all of the dataframe operations done in C extensions in these cases?
  - jzwinck 9y ago
    While a lot of NumPy is C and Fortran, Pandas is mostly pure Python and some Cython. And mostly it does not release the GIL. You often end up having to implement your own C extensions or use Numba for the core of your processing. Even with BLAS enabled, NumPy has almost zero intrinsic parallelism, np.dot() being the notable exception which releases the GIL and uses multicore by itself.
    - throwaway613834 9y ago
      > Even with BLAS enabled, NumPy has almost zero intrinsic parallelism, np.dot() being the notable exception which releases the GIL and uses multicore by itself. Is there any sort of list (comprehensive or otherwise) that denotes which NumPy functions are parallelism-friendly? I mean this whether it's in terms of releasing the GIL, in terms of SIMD support, or in terms of being multi-core.
      - jzwinck 9y ago
        Why are you asking this using a throwaway? np.dot() is multicore. np.load () (and family) releases the GIL. SIMD mostly depends on the build system, so if you want it you might need to build NumPy from source. https://stackoverflow.com/questions/24022723/where-can-i-find-a-list-of-numpy-functions-which-release-the-gil https://stackoverflow.com/questions/24022723/where-can-i-fin...
lstyls 9y ago
This is a good point and one of the few convincing arguments I've heard against the GIL. Thanks for providing so much detail! Did you consider just mounting a ramdisk and storing data as files? At first glance it seems like a decent fit for sharing read-only data in memory.
- darpa_escapee 9y ago
  From my experience, tmpfs adds an overhead. If I use an in-menory database for SQLite, it is about twice as fast to interact with than accessing a database file loaded into a tmpfs.
riri-au 9y ago
Agreed, shared programming is an immensely useful feature for numerical programming, including data science and machine learning. Lots of people will say that those should be written in C++, but I think the rise of machine learning & data science in high level languages argues against their point.
- bhntr3 9y ago
  Yeah. Python is pretty standard now. To implement high throughput scoring on models written in python, I have to run multiple processes with one copy of the python model for each process. For large models like random forests, this can eat up a lot of memory. Ideally, it would be a single model in memory with access from multiple threads. But that won't work right now cause GIL.
sitkack 9y ago
The other glaring issue with CPython is all the globals, meaning there can be only ONE Python VM running in a process space. The closest work has been done on PyParallel https://news.ycombinator.com/item?id=7861942 https://news.ycombinator.com/item?id=7861942 but afaik it is only for windows.

sametmax 9y ago

But then multi-interpreters would allow that, and the article discard it as a valid solution. I find it harsh. It seems much easier to implement, doesn't have the same serialization problem than multiprocessing has and allow to utilize all the CPUs. Yes it's not as good proper threads because you do have more overhead, but it's an order of magnitude better than what we currently have, while being way easier to do that getting rid of the GIL. Too bad the current project is on hold.

sitkack 9y ago
The points against sub-interpreters or multi-interpreters is a false dichotomy. There are plenty of scenarios where having multiple interpreters in the same address space would be valuable. Queues could be effectively to communicate between interpreters, no sharing needed. Where sharing is required, those structures can be on their own non-VM heap with the required locks.

detroitcoder 9y ago

^This. It is a very common usecase for applications I work with to create a very large in memory read-only pd dataframe and then put a flask interface to operations on that dataframe using gunicorn and expose as an API. If I use async workers, the dataframe operations are bound by GIL restraints. If I use sync workers, each process needs a copy of the pd dataframe which the server cannot handle (I have never seen pre-fork shared memory work for this problem). I don't want to introduce another technology to solve this problem.

sambe 9y ago
I rarely hear people complain about genuine use-cases but this would seem to be one. However, aren't most/all of the dataframe operations done in C extensions in these cases?
- jzwinck 9y ago
  While a lot of NumPy is C and Fortran, Pandas is mostly pure Python and some Cython. And mostly it does not release the GIL. You often end up having to implement your own C extensions or use Numba for the core of your processing. Even with BLAS enabled, NumPy has almost zero intrinsic parallelism, np.dot() being the notable exception which releases the GIL and uses multicore by itself.
  - throwaway613834 9y ago
    > Even with BLAS enabled, NumPy has almost zero intrinsic parallelism, np.dot() being the notable exception which releases the GIL and uses multicore by itself. Is there any sort of list (comprehensive or otherwise) that denotes which NumPy functions are parallelism-friendly? I mean this whether it's in terms of releasing the GIL, in terms of SIMD support, or in terms of being multi-core.
    - jzwinck 9y ago
      Why are you asking this using a throwaway? np.dot() is multicore. np.load () (and family) releases the GIL. SIMD mostly depends on the build system, so if you want it you might need to build NumPy from source. https://stackoverflow.com/questions/24022723/where-can-i-find-a-list-of-numpy-functions-which-release-the-gil https://stackoverflow.com/questions/24022723/where-can-i-fin...

lstyls 9y ago

This is a good point and one of the few convincing arguments I've heard against the GIL. Thanks for providing so much detail! Did you consider just mounting a ramdisk and storing data as files? At first glance it seems like a decent fit for sharing read-only data in memory.

darpa_escapee 9y ago
From my experience, tmpfs adds an overhead. If I use an in-menory database for SQLite, it is about twice as fast to interact with than accessing a database file loaded into a tmpfs.

riri-au 9y ago

Agreed, shared programming is an immensely useful feature for numerical programming, including data science and machine learning. Lots of people will say that those should be written in C++, but I think the rise of machine learning & data science in high level languages argues against their point.

bhntr3 9y ago
Yeah. Python is pretty standard now. To implement high throughput scoring on models written in python, I have to run multiple processes with one copy of the python model for each process. For large models like random forests, this can eat up a lot of memory. Ideally, it would be a single model in memory with access from multiple threads. But that won't work right now cause GIL.

sitkack 9y ago

The other glaring issue with CPython is all the globals, meaning there can be only ONE Python VM running in a process space. The closest work has been done on PyParallel https://news.ycombinator.com/item?id=7861942 https://news.ycombinator.com/item?id=7861942 but afaik it is only for windows.