5 ms·

Every time the GIL removal is discussed, I notice a tidal wave of negativity in the comments, which is completely surprising to me. I suspect some critics prima

by w-m 3y ago

Every time the GIL removal is discussed, I notice a tidal wave of negativity in the comments, which is completely surprising to me. I suspect some critics primarily deal with thread-parallel programs that focus on asynchronous data sharing and task signaling. In languages like C++, handling these with threads and locks is tricky, and you're all but guaranteed to produce subtle race conditions. I'm totally with you on this – I wouldn't want to manage this in Python either. However, consider parallel data processing, a task I face so often in computer vision. In C++, I can slap a `pragma omp parallel for` onto a for loop processing a list of images/videos/meshes/csvs, increasing the speed of this section of the program by the number of CPU cores. I do this when the problem is well defined (one input, one output each), and it hasn't blown up in my face once. In Python, on the other hand, it feels cumbersome when spinning up worker processes for such tasks, not to mention harder to debug. For this kind of work, removing the GIL would be a massive relief. It's good that people are highlighting the issues. But to add my weight to the scale, the GIL is the main barrier in Python to stand in my way to speed up my programs. So I'm very happy that they're investing their resources to build a proper way of doing thread-parallelism.

Eisenstein 3y ago
I want to let you know that I agree with you 100% and face the same issues. It is baffling to me how this has not been solved by now, especially with Python's dominance in computer vision and machine learning.
deleted 3y ago
[deleted]
dagw 3y ago
It's worth remembering that the first serious gil removal patch was submitted against python 1.4, so it's not like people haven't been trying. There have also been (not 100% CPython compatible) python implementations without the GIL that people have could have used, but none of them got any traction. I suspect the negativity you're seeing comes mainly from two main sources. First people are terrified of another 2->3 situation. If removing the GIL ends up breaking any existing libraries or causing significant single-core performance degradations then that could do seriously damage to python and no one wants to go through that again. Secondly people who have been working on this for a long time get annoyed when people make removing the GIL sound easy and imply that the only reason it hasn't been done is because the developers are lazy, incompetent or haven't thought of it. Various people have been trying for literally decades, but so far no one has managed to come up with a solution that is both backwards compatible and performant. Everybody agrees that removing the GIL would be fantastic. However most of the core developers aren't willing to sacrifice backwards compatibility with existing code to get there.
School-Cotton 3y ago
I guess I think of highly parallel CPU-bound work as being fundamentally against the ethos of Python, which is to be a scripting/glue language on top of native extensions. Thus anything (like GIL removal) that makes it harder to write native extensions is harmful. Even if the GIL is removed, the task you’re talking about would be dozens of times slower in Python than in a better-suited language, so why not write it in C++ or Rust and then call into that from Python? I could be wrong or missing something, and I’m not a Python developer so I don’t really have a dog in the fight, but I just wanted to try to explain where some of the criticism is coming from.
- w-m 3y ago
  It's exactly the high-performance native extensions that require the GIL removal to allow for a full utilization of the machine. Although already successful as glue for these libraries, Python is ultimately hamstrung by the GIL on many-core machines. Using the GIL removal to start writing algorithms in interpreted Python is not the intention. I warmly recommend reading the motivation section of the PEP. It is incredibly well written, quoting issues from PyTorch, scikit and NumPy: https://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/
  - School-Cotton 3y ago
    That doesn't make sense to me, since native extensions can and do drop the GIL. Try calling `torch.set_num_threads(16)` and then doing some CPU-intensive operation on a large tensor; you will observe the Python process using 16 cores. What am I missing?
    - w-m 3y ago
      I think you are missing that the world is a little messy. You are correct that native extensions can and do drop the GIL. But data needs to be loaded, it needs to be preprocessed (does your image downsampling library drop the GIL? Does the video decoder drop the GIL? How do you find out? You need to do a little string processing on your data, well then you're out of luck.). Hardware devices need to be managed. Progress needs to be logged, and operations need to be debugged. There isn't a perfect kernel that can be run for every operation, you may have to run a little Python code before calling into your native code. Again, I would encourage you to read the motivation section of the PEP, which shows a multitude of frustrations with the current state. The issue is subtle enough that I'm struggling to summarize it in a single sentence.
tgv 3y ago
Preface: I rarely use Python. But to me, the logical, manageable thing seems to be a double implementation: one version with, one version without gil. Pick the one that suits you. Computer vision often has tasks independent of the global state; pure functions, if you will. But many applications have different "state" pools all over the place. JIRA is written in Python. Their library for accessing their API alone is 8k lines of Python. I suspect JIRA's code base won't be all pure functions. A large group (probably a comfortable majority) uses Python at a basic level. They may use numpy and matplot, but that's about it. They will notice a slow down when the GIL is removed. Don't expect smiles. More advanced users may use concurrent code. If you use an external C library, chances are it has to be updated to work with a no-gil python version. This won't go swimmingly. It puts the burden on the maintainers of those libs, and they may not be able or willing to fix the problem. That will lead to unhappy faces. And then there will be the subtle bugs in complex systems. Python programmers are not really used to synchronicity problems, so they'll slap locks all over the place, leading to slow down or dead-locks. More unhappy faces. So I see the commitment to a single, no-GIL version as a serious threat to Python.
liuliu 3y ago
I am thinking about this. I think our distinction between IO-bounded tasks v.s. compute-bounded tasks helped design in some ways. However, as time goes by, the code you are dealing with can bounce between these two modes, and that is where all the limitations seem good for you start to be bad. That's what I suspect happening on the Python / scientific-computing community. They thought (and some people in this thread still think) that Python can be the driver to wait for compute-bounded task to finish (effectively, making the Python code to be IO-bounded / i.e. waiting for another language to finish). But over time, it changed. People write more code in the Python driver, and suddenly, the very optimized C code is no longer the bottleneck, the Python code is. We've seen this bouncing pattern enough times and really need to think through whether we want to have two different design patterns to deal with IO-bounded problem v.s. compute-bounded problem.

by w-m 3y ago

Eisenstein 3y ago

I want to let you know that I agree with you 100% and face the same issues. It is baffling to me how this has not been solved by now, especially with Python's dominance in computer vision and machine learning.

deleted 3y ago

[deleted]

dagw 3y ago

It's worth remembering that the first serious gil removal patch was submitted against python 1.4, so it's not like people haven't been trying. There have also been (not 100% CPython compatible) python implementations without the GIL that people have could have used, but none of them got any traction. I suspect the negativity you're seeing comes mainly from two main sources. First people are terrified of another 2->3 situation. If removing the GIL ends up breaking any existing libraries or causing significant single-core performance degradations then that could do seriously damage to python and no one wants to go through that again. Secondly people who have been working on this for a long time get annoyed when people make removing the GIL sound easy and imply that the only reason it hasn't been done is because the developers are lazy, incompetent or haven't thought of it. Various people have been trying for literally decades, but so far no one has managed to come up with a solution that is both backwards compatible and performant. Everybody agrees that removing the GIL would be fantastic. However most of the core developers aren't willing to sacrifice backwards compatibility with existing code to get there.

School-Cotton 3y ago

I guess I think of highly parallel CPU-bound work as being fundamentally against the ethos of Python, which is to be a scripting/glue language on top of native extensions. Thus anything (like GIL removal) that makes it harder to write native extensions is harmful. Even if the GIL is removed, the task you’re talking about would be dozens of times slower in Python than in a better-suited language, so why not write it in C++ or Rust and then call into that from Python? I could be wrong or missing something, and I’m not a Python developer so I don’t really have a dog in the fight, but I just wanted to try to explain where some of the criticism is coming from.

w-m 3y ago
It's exactly the high-performance native extensions that require the GIL removal to allow for a full utilization of the machine. Although already successful as glue for these libraries, Python is ultimately hamstrung by the GIL on many-core machines. Using the GIL removal to start writing algorithms in interpreted Python is not the intention. I warmly recommend reading the motivation section of the PEP. It is incredibly well written, quoting issues from PyTorch, scikit and NumPy: https://peps.python.org/pep-0703/ https://peps.python.org/pep-0703/
- School-Cotton 3y ago
  That doesn't make sense to me, since native extensions can and do drop the GIL. Try calling `torch.set_num_threads(16)` and then doing some CPU-intensive operation on a large tensor; you will observe the Python process using 16 cores. What am I missing?
  - w-m 3y ago
    I think you are missing that the world is a little messy. You are correct that native extensions can and do drop the GIL. But data needs to be loaded, it needs to be preprocessed (does your image downsampling library drop the GIL? Does the video decoder drop the GIL? How do you find out? You need to do a little string processing on your data, well then you're out of luck.). Hardware devices need to be managed. Progress needs to be logged, and operations need to be debugged. There isn't a perfect kernel that can be run for every operation, you may have to run a little Python code before calling into your native code. Again, I would encourage you to read the motivation section of the PEP, which shows a multitude of frustrations with the current state. The issue is subtle enough that I'm struggling to summarize it in a single sentence.

tgv 3y ago

Preface: I rarely use Python. But to me, the logical, manageable thing seems to be a double implementation: one version with, one version without gil. Pick the one that suits you. Computer vision often has tasks independent of the global state; pure functions, if you will. But many applications have different "state" pools all over the place. JIRA is written in Python. Their library for accessing their API alone is 8k lines of Python. I suspect JIRA's code base won't be all pure functions. A large group (probably a comfortable majority) uses Python at a basic level. They may use numpy and matplot, but that's about it. They will notice a slow down when the GIL is removed. Don't expect smiles. More advanced users may use concurrent code. If you use an external C library, chances are it has to be updated to work with a no-gil python version. This won't go swimmingly. It puts the burden on the maintainers of those libs, and they may not be able or willing to fix the problem. That will lead to unhappy faces. And then there will be the subtle bugs in complex systems. Python programmers are not really used to synchronicity problems, so they'll slap locks all over the place, leading to slow down or dead-locks. More unhappy faces. So I see the commitment to a single, no-GIL version as a serious threat to Python.

liuliu 3y ago

I am thinking about this. I think our distinction between IO-bounded tasks v.s. compute-bounded tasks helped design in some ways. However, as time goes by, the code you are dealing with can bounce between these two modes, and that is where all the limitations seem good for you start to be bad. That's what I suspect happening on the Python / scientific-computing community. They thought (and some people in this thread still think) that Python can be the driver to wait for compute-bounded task to finish (effectively, making the Python code to be IO-bounded / i.e. waiting for another language to finish). But over time, it changed. People write more code in the Python driver, and suddenly, the very optimized C code is no longer the bottleneck, the Python code is. We've seen this bouncing pattern enough times and really need to think through whether we want to have two different design patterns to deal with IO-bounded problem v.s. compute-bounded problem.