12 ms·

I find the current focus on GIL-less Python really strange. The Faster cPython team set an ambitious goal of increasing cPython performance 50% with each releas

I find the current focus on GIL-less Python really strange. The Faster cPython team set an ambitious goal of increasing cPython performance 50% with each release. 3.11 contained some real improvements, but nowhere near 50%. And for much of our testing, 3.12 is either flat or slower. True multi-threading would be great, but I would much rather have improved single threaded performance first. I of course respect that our needs may not represent everyone else's, and we are grateful for all the work that has gone into making Python a great language. But what am I missing?

benrutter 3y ago
My understanding is they are two seperate projects within cpython and don't necessarily have the same people working on them. I agree with you that if it was one or the other, most use cases are gonna be better met with straight up faster single threaded code. But why not have both?
- slashdev 3y ago
  Both is not really on the table. Getting rid of the Gil will slow down single threaded code.
  - rowanG077 3y ago
    Why would removing the GIL slow down single threaded code?
    - edgyquant 3y ago
      I believe without a GIL objects need to be thread safe which comes at a price
    - viraptor 3y ago
      You still need to run the same memory management/accounting for all the objects as when you run multithreaded, because at any point a new thread may be started. And it takes more time to make all MT access safe then to prevent other threads from running.
    - Austizzle 3y ago
      As far as I know, in order to make everything thread safe in the same way the gil did, they need to add locks in a lot of places to make sure that no two objects can be modified at the same time. Adding those locks will slow down execution
    - Jtsummers 3y ago
      No-GIL forces the addition of locks and other concurrency controls into the execution path of single-threaded code that's not there now. It's the same kind of hit you get going from thread-unsafe code (hopefully intended for single-threaded execution) to thread-safe in any other project (since, well, it's the same; the GIL means they could write code with single-threaded assumptions). Checking and taking locks is not a free action.
      - rowanG077 3y ago
        I thought the point of No-GIL was to force users of python to provide proper locking.
        Jtsummers 3y ago
        To achieve no-GIL, libraries and extensions along with the underlying language implementation will need to add in the proper locking. For users, it'll be mixed. Users writing single-threaded code shouldn't have to change a single line of code, but they'll see (sans the concurrent efforts to speed up the underlying implementation) slower performance due to everything done to achieve no-GIL (the actual net effect will be a performance boost due to that concurrent effort over time). If they're writing multi-threaded code, then they should be writing it to be threadsafe now, not assuming that the GIL will protect them (because it doesn't guarantee it now anyways). So nothing should actually change for most users of Python if they're writing correct code today.
        kortex 3y ago
        Python's reference counting garbage collector would also need concurrency controls.
  - Jtsummers 3y ago
    The last numbers I saw put the performance penalty of no-GIL at under 10%, so both are still on the table since Python has a lot of single-threaded performance left to recover. You can get both a much faster-than-current Python in single-threaded and real threads.
    - alfalfasprout 3y ago
      Yep and when the faster cpython team gets to implementing advanced JITing (which GVR has said is planned) that'll be huge.
    - slashdev 3y ago
      Obviously you can recover it, but they will Be a performance hit. Single threaded without GIL will always be slower than with it. I’ll be surprised if they can keep it to only 10%. It will depend on the workload and there will Be some pathological cases.
  - alfalfasprout 3y ago
    > Both is not really on the table Incorrect. > Getting rid of the Gil will slow down single threaded code Correct. But that's before fastercpython improves single threaded performance closing that gap if not reversing it.
softwaredoug 3y ago
The computationally expensive parts of “Python” are done in libraries like numpy, tensorflow, etc. Python makes it really handy to play with low level abstractions in a higher level language. So yeah I’ve never stressed much about the GIL as a long time pythons dev
- jhj 3y ago
  Yes but coordinating the computationally expensive parts usually needs to be done across multiple CPU threads. Typically one should feed each GPU from a different CPU thread, even in native C/C++ land, as in many cases the kernels being run on the GPU may not be entirely heavy compute but instead on the order of the kernel launch overhead from the host (1-4 us); not every kernel run would be orders of magnitude longer runtime than the overheads involved. Data loaders and other non-computationally expensive parts (I/O latency or throughput bound things) need to be done as well, which make sense to multi-thread as well.
- deleted 3y ago
  [deleted]
- mardifoufs 3y ago
  > The computationally expensive parts of “Python” are done in libraries like numpy, tensorflow, etc. Python makes it really handy to play with low level abstractions in a higher level language. > So yeah I’ve never stressed much about the GIL as a long time pythons dev You almost always have to use python objects at some point or another, even with very high perf libraries. Unless you are using python as nothing else than glue code, and loading and preprocessing the data entirely outside of your python code. At some point (and I'm not talking about extreme scales here, just feeding a single datqaloader can be enough) you hit a very hard performance bottleneck. Sure you can just use more external code at that point but I think the intention is to make python at least more suitable for the non compute intensive stuff. Like, it is already suitable right now but it's not getting better, while everything around it is (I'm not talking about other languages, what I mean is that the tools and libraries are getting better so without improvement to the language the bottleneck will just get worse)
- fulafel 3y ago
bratao 3y ago
In my opinion, Python needs to have an urgent answer to the use of multiple cores. AMD just launched a CPU with 96 cores. Today the use on multiple cores is through the use multiprocess, which has many limitations. I understand that the multiple interpreters could come with something like Goroutines, but I still like the real multi thread option more.
- brianwawok 3y ago
  My server can easily fill all 96 running standard Python under gunicorn.
  - NegativeLatency 3y ago
    and if you use and async framework it's even more capable
  - iterateoften 3y ago
    gunicorn uses multiple processes
    - aeyes 3y ago
      You can still share most of the memory if you load everything before the fork so why is having multiple processes a problem?
      - rtpg 3y ago
        Ive found in practice that memory sharing doesn’t work as much as one would want, and multiple processes cost hundreds of megs (annoying!)
      - skrause 3y ago
        One operating system I have to deploy on (Windows) has neither fork() nor gunicorn.
      - dontlaugh 3y ago
euiq 3y ago
I think you're missing that your needs are different from those of the people quoted in PEP 703: <https://peps.python.org/pep-0703/#motivation https://peps.python.org/pep-0703/#motivation>
- qeternity 3y ago
  I literally said I understand that my needs are different. I get why the PEP exists. I don't get why it's receiving such priority.
rvanlaar 3y ago
There's a GIL presentation [1] from 2010 which shows Python 3.2 on a 1 CPU machine was faster than on a dual core. In all, there might be reason to expect GILless python to be faster single core in certain scenario's. [1] https://youtu.be/Obt-vMVdM8s?t=2047 https://youtu.be/Obt-vMVdM8s?t=2047
alfalfasprout 3y ago
They're completely different goals. Yeah in theory multithreaded python lets you speed up certain programs... but the way in which it does so is important. With nogil python you can effectively have multiple threads that eg; call out to C code while having shared state accessible as python objects. This is pretty key for ML-- in fact this current incarnation of the PEP came from the PyTorch team. Single threaded performance is important too but there have already been lots of decent workarounds for critical sections (eg; numba, Cython, and now things like Mojo). The ordering is important too-- a lot of the faster cpython work would be thrown away completely if nogil came about. So the teams have had to coordinate. In the ideal world that means both nogil mode + improvements to single threaded performance (Guido even hinted sophisticated JITing is being considered).
deleted 3y ago
[deleted]

benrutter 3y ago

My understanding is they are two seperate projects within cpython and don't necessarily have the same people working on them. I agree with you that if it was one or the other, most use cases are gonna be better met with straight up faster single threaded code. But why not have both?

slashdev 3y ago
Both is not really on the table. Getting rid of the Gil will slow down single threaded code.
- rowanG077 3y ago
  Why would removing the GIL slow down single threaded code?
  - edgyquant 3y ago
    I believe without a GIL objects need to be thread safe which comes at a price
  - viraptor 3y ago
    You still need to run the same memory management/accounting for all the objects as when you run multithreaded, because at any point a new thread may be started. And it takes more time to make all MT access safe then to prevent other threads from running.
  - Austizzle 3y ago
    As far as I know, in order to make everything thread safe in the same way the gil did, they need to add locks in a lot of places to make sure that no two objects can be modified at the same time. Adding those locks will slow down execution
  - Jtsummers 3y ago
    No-GIL forces the addition of locks and other concurrency controls into the execution path of single-threaded code that's not there now. It's the same kind of hit you get going from thread-unsafe code (hopefully intended for single-threaded execution) to thread-safe in any other project (since, well, it's the same; the GIL means they could write code with single-threaded assumptions). Checking and taking locks is not a free action.
    - rowanG077 3y ago
      I thought the point of No-GIL was to force users of python to provide proper locking.
      - Jtsummers 3y ago
        To achieve no-GIL, libraries and extensions along with the underlying language implementation will need to add in the proper locking. For users, it'll be mixed. Users writing single-threaded code shouldn't have to change a single line of code, but they'll see (sans the concurrent efforts to speed up the underlying implementation) slower performance due to everything done to achieve no-GIL (the actual net effect will be a performance boost due to that concurrent effort over time). If they're writing multi-threaded code, then they should be writing it to be threadsafe now, not assuming that the GIL will protect them (because it doesn't guarantee it now anyways). So nothing should actually change for most users of Python if they're writing correct code today.
      - kortex 3y ago
        Python's reference counting garbage collector would also need concurrency controls.
- Jtsummers 3y ago
  The last numbers I saw put the performance penalty of no-GIL at under 10%, so both are still on the table since Python has a lot of single-threaded performance left to recover. You can get both a much faster-than-current Python in single-threaded and real threads.
  - alfalfasprout 3y ago
    Yep and when the faster cpython team gets to implementing advanced JITing (which GVR has said is planned) that'll be huge.
  - slashdev 3y ago
    Obviously you can recover it, but they will Be a performance hit. Single threaded without GIL will always be slower than with it. I’ll be surprised if they can keep it to only 10%. It will depend on the workload and there will Be some pathological cases.
- alfalfasprout 3y ago
  > Both is not really on the table Incorrect. > Getting rid of the Gil will slow down single threaded code Correct. But that's before fastercpython improves single threaded performance closing that gap if not reversing it.

softwaredoug 3y ago

The computationally expensive parts of “Python” are done in libraries like numpy, tensorflow, etc. Python makes it really handy to play with low level abstractions in a higher level language. So yeah I’ve never stressed much about the GIL as a long time pythons dev

jhj 3y ago
Yes but coordinating the computationally expensive parts usually needs to be done across multiple CPU threads. Typically one should feed each GPU from a different CPU thread, even in native C/C++ land, as in many cases the kernels being run on the GPU may not be entirely heavy compute but instead on the order of the kernel launch overhead from the host (1-4 us); not every kernel run would be orders of magnitude longer runtime than the overheads involved. Data loaders and other non-computationally expensive parts (I/O latency or throughput bound things) need to be done as well, which make sense to multi-thread as well.
deleted 3y ago
[deleted]
mardifoufs 3y ago
> The computationally expensive parts of “Python” are done in libraries like numpy, tensorflow, etc. Python makes it really handy to play with low level abstractions in a higher level language. > So yeah I’ve never stressed much about the GIL as a long time pythons dev You almost always have to use python objects at some point or another, even with very high perf libraries. Unless you are using python as nothing else than glue code, and loading and preprocessing the data entirely outside of your python code. At some point (and I'm not talking about extreme scales here, just feeding a single datqaloader can be enough) you hit a very hard performance bottleneck. Sure you can just use more external code at that point but I think the intention is to make python at least more suitable for the non compute intensive stuff. Like, it is already suitable right now but it's not getting better, while everything around it is (I'm not talking about other languages, what I mean is that the tools and libraries are getting better so without improvement to the language the bottleneck will just get worse)
fulafel 3y ago

bratao 3y ago

In my opinion, Python needs to have an urgent answer to the use of multiple cores. AMD just launched a CPU with 96 cores. Today the use on multiple cores is through the use multiprocess, which has many limitations. I understand that the multiple interpreters could come with something like Goroutines, but I still like the real multi thread option more.

brianwawok 3y ago
My server can easily fill all 96 running standard Python under gunicorn.
- NegativeLatency 3y ago
  and if you use and async framework it's even more capable
- iterateoften 3y ago
  gunicorn uses multiple processes
  - aeyes 3y ago
    You can still share most of the memory if you load everything before the fork so why is having multiple processes a problem?
    - rtpg 3y ago
      Ive found in practice that memory sharing doesn’t work as much as one would want, and multiple processes cost hundreds of megs (annoying!)
    - skrause 3y ago
      One operating system I have to deploy on (Windows) has neither fork() nor gunicorn.
    - dontlaugh 3y ago

euiq 3y ago

I think you're missing that your needs are different from those of the people quoted in PEP 703: <https://peps.python.org/pep-0703/#motivation https://peps.python.org/pep-0703/#motivation>

qeternity 3y ago
I literally said I understand that my needs are different. I get why the PEP exists. I don't get why it's receiving such priority.

rvanlaar 3y ago

There's a GIL presentation [1] from 2010 which shows Python 3.2 on a 1 CPU machine was faster than on a dual core. In all, there might be reason to expect GILless python to be faster single core in certain scenario's. [1] https://youtu.be/Obt-vMVdM8s?t=2047 https://youtu.be/Obt-vMVdM8s?t=2047

alfalfasprout 3y ago

They're completely different goals. Yeah in theory multithreaded python lets you speed up certain programs... but the way in which it does so is important. With nogil python you can effectively have multiple threads that eg; call out to C code while having shared state accessible as python objects. This is pretty key for ML-- in fact this current incarnation of the PEP came from the PyTorch team. Single threaded performance is important too but there have already been lots of decent workarounds for critical sections (eg; numba, Cython, and now things like Mojo). The ordering is important too-- a lot of the faster cpython work would be thrown away completely if nogil came about. So the teams have had to coordinate. In the ideal world that means both nogil mode + improvements to single threaded performance (Guido even hinted sophisticated JITing is being considered).

deleted 3y ago

[deleted]