15 ms·

Ask HN: Who's building on Python NoGIL?

141 points by grandimam 2y ago

I am interested in knowing the things the community is building specifically taking into account the NoGIL aspect of python. Like is someone building frameworks around using Threads instead of Async?

tgma 2y ago
Started working on a no-gil gRPC Python implementation but super low priority. Has anyone started deploying nogil at scale in prod?
- grandimam 2y ago
  No, I am not personally aware of anyone using it prod.
- melodyogonna 2y ago
  I don't think it is ready to be used in prod. The feature is still experimental
shlomo_z 2y ago
I have the same question! I love Python and asynchronous stuff, and I do not know too much about threading. Is threading potentially better for IO bound tasks than async?
- MathMonkeyMan 2y ago
  Potentially, but probably not. The benefit of a parallel-enabled interpreter would be two CPU cores executing bytecode instructions at the same time in the same interpreter. So, you could have one python thread working on one set of data, and another python thread working on another set of data, and the two threads would not interfere with each other much or at all. Today, with the global interpreter lock, only one of those threads can be executing bytecode at a time.
  - shlomo_z 2y ago
    > Today, with the global interpreter lock, only one of those threads can be executing bytecode at a time. Yes, but Python now has a version without GIL, which prompted this post in the first place. So my question is: Now, if I use a version of Python 3.13 without GIL, can a threaded Flask app do better than an AIOHTTP server.
    - elashri 2y ago
      I think it will depend on which task we are talking about. For For CPU-bound tasks (i.e heavy computation, data processing), The No-GIL Flask with threading would likely perform better than AIOHTTP since it can truly parallelize computation across cores.Now for I/O-bound tasks (database queries, API calls or file operations) then AIOHTTP would still likely be more efficient due to its lower overhead and memory. So for your original question > Is threading potentially better for IO bound tasks than async? async will be better in general, potentially due to Async co-routines using far less memory than threads and being better under high concurrency. But that's all will depend on the details of implementation of the server.
- deleted 2y ago
  [deleted]
- Uptrenda 2y ago
  Quite honestly I'd tell you not to mix threads with asyncio. As you note: IO bound tasks aren't CPU hogs and there's little benefit to mixing it with threads. It will lead to unnecessary bugs, complexity, and problems with event loop management. Asyncio can run tens of thousands of tasks if its used properly. If you think something will block it you should check out "process pool executors." Note that its very tricky to share resources like sockets between processes so its kind of another reason to avoid stuff like this. I think Python 3.14 will have interpreter pools for even more concurrency options.
  - solidasparagus 2y ago
    > there's little benefit to mixing it with threads > If you think something will block it you should check out "process pool executors." Note that its very tricky to share resources like sockets between processes so its kind of another reason to avoid stuff like this. Isn't that the benefit of no-gil? The ability to run CPU-intensive operations without incurring the overhead and friction of multiprocessing? Now you can do multicore processing while also having shared memory
- KaiserPro 2y ago
  > Is threading potentially better for IO bound tasks than async? Its a pick your poison type of deal. I _personally_ hate async. Its a suprise goto, which I don't like. However in my testing asyncio performs the same as threading. (this was with socket based IO) If you loose the GIL, then threading _should_ be much faster, as it should scale to how many cores you have. But thats only if you have enough IO to saturate a core.
throwaway81523 2y ago
I've always used threads despite the GIL. I haven't tried NoGIL and am waiting to find out how many bugs it surfaces. I do get the impression that multi-threaded Python code is full of hazards that the GIL covers up. There will have to be locks inserted all over the place. CPython should have simply been retired as part of the 2 to 3 transition. It was great in its day, on 1-core machines with the constraints of that era. I have a feeling of tragedy that this didn't happen and now it can never be repaired. I probably wouldn't use Python for web projects these days. I haven't done anything in Elixir yet but it looks like about the best option. (I've used Erlang so I think I have a decent idea of what I'd be getting into with Elixir).
- tgma 2y ago
  In a strange way, Python being so bad at interpreting bytecodes and limited by GIL, plus being good at interfacing with C cheaply (unlike Go and Java,) induced a programming style that is extremely suited for data-parallel computing which is the way to efficiently scale compute in today's SIMD/GPU world. If you wanted to be efficient, you had to prepare your data ahead of time and hand it off. Any intermediate interaction with that data would ruin your performance. That's mostly how efficient Python libraries and ecosystem are built. Weakness may have turned into a strength.
  - grandimam 2y ago
    Can you elaborate more on the data-parallel computing part?
    - francocalvo 2y ago
      He's probably talking about libraries like PySpark or PyFlink which are used a lot
      - whoiscroberts 2y ago
        Pyflink seems promising, I love vanilla flink but as soon as you need to debug your pyflink job pyflink becomes a hurdle. That translation layer between Python and Java can be opaque.
bionhoward 2y ago
PyO3 0.23.0 was a big release I’ve been tinkering with extensively. Support for “free-threaded Python” is a headline feature, and I imagine NoGIL Python will be extremely nice for Rust interoperability, so there is definitely interest in that crate. Also could be huge for queueing data for GPUs, api servers, and bulk data fetching. For whatever reason (maybe post 2to3 PTSD), Python community seems not extremely eager to jump on latest versions of Python and it often takes a long time for popular libraries to support the latest and greatest, so I’d recommend patience and baby steps https://github.com/PyO3/pyo3/releases/tag/v0.23.0 https://github.com/PyO3/pyo3/releases/tag/v0.23.0
- hamandcheese 2y ago
  > For whatever reason (maybe post 2to3 PTSD), Python community seems not extremely eager to jump on latest versions of Python Well, you'd think after the 2 to 3 debacle, python might take backwards compatibility more seriously, but they don't. Follow semver, and stop breaking things on 3.x. If it's deprecated in 3.x, don't remove it until 4.
  - brianwawok 2y ago
    Meh. I’d rather we move faster and make the world better even if we have to update code a bit more. Never that hard.
    - KronisLV 2y ago
      I’ve seen woefully outdated enterprise projects that are unlikely to ever be updated to new runtimes (think Java 8 to anything newer, alongside the limitations JDK version support brings), due to nobody having the manpower to port up to a million lines over while hoping for it to be bug for bug compatible. Moving fast is really nice when you can get paid for shipping and have the project either be retired or someone else’s problem in a decade.
    - KaiserPro 2y ago
      Updates cost money, or time. Most places don't have the resources to keep updating everything for little benefit. (just look at node its terrible to support. )
0xDEADFED5 2y ago
Waiting for CFFI or pywin32 free-threaded support since I don't have time to work on CFFI myself
upghost 2y ago
This is going to be bananas for libpython-clj[1]. One of the biggest limiting factors right now is that you can't mix Java/Clojure concurrency with Python concurrency, you need to have a really clear separation of concurrency models. But with this, you will be able to freely mix Clojure and Python concurrency. Just from a compositional standpoint, Clojure atoms and core.async with Python functions will be fantastic. More practically, this will unlock a lot of performance gains with PyTorch and Tensorflow which historically we've had to lock to single threaded mode. Yay! [1]: https://github.com/clj-python/libpython-clj https://github.com/clj-python/libpython-clj
- deleted 2y ago
  [deleted]
- jwindle47 2y ago
  I’m here for it :) love the Clojure approach to symbiosis. Parens consume all the things!
  - bobxmax 2y ago
    Neat, guess we'll finally see if cross-language concurrency stops being such a pain.
carlsborg 2y ago
Its merged into CPython 3.13 but labeled as experimental. Single threaded cpu bound workloads suffer in benchmarks (vs i/o workloads) till they put back the specializing adaptive interpreter (PEP 659) in 3.14. Docs say a 40% hit now, target is 10% at next release. C extensions will have to be re-built and ported to support free threaded mode. Some interesting and impactful bits of open source work for those with a c++ multithreading background.
- santiagobasulto 2y ago
  May I ask which benchmarks you saw? I was looking for some reliable one and couldn’t find them.
  - throwaway10816 2y ago
    [flagged]
  - carlsborg 2y ago
    The official docs reference those numbers.
shiv_agarwal 2y ago
[flagged]
trollbridge 2y ago
In new code I try to use threads, but certain things like yield which rely on async are simply too common and useful to stop using. So far in production if I need to use multiple cores, I use multiple processes and design apps that way. The discipline this imposes does seem to result in better apps than I wrote in an environment like Java with tons of threads.
- bhouston 2y ago
  > In new code I try to use threads, but certain things like yield which rely on async are simply too common and useful to stop using. Huh? I python you have to choose either threads or asynchronous/await? Why not combine both of them? I am so confused. C# allows for both to be combined quite naturally. And JavaScript as well allows for workers with async/await.
  - elcomet 2y ago
    What do you mean? Async/await uses threads
    - bhouston 2y ago
      I was trying to make sense of this sentence of the original commenter: "In new code I try to use threads, but certain things like yield which rely on async are simply too common and useful to stop using." Seems to suggest that threads and yield/async were mutually exclusive. I misunderstood. I will move on.
      - te_chris 2y ago
        To use async await you have to execute the code in a run loop.
        nwatson 2y ago
        ... and you can push CPU-intensive stuff off to a thread-executor to avoid blocking the async run-loop
PaulHoule 2y ago
My RSS reader is written in async Python but I think it was a mistake and I built my image sorter to use gunicorn which means I have to run it inside WSL on Windows but actually it works really well. My "image sorter" is actually a lot of different things (it has a webcrawler in it, a tagging system, will probably take over the RSS reader's job someday) but it does an unholy mix of )(i) "things that require significant CPU" (like... math) and (ii) "things that require just a touch of CPU" like serving images. I found that (i) was blocking (ii) making the image sorter unusable. So far though that is processes and not threads. For the last few weeks for the hell of it I've been writing a small and very pedagogical chess playing program in Python (trying to outdo Lisp) and once I got the signs figured out in the alpha-beta negamax algorithm it can now beat my tester most of the time. (When I had the signs wrong it managed to find the fool's mate which is not too surprising in retrospect since it looks ahead enough plies) That was my major goal but I'd also like to try an MCTS chess program which is more of a leap into the unknown. Unlike alpha-beta MCTS can be almost trivially parallelized (run 16 threads of it for, say, 0.1 s, merge the trees, repeat, ...) and threads would be a convenient way to handle concurrency here although multiprocessing out to be good enough. So I am thinking about using a non-GIL Python but on the other hand I could also rewrite in Java and get a 50x or so speedup and great thread support. (Note the problem here is that unlike games where you fill up a board, chess doesn't really progress when you play out random moves. With random moves for instance you can't reproduce White's advantage at the beginning of the game and if your evaluation function can't see that you are doing really bad. I need a weak player for the playouts that plays well enough that it can take advantage of situations that real players can take advantage of at least some of the time. A really good move ordering function for an alpha-beta search might do the trick.
Reclaimer 2y ago
trying to make a framework for this called clusterops. https://github.com/The-Swarm-Corporation/ClusterOps https://github.com/The-Swarm-Corporation/ClusterOps

141 points by grandimam 2y ago

I am interested in knowing the things the community is building specifically taking into account the NoGIL aspect of python. Like is someone building frameworks around using Threads instead of Async?

tgma 2y ago

Started working on a no-gil gRPC Python implementation but super low priority. Has anyone started deploying nogil at scale in prod?

grandimam 2y ago
No, I am not personally aware of anyone using it prod.
melodyogonna 2y ago
I don't think it is ready to be used in prod. The feature is still experimental

shlomo_z 2y ago

I have the same question! I love Python and asynchronous stuff, and I do not know too much about threading. Is threading potentially better for IO bound tasks than async?

MathMonkeyMan 2y ago
Potentially, but probably not. The benefit of a parallel-enabled interpreter would be two CPU cores executing bytecode instructions at the same time in the same interpreter. So, you could have one python thread working on one set of data, and another python thread working on another set of data, and the two threads would not interfere with each other much or at all. Today, with the global interpreter lock, only one of those threads can be executing bytecode at a time.
- shlomo_z 2y ago
  > Today, with the global interpreter lock, only one of those threads can be executing bytecode at a time. Yes, but Python now has a version without GIL, which prompted this post in the first place. So my question is: Now, if I use a version of Python 3.13 without GIL, can a threaded Flask app do better than an AIOHTTP server.
  - elashri 2y ago
    I think it will depend on which task we are talking about. For For CPU-bound tasks (i.e heavy computation, data processing), The No-GIL Flask with threading would likely perform better than AIOHTTP since it can truly parallelize computation across cores.Now for I/O-bound tasks (database queries, API calls or file operations) then AIOHTTP would still likely be more efficient due to its lower overhead and memory. So for your original question > Is threading potentially better for IO bound tasks than async? async will be better in general, potentially due to Async co-routines using far less memory than threads and being better under high concurrency. But that's all will depend on the details of implementation of the server.
deleted 2y ago
[deleted]
Uptrenda 2y ago
Quite honestly I'd tell you not to mix threads with asyncio. As you note: IO bound tasks aren't CPU hogs and there's little benefit to mixing it with threads. It will lead to unnecessary bugs, complexity, and problems with event loop management. Asyncio can run tens of thousands of tasks if its used properly. If you think something will block it you should check out "process pool executors." Note that its very tricky to share resources like sockets between processes so its kind of another reason to avoid stuff like this. I think Python 3.14 will have interpreter pools for even more concurrency options.
- solidasparagus 2y ago
  > there's little benefit to mixing it with threads > If you think something will block it you should check out "process pool executors." Note that its very tricky to share resources like sockets between processes so its kind of another reason to avoid stuff like this. Isn't that the benefit of no-gil? The ability to run CPU-intensive operations without incurring the overhead and friction of multiprocessing? Now you can do multicore processing while also having shared memory
KaiserPro 2y ago
> Is threading potentially better for IO bound tasks than async? Its a pick your poison type of deal. I _personally_ hate async. Its a suprise goto, which I don't like. However in my testing asyncio performs the same as threading. (this was with socket based IO) If you loose the GIL, then threading _should_ be much faster, as it should scale to how many cores you have. But thats only if you have enough IO to saturate a core.

throwaway81523 2y ago

I've always used threads despite the GIL. I haven't tried NoGIL and am waiting to find out how many bugs it surfaces. I do get the impression that multi-threaded Python code is full of hazards that the GIL covers up. There will have to be locks inserted all over the place. CPython should have simply been retired as part of the 2 to 3 transition. It was great in its day, on 1-core machines with the constraints of that era. I have a feeling of tragedy that this didn't happen and now it can never be repaired. I probably wouldn't use Python for web projects these days. I haven't done anything in Elixir yet but it looks like about the best option. (I've used Erlang so I think I have a decent idea of what I'd be getting into with Elixir).

tgma 2y ago
In a strange way, Python being so bad at interpreting bytecodes and limited by GIL, plus being good at interfacing with C cheaply (unlike Go and Java,) induced a programming style that is extremely suited for data-parallel computing which is the way to efficiently scale compute in today's SIMD/GPU world. If you wanted to be efficient, you had to prepare your data ahead of time and hand it off. Any intermediate interaction with that data would ruin your performance. That's mostly how efficient Python libraries and ecosystem are built. Weakness may have turned into a strength.
- grandimam 2y ago
  Can you elaborate more on the data-parallel computing part?
  - francocalvo 2y ago
    He's probably talking about libraries like PySpark or PyFlink which are used a lot
    - whoiscroberts 2y ago
      Pyflink seems promising, I love vanilla flink but as soon as you need to debug your pyflink job pyflink becomes a hurdle. That translation layer between Python and Java can be opaque.

bionhoward 2y ago

PyO3 0.23.0 was a big release I’ve been tinkering with extensively. Support for “free-threaded Python” is a headline feature, and I imagine NoGIL Python will be extremely nice for Rust interoperability, so there is definitely interest in that crate. Also could be huge for queueing data for GPUs, api servers, and bulk data fetching. For whatever reason (maybe post 2to3 PTSD), Python community seems not extremely eager to jump on latest versions of Python and it often takes a long time for popular libraries to support the latest and greatest, so I’d recommend patience and baby steps https://github.com/PyO3/pyo3/releases/tag/v0.23.0 https://github.com/PyO3/pyo3/releases/tag/v0.23.0

hamandcheese 2y ago
> For whatever reason (maybe post 2to3 PTSD), Python community seems not extremely eager to jump on latest versions of Python Well, you'd think after the 2 to 3 debacle, python might take backwards compatibility more seriously, but they don't. Follow semver, and stop breaking things on 3.x. If it's deprecated in 3.x, don't remove it until 4.
- brianwawok 2y ago
  Meh. I’d rather we move faster and make the world better even if we have to update code a bit more. Never that hard.
  - KronisLV 2y ago
    I’ve seen woefully outdated enterprise projects that are unlikely to ever be updated to new runtimes (think Java 8 to anything newer, alongside the limitations JDK version support brings), due to nobody having the manpower to port up to a million lines over while hoping for it to be bug for bug compatible. Moving fast is really nice when you can get paid for shipping and have the project either be retired or someone else’s problem in a decade.
  - KaiserPro 2y ago
    Updates cost money, or time. Most places don't have the resources to keep updating everything for little benefit. (just look at node its terrible to support. )

0xDEADFED5 2y ago

Waiting for CFFI or pywin32 free-threaded support since I don't have time to work on CFFI myself

upghost 2y ago

This is going to be bananas for libpython-clj[1]. One of the biggest limiting factors right now is that you can't mix Java/Clojure concurrency with Python concurrency, you need to have a really clear separation of concurrency models. But with this, you will be able to freely mix Clojure and Python concurrency. Just from a compositional standpoint, Clojure atoms and core.async with Python functions will be fantastic. More practically, this will unlock a lot of performance gains with PyTorch and Tensorflow which historically we've had to lock to single threaded mode. Yay! [1]: https://github.com/clj-python/libpython-clj https://github.com/clj-python/libpython-clj

deleted 2y ago
[deleted]
jwindle47 2y ago
I’m here for it :) love the Clojure approach to symbiosis. Parens consume all the things!
- bobxmax 2y ago
  Neat, guess we'll finally see if cross-language concurrency stops being such a pain.

carlsborg 2y ago

Its merged into CPython 3.13 but labeled as experimental. Single threaded cpu bound workloads suffer in benchmarks (vs i/o workloads) till they put back the specializing adaptive interpreter (PEP 659) in 3.14. Docs say a 40% hit now, target is 10% at next release. C extensions will have to be re-built and ported to support free threaded mode. Some interesting and impactful bits of open source work for those with a c++ multithreading background.

santiagobasulto 2y ago
May I ask which benchmarks you saw? I was looking for some reliable one and couldn’t find them.
- throwaway10816 2y ago
  [flagged]
- carlsborg 2y ago
  The official docs reference those numbers.

shiv_agarwal 2y ago

[flagged]

trollbridge 2y ago

In new code I try to use threads, but certain things like yield which rely on async are simply too common and useful to stop using. So far in production if I need to use multiple cores, I use multiple processes and design apps that way. The discipline this imposes does seem to result in better apps than I wrote in an environment like Java with tons of threads.

bhouston 2y ago
> In new code I try to use threads, but certain things like yield which rely on async are simply too common and useful to stop using. Huh? I python you have to choose either threads or asynchronous/await? Why not combine both of them? I am so confused. C# allows for both to be combined quite naturally. And JavaScript as well allows for workers with async/await.
- elcomet 2y ago
  What do you mean? Async/await uses threads
  - bhouston 2y ago
    I was trying to make sense of this sentence of the original commenter: "In new code I try to use threads, but certain things like yield which rely on async are simply too common and useful to stop using." Seems to suggest that threads and yield/async were mutually exclusive. I misunderstood. I will move on.
    - te_chris 2y ago
      To use async await you have to execute the code in a run loop.
      - nwatson 2y ago
        ... and you can push CPU-intensive stuff off to a thread-executor to avoid blocking the async run-loop

PaulHoule 2y ago

My RSS reader is written in async Python but I think it was a mistake and I built my image sorter to use gunicorn which means I have to run it inside WSL on Windows but actually it works really well. My "image sorter" is actually a lot of different things (it has a webcrawler in it, a tagging system, will probably take over the RSS reader's job someday) but it does an unholy mix of )(i) "things that require significant CPU" (like... math) and (ii) "things that require just a touch of CPU" like serving images. I found that (i) was blocking (ii) making the image sorter unusable. So far though that is processes and not threads. For the last few weeks for the hell of it I've been writing a small and very pedagogical chess playing program in Python (trying to outdo Lisp) and once I got the signs figured out in the alpha-beta negamax algorithm it can now beat my tester most of the time. (When I had the signs wrong it managed to find the fool's mate which is not too surprising in retrospect since it looks ahead enough plies) That was my major goal but I'd also like to try an MCTS chess program which is more of a leap into the unknown. Unlike alpha-beta MCTS can be almost trivially parallelized (run 16 threads of it for, say, 0.1 s, merge the trees, repeat, ...) and threads would be a convenient way to handle concurrency here although multiprocessing out to be good enough. So I am thinking about using a non-GIL Python but on the other hand I could also rewrite in Java and get a 50x or so speedup and great thread support. (Note the problem here is that unlike games where you fill up a board, chess doesn't really progress when you play out random moves. With random moves for instance you can't reproduce White's advantage at the beginning of the game and if your evaluation function can't see that you are doing really bad. I need a weak player for the playouts that plays well enough that it can take advantage of situations that real players can take advantage of at least some of the time. A really good move ordering function for an alpha-beta search might do the trick.

Reclaimer 2y ago

trying to make a framework for this called clusterops. https://github.com/The-Swarm-Corporation/ClusterOps https://github.com/The-Swarm-Corporation/ClusterOps