5 ms·

The article gives a good summary of the quite complex landscape of concurrency in python. There's more to it, for example gil-free c-extensions, subprocesses an

The article gives a good summary of the quite complex landscape of concurrency in python. There's more to it, for example gil-free c-extensions, subprocesses and cross-machine (plus IPC) communication. But I'm particularly bothered by the fact that many articles and tutorials look at concurrency as if it's only about factoring primes or writing a web server with many (perhaps even idempotent) parallel requests. In reality, people will often want and need to combine multiple of these approaches, and then it gets VERY messy. I.e. try to combine a multiprocessing executor with multiple asyncio loops and boom you're in some very deep waters. One project that does this (async loops inside multiple processes) is proxy.py - very enlightening to read its code base [1]. But I really, really wish python would do more to provide simple and robust abstractions for these kinds of tasks. My dream would be a robust actor system similar to erlang, but we'll probably never get that. [1] https://github.com/abhinavsingh/proxy.py https://github.com/abhinavsingh/proxy.py

matsemann 4y ago
It's actually frustratingly complex. Right now I'm working on a "bridge" that receives http requests, and then needs to send that on an existing websocket connection to another system then wait for some responses, send some more on the ws etc and keep monitoring the ws. So the idea is basically to have multiple permanent websocket workers (one for each machine we need to speak with), that get tasks sent to them. Some added complexity is that each machine can only ever have one socket opened at a time. If I do it using asyncio, I end up with the issue of GIL making it so that I can't really do stuff concurrently, which sucks as the number of incoming requests and machines I need to bridge to increases. Or if I do it using multiple or sub-processes they can no longer communicate, I may risk having multiple processes trying to establish a connection, or processes dying I have to handle manually vs k8s doing it. Or I can do it with different deploys, and having a queue/db or whatever inbetween, drastically increasing complexity. So hard to combine different modes in the same app. While in java land I would just have fired up a webserver and made a thread+worker per ws connection and it would've scaled to thousands without issue.
- mrweasel 4y ago
  Then why not use Java? I love Python, it’s my language of choice for most things, but as you rightly point out, for some things it just invites complexity.
  - rmbyrro 4y ago
    Same I was thinking. As it is today, Python just isn't a (relatively) good tool for concurrency. Use Go, Java. Clojure has some of the most interesting abstractions in this area, but the lisp syntax scares people away.
- gspetr 4y ago
  How do you test all of it?
- uniqueuid 4y ago
  Part of the problem - to me at least - is that in practice, concurrency with python is pretty much only heavy (slow) multiprocessing.queues. It's what multiprocessing.pool and the corresponding executors use. At the same time, my experience (along with that of many others) has been that these queues are not exactly robust - there seem to be many horrible edge cases where queues and/or processes get stuck and it's really hard to have a robust all-python app that recovers from e.g. a dead child (!). Ask ML frameworks, for example, how they handle spawning while holding GPU contexts and the like. It's a nightmare. So if we had a single robust, elegant and half-way fast abstraction for IPC, that would solve a lot of the existing issues. And it needs to be bi-directional of course.
- js2 4y ago
  It sounds like your application is I/O bound. That's an ideal use case for threads because the GIL is not held during I/O, so threads allow you to multiplex across lots of connections. I have a gunicorn server that's part of an internal processing pipeline. It receives an HTTP post from a Java client, grabs some files from S3, caches them to disk, does some processing on the POST and returns a reply. It spends most of its time blocked on I/O. I run it with 1000 threads per process no problem because that's what the Java side's thread-count is set to. I also have a related server that receivs HTTP POSTs from millions of clients all over the Internet via an AWS ALB. For each POST, it validates the data, splits the POST into three files that are each written to S3 and adds an event to SQS. For this application, I used Falcon and gevent which has me I/O bound. I tested it with threads but that ended up being CPU bound. I also tested with pypy but again, that ended up CPU bound. Gevent got me the best concurrency. Anyway, unless you've tested you can't just assume the GIL will be a problem. I've been writing Python for two decades using it in a variety of applications and I can count on one hand the number of times the GIL has been an issue.
thenipper 4y ago
It adds some overhead but half the time I’m lazy and just use Ray: https://www.ray.io/ https://www.ray.io/ to handle my concurrency for me. That being I agree with you some better abstractions would be nice.
mountainriver 4y ago
There is an effort to bring true parallelism to python, we’ll see how it goes, and yeah it’s desperately needed
wpietri 4y ago
Yes, I was surprised how good it is! I especially liked how it started out with CPU-bound vs IO-bound. It's such a key distinction, but a lot of people just getting into concurrency will not have thought much about that. (And agreed on an actor system. It's definitely the approach that best fits my brain.)
pyuser583 4y ago
Python does have an actor framework: thespian. It’s not bad, but not Erlang either.

matsemann 4y ago

It's actually frustratingly complex. Right now I'm working on a "bridge" that receives http requests, and then needs to send that on an existing websocket connection to another system then wait for some responses, send some more on the ws etc and keep monitoring the ws. So the idea is basically to have multiple permanent websocket workers (one for each machine we need to speak with), that get tasks sent to them. Some added complexity is that each machine can only ever have one socket opened at a time. If I do it using asyncio, I end up with the issue of GIL making it so that I can't really do stuff concurrently, which sucks as the number of incoming requests and machines I need to bridge to increases. Or if I do it using multiple or sub-processes they can no longer communicate, I may risk having multiple processes trying to establish a connection, or processes dying I have to handle manually vs k8s doing it. Or I can do it with different deploys, and having a queue/db or whatever inbetween, drastically increasing complexity. So hard to combine different modes in the same app. While in java land I would just have fired up a webserver and made a thread+worker per ws connection and it would've scaled to thousands without issue.

mrweasel 4y ago
Then why not use Java? I love Python, it’s my language of choice for most things, but as you rightly point out, for some things it just invites complexity.
- rmbyrro 4y ago
  Same I was thinking. As it is today, Python just isn't a (relatively) good tool for concurrency. Use Go, Java. Clojure has some of the most interesting abstractions in this area, but the lisp syntax scares people away.
gspetr 4y ago
How do you test all of it?
uniqueuid 4y ago
Part of the problem - to me at least - is that in practice, concurrency with python is pretty much only heavy (slow) multiprocessing.queues. It's what multiprocessing.pool and the corresponding executors use. At the same time, my experience (along with that of many others) has been that these queues are not exactly robust - there seem to be many horrible edge cases where queues and/or processes get stuck and it's really hard to have a robust all-python app that recovers from e.g. a dead child (!). Ask ML frameworks, for example, how they handle spawning while holding GPU contexts and the like. It's a nightmare. So if we had a single robust, elegant and half-way fast abstraction for IPC, that would solve a lot of the existing issues. And it needs to be bi-directional of course.
js2 4y ago
It sounds like your application is I/O bound. That's an ideal use case for threads because the GIL is not held during I/O, so threads allow you to multiplex across lots of connections. I have a gunicorn server that's part of an internal processing pipeline. It receives an HTTP post from a Java client, grabs some files from S3, caches them to disk, does some processing on the POST and returns a reply. It spends most of its time blocked on I/O. I run it with 1000 threads per process no problem because that's what the Java side's thread-count is set to. I also have a related server that receivs HTTP POSTs from millions of clients all over the Internet via an AWS ALB. For each POST, it validates the data, splits the POST into three files that are each written to S3 and adds an event to SQS. For this application, I used Falcon and gevent which has me I/O bound. I tested it with threads but that ended up being CPU bound. I also tested with pypy but again, that ended up CPU bound. Gevent got me the best concurrency. Anyway, unless you've tested you can't just assume the GIL will be a problem. I've been writing Python for two decades using it in a variety of applications and I can count on one hand the number of times the GIL has been an issue.

thenipper 4y ago

It adds some overhead but half the time I’m lazy and just use Ray: https://www.ray.io/ https://www.ray.io/ to handle my concurrency for me. That being I agree with you some better abstractions would be nice.

mountainriver 4y ago

There is an effort to bring true parallelism to python, we’ll see how it goes, and yeah it’s desperately needed

wpietri 4y ago

Yes, I was surprised how good it is! I especially liked how it started out with CPU-bound vs IO-bound. It's such a key distinction, but a lot of people just getting into concurrency will not have thought much about that. (And agreed on an actor system. It's definitely the approach that best fits my brain.)

pyuser583 4y ago

Python does have an actor framework: thespian. It’s not bad, but not Erlang either.