3 ms·

It's actually frustratingly complex. Right now I'm working on a "bridge" that receives http requests, and then needs to send that on an existing websocket conn

It's actually frustratingly complex. Right now I'm working on a "bridge" that receives http requests, and then needs to send that on an existing websocket connection to another system then wait for some responses, send some more on the ws etc and keep monitoring the ws. So the idea is basically to have multiple permanent websocket workers (one for each machine we need to speak with), that get tasks sent to them. Some added complexity is that each machine can only ever have one socket opened at a time. If I do it using asyncio, I end up with the issue of GIL making it so that I can't really do stuff concurrently, which sucks as the number of incoming requests and machines I need to bridge to increases. Or if I do it using multiple or sub-processes they can no longer communicate, I may risk having multiple processes trying to establish a connection, or processes dying I have to handle manually vs k8s doing it. Or I can do it with different deploys, and having a queue/db or whatever inbetween, drastically increasing complexity. So hard to combine different modes in the same app. While in java land I would just have fired up a webserver and made a thread+worker per ws connection and it would've scaled to thousands without issue.

mrweasel 4y ago
Then why not use Java? I love Python, it’s my language of choice for most things, but as you rightly point out, for some things it just invites complexity.
- rmbyrro 4y ago
  Same I was thinking. As it is today, Python just isn't a (relatively) good tool for concurrency. Use Go, Java. Clojure has some of the most interesting abstractions in this area, but the lisp syntax scares people away.
gspetr 4y ago
How do you test all of it?
uniqueuid 4y ago
Part of the problem - to me at least - is that in practice, concurrency with python is pretty much only heavy (slow) multiprocessing.queues. It's what multiprocessing.pool and the corresponding executors use. At the same time, my experience (along with that of many others) has been that these queues are not exactly robust - there seem to be many horrible edge cases where queues and/or processes get stuck and it's really hard to have a robust all-python app that recovers from e.g. a dead child (!). Ask ML frameworks, for example, how they handle spawning while holding GPU contexts and the like. It's a nightmare. So if we had a single robust, elegant and half-way fast abstraction for IPC, that would solve a lot of the existing issues. And it needs to be bi-directional of course.
js2 4y ago
It sounds like your application is I/O bound. That's an ideal use case for threads because the GIL is not held during I/O, so threads allow you to multiplex across lots of connections. I have a gunicorn server that's part of an internal processing pipeline. It receives an HTTP post from a Java client, grabs some files from S3, caches them to disk, does some processing on the POST and returns a reply. It spends most of its time blocked on I/O. I run it with 1000 threads per process no problem because that's what the Java side's thread-count is set to. I also have a related server that receivs HTTP POSTs from millions of clients all over the Internet via an AWS ALB. For each POST, it validates the data, splits the POST into three files that are each written to S3 and adds an event to SQS. For this application, I used Falcon and gevent which has me I/O bound. I tested it with threads but that ended up being CPU bound. I also tested with pypy but again, that ended up CPU bound. Gevent got me the best concurrency. Anyway, unless you've tested you can't just assume the GIL will be a problem. I've been writing Python for two decades using it in a variety of applications and I can count on one hand the number of times the GIL has been an issue.
liveoneggs 4y ago
> It's actually frustratingly complex. Kind of like every other part of the python ecosystem

mrweasel 4y ago

Then why not use Java? I love Python, it’s my language of choice for most things, but as you rightly point out, for some things it just invites complexity.

rmbyrro 4y ago
Same I was thinking. As it is today, Python just isn't a (relatively) good tool for concurrency. Use Go, Java. Clojure has some of the most interesting abstractions in this area, but the lisp syntax scares people away.

gspetr 4y ago

How do you test all of it?

uniqueuid 4y ago

Part of the problem - to me at least - is that in practice, concurrency with python is pretty much only heavy (slow) multiprocessing.queues. It's what multiprocessing.pool and the corresponding executors use. At the same time, my experience (along with that of many others) has been that these queues are not exactly robust - there seem to be many horrible edge cases where queues and/or processes get stuck and it's really hard to have a robust all-python app that recovers from e.g. a dead child (!). Ask ML frameworks, for example, how they handle spawning while holding GPU contexts and the like. It's a nightmare. So if we had a single robust, elegant and half-way fast abstraction for IPC, that would solve a lot of the existing issues. And it needs to be bi-directional of course.

js2 4y ago

It sounds like your application is I/O bound. That's an ideal use case for threads because the GIL is not held during I/O, so threads allow you to multiplex across lots of connections. I have a gunicorn server that's part of an internal processing pipeline. It receives an HTTP post from a Java client, grabs some files from S3, caches them to disk, does some processing on the POST and returns a reply. It spends most of its time blocked on I/O. I run it with 1000 threads per process no problem because that's what the Java side's thread-count is set to. I also have a related server that receivs HTTP POSTs from millions of clients all over the Internet via an AWS ALB. For each POST, it validates the data, splits the POST into three files that are each written to S3 and adds an event to SQS. For this application, I used Falcon and gevent which has me I/O bound. I tested it with threads but that ended up being CPU bound. I also tested with pypy but again, that ended up CPU bound. Gevent got me the best concurrency. Anyway, unless you've tested you can't just assume the GIL will be a problem. I've been writing Python for two decades using it in a variety of applications and I can count on one hand the number of times the GIL has been an issue.

liveoneggs 4y ago

> It's actually frustratingly complex. Kind of like every other part of the python ecosystem