3 ms·
It seems to me that all that this boils down to is the fact that await/async makes back pressure something you need to deal with explicitly. Having the default
by Doxin 7y ago
It seems to me that all that this boils down to is the fact that await/async makes back pressure something you need to deal with explicitly. Having the default be buffering isn't ideal but since each application will have their own idea of what to do with backpressure it'd be hard to figure out a different default that works better.
In any case all this can be solved without major rewrites by making sure every awaitable is awaited at some point in the future. Instead of doing this:
while connection.accept():
handle_connection()
you might do something like this:
connection_pool = []
while connection.accept():
connection_pool.append(handle_connection())
if len(connection_pool)>=MAX_CONNECTIONS:
await wait_any(connection_pool)
And there you go. Anytime there's more than MAX_CONNECTIONS the program stops accepting new connections, providing back pressure. It's more code but it's also defining exactly HOW to provide back pressure. Your specific use case might warrant providing back pressure not based on connection count but cpu usage or average response times. You might want to have a single global maximum connection count instead of one per thread. All of these aren't much more complex than what I've shown above as long as you keep the cardinal rule in mind: any awaitable MUST be awaited.
And in fact in python -- and other programming languages too I bet -- you get a warning on exit if there are any awaitables that never got awaited. In my opinion that should be an error. I can't think of a single scenario where it'd be proper form to never await an awaitable. You might await immediately, sometime in the future, or at the end of the program. But you never don't await at all.
Threading isn't any easer, or harder. If you spawn a thread for each connection you run into the same issue as await unless you do something about it. If you use a pool of threads you get backpressure for free, but the same goes for a pool of awaitables!
tl;dr: async/await has the exact same problems as threads, the tooling around async/await is just less mature. Rewriting async code to provide back pressure is near trivial.