5 ms·
One thing that plagues us about async Python in production is the propensity for a single synchronous call to bring down a server. The problem tends to be diffi
by weberc2 8y ago
One thing that plagues us about async Python in production is the propensity for a single synchronous call to bring down a server. The problem tends to be difficult to debug since it often manifests itself in timeouts for requests that aren't actually issuing the blocking request, but for which can't be serviced because the event loop is blocked nonetheless. It's also easy to add bugs in async code (e.g., forgetting to 'await' and getting back a promise instead of a response object). Of course, we could stick with sync Python, but that has lots of other problems (namely it's difficult to pack Python server processes onto hosts with passable efficiency).
Elixir and Go just skirt these problems altogether because all I/O is async under the hood (no need for "await" keywords), and both languages support efficient concurrency and parallelism. One process can fully utilize a host.
Of course, this doesn't speak to the quality of available web frameworks, which may or may not matter for your application.
- jashmatthews 8y agoNot a Python dev but I worked a lot with Ruby performance a lot for the past few years. Do you really need async IO? Even at Google scale[1] it's kinda a waste of time for web/job servers and only required for proxying etc. The overhead of threads is massively overstated [2] Ruby scales a long way using Puma or Phusion Passenger with one process per core (just like NodeJS) and adding threads until you hit CPU saturation. Python must have something similar? [1] https://www.slideshare.net/e456/tyma-paulmultithreaded1 https://www.slideshare.net/e456/tyma-paulmultithreaded1 [2] https://eli.thegreenplace.net/2018/measuring-context-switching-and-memory-overheads-for-linux-threads/ https://eli.thegreenplace.net/2018/measuring-context-switchi...
- deleted 8y ago[deleted]
- optimuspaul 8y ago> One thing that plagues us about async Python in production is the propensity for a single synchronous call to bring down a server. Probably need a better framework. I've never had such a problem come up unless I was trying to be clever and not use an established framework.
- weberc2 8y agoYou don’t understand the problem you’re criticizing. If you’re using an async web framework like aiohttp and something in the depths of one of your routes does sync IO, it will block your event loop. Nothing clever required. There isn’t a web framework that I’m aware of which magically makes all sync requests async.
- jashmatthews 8y ago> There isn’t a web framework that I’m aware of which magically makes all sync requests async. Waitress gets part of the way there. IIUC Request/response is done via asyncio but the actual web app part uses a thread so you can't accidentally block the whole thing.
- weberc2 8y agoInteresting. I wonder how it manages the threading with the GIL and all that.
- jashmatthews 8y agoThread per request works because the GIL is released during blocking IO. You only need one process per core and each process runs multiple threads. It saves a massive amount of RAM compared to trying to cram 50 (g)unicorns onto a server.
- optimuspaul 8y agoFunny that you insult me and then backup my vague comment with more specifics. My point is that you need to be aware of what is blocking all the way down, all frameworks you are using. It is possible, maybe not magical or trivial, to making everything async. Please don't assume you know what someone understands. Try asking questions when comments seem vague or stupid like mine did.