21 ms·
Python wsgi servers are generally based on processes and not threads since threads would trigger GIL considerations. While your process is making the single API
by BlackFly 7y ago
Python wsgi servers are generally based on processes and not threads since threads would trigger GIL considerations. While your process is making the single API call, the process is not serving anyone else, ergo you can only have so many simultaneous requests in flight as you have workers. Threads have no benefit over asyncio in python due to the GIL and have costs over ascyncio due to setting up the thread context with the OS.
With asyncio based implementations, while the coroutine is waiting for the response on the API call, the process's event loop can check on the coroutine which is handling the network socket and start processing another request. Ideally it should have no impact on the amount of time the original request takes to process but you can see how it might since when the API call resolves, the process may be working on the other request instead of continuing with the original coroutine.
Asynchronous code in general can allow you to extend your bandwidth past the number of concurrent executions your program can maintain (whether they be processes in python or threads in other languages) by doing additional work while it is waiting for something to complete. Ideally it should not cost much in terms of latency.