4 ms·
Gradually migrating Python code to asyncio
- uniqueuid 5y agoThe post is a good introduction, and in particular this sentence summarizes the core problem: "You can always call normal functions from coroutines, and you can always await coroutines from other coroutines, but scheduling coroutines from normal functions requires some care." Helper functions such as run_in_executor(), ensure_future() etc. are absolutely critical for navigating the added complexity. However! Im my experience, it is inevitable that one needs to re-consider the entire architecture of an app when adding async parts. The reason is that you will end up implicitly creating and removing choke points that may be surprising. For example, if you had a thread pool before, the concurrency was cumbersome but easy to reason about. If you're now sending coroutines to the loop, you might end up having a large number (thousands, millions) waiting for some shared resource in your code or consuming memory. Async python has a lot of advantages and I honestly like the style of programming it encourages, but I feel that many of the examples and tutorials are missing the point by showing map/reduce-style workflows which are trivial anyways. The hard parts are continuously running systems with loops and complex logic flow.