4 ms·
Non-blocking HTTP request is the bread and butter use case for asyncio. Most JS projects are doing something like this, and they don't need to manage threads fo
by hot_gril 2y ago
Non-blocking HTTP request is the bread and butter use case for asyncio. Most JS projects are doing something like this, and they don't need to manage threads for it. You want to manage your own thread pool for this, or are you going to spawn and kill a thread every time you make a request?
- BugsJustFindMe 2y ago> Non-blocking HTTP request is the bread and butter use case for asyncio And the amount of contorting that has to be done for it in Python would be hilarious if it weren't so sad. > Most JS projects I don't know what JavaScript does, but I do know that Python is not JavaScript. > You want to manage your own thread pool for this... In Python, concurrent futures' ThreadPoolExecutor is actually nice to use and doesn't require rewriting existing worker code. It's already done, has a clean interface, and was part of the standard library before asyncio was.
- stavros 2y agoI feel you. I know asyncio is "the future", but I usually just want to write a background task, and really hate all the gymnastics I have to do with the color of my functions.
- BugsJustFindMe 2y agoI feel like "asyncio is the future" was invented by the same people who think it's totally normal to switch to a new javascript web framework every 6 months.
- hot_gril 2y agoJS had an event loop since the start. It's an old concept that Python seems to have lifted, as did Rust. I used Python for a decade and never really liked the way it did threads.
- zo1 2y agoPython's reactor pattern, or event loop as you call it, started with the "Twisted" framework or library. And that was first published in 2003. That's a full 6 years before Node.js was released which I assume was the first time anything event-loopy started happening in the JS world. I forgot to mention that it came into prominence in the Python world through the Tornado http server library that did the same thing. Slowly over time, more and more language features were added to give native or first-class-citizen support to what a lot of people were doing behind the scenes (in sometimes very contrived abuses of generator functions).
- hot_gril 2y agoYeah, this pattern is old. But JS is the only common language (at least today) that went so all-in with it.
- stavros 2y agoI agree, I find Go's way much easier to reason about. It's all just functions.
- hot_gril 2y agoThreadPoolExecutor is the most similar thing to asyncio: It hands out promises, and when you call .result(), it's the same as await. JS even made its own promises implicitly compatible with async/await. I'm mentioning what JS does because you're describing a very common JS use case, and Python isn't all that different. If you have async stuff happening all over the place, what do you use, a global ThreadPoolExecutor? It's not bad, but a bit more cumbersome and probably less efficient. You're running multiple OS threads that are locking, vs a single-threaded event loop. Gets worse the more long-running blocking calls there are. Also, I was originally asking about free threads. GIL isn't a problem if you're just waiting on I/O. If you want to compute on multiple cores at once, there's multiprocessing, or more likely you're using stuff like numpy that uses C threads anyway.
- BugsJustFindMe 2y ago> Python isn't all that different Again, Python's implementation of asyncio does not allow you to background worker code without explicitly altering that worker code to be aware of asyncio. Threads do. They just don't occupy the same space. > Also, I was originally asking about free threads...there's multiprocessing Eh, the obvious reason to not want to use separate processes is a desire for some kind of shared state without the cost or burden of IPC. The fact that you suggested multiprocessing.Pool instead of concurrent_futures.ProcessPoolExecutor and asked about manual pool management feels like it tells me a little bit about where your head is at here wrt Python.
- hot_gril 2y agoBasically true in JS too. You're not supposed to do blocking calls in async code. You also can't "await" an async call inside a non-async func, though you could fire-and-forget it. Right, but how often does a Python program have complex shared state across threads, rather than some simple fan-out-fan-in, and also need to take advantage of multiple cores?
- nilamo 2y agoThe primary thing that tripped me up about async/await, specifically only in Python, is that the called function does not begin running until you await it. Before that moment, it's just an unstarted generator. To make background jobs, I've used the class-based version to start a thread, then the magic method that's called on await simply joins the thread. Which is a lot of boilerplate to get a little closer to how async works in (at least) js and c#.