10 ms·

I wish Gunicorn with Gevent had won the “async vs. coroutine Python” battle. The Developer UX is so much better with coroutines and just as scalable as asyncio

I wish Gunicorn with Gevent had won the “async vs. coroutine Python” battle. The Developer UX is so much better with coroutines and just as scalable as asyncio (async/await). Many people misunderstand asyncio thinking it will make your code faster, when what it does do is make it more scalable (if you have allot of IO for example). But at the expense of developer UX. Mixing async and sync code within Python is a nightmare. I am completely unconvinced by the move to asyncio everywhere in Python. I have been using Gunicorn with Gevent for nearly 10 years and never had a problem with it. Languages like JavaScript that have been async from the start work well as you (mostly) only have an async api. https://docs.gunicorn.org/en/latest/design.html#async-workers https://docs.gunicorn.org/en/latest/design.html#async-worker... http://www.gevent.org/ http://www.gevent.org/

sandGorgon 5y ago
it kinda has - we use gunicorn with fastapi using the uvicorn asgi workers https://fastapi.tiangolo.com/deployment/server-workers/ https://fastapi.tiangolo.com/deployment/server-workers/
- qff 5y ago
  Green threads (gunicorn/gevent) allows you to write Python the way you always have and gain many of the benefits of async. Uvicorn and ASGI do not. For those you need to handle async explictly.
  - sandGorgon 5y ago
    i hear you - however the framework captures most of the abstraction entirely. In addition, most of the bottlenecks are actually i/o (db queries, etc) where its entirely worth doing it explicitly versus magic. in a general programming language sense...ur right. But from the POV of what gunicorn/fastapi are actually used for - web frameworks - its actually quite good.
- ramraj07 5y ago
  The list of terms in that sentence: Gunicorn Fastapi Uvicorn Asgi I suppose also nginx docker?
  - dgellow 5y ago
    Edit: the grandparent comment has now been updated to clarify, so my comment here is not relevant anymore. ASGI is similar to WSGI but for async oriented web servers. The person you respond to is saying that they use Uvicorn via the ASGI interface. See https://asgi.readthedocs.io/en/latest/ https://asgi.readthedocs.io/en/latest/.
  - orf 5y ago
    The list of letters in that sentence: a c d e f g h i k l m n o p r s t u v x I suppose also newlines?
- floatrock 5y ago
  I've always been a bit confused by this deployment model. gunicorn bills itself as "Gunicorn 'Green Unicorn' is a Python WSGI HTTP Server for UNIX. It's a pre-fork worker model." uvicorn's one-liner "Uvicorn is a lightning-fast ASGI server implementation, using uvloop and httptools." So it seems redundant at first to be running both, but what the gunicorn+uvicorn-worker-process pattern really does is throw away the "WSGI HTTP Server" part of gunicorn and just use the "pre-fork worker" process management part, with the underlying processes actually being ASGI. If you run in a different kind of process-management environment like k8s, does running gunicorn+uvicorn give you anything else?
  - sandGorgon 5y ago
    Gunicorn fundamentally runs everything as a worker process. You have sync, gthread, gevent, evenlet, tornado (tornado was also a popular server). UvicornWorker is another type of gunicorn worker... incidentally authored by the same guys who built uvicorn. Nothing strange at all. But ur question is genuine - but it is orthogonal to the uvicorn part of this answer. Fundamentally, in a k8s model...why bother with gunicorn? The answer: I don't know. I do love the control that gunicorn gives me...so I'd probably use it even if I was using a single worker. However there is a complex answer with Threads vs cores vs pods that I'm unqualified to give.
korijn 5y ago
I have been using the async/await syntax in C# for more than ten years and never had a problem with it. What is your point?
- samwillis 5y ago
  Most of Pythons standard library is sync, the minute you want to use a sync api from within async/await code it becomes a nightmare to manage. If you are only using async library’s then async/await works perfectly.
  - philote 5y ago
    It's straightforward to call sync functions from async ones. Are you referring to the fact that those sync functions could block and cause the async code to perform worse? I admit it's a few more lines of code to write a script that runs async code. I've worked on several projects recently where async was used throughout and haven't had issues. There are many async libraries now, like ones to utilize GCP services, make HTTP calls, connect to Redis, etc.
    - samwillis 5y ago
      Yes, I mean blocking sync calls from async code.
  - omnicognate 5y ago
    Sounds like you're referring to the "coloured functions" "problem", but it's the other way round. Calling a sync API from an async function is no different to calling it from a sync one. Calling an async function from a sync one requires that you make the caller async. Personally, I like that. An asynchronous function is different to a synchronous one, and if you call an asynchronous function the caller is asynchronous. The "colouring" just reflects the realities of the problem domain. I'm not keen on libraries that monkeypatch the world to hide that fact, as gevent does. However, some people prefer those libraries. Which is fine. They both exist, use what you like. There's no need to have wars about it.
BlueTemplar 5y ago
Wait, but JavaScript only had async/await added recently too, with ES6 ? (Or was that "just" about an easier way to use these features?)
- samwillis 5y ago
  Async/await in JavaScript is, a very welcome, syntactic sugar over a callback api. The point is that almost all JavaScript library’s have always been async, traditionally with callback based apis (now async/await). You never mixed async and sync code.
  - renke1 5y ago
    That's reason why I think the push to follow event-driven (reactive) approaches in ecosystems like Java is doomed to fail (okay, that may be a bit harsh) because there is lot of existing blocking code. It will take a long time until all of the old libraries catch up. It kind of splits the whole ecosystem into half. Specifically in Java I think it's fine we are relying on blocking APIs (and threads). For most stuff it scales properly, doesn't feel unnatural and is easier to understand. I say this as a big fan of JavaScript, I just don't see the approach being introduced after the fact as a good idea.
akvadrako 5y ago
Even better is to just use threads. Linux can handle a million threads just fine, so even if 99.99% of your clients are idle, threading won't be the bottleneck.
- NovemberWhiskey 5y ago
  It's not the time when most of your threads are idle that's the problem; it's when you've accepted a million incoming socket connections, given one connection to each thread and now you're thrashing wildly trying to get any useful work done as your clients timeout, reconnect and repeat.
  - otabdeveloper4 5y ago
    A green thread implementation will thrash just as much in your hypothetical scenario. The only advantage of green threading is the capability to do problem-specific optimized scheduling. If you're not doing this then you don't need green threads.
    - akvadrako 5y ago
      You are also optimize the scheduling for real threads, for example by increasing sched_latency_ns and sched_min_granularity_ns.
    - pm90 5y ago
      > The only advantage of green threading is the capability to do problem-specific optimized scheduling. Isn’t the main advantage simply that you avoid making an absurd number of costly system calls?
      - camnora 5y ago
        This article does a pretty good job of explaining how green events work. It seems that each of the threads will attempt to accept any/all incoming requests. https://rachelbythebay.com/w/2020/03/07/costly/ https://rachelbythebay.com/w/2020/03/07/costly/
BlueTemplar 5y ago
Wait, but JavaScript only had async/await added recently too, with ES6 ? (Or was that "just" about an easier way to use these features?)! EDIT: Argh, double posting, did I hit "back" by mistake ?
- sammorrowdrums 5y ago
  They are effectively sugar on JS promises which are effectively sugar around async event callbacks. This is why node rose so quickly in popularity as a server. The async IO handling could be leveraged with callbacks to scale low number of processes to handle large number of requests. Plus easy to learn and lots of people knew some JS. JS always had an event loop (in browser so async events work) and in nodeJS it can be used for async IO. So JS has basically always handled async of sorts (although that has improved significantly).
ochoseis 5y ago
Yesterday a friend just sent me a great post on tuning gevent with gunicorn: https://eng.lyft.com/gevent-part-3-performance-e64303fa102b https://eng.lyft.com/gevent-part-3-performance-e64303fa102b gevent workers are best suited when your app spends a lot of time waiting on I/O (as you note), and they recommend setting `--worker-connections` to the ratio of {network IO time}/{CPU time}. If you set that too high, then it can result in higher latency for requests.
lvass 5y ago
>I am completely unconvinced by the move to asyncio everywhere in Python Me too, now. I've just been bitten by asyncio and it wasn't nice. Running many concurrent tasks a few of which are websockets, the state of Python websocket libraries that work with asyncio is terrible. The sockets would very silently crash until I added asyncio.sleep(0) between the more expensive CPU tasks. It's incredibly hard to debug and exceptions don't get raised properly to outside an event loop, silently making your program run in a zombie state. Even stack traces instantly become a mess when you add async. Bringing something like pandas to an environment like this is probably an immense headache.
- O5vYtytb 5y ago
  Sounds like you need some structured concurrency. Check out Trio or AnyIO. https://trio.readthedocs.io/ https://trio.readthedocs.io/ It's too bad we got asyncio as the standard library module, though some SC concepts are going to make it into the library in future Python versions.
  - nepthar 5y ago
    Small plug for Kotlin as well! Yes, there's new concepts to learn with structured concurrency, but I've been mostly enjoying how quickly it gets out of your way. once properly configured :)
  - vasachi 5y ago
    How would Trio help with this use-case?
    - O5vYtytb 5y ago
      Structured concurrency gives you exceptions that bubble up the stack like a normal program does, and it cancels other tasks along the way so you don't have zombies. Debugging is thusly easier as well since you keep tasks into a scope (nursery, cancel scope).
- samwillis 5y ago
  > The sockets would very silently crash until I added asyncio.sleep(0) between the more expensive CPU tasks. That is exactly the problem with asyncio, the minute you want to do something a little more cpu heavy it blocks. Although admittedly, you would probably see the same problem with gevent, and have to use the same sleep(0) trick. Point is for good scalability you need both traditional threads and either coroutines/asyncio to work together.
whalesalad 5y ago
I have also often wondered the same thing. I have been running wild with gevent/gunicorn for years and have struggled to adopt native async that is part of the core lib. A pet project I would like to get started on would utilize gevent and a dynamic pool of processes to recreate an actor model environment on top of Python. That is - I hope to engineer around the GIL by distributing work to distinct processes, but at the end of the day the user just interacts with one "runtime". Trying to improve my academic knowledge of scheduling algorithms and bin-packing techniques. I have some client projects that would greatly benefit from a move to Erlang/Elixir, but the cost/time/risk of forklifting the entire project to a new runtime is too high. I've used dramatiq.io which is advertised as an actor system, but it really isn't (actors do not have persistent state, no semantics for at-most-one, etc) I think there is a void to be filled here. Ostensibly the underlying concurrency method is not important here, and could be stubbed out, but I will definitely get it started with gevent.
the__alchemist 5y ago
What you lament is common in parts of Rust as well - especially in web-server code. Also making inroads into embedded.
- mikepurvis 5y ago
  Seems like it would be a pretty natural fit in many embedded scenarios— particularly where your entire program is basically just a bunch of threads waiting for things to happen. FreeRTOS is that anyway, except it's longjmping between heap- or statically-allocated stacks, and you have to unblock worker threads by poking them from ISRs. All of this is workable but I could definitely imagine it being more sane with first-class language support for the fundamental cooperative multitasking model that's at work.
- jabl 5y ago
  As a minor counterpoint, the recently announced Hubris OS, implemented in Rust, is entirely synchronous. http://cliffle.com/blog/on-hubris-and-humility/ http://cliffle.com/blog/on-hubris-and-humility/ https://oxide.computer/blog/hubris-and-humility https://oxide.computer/blog/hubris-and-humility
  - the__alchemist 5y ago
    I'd love to see where this goes! I don't have an immediate use case since my current projects aren't complex enough to warrant an OS. Would love to have that in the toolkit.
  - talideon 5y ago
    Hubris is _very_ different, however. Each task is fully static in terms of its resources allocation, and multitasking is essentially cooperative. Now, I'm willing to be corrected on this, but a lot about it reminds me of TinyOS written with larger systems in mind.
- odiroot 5y ago
  I already write async Python on embedded for my pet projects. It actually makes a lot of sense -- with events from GPIO etc.
prionassembly 5y ago
Almost no one really understands asyncio (I pretty much don't), but syntax like "async def" and "async with" makes you feel like you do, and stuff like FastAPI encourages you to type "async". Once in a while someone says that "yes, you can do logic programming in PHP or OOP in Haskell". Maybe, but syntax is more than a nudge, it's a finger on the scales.
- joconde 5y ago
  > Almost no one really understands asyncio I'd very much like a source for that, because I don't feel like it's very hard to understand. > syntax like "async def" and "async with" makes you feel like you do, and stuff like FastAPI encourages you to type "async" FastAPI's docs tell you to use "def" if you don't know what you're doing. [1] says "If you just don't know, use normal def." It's also pretty simple, I think. If your function blocks, use "def", and let FastAPI run it in a thread pool to avoid blocking the event loop. If it doesn't block, use "async", and take advantage of concurrent execution without needing a new thread. [1] https://fastapi.tiangolo.com/async/ https://fastapi.tiangolo.com/async/
  - davidkhess 5y ago
    Not that this validates the claim, but this was a pretty influential post by the author of Flask circa 2016: https://lucumr.pocoo.org/2016/10/30/i-dont-understand-asyncio/ https://lucumr.pocoo.org/2016/10/30/i-dont-understand-asynci...
    - joconde 5y ago
      I agree that it's kind of a mess, but it's getting better with simpler APIs (like asyncio.to_thread()), and in practice you can just use the primitives you want and ignore the rest. I haven't read the post in detail though, thanks for the link! Seems interesting.
  - zippergz 5y ago
    FastAPI's docs tell you to use "def," yet the most basic Hello World example in the tutorial uses "async def." In fact most, if not all, of the examples in the tutorial use "async def." So I don't think it's surprising if developers new to FastAPI tend to use async by default.
aeyes 5y ago
The downside is that monkey patching the world with gevent was hard to pull off and even harder to verify. I was running a large Django installation on gunicorn+gevent for some years and I saw almost no difference in throughput or response time when I made the switch to sync workers. I was glad once I could remove all the monkey patches which really did nothing for my use case. To this day I am not sure how anyone is getting better performance from using async over sync for something like Django. If you run 1000 sync gunicorn processes, I'm sure the OS scheduler is doing a better job in assigning them CPU time slices than what you could ever do with gevent. There is a place for gevent/asyncio, for example if you want to fire off some http requests or DB calls in parallel but you'll have to do it manually no matter what gunicorn worker you run on.
- samwillis 5y ago
  Agree on the hackish nature of the monkey patching. My wish is that Gevent had been adopted into core so that it “just worked” with no monkey patching. Or at least a lower level api was exposed allowing different concurrency/coroutine implementations for the same apis. I have used Gevent extensively in two places; long running eventsource/sse Django views (it’s just implicit due to the Gunicorn worker) and views with hundreds of db and http api calls that are made concurrently before returning a response. Both worked well from my experience, the latter needed coroutines to be spun up for each api/db call.
  - hedgehog 5y ago
    For a while gevent was my go to for projects needing some kind of pub/sub service (different telemetry and device management applications). Worked great in that case but then Go came along, it works even better for those cases.
  - dmw_ng 5y ago
    > My wish is that Gevent had been adopted into core so that it “just worked” with no monkey patching The problem with Gevent is no matter how clean the stack switching, the major liability is other code sharing that stack -- the complete universe of native Python extensions is huge, and some of it most certainly makes assumptions about the stack. I've used Gevent/greenlet for years and never encountered issues, but it's important not to be under any illusions about the kind of hellish crashes that might be lurking just around the corner as more third party code gets added to a project.
rdtsc 5y ago
> The Developer UX is so much better with coroutines and just as scalable as asyncio (async/await). The sad thing is, this was all known because Twisted had been around for years before they made the decision to add async/await. Deferreds, waiting for deferreds, callbacks, deferred semaphores [1]. Not to mention the bifurcated state of libraries, a few supporting Twisted and most not supporting it. And then, when gevent/greenlet/gunicorn came around it was like a breath of fresh air. I remember the day I replaced all the Twisted stuff with a monkeypatch snippet + gevent with a good 20% of code reduction with all the boilerplate gone. And then, Python 3 rolled around and it all came back in the form of async/await. By that point I moved on to a different ecosystem (BEAM VM Elixir/Erlang) so haven't followed much what was happening. But just point out, you're not the only one who noticed it. I remember commenting on it some years ago as well [2] [1] https://twistedmatrix.com/documents/current/api/twisted.internet.defer.DeferredSemaphore.html https://twistedmatrix.com/documents/current/api/twisted.inte... [2] https://news.ycombinator.com/item?id=12831271 https://news.ycombinator.com/item?id=12831271
davidkhess 5y ago
Ditto. I started out on Twisted back in the day and stayed with it for 6 years all they way up until they introduced deferreds combined with yield/generators (the same technique which is the heart of today's asyncio). It was great at the time but ultimately tiresome. It introduced framework specific incantations all over the place and any framework using I/O had to be Twisted-aware in order to use it straight-forwardly. The exact same complaints I have today of asycnio. I went to gevent 10 years ago and have never regretted it. If you have some reasonable knowledge of threading and synchronization then it is a really nice developer experience.
nabla9 5y ago
All old is new again, in different context. Windows 3.X programming was famously painful because it didn't have per-emptive processes. Each program had to yield the control. Windows API was just cooperative async messaging. If you want to scale massively with async, do it Erlang way.
fearface 5y ago
JavaScript didn’t not have any async or parallelization keywords at all. Essentially it’s just passing a delegate that is called later. This was stabdardized into Promises and all the async/await is another wrapper around the promises. It works well, because in the end, it’s just passing a function that should be called when the process finished.
schmichael 5y ago
It was probably over 10 years ago now that I had my hallway moment with Guido van Rossum (creator of Python) at a PyCon. I gave him what was probably an incomprehensible pitch about the shortcomings of generators as coroutines for concurrent network server programming and why I thought stackless python's approach was what Python should adopt as its primary approach to concurrency. Guido listened attentively despite my rambling being essentially a poor rehashing of a proposal rejected 10 years prior! I was too starstruck to remember a concrete thing he replied with, but I remember it being kind and detailed. Was an incredibly formative experience for me about how those in a position of authority and privilege should behave. I doubt I was "right" in any universal sense. Green threads make FFI difficult and costly, and cheap and easy FFI has always been one of Python's most compelling features. I did switch to being a Go programmer as soon as I could find a job using it and haven't looked back. I don't think there's one right abstraction for concurrency, but I think I could live with Go's for the rest of my career and be happy. All of this to say: +1 to gevent.
quacker 5y ago
Completely agree on Gevent. The UX was outstanding. In general, I personally don't love the direction Python 3 has taken with asyncio, type hints, and more and more syntactic sugar. Meanwhile, no real improvements to packaging or to performance of pure Python code. But I'm just a bystander and Python is a big ship to steer, so I get it. Languages like JavaScript that have been async from the start work well This is one of several reasons I've enjoyed gradually switching over to Go from Python. Concurrency is more or less a pleasure in Go. It equals the great UX of Gevent (i.e. Goroutines) but baked into the language and multiplexed onto many real threads (versus only one thread in Gevent).

sandGorgon 5y ago

it kinda has - we use gunicorn with fastapi using the uvicorn asgi workers https://fastapi.tiangolo.com/deployment/server-workers/ https://fastapi.tiangolo.com/deployment/server-workers/

qff 5y ago
Green threads (gunicorn/gevent) allows you to write Python the way you always have and gain many of the benefits of async. Uvicorn and ASGI do not. For those you need to handle async explictly.
- sandGorgon 5y ago
  i hear you - however the framework captures most of the abstraction entirely. In addition, most of the bottlenecks are actually i/o (db queries, etc) where its entirely worth doing it explicitly versus magic. in a general programming language sense...ur right. But from the POV of what gunicorn/fastapi are actually used for - web frameworks - its actually quite good.
ramraj07 5y ago
The list of terms in that sentence: Gunicorn Fastapi Uvicorn Asgi I suppose also nginx docker?
- dgellow 5y ago
  Edit: the grandparent comment has now been updated to clarify, so my comment here is not relevant anymore. ASGI is similar to WSGI but for async oriented web servers. The person you respond to is saying that they use Uvicorn via the ASGI interface. See https://asgi.readthedocs.io/en/latest/ https://asgi.readthedocs.io/en/latest/.
- orf 5y ago
  The list of letters in that sentence: a c d e f g h i k l m n o p r s t u v x I suppose also newlines?
floatrock 5y ago
I've always been a bit confused by this deployment model. gunicorn bills itself as "Gunicorn 'Green Unicorn' is a Python WSGI HTTP Server for UNIX. It's a pre-fork worker model." uvicorn's one-liner "Uvicorn is a lightning-fast ASGI server implementation, using uvloop and httptools." So it seems redundant at first to be running both, but what the gunicorn+uvicorn-worker-process pattern really does is throw away the "WSGI HTTP Server" part of gunicorn and just use the "pre-fork worker" process management part, with the underlying processes actually being ASGI. If you run in a different kind of process-management environment like k8s, does running gunicorn+uvicorn give you anything else?
- sandGorgon 5y ago
  Gunicorn fundamentally runs everything as a worker process. You have sync, gthread, gevent, evenlet, tornado (tornado was also a popular server). UvicornWorker is another type of gunicorn worker... incidentally authored by the same guys who built uvicorn. Nothing strange at all. But ur question is genuine - but it is orthogonal to the uvicorn part of this answer. Fundamentally, in a k8s model...why bother with gunicorn? The answer: I don't know. I do love the control that gunicorn gives me...so I'd probably use it even if I was using a single worker. However there is a complex answer with Threads vs cores vs pods that I'm unqualified to give.

korijn 5y ago

I have been using the async/await syntax in C# for more than ten years and never had a problem with it. What is your point?

samwillis 5y ago
Most of Pythons standard library is sync, the minute you want to use a sync api from within async/await code it becomes a nightmare to manage. If you are only using async library’s then async/await works perfectly.
- philote 5y ago
  It's straightforward to call sync functions from async ones. Are you referring to the fact that those sync functions could block and cause the async code to perform worse? I admit it's a few more lines of code to write a script that runs async code. I've worked on several projects recently where async was used throughout and haven't had issues. There are many async libraries now, like ones to utilize GCP services, make HTTP calls, connect to Redis, etc.
  - samwillis 5y ago
    Yes, I mean blocking sync calls from async code.
- omnicognate 5y ago
  Sounds like you're referring to the "coloured functions" "problem", but it's the other way round. Calling a sync API from an async function is no different to calling it from a sync one. Calling an async function from a sync one requires that you make the caller async. Personally, I like that. An asynchronous function is different to a synchronous one, and if you call an asynchronous function the caller is asynchronous. The "colouring" just reflects the realities of the problem domain. I'm not keen on libraries that monkeypatch the world to hide that fact, as gevent does. However, some people prefer those libraries. Which is fine. They both exist, use what you like. There's no need to have wars about it.

BlueTemplar 5y ago

Wait, but JavaScript only had async/await added recently too, with ES6 ? (Or was that "just" about an easier way to use these features?)

samwillis 5y ago
Async/await in JavaScript is, a very welcome, syntactic sugar over a callback api. The point is that almost all JavaScript library’s have always been async, traditionally with callback based apis (now async/await). You never mixed async and sync code.
- renke1 5y ago
  That's reason why I think the push to follow event-driven (reactive) approaches in ecosystems like Java is doomed to fail (okay, that may be a bit harsh) because there is lot of existing blocking code. It will take a long time until all of the old libraries catch up. It kind of splits the whole ecosystem into half. Specifically in Java I think it's fine we are relying on blocking APIs (and threads). For most stuff it scales properly, doesn't feel unnatural and is easier to understand. I say this as a big fan of JavaScript, I just don't see the approach being introduced after the fact as a good idea.

akvadrako 5y ago

Even better is to just use threads. Linux can handle a million threads just fine, so even if 99.99% of your clients are idle, threading won't be the bottleneck.

NovemberWhiskey 5y ago
It's not the time when most of your threads are idle that's the problem; it's when you've accepted a million incoming socket connections, given one connection to each thread and now you're thrashing wildly trying to get any useful work done as your clients timeout, reconnect and repeat.
- otabdeveloper4 5y ago
  A green thread implementation will thrash just as much in your hypothetical scenario. The only advantage of green threading is the capability to do problem-specific optimized scheduling. If you're not doing this then you don't need green threads.
  - akvadrako 5y ago
    You are also optimize the scheduling for real threads, for example by increasing sched_latency_ns and sched_min_granularity_ns.
  - pm90 5y ago
    > The only advantage of green threading is the capability to do problem-specific optimized scheduling. Isn’t the main advantage simply that you avoid making an absurd number of costly system calls?
    - camnora 5y ago
      This article does a pretty good job of explaining how green events work. It seems that each of the threads will attempt to accept any/all incoming requests. https://rachelbythebay.com/w/2020/03/07/costly/ https://rachelbythebay.com/w/2020/03/07/costly/

BlueTemplar 5y ago

Wait, but JavaScript only had async/await added recently too, with ES6 ? (Or was that "just" about an easier way to use these features?)! EDIT: Argh, double posting, did I hit "back" by mistake ?

sammorrowdrums 5y ago
They are effectively sugar on JS promises which are effectively sugar around async event callbacks. This is why node rose so quickly in popularity as a server. The async IO handling could be leveraged with callbacks to scale low number of processes to handle large number of requests. Plus easy to learn and lots of people knew some JS. JS always had an event loop (in browser so async events work) and in nodeJS it can be used for async IO. So JS has basically always handled async of sorts (although that has improved significantly).

ochoseis 5y ago

Yesterday a friend just sent me a great post on tuning gevent with gunicorn: https://eng.lyft.com/gevent-part-3-performance-e64303fa102b https://eng.lyft.com/gevent-part-3-performance-e64303fa102b gevent workers are best suited when your app spends a lot of time waiting on I/O (as you note), and they recommend setting `--worker-connections` to the ratio of {network IO time}/{CPU time}. If you set that too high, then it can result in higher latency for requests.

lvass 5y ago

>I am completely unconvinced by the move to asyncio everywhere in Python Me too, now. I've just been bitten by asyncio and it wasn't nice. Running many concurrent tasks a few of which are websockets, the state of Python websocket libraries that work with asyncio is terrible. The sockets would very silently crash until I added asyncio.sleep(0) between the more expensive CPU tasks. It's incredibly hard to debug and exceptions don't get raised properly to outside an event loop, silently making your program run in a zombie state. Even stack traces instantly become a mess when you add async. Bringing something like pandas to an environment like this is probably an immense headache.

O5vYtytb 5y ago
Sounds like you need some structured concurrency. Check out Trio or AnyIO. https://trio.readthedocs.io/ https://trio.readthedocs.io/ It's too bad we got asyncio as the standard library module, though some SC concepts are going to make it into the library in future Python versions.
- nepthar 5y ago
  Small plug for Kotlin as well! Yes, there's new concepts to learn with structured concurrency, but I've been mostly enjoying how quickly it gets out of your way. once properly configured :)
- vasachi 5y ago
  How would Trio help with this use-case?
  - O5vYtytb 5y ago
    Structured concurrency gives you exceptions that bubble up the stack like a normal program does, and it cancels other tasks along the way so you don't have zombies. Debugging is thusly easier as well since you keep tasks into a scope (nursery, cancel scope).
samwillis 5y ago
> The sockets would very silently crash until I added asyncio.sleep(0) between the more expensive CPU tasks. That is exactly the problem with asyncio, the minute you want to do something a little more cpu heavy it blocks. Although admittedly, you would probably see the same problem with gevent, and have to use the same sleep(0) trick. Point is for good scalability you need both traditional threads and either coroutines/asyncio to work together.

whalesalad 5y ago

I have also often wondered the same thing. I have been running wild with gevent/gunicorn for years and have struggled to adopt native async that is part of the core lib. A pet project I would like to get started on would utilize gevent and a dynamic pool of processes to recreate an actor model environment on top of Python. That is - I hope to engineer around the GIL by distributing work to distinct processes, but at the end of the day the user just interacts with one "runtime". Trying to improve my academic knowledge of scheduling algorithms and bin-packing techniques. I have some client projects that would greatly benefit from a move to Erlang/Elixir, but the cost/time/risk of forklifting the entire project to a new runtime is too high. I've used dramatiq.io which is advertised as an actor system, but it really isn't (actors do not have persistent state, no semantics for at-most-one, etc) I think there is a void to be filled here. Ostensibly the underlying concurrency method is not important here, and could be stubbed out, but I will definitely get it started with gevent.

the__alchemist 5y ago

What you lament is common in parts of Rust as well - especially in web-server code. Also making inroads into embedded.

mikepurvis 5y ago
Seems like it would be a pretty natural fit in many embedded scenarios— particularly where your entire program is basically just a bunch of threads waiting for things to happen. FreeRTOS is that anyway, except it's longjmping between heap- or statically-allocated stacks, and you have to unblock worker threads by poking them from ISRs. All of this is workable but I could definitely imagine it being more sane with first-class language support for the fundamental cooperative multitasking model that's at work.
jabl 5y ago
As a minor counterpoint, the recently announced Hubris OS, implemented in Rust, is entirely synchronous. http://cliffle.com/blog/on-hubris-and-humility/ http://cliffle.com/blog/on-hubris-and-humility/ https://oxide.computer/blog/hubris-and-humility https://oxide.computer/blog/hubris-and-humility
- the__alchemist 5y ago
  I'd love to see where this goes! I don't have an immediate use case since my current projects aren't complex enough to warrant an OS. Would love to have that in the toolkit.
- talideon 5y ago
  Hubris is _very_ different, however. Each task is fully static in terms of its resources allocation, and multitasking is essentially cooperative. Now, I'm willing to be corrected on this, but a lot about it reminds me of TinyOS written with larger systems in mind.
odiroot 5y ago
I already write async Python on embedded for my pet projects. It actually makes a lot of sense -- with events from GPIO etc.

prionassembly 5y ago

Almost no one really understands asyncio (I pretty much don't), but syntax like "async def" and "async with" makes you feel like you do, and stuff like FastAPI encourages you to type "async". Once in a while someone says that "yes, you can do logic programming in PHP or OOP in Haskell". Maybe, but syntax is more than a nudge, it's a finger on the scales.

joconde 5y ago
> Almost no one really understands asyncio I'd very much like a source for that, because I don't feel like it's very hard to understand. > syntax like "async def" and "async with" makes you feel like you do, and stuff like FastAPI encourages you to type "async" FastAPI's docs tell you to use "def" if you don't know what you're doing. [1] says "If you just don't know, use normal def." It's also pretty simple, I think. If your function blocks, use "def", and let FastAPI run it in a thread pool to avoid blocking the event loop. If it doesn't block, use "async", and take advantage of concurrent execution without needing a new thread. [1] https://fastapi.tiangolo.com/async/ https://fastapi.tiangolo.com/async/
- davidkhess 5y ago
  Not that this validates the claim, but this was a pretty influential post by the author of Flask circa 2016: https://lucumr.pocoo.org/2016/10/30/i-dont-understand-asyncio/ https://lucumr.pocoo.org/2016/10/30/i-dont-understand-asynci...
  - joconde 5y ago
    I agree that it's kind of a mess, but it's getting better with simpler APIs (like asyncio.to_thread()), and in practice you can just use the primitives you want and ignore the rest. I haven't read the post in detail though, thanks for the link! Seems interesting.
- zippergz 5y ago
  FastAPI's docs tell you to use "def," yet the most basic Hello World example in the tutorial uses "async def." In fact most, if not all, of the examples in the tutorial use "async def." So I don't think it's surprising if developers new to FastAPI tend to use async by default.

aeyes 5y ago

The downside is that monkey patching the world with gevent was hard to pull off and even harder to verify. I was running a large Django installation on gunicorn+gevent for some years and I saw almost no difference in throughput or response time when I made the switch to sync workers. I was glad once I could remove all the monkey patches which really did nothing for my use case. To this day I am not sure how anyone is getting better performance from using async over sync for something like Django. If you run 1000 sync gunicorn processes, I'm sure the OS scheduler is doing a better job in assigning them CPU time slices than what you could ever do with gevent. There is a place for gevent/asyncio, for example if you want to fire off some http requests or DB calls in parallel but you'll have to do it manually no matter what gunicorn worker you run on.

samwillis 5y ago
Agree on the hackish nature of the monkey patching. My wish is that Gevent had been adopted into core so that it “just worked” with no monkey patching. Or at least a lower level api was exposed allowing different concurrency/coroutine implementations for the same apis. I have used Gevent extensively in two places; long running eventsource/sse Django views (it’s just implicit due to the Gunicorn worker) and views with hundreds of db and http api calls that are made concurrently before returning a response. Both worked well from my experience, the latter needed coroutines to be spun up for each api/db call.
- hedgehog 5y ago
  For a while gevent was my go to for projects needing some kind of pub/sub service (different telemetry and device management applications). Worked great in that case but then Go came along, it works even better for those cases.
- dmw_ng 5y ago
  > My wish is that Gevent had been adopted into core so that it “just worked” with no monkey patching The problem with Gevent is no matter how clean the stack switching, the major liability is other code sharing that stack -- the complete universe of native Python extensions is huge, and some of it most certainly makes assumptions about the stack. I've used Gevent/greenlet for years and never encountered issues, but it's important not to be under any illusions about the kind of hellish crashes that might be lurking just around the corner as more third party code gets added to a project.

rdtsc 5y ago

> The Developer UX is so much better with coroutines and just as scalable as asyncio (async/await). The sad thing is, this was all known because Twisted had been around for years before they made the decision to add async/await. Deferreds, waiting for deferreds, callbacks, deferred semaphores [1]. Not to mention the bifurcated state of libraries, a few supporting Twisted and most not supporting it. And then, when gevent/greenlet/gunicorn came around it was like a breath of fresh air. I remember the day I replaced all the Twisted stuff with a monkeypatch snippet + gevent with a good 20% of code reduction with all the boilerplate gone. And then, Python 3 rolled around and it all came back in the form of async/await. By that point I moved on to a different ecosystem (BEAM VM Elixir/Erlang) so haven't followed much what was happening. But just point out, you're not the only one who noticed it. I remember commenting on it some years ago as well [2] [1] https://twistedmatrix.com/documents/current/api/twisted.internet.defer.DeferredSemaphore.html https://twistedmatrix.com/documents/current/api/twisted.inte... [2] https://news.ycombinator.com/item?id=12831271 https://news.ycombinator.com/item?id=12831271

davidkhess 5y ago

Ditto. I started out on Twisted back in the day and stayed with it for 6 years all they way up until they introduced deferreds combined with yield/generators (the same technique which is the heart of today's asyncio). It was great at the time but ultimately tiresome. It introduced framework specific incantations all over the place and any framework using I/O had to be Twisted-aware in order to use it straight-forwardly. The exact same complaints I have today of asycnio. I went to gevent 10 years ago and have never regretted it. If you have some reasonable knowledge of threading and synchronization then it is a really nice developer experience.

nabla9 5y ago

All old is new again, in different context. Windows 3.X programming was famously painful because it didn't have per-emptive processes. Each program had to yield the control. Windows API was just cooperative async messaging. If you want to scale massively with async, do it Erlang way.

fearface 5y ago

JavaScript didn’t not have any async or parallelization keywords at all. Essentially it’s just passing a delegate that is called later. This was stabdardized into Promises and all the async/await is another wrapper around the promises. It works well, because in the end, it’s just passing a function that should be called when the process finished.

schmichael 5y ago

It was probably over 10 years ago now that I had my hallway moment with Guido van Rossum (creator of Python) at a PyCon. I gave him what was probably an incomprehensible pitch about the shortcomings of generators as coroutines for concurrent network server programming and why I thought stackless python's approach was what Python should adopt as its primary approach to concurrency. Guido listened attentively despite my rambling being essentially a poor rehashing of a proposal rejected 10 years prior! I was too starstruck to remember a concrete thing he replied with, but I remember it being kind and detailed. Was an incredibly formative experience for me about how those in a position of authority and privilege should behave. I doubt I was "right" in any universal sense. Green threads make FFI difficult and costly, and cheap and easy FFI has always been one of Python's most compelling features. I did switch to being a Go programmer as soon as I could find a job using it and haven't looked back. I don't think there's one right abstraction for concurrency, but I think I could live with Go's for the rest of my career and be happy. All of this to say: +1 to gevent.

quacker 5y ago

Completely agree on Gevent. The UX was outstanding. In general, I personally don't love the direction Python 3 has taken with asyncio, type hints, and more and more syntactic sugar. Meanwhile, no real improvements to packaging or to performance of pure Python code. But I'm just a bystander and Python is a big ship to steer, so I get it. Languages like JavaScript that have been async from the start work well This is one of several reasons I've enjoyed gradually switching over to Go from Python. Concurrency is more or less a pleasure in Go. It equals the great UX of Gevent (i.e. Goroutines) but baked into the language and multiplexed onto many real threads (versus only one thread in Gevent).