3 ms·

> instead we got a natively supported, robust, easy-to-use concurrency paradigm built around green/virtual threading that accommodates both IO and CPU bound wor

by btown 5y ago

> instead we got a natively supported, robust, easy-to-use concurrency paradigm built around green/virtual threading that accommodates both IO and CPU bound work Minus the "natively supported" part, we have this today in http://www.gevent.org/ http://www.gevent.org/ ! It's so, so empowering to be able to access the entire historical body of work of synchronous-I/O Python libraries, and with a single monkey patch cause every I/O operation, no matter how deep in the stack, to yield to your greenlet pool without code changes. We fire up one process per core (gevent doesn't have good support for multiprocessing, but if you're relying on that, you're stuck on one machine anyways), spend perhaps 1 person-day a quarter dealing with its quirks, and in turn we never need to worry about the latencies of external services; our web servers and batch workers have throughput limited only by CPU and RAM, for which there's relatively little (though nonzero) overhead. IMO Python should have leaned into official adoption of gevent. It may not beat asyncio in raw performance numbers because asyncio can rely on custom-built bytecode instructions, whereas gevent has "userspace" code that must execute upon every yield. And, as with asyncio, you have to be careful about CPU-intensive code that may prevent you from yielding. But it's perfect for most horizontal-scaling soft-realtime web-style use cases.

Buttons840 5y ago
> our web servers and batch workers have throughput limited only by CPU and RAM Are you able to fully utilize a multicore processor? I'm not familiar with gevent.
- js2 5y ago
  This is why you run one process per core, and you'll typically have something like nginx+uWSGI distribute requests across them. I use this combination with https://falconframework.org/ https://falconframework.org/ and boto3 to spool HTTP POST requests to S3 and SQS and am pretty happy with it. uWSGI supports gevent https://uwsgi-docs.readthedocs.io/en/latest/Gevent.html https://uwsgi-docs.readthedocs.io/en/latest/Gevent.html Falcon also benefits from Cython acceleration. It's been a while but at the time I tested against PyPy and either it was slower or had quirks I was unable to resolve (don't recall w/o consulting my notes).
  - btown 5y ago
    Yep, we use Gunicorn in full gevent mode, tuned to spawn and route to a gevent-patched Django process per core, each of which can handle as many concurrent requests as will fit in (its slice of) RAM. A far cry from the one-request-per-thread days of yore!
    - sandGorgon 5y ago
      you use gunicorn as a reverse proxy - it seems that you are not running Gunicorn to spawn django processes is it ? I'm wondering how would this work in a kubernetes/docker world.
      - btown 5y ago
        Gunicorn both acts as a reverse proxy and a master process that forks workers - each of which, in our case, loads Django. https://docs.gunicorn.org/en/stable/design.html https://docs.gunicorn.org/en/stable/design.html
unityByFreedom 5y ago
Gevent is indeed amazing. I just started using it. Isn't the point of this post's effort to bake in its functionality? Handling exceptions with gevent doesn't seem trivial to me, for example, and it'd be nice to not have to monkey patch, as easy as it is.
- btown 5y ago
  Two different parts of the problem! Gevent is all about saying "I/O shouldn't keep my current thread from doing useful work." And if you're I/O bound, that's great! But if you have a lot of trivially parallelizable CPU work to do, it won't help much there, because gevent does nothing about the GIL (global interpreter lock). The OP is trying to solve the GIL problem, which is saying: "activity on other threads, which might touch objects in memory that my thread cares about, shouldn't keep my current thread from doing useful work." Right now, Python needs to be super conservative to keep threads from stomping on each others' memory, and this should make it a lot more feasible to do that in a sane way. As the comments in the OP post itself suggest, though, this isn't necessarily a sufficient solution. If you're looking for the real solution to "let other threads borrow my memory without global locks," the Rust language and its borrow checker is really the holy grail there. Regarding gevent exceptions, I find them actually quite natural to work with. If you're looking at old tutorials that talk about linking callbacks to handle exceptions, in practice I've never needed to do that - that stuff is primarily useful to those writing things like gunicorn itself. Pool.imap[_unordered], greenlet.get(), and more will simply re-raise exceptions in the caller's greenlet if there are exceptions in the farmed-out greenlets. And if you need even more control, you can try-catch within the functions that are running inside your greenlets, and return results that may either be a success or a failure.
smetj 5y ago
Gevent is the best thing which ever happened within the async world of Python. It's just great to work with. Rock solid. I have high IO production software running for years with gevent as the workhorse, copes with high load, no maintenance, brilliant. Then came Asyncio, which I personally disliked for the simple reason it became so popular that everybody thinks it's necessary to write an asyncio version of their library/module. The result is it now tears everything in 2. Asyncio code and non-asyncio code ... Everything like Asyncio, Curio, Trio are impressive and interesting but their interfaces are way too involved for the intended result and it should never be like that. (Gevent suffered less from that). The only way forward is proper thread support without the GIL and therefor I applaud this effort.
- ayush--s 5y ago
  It's really easy to shoot yourself in the foot by using global mutable variables. How do you guard against those when using gevent in production?
  - sillysaurusx 5y ago
    Global state isn't a problem for multithreaded code. State updates are the problem; it doesn't matter whether it's global or local. If you can solve the local case, you can solve the global case. Pick your technique; any ol' technique is just as good as any other.
  - btown 5y ago
    It's actually quite doable in gevent, because you have a guarantee that only one thread will ever be touching those variables ever. You can have 5 or 50 lines of code and be guaranteed that they will operate atomically, read their writes, be immune to interruption, all that good stuff... as long as they don't do any I/O. Of course, the difference from a platform like Node.js or asyncio (where every async/await yield must be explicit all the way down) is that one of your libraries calling `logger.info(...)` might cause I/O, and then cause an implicit yield to the event loop, and break your atomicity without you knowing about it. But if you don't log, and you just work in memory, with code you own or have audited to not do I/O, the sky's the limit. And you almost always want this kind of well-tested, non-logging, high-performance abstraction layer around global mutable state access anyways.
jgehrcke 5y ago
This! > gevent doesn't have good support for multiprocessing This is where my little library https://github.com/jgehrcke/gipc https://github.com/jgehrcke/gipc might help! It's been working well for people since 2012 :) By the way, gevent is huge for the Python ecosystem and I would love to acknowledge both Denis Bilenko and Jason Madden who have done a great job over the years. I am in particular impressed by Jason's maintenance work -- creating a new project is always exciting. Maintaining it over the years is sometimes just tedious, hard work. Jason has impressed me a lot with his one man show there. Not sure if everybody knows, but gevent was for starters built on top of "only" libev -- another super impressive project built and maintained by practically a single person: Marc Lehmann. I learned a lot from his work, too. So so so many serious software projects build on top of libev. Only 'recently' gevent included support for libuv, the event loop that originated from the NodeJS ecosystem. I find it super interesting to see that CPython+gevent+libuv is highly similar to running NodeJS! By the way, the LWN article is once again a wonderfully written piece by Jonathan Corbet. His way to doing technical writing is simply great, I have enjoyed it very much over the years and learned a lot.

by btown 5y ago

Buttons840 5y ago

> our web servers and batch workers have throughput limited only by CPU and RAM Are you able to fully utilize a multicore processor? I'm not familiar with gevent.

js2 5y ago
This is why you run one process per core, and you'll typically have something like nginx+uWSGI distribute requests across them. I use this combination with https://falconframework.org/ https://falconframework.org/ and boto3 to spool HTTP POST requests to S3 and SQS and am pretty happy with it. uWSGI supports gevent https://uwsgi-docs.readthedocs.io/en/latest/Gevent.html https://uwsgi-docs.readthedocs.io/en/latest/Gevent.html Falcon also benefits from Cython acceleration. It's been a while but at the time I tested against PyPy and either it was slower or had quirks I was unable to resolve (don't recall w/o consulting my notes).
- btown 5y ago
  Yep, we use Gunicorn in full gevent mode, tuned to spawn and route to a gevent-patched Django process per core, each of which can handle as many concurrent requests as will fit in (its slice of) RAM. A far cry from the one-request-per-thread days of yore!
  - sandGorgon 5y ago
    you use gunicorn as a reverse proxy - it seems that you are not running Gunicorn to spawn django processes is it ? I'm wondering how would this work in a kubernetes/docker world.
    - btown 5y ago
      Gunicorn both acts as a reverse proxy and a master process that forks workers - each of which, in our case, loads Django. https://docs.gunicorn.org/en/stable/design.html https://docs.gunicorn.org/en/stable/design.html

unityByFreedom 5y ago

Gevent is indeed amazing. I just started using it. Isn't the point of this post's effort to bake in its functionality? Handling exceptions with gevent doesn't seem trivial to me, for example, and it'd be nice to not have to monkey patch, as easy as it is.

btown 5y ago
Two different parts of the problem! Gevent is all about saying "I/O shouldn't keep my current thread from doing useful work." And if you're I/O bound, that's great! But if you have a lot of trivially parallelizable CPU work to do, it won't help much there, because gevent does nothing about the GIL (global interpreter lock). The OP is trying to solve the GIL problem, which is saying: "activity on other threads, which might touch objects in memory that my thread cares about, shouldn't keep my current thread from doing useful work." Right now, Python needs to be super conservative to keep threads from stomping on each others' memory, and this should make it a lot more feasible to do that in a sane way. As the comments in the OP post itself suggest, though, this isn't necessarily a sufficient solution. If you're looking for the real solution to "let other threads borrow my memory without global locks," the Rust language and its borrow checker is really the holy grail there. Regarding gevent exceptions, I find them actually quite natural to work with. If you're looking at old tutorials that talk about linking callbacks to handle exceptions, in practice I've never needed to do that - that stuff is primarily useful to those writing things like gunicorn itself. Pool.imap[_unordered], greenlet.get(), and more will simply re-raise exceptions in the caller's greenlet if there are exceptions in the farmed-out greenlets. And if you need even more control, you can try-catch within the functions that are running inside your greenlets, and return results that may either be a success or a failure.

smetj 5y ago

Gevent is the best thing which ever happened within the async world of Python. It's just great to work with. Rock solid. I have high IO production software running for years with gevent as the workhorse, copes with high load, no maintenance, brilliant. Then came Asyncio, which I personally disliked for the simple reason it became so popular that everybody thinks it's necessary to write an asyncio version of their library/module. The result is it now tears everything in 2. Asyncio code and non-asyncio code ... Everything like Asyncio, Curio, Trio are impressive and interesting but their interfaces are way too involved for the intended result and it should never be like that. (Gevent suffered less from that). The only way forward is proper thread support without the GIL and therefor I applaud this effort.

ayush--s 5y ago
It's really easy to shoot yourself in the foot by using global mutable variables. How do you guard against those when using gevent in production?
- sillysaurusx 5y ago
  Global state isn't a problem for multithreaded code. State updates are the problem; it doesn't matter whether it's global or local. If you can solve the local case, you can solve the global case. Pick your technique; any ol' technique is just as good as any other.
- btown 5y ago
  It's actually quite doable in gevent, because you have a guarantee that only one thread will ever be touching those variables ever. You can have 5 or 50 lines of code and be guaranteed that they will operate atomically, read their writes, be immune to interruption, all that good stuff... as long as they don't do any I/O. Of course, the difference from a platform like Node.js or asyncio (where every async/await yield must be explicit all the way down) is that one of your libraries calling `logger.info(...)` might cause I/O, and then cause an implicit yield to the event loop, and break your atomicity without you knowing about it. But if you don't log, and you just work in memory, with code you own or have audited to not do I/O, the sky's the limit. And you almost always want this kind of well-tested, non-logging, high-performance abstraction layer around global mutable state access anyways.

jgehrcke 5y ago

This! > gevent doesn't have good support for multiprocessing This is where my little library https://github.com/jgehrcke/gipc https://github.com/jgehrcke/gipc might help! It's been working well for people since 2012 :) By the way, gevent is huge for the Python ecosystem and I would love to acknowledge both Denis Bilenko and Jason Madden who have done a great job over the years. I am in particular impressed by Jason's maintenance work -- creating a new project is always exciting. Maintaining it over the years is sometimes just tedious, hard work. Jason has impressed me a lot with his one man show there. Not sure if everybody knows, but gevent was for starters built on top of "only" libev -- another super impressive project built and maintained by practically a single person: Marc Lehmann. I learned a lot from his work, too. So so so many serious software projects build on top of libev. Only 'recently' gevent included support for libuv, the event loop that originated from the NodeJS ecosystem. I find it super interesting to see that CPython+gevent+libuv is highly similar to running NodeJS! By the way, the LWN article is once again a wonderfully written piece by Jonathan Corbet. His way to doing technical writing is simply great, I have enjoyed it very much over the years and learned a lot.