4 ms·

Gevent is indeed amazing. I just started using it. Isn't the point of this post's effort to bake in its functionality? Handling exceptions with gevent doesn't s

Gevent is indeed amazing. I just started using it. Isn't the point of this post's effort to bake in its functionality? Handling exceptions with gevent doesn't seem trivial to me, for example, and it'd be nice to not have to monkey patch, as easy as it is.

btown 5y ago
Two different parts of the problem! Gevent is all about saying "I/O shouldn't keep my current thread from doing useful work." And if you're I/O bound, that's great! But if you have a lot of trivially parallelizable CPU work to do, it won't help much there, because gevent does nothing about the GIL (global interpreter lock). The OP is trying to solve the GIL problem, which is saying: "activity on other threads, which might touch objects in memory that my thread cares about, shouldn't keep my current thread from doing useful work." Right now, Python needs to be super conservative to keep threads from stomping on each others' memory, and this should make it a lot more feasible to do that in a sane way. As the comments in the OP post itself suggest, though, this isn't necessarily a sufficient solution. If you're looking for the real solution to "let other threads borrow my memory without global locks," the Rust language and its borrow checker is really the holy grail there. Regarding gevent exceptions, I find them actually quite natural to work with. If you're looking at old tutorials that talk about linking callbacks to handle exceptions, in practice I've never needed to do that - that stuff is primarily useful to those writing things like gunicorn itself. Pool.imap[_unordered], greenlet.get(), and more will simply re-raise exceptions in the caller's greenlet if there are exceptions in the farmed-out greenlets. And if you need even more control, you can try-catch within the functions that are running inside your greenlets, and return results that may either be a success or a failure.

btown 5y ago

Two different parts of the problem! Gevent is all about saying "I/O shouldn't keep my current thread from doing useful work." And if you're I/O bound, that's great! But if you have a lot of trivially parallelizable CPU work to do, it won't help much there, because gevent does nothing about the GIL (global interpreter lock). The OP is trying to solve the GIL problem, which is saying: "activity on other threads, which might touch objects in memory that my thread cares about, shouldn't keep my current thread from doing useful work." Right now, Python needs to be super conservative to keep threads from stomping on each others' memory, and this should make it a lot more feasible to do that in a sane way. As the comments in the OP post itself suggest, though, this isn't necessarily a sufficient solution. If you're looking for the real solution to "let other threads borrow my memory without global locks," the Rust language and its borrow checker is really the holy grail there. Regarding gevent exceptions, I find them actually quite natural to work with. If you're looking at old tutorials that talk about linking callbacks to handle exceptions, in practice I've never needed to do that - that stuff is primarily useful to those writing things like gunicorn itself. Pool.imap[_unordered], greenlet.get(), and more will simply re-raise exceptions in the caller's greenlet if there are exceptions in the farmed-out greenlets. And if you need even more control, you can try-catch within the functions that are running inside your greenlets, and return results that may either be a success or a failure.