4 ms·

I am not sure when this article was written, but I really wouldn't recommend threads for even IO-bound tasks. Since threads don't escape the GIL the added cont

I am not sure when this article was written, but I really wouldn't recommend threads for even IO-bound tasks. Since threads don't escape the GIL the added contention with OS thread scheduling (context switching) is a performance overhead that coroutines (asyncio) don't have. I haven't benchmarked threads vs. coroutines for IO-bound tasks...but my gut feeling is coroutines are going to generally be better because of the lack of thread switching overhead. So for me, there really are only 2 concurrency choices: coroutines or multiprocessing. And generally, if I find myself reaching for multiprocessing I seriously evaluate if the logic shouldn't be ported to a different language.

lelandbatey 4y ago
I wouldn't quite say that there's only 2 concurrency choices, but I agree that there's only 2 concurrency choices in Python if you are operating under heavy load. The multi-threaded concurrency style can be a great choice if your performance requirements are low and you only want a small amount of concurrency. Then it can be very easy to emulate a Golang-esq model, by having just a few threads which are communicating by writing data via queues to one another.
hbrn 4y ago
But threads do release GIL on IO. There's a slight overhead of course, but if you're not running hundreds of servers and thousands of threads, the overhead can be negligible. I've seen more than once people rewriting thread-based code to coroutines, despite not having any noticeable benefits other than being more sexy. In fact I was one of those people almost a decade ago, rewriting the whole networking portion of the app only to achieve ~5% performance improvement. One should always measure the overhead instead assuming how big it is.
- gshulegaard 4y ago
  I am not expert, and this might be dated information, but historically there is a "thrashing" introduced between the GIL and OS scheduled threads which has surprising performance implications. David Beazley does a much better illustrating the nuances than I ever could: https://www.youtube.com/watch?v=Obt-vMVdM8s https://www.youtube.com/watch?v=Obt-vMVdM8s Including a wild phenomenon where a threaded Python program's performance can degrade by increasing number of CPU cores available to a system. Coroutines avoid this by not requiring individual thread scheduling by the OS. So they effectively release the GIL like threads on IO but avoid OS level thread switching.
  - hbrn 4y ago
    It is a bit dated (he talks about GIL improvements in 3.2), but regardless I think it only proves my point. The examples David shows are threads with zero IO unable to properly cooperate. Having threads that are mostly doing IO would actually allow them to cooperate natively by releasing GIL. Another interesting example he talks about is IO-bound threads competing with CPU-bound threads, where CPU-threads are getting seemingly unfair advantage. But the same example would only get worse with coroutine implementation: since there's only one thread, CPU bound code would just block IO forever. IO-bound code would not just be slow, it would never execute. What David illustrated is that GIL makes it harder to mix CPU- and IO-bound tasks in the same process. But coroutines are not the solution here. Solution is not to mix them.
    - gshulegaard 4y ago
      Right so the GIL improvements which (probably?) made it into 3.2 solves thrashing but introduces latency during "negotiation" for the GIL. > What David illustrated is that GIL makes it harder to mix CPU- and IO-bound tasks in the same process. But coroutines are not the solution here. Solution is not to mix them. Which I believe is my point and why I forwarded the personal notion that there really is only two acceptable concurrency models in Python. Coroutines solve cooperative scheduling for things that can be cooperatively scheduled. Threads _could_, but in Python specifically threads are strictly inferior as they have additional overhead by nature. In slightly different phrasing, Threads can be nearly or even acceptably equivalent to coroutines in performance, but they will never be better so long as Python has a GIL. For everything else there is Multiprocessing (for example segmenting CPU processing from IO over separate processes).
      - hbrn 4y ago
        > threads are strictly inferior as they have additional overhead by nature In terms of performance, sure. But performance is rarely your only priority. A lot of the times you already have existing code that can be switched to threads with 2 lines of code. Instead, folks choose to rewrite the whole app in asyncio/nodejs because threads aren't sexy. Hey, I've done it. But today I'd rather have inferior working app in one day, than a completely rewritten app in a few weeks, with a mental model that half engineers don't understand and don't have experience with. Maybe in a few years you will be forced to rewrite it in asyncio. And that would be the right time to do it: when it's a business demand, not when few perfectionists are annoyed.
- edf13 4y ago
  5% performance increase is worthwhile though?
  - hbrn 4y ago
    Only if it's free. The costs were quite high: Time spend rewriting networking Code because more complicated: cooperation was moved to code instead of being handled by OS (to be fair this was way before asyncio, it was gevent). Code became alien to the team. I was the only one who knew how it worked. In the end, the only problem I solved was the problem we didn't have: "not using coroutines". Sure, threads suck at scale. But you should build things for scale when it's necessary, not when it's fun.

lelandbatey 4y ago

I wouldn't quite say that there's only 2 concurrency choices, but I agree that there's only 2 concurrency choices in Python if you are operating under heavy load. The multi-threaded concurrency style can be a great choice if your performance requirements are low and you only want a small amount of concurrency. Then it can be very easy to emulate a Golang-esq model, by having just a few threads which are communicating by writing data via queues to one another.

hbrn 4y ago

But threads do release GIL on IO. There's a slight overhead of course, but if you're not running hundreds of servers and thousands of threads, the overhead can be negligible. I've seen more than once people rewriting thread-based code to coroutines, despite not having any noticeable benefits other than being more sexy. In fact I was one of those people almost a decade ago, rewriting the whole networking portion of the app only to achieve ~5% performance improvement. One should always measure the overhead instead assuming how big it is.

gshulegaard 4y ago
I am not expert, and this might be dated information, but historically there is a "thrashing" introduced between the GIL and OS scheduled threads which has surprising performance implications. David Beazley does a much better illustrating the nuances than I ever could: https://www.youtube.com/watch?v=Obt-vMVdM8s https://www.youtube.com/watch?v=Obt-vMVdM8s Including a wild phenomenon where a threaded Python program's performance can degrade by increasing number of CPU cores available to a system. Coroutines avoid this by not requiring individual thread scheduling by the OS. So they effectively release the GIL like threads on IO but avoid OS level thread switching.
- hbrn 4y ago
  It is a bit dated (he talks about GIL improvements in 3.2), but regardless I think it only proves my point. The examples David shows are threads with zero IO unable to properly cooperate. Having threads that are mostly doing IO would actually allow them to cooperate natively by releasing GIL. Another interesting example he talks about is IO-bound threads competing with CPU-bound threads, where CPU-threads are getting seemingly unfair advantage. But the same example would only get worse with coroutine implementation: since there's only one thread, CPU bound code would just block IO forever. IO-bound code would not just be slow, it would never execute. What David illustrated is that GIL makes it harder to mix CPU- and IO-bound tasks in the same process. But coroutines are not the solution here. Solution is not to mix them.
  - gshulegaard 4y ago
    Right so the GIL improvements which (probably?) made it into 3.2 solves thrashing but introduces latency during "negotiation" for the GIL. > What David illustrated is that GIL makes it harder to mix CPU- and IO-bound tasks in the same process. But coroutines are not the solution here. Solution is not to mix them. Which I believe is my point and why I forwarded the personal notion that there really is only two acceptable concurrency models in Python. Coroutines solve cooperative scheduling for things that can be cooperatively scheduled. Threads _could_, but in Python specifically threads are strictly inferior as they have additional overhead by nature. In slightly different phrasing, Threads can be nearly or even acceptably equivalent to coroutines in performance, but they will never be better so long as Python has a GIL. For everything else there is Multiprocessing (for example segmenting CPU processing from IO over separate processes).
    - hbrn 4y ago
      > threads are strictly inferior as they have additional overhead by nature In terms of performance, sure. But performance is rarely your only priority. A lot of the times you already have existing code that can be switched to threads with 2 lines of code. Instead, folks choose to rewrite the whole app in asyncio/nodejs because threads aren't sexy. Hey, I've done it. But today I'd rather have inferior working app in one day, than a completely rewritten app in a few weeks, with a mental model that half engineers don't understand and don't have experience with. Maybe in a few years you will be forced to rewrite it in asyncio. And that would be the right time to do it: when it's a business demand, not when few perfectionists are annoyed.
edf13 4y ago
5% performance increase is worthwhile though?
- hbrn 4y ago
  Only if it's free. The costs were quite high: Time spend rewriting networking Code because more complicated: cooperation was moved to code instead of being handled by OS (to be fair this was way before asyncio, it was gevent). Code became alien to the team. I was the only one who knew how it worked. In the end, the only problem I solved was the problem we didn't have: "not using coroutines". Sure, threads suck at scale. But you should build things for scale when it's necessary, not when it's fun.