5 ms·

The problem with "green threads" is that their context switching is unpredictable. You never know where the context is switched, potentially confusing C APIs an

The problem with "green threads" is that their context switching is unpredictable. You never know where the context is switched, potentially confusing C APIs and such. Asyncio "threads", on the other hand, never switch context between "yield from" instructions. This makes it usable with the most bungled and complicated C APIs (e.g. Blender...), and also lets us reason more clearly about the concurrent behavior.

harlowja 11y ago
I'm going to go out on a limb and say this is a fallacy, you have just turned yourself (the programmer) into the context switching routine that the operating system (or python itself) does for you. IMHO computers are good (and can be made better) at scheduling and we should really avoid trying to do it ourselves manually... Getting used to thinking about where code can be context switched is IMHO worth more than shoving it under a rug and turning the programmer into the scheduler...
- bayesianhorse 11y ago
  It's not as bad as you think it is. First thing: Asyncio doesn't force the programmer to schedule anything. The event loop does that. Also, this kind of message-passing concurrency is not meant for cpu-intensive work, but rather for situations where there's nothing to do in a particular task much of the time. Concurrent threads/tasks in frameworks like NodeJs, Tornado and Asyncio are very good at doing nothing. For example in a web server, a traditionally threaded "worker" spends a lot of time waiting for a connection, waiting for database requests to finish, and so on. The operating system can use that time to execute other threads. But the memory of the thread sits idle during that time. Scheduling in asyncio is not imposed by the operating system or the programmer, and only partly by the event loop, but largely because of external circumstances like network latency/bandwith, how long do the database queries take, and how fast the user is acting.
  - harlowja 11y ago
    Who writes the 'yield from' or 'async' or @coroutine (yield points, aka context switching 'places', that u have to pick correctly)...? I thought that was the programmer ;-) As a comparison to glyph http://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/ http://techspot.zzzeek.org/2015/02/15/asynchronous-python-an... (the creator/author of sqlalchemy) is a pretty good read :-)
    - bayesianhorse 11y ago
      Declaring in which line to switch context does not a scheduler make. Also, "yield from" does not just mean context switching but also returns a coroutine's result. I have written quite a lot of asyncio and tornado code by now, and I can't remember a single instance where I had to yield/yield from/wait just because the computation would freeze up the thread or increase latency. Coroutines really aren't about explicit scheduling, at the very most you can specify what code is executed without switching. And the latter is absolutely necessary in some situations. For example, threads or greenlets freak out Blender.

harlowja 11y ago

I'm going to go out on a limb and say this is a fallacy, you have just turned yourself (the programmer) into the context switching routine that the operating system (or python itself) does for you. IMHO computers are good (and can be made better) at scheduling and we should really avoid trying to do it ourselves manually... Getting used to thinking about where code can be context switched is IMHO worth more than shoving it under a rug and turning the programmer into the scheduler...

bayesianhorse 11y ago
It's not as bad as you think it is. First thing: Asyncio doesn't force the programmer to schedule anything. The event loop does that. Also, this kind of message-passing concurrency is not meant for cpu-intensive work, but rather for situations where there's nothing to do in a particular task much of the time. Concurrent threads/tasks in frameworks like NodeJs, Tornado and Asyncio are very good at doing nothing. For example in a web server, a traditionally threaded "worker" spends a lot of time waiting for a connection, waiting for database requests to finish, and so on. The operating system can use that time to execute other threads. But the memory of the thread sits idle during that time. Scheduling in asyncio is not imposed by the operating system or the programmer, and only partly by the event loop, but largely because of external circumstances like network latency/bandwith, how long do the database queries take, and how fast the user is acting.
- harlowja 11y ago
  Who writes the 'yield from' or 'async' or @coroutine (yield points, aka context switching 'places', that u have to pick correctly)...? I thought that was the programmer ;-) As a comparison to glyph http://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/ http://techspot.zzzeek.org/2015/02/15/asynchronous-python-an... (the creator/author of sqlalchemy) is a pretty good read :-)
  - bayesianhorse 11y ago
    Declaring in which line to switch context does not a scheduler make. Also, "yield from" does not just mean context switching but also returns a coroutine's result. I have written quite a lot of asyncio and tornado code by now, and I can't remember a single instance where I had to yield/yield from/wait just because the computation would freeze up the thread or increase latency. Coroutines really aren't about explicit scheduling, at the very most you can specify what code is executed without switching. And the latter is absolutely necessary in some situations. For example, threads or greenlets freak out Blender.