4 ms·

> A green thread implementation will thrash just as much in your hypothetical scenario. The million threads risk slowing each other down with context switches,

> A green thread implementation will thrash just as much in your hypothetical scenario. The million threads risk slowing each other down with context switches, preventing any from finishing in time. An async event loop will focus on one connection at a time until `await`, and at least finish the tasks it attempts. I haven't been in this situation, but async seems to have real advantages here. With OS threads, you don't control scheduling without costly synchronisation primitives.

otabdeveloper4 5y ago
A context switch happens when a thread blocks on I/O. An async event loop will context switch when 'await' happens. (Which is, also, usually when it wants to wait for I/O.) You're assuming that an async event context switch is somehow vastly less costly than an OS context switch, which isn't true in the general case. (And unless you really went out of your way to make it happen then yours is the general case.)
- foxfluff 5y ago
  IME async "context switches" are indeed vastly less costly, and I don't see why it wouldn't be. And I think calling it context switch is reading too much into it, when it's really just the program continuing on to operate on different data.
- joconde 5y ago
  IIRC, coroutines are split internally into function calls, so switching to a task is actually just a function call from the event loop, in the same call stack. Threads have separate stacks. Surely this costs at least a little, and adds up with the number of threads?
  - akvadrako 5y ago
    It does cost a little RAM, but stack size is rarely the bottleneck, especially when using a language like Python.
    - joconde 5y ago
      But we have to switch from one stack to the other. If we do that more than necessary on thousands of threads, I imagine the cache will be used less efficiently. The problem is runtime, not RAM usage.
      - akvadrako 5y ago
        In either case you need to load the context for other requests. It isn't like you need to load the whole stack either. Maybe using a stack means using more cache lines than coroutines, but you'd have to be right at the edge of capacity for active requests for it to matter.
        joconde 5y ago
        > In either case you need to load the context for other requests. For async tasks, it means return, then call. For threads, it means moving to an entirely different stack, somewhere else in memory. “Loading context” is more expensive in the latter case. > Maybe using a stack means using more cache lines than coroutines, but you'd have to be right at the edge of capacity for active requests for it to matter. Why? Cache doesn’t only speed things up when capacity is full. Anything we have to reload from RAM will take time to load.
        akvadrako 5y ago
        To do anything with a request you are going to have a context which will function just like the stack of a thread. The only question is how many cache lines each takes. It only matters if you are at capacity because otherwise the active requests will be cached.
        joconde 5y ago
        > To do anything with a request you are going to have a context which will function just like the stack of a thread. But a thread has a separate call stack. Returning and calling another function in the same stack just involves moving the stack pointer by a few bytes. Switching to another thread’s stack will invalidate much more cache, while running an async task until await will make the best use of the existing cache, and switch (much closer in RAM than in another thread) only when it makes sense, i.e. the previous task started waiting on something. > It only matters if you are at capacity because otherwise the active requests will be cached. CPU cache is a few megabytes. That definitely won’t hold hundreds of requests if they all transfer substantial data, even if you can handle thousands or more.
- ris 5y ago
  > You're assuming that an async event context switch is somehow vastly less costly than an OS context switch They are, and this is the entire premise green threads are built on. An OS context switch has to do a lot more including switching page tables and possibly performing a TLB flush. (I do agree that the majority of python projects will get by just fine with regular threads though)

otabdeveloper4 5y ago

A context switch happens when a thread blocks on I/O. An async event loop will context switch when 'await' happens. (Which is, also, usually when it wants to wait for I/O.) You're assuming that an async event context switch is somehow vastly less costly than an OS context switch, which isn't true in the general case. (And unless you really went out of your way to make it happen then yours is the general case.)

foxfluff 5y ago
IME async "context switches" are indeed vastly less costly, and I don't see why it wouldn't be. And I think calling it context switch is reading too much into it, when it's really just the program continuing on to operate on different data.
joconde 5y ago
IIRC, coroutines are split internally into function calls, so switching to a task is actually just a function call from the event loop, in the same call stack. Threads have separate stacks. Surely this costs at least a little, and adds up with the number of threads?
- akvadrako 5y ago
  It does cost a little RAM, but stack size is rarely the bottleneck, especially when using a language like Python.
  - joconde 5y ago
    But we have to switch from one stack to the other. If we do that more than necessary on thousands of threads, I imagine the cache will be used less efficiently. The problem is runtime, not RAM usage.
    - akvadrako 5y ago
      In either case you need to load the context for other requests. It isn't like you need to load the whole stack either. Maybe using a stack means using more cache lines than coroutines, but you'd have to be right at the edge of capacity for active requests for it to matter.
      - joconde 5y ago
        > In either case you need to load the context for other requests. For async tasks, it means return, then call. For threads, it means moving to an entirely different stack, somewhere else in memory. “Loading context” is more expensive in the latter case. > Maybe using a stack means using more cache lines than coroutines, but you'd have to be right at the edge of capacity for active requests for it to matter. Why? Cache doesn’t only speed things up when capacity is full. Anything we have to reload from RAM will take time to load.
        akvadrako 5y ago
        To do anything with a request you are going to have a context which will function just like the stack of a thread. The only question is how many cache lines each takes. It only matters if you are at capacity because otherwise the active requests will be cached.
        joconde 5y ago
        > To do anything with a request you are going to have a context which will function just like the stack of a thread. But a thread has a separate call stack. Returning and calling another function in the same stack just involves moving the stack pointer by a few bytes. Switching to another thread’s stack will invalidate much more cache, while running an async task until await will make the best use of the existing cache, and switch (much closer in RAM than in another thread) only when it makes sense, i.e. the previous task started waiting on something. > It only matters if you are at capacity because otherwise the active requests will be cached. CPU cache is a few megabytes. That definitely won’t hold hundreds of requests if they all transfer substantial data, even if you can handle thousands or more.
ris 5y ago
> You're assuming that an async event context switch is somehow vastly less costly than an OS context switch They are, and this is the entire premise green threads are built on. An OS context switch has to do a lot more including switching page tables and possibly performing a TLB flush. (I do agree that the majority of python projects will get by just fine with regular threads though)