5 ms·

A green thread implementation will thrash just as much in your hypothetical scenario. The only advantage of green threading is the capability to do problem-spe

A green thread implementation will thrash just as much in your hypothetical scenario. The only advantage of green threading is the capability to do problem-specific optimized scheduling. If you're not doing this then you don't need green threads.

akvadrako 5y ago
You are also optimize the scheduling for real threads, for example by increasing sched_latency_ns and sched_min_granularity_ns.
pm90 5y ago
> The only advantage of green threading is the capability to do problem-specific optimized scheduling. Isn’t the main advantage simply that you avoid making an absurd number of costly system calls?
- camnora 5y ago
  This article does a pretty good job of explaining how green events work. It seems that each of the threads will attempt to accept any/all incoming requests. https://rachelbythebay.com/w/2020/03/07/costly/ https://rachelbythebay.com/w/2020/03/07/costly/
  - dekhn 5y ago
    that's an implementation detail, mostly because Linux has taken too long to copy WaitForMultipleObjects https://stackoverflow.com/questions/13913882/what-is-the-equivalent-linux-api-for-waitformultipleobjects-and-wsaenumnetwork https://stackoverflow.com/questions/13913882/what-is-the-equ...
joconde 5y ago
> A green thread implementation will thrash just as much in your hypothetical scenario. The million threads risk slowing each other down with context switches, preventing any from finishing in time. An async event loop will focus on one connection at a time until `await`, and at least finish the tasks it attempts. I haven't been in this situation, but async seems to have real advantages here. With OS threads, you don't control scheduling without costly synchronisation primitives.
- otabdeveloper4 5y ago
  A context switch happens when a thread blocks on I/O. An async event loop will context switch when 'await' happens. (Which is, also, usually when it wants to wait for I/O.) You're assuming that an async event context switch is somehow vastly less costly than an OS context switch, which isn't true in the general case. (And unless you really went out of your way to make it happen then yours is the general case.)
  - foxfluff 5y ago
    IME async "context switches" are indeed vastly less costly, and I don't see why it wouldn't be. And I think calling it context switch is reading too much into it, when it's really just the program continuing on to operate on different data.
  - joconde 5y ago
    IIRC, coroutines are split internally into function calls, so switching to a task is actually just a function call from the event loop, in the same call stack. Threads have separate stacks. Surely this costs at least a little, and adds up with the number of threads?
    - akvadrako 5y ago
      It does cost a little RAM, but stack size is rarely the bottleneck, especially when using a language like Python.
      - joconde 5y ago
        But we have to switch from one stack to the other. If we do that more than necessary on thousands of threads, I imagine the cache will be used less efficiently. The problem is runtime, not RAM usage.
        akvadrako 5y ago
        In either case you need to load the context for other requests. It isn't like you need to load the whole stack either. Maybe using a stack means using more cache lines than coroutines, but you'd have to be right at the edge of capacity for active requests for it to matter.
isignal 5y ago
Agree that that kernel can handle many sleeping threads fine. The main problem with a million threads is that they all take up some fixed memory for stack. 1 MB/stack as a rough estimate. Go’s green threads use some kind of dynamic stack implementation to solve this problem. Dynamic stack requires a language specific implementation, for ex, with C pointers it would be difficult. Depending on how python C modules work, a similar solution would be needed to just offload to the kernel.
- akvadrako 5y ago
  Your threads won't be using 1MB per stack unless you are running deeply recursive algorithms, but then you'll be using lots of CPU. The kernel might allocate that much, but it's just virtual memory.

akvadrako 5y ago

You are also optimize the scheduling for real threads, for example by increasing sched_latency_ns and sched_min_granularity_ns.

pm90 5y ago

> The only advantage of green threading is the capability to do problem-specific optimized scheduling. Isn’t the main advantage simply that you avoid making an absurd number of costly system calls?

camnora 5y ago
This article does a pretty good job of explaining how green events work. It seems that each of the threads will attempt to accept any/all incoming requests. https://rachelbythebay.com/w/2020/03/07/costly/ https://rachelbythebay.com/w/2020/03/07/costly/
- dekhn 5y ago
  that's an implementation detail, mostly because Linux has taken too long to copy WaitForMultipleObjects https://stackoverflow.com/questions/13913882/what-is-the-equivalent-linux-api-for-waitformultipleobjects-and-wsaenumnetwork https://stackoverflow.com/questions/13913882/what-is-the-equ...

joconde 5y ago

> A green thread implementation will thrash just as much in your hypothetical scenario. The million threads risk slowing each other down with context switches, preventing any from finishing in time. An async event loop will focus on one connection at a time until `await`, and at least finish the tasks it attempts. I haven't been in this situation, but async seems to have real advantages here. With OS threads, you don't control scheduling without costly synchronisation primitives.

otabdeveloper4 5y ago
A context switch happens when a thread blocks on I/O. An async event loop will context switch when 'await' happens. (Which is, also, usually when it wants to wait for I/O.) You're assuming that an async event context switch is somehow vastly less costly than an OS context switch, which isn't true in the general case. (And unless you really went out of your way to make it happen then yours is the general case.)
- foxfluff 5y ago
  IME async "context switches" are indeed vastly less costly, and I don't see why it wouldn't be. And I think calling it context switch is reading too much into it, when it's really just the program continuing on to operate on different data.
- joconde 5y ago
  IIRC, coroutines are split internally into function calls, so switching to a task is actually just a function call from the event loop, in the same call stack. Threads have separate stacks. Surely this costs at least a little, and adds up with the number of threads?
  - akvadrako 5y ago
    It does cost a little RAM, but stack size is rarely the bottleneck, especially when using a language like Python.
    - joconde 5y ago
      But we have to switch from one stack to the other. If we do that more than necessary on thousands of threads, I imagine the cache will be used less efficiently. The problem is runtime, not RAM usage.
      - akvadrako 5y ago
        In either case you need to load the context for other requests. It isn't like you need to load the whole stack either. Maybe using a stack means using more cache lines than coroutines, but you'd have to be right at the edge of capacity for active requests for it to matter.

isignal 5y ago

Agree that that kernel can handle many sleeping threads fine. The main problem with a million threads is that they all take up some fixed memory for stack. 1 MB/stack as a rough estimate. Go’s green threads use some kind of dynamic stack implementation to solve this problem. Dynamic stack requires a language specific implementation, for ex, with C pointers it would be difficult. Depending on how python C modules work, a similar solution would be needed to just offload to the kernel.

akvadrako 5y ago
Your threads won't be using 1MB per stack unless you are running deeply recursive algorithms, but then you'll be using lots of CPU. The kernel might allocate that much, but it's just virtual memory.