3 ms·
Whether a single server can handle a million threads depends on how often those threads need to wake up, and how much work they need to do before going back to
by wtallis 5y ago
Whether a single server can handle a million threads depends on how often those threads need to wake up, and how much work they need to do before going back to sleep. Thread pools don't even come close to working well for storage IO, because dozens or hundreds of threads per core each blocking on just a few kB of IO that the SSDs can handle in milliseconds or less means you spend far too much time on context switches. Network IO is usually lower in aggregate throughput and much higher in latency, so each thread spends relatively more time sleeping and therefore you can fit more of them into a single machine's available CPU time. But a million threads on one box imposes significant constraints on how active those threads can be.
- ai-dev 5y ago> But a million threads on one box imposes significant constraints on how active those threads can be. Wouldn't the same constraints be imposed on a million async tasks? What allows async tasks to be more active than threads? Is it the scheduling model, the overhead of context switching, or something else? EDIT: I reread and saw that you mentioned context switching. I guess my question would then be the same as here: https://news.ycombinator.com/item?id=29452759 https://news.ycombinator.com/item?id=29452759. Is the claim that the context switching overhead when due to I/O readiness the same true? I'm mostly thinking about network servers, where I think most? context switches would be due to I/O readiness.
- xxpor 5y agoIf nothing else, a green thread context switch is much cheaper than a real context switch because you don't have the security overhead of flushing caches/tlb/etc