4 ms·

> In an ideal world, the number of threads in a system would be the same as the number of logical processors ... no. Not if you rely on blocking system calls.

> In an ideal world, the number of threads in a system would be the same as the number of logical processors ... no. Not if you rely on blocking system calls. Or wait for anything that isn't the CPU. Or run more than one application per CPU on your system. Or manage concurrent code flows at any level higher than source code.

tcfhgj 5y ago
in an ideal world you don't use blocking system calls?
- colanderman 5y ago
  Yep, that's a great goal, and I/O-intesive applications do this, managing concurrent workflows with custom user-mode schedulers. But now you have more than one application on the machine? Or even, code from multiple vendors which all need to issue I/O concurrently? How do you integrate their work with your custom scheduler? You need some centralized, system-wide scheduler everyone agrees on. That's what threads are.
  - tcfhgj 5y ago
    Does it make sense to ask for a system wide scheduler that manages tasks (tasks in terms of the state machine behind async await) instead of threads? Currently every app that uses tasks has it's own executor with a corresponding thread pool, which may be more efficiently used if it was shared by using a single async executor across all processes. of course, it requires to agree on one executor which doesn't sound realistic, but would it make sense in general?
    - colanderman 5y ago
      This is essentially how an Erlang system works. There's a single executor with one "CPU" thread per CPU, and a bunch of "I/O" threads, the sum of which are shared among all Erlang applications executing under the runtime. Erlang can get away with this because it's interpreted. But also, this is essentially what threads are... the main advantage of threads being, they coordinate raw assembly code, instead of something interpreted or JITted. OS schedulers are pretty well optimized by this point. The major overheads relate to maintaining security boundaries (e.g. swapping out the TLBs on context switch) and the cost of starting up a thread. Just having threads sitting around doesn't really cost anything.
      - toast0 5y ago
        > Erlang can get away with this because it's interpreted. Erlang gets away with this because of its programming model; it doesn't matter if it's interpreted or JITed or if you managed to compile it ahead of time. The relevant parts of the programming model are that any function call can yield, and because it's a functional language, there's no looping without function calls, so you can't stay busy forever without yielding. The value of green thread over native threads is the difference in numbers. I haven't tried, but I imagine most OSes would fall over if you did a native thread per connection and you had a million connections; some OSes would probably hit a thread limit before then. Of course, if you've got a million connections, you're probably starting many per second, so startup cost is important (although maybe you could do a pool if it's cheaper to clean up an otherwise dying thread than kill it and start a new one) Edit: read the article; that was a weird rant; I'm sure there are better things to rant about on an idle windows machine (like how much disk I/O they do... especially on spinning drives with Windows 10)
    - Findecanor 5y ago
      Microsoft's research OS Midori supposedly did everything with async/await and got down to about one thread per processor (or at least one thread per process and processor). In Midori all programs were written in a special dialect of C#. Safety would have been ensured by the language, but all programs it actually ran were probably also written by very competent programmers (which doesn't happen in real life ...). If I understand things correctly, the language would have imposed an early example of "Structured Concurrency" (although that term hadn't been invented yet) <http://joeduffyblog.com/2015/11/19/asynchronous-everything/ http://joeduffyblog.com/2015/11/19/asynchronous-everything/>
- astrange 5y ago
  There's some fine reasons to use blocking I/O. Often you don't have anything to do until it comes back, so it's the most natural way to write it. For a very short system call, it's faster to make it blocking because you can resume the thread instantly without the overhead of however you send/handle an async reply. It avoids space leaks, because a blocking call will… block if it can't make progress, so you can't send an infinite amount of async calls. And it means backtraces in gdb work.
jayd16 5y ago
It's ideal in the sense that you don't need to block to wait and you can avoid context switches. But obviously that means abandoning some common patterns.
- colanderman 5y ago
  "Common patterns" including, running more than one application on a system. I've worked on several thread-per-core systems, including embedded systems in which each thread owns a core and does not make system calls or context switch. You can do that, and it's ideal performance-wise, but it does not apply to multi-process systems -- how else are they to coordinate workflows, than through kernel-visible concurrency constructs: i.e., threads? In language implementations with built-in user-mode schedulers like Go and Erlang, you get closer to this ideal for, as they will spin up a thread per core for CPU-intensive workloads. But even there, blocking system calls are still the norm (at least with Linux, which is only very recently moving toward a generic async system call system, with io_uring), requiring many more threads per core to saturate I/O workloads. And that still doesn't address the multi-process coordination problem.
  - jayd16 5y ago
    All perfectly valid but I think you're letting perfect be the enemy of good here. Even if your app can be preempted, green threading is still useful.
    - colanderman 5y ago
      I don't disagree. But green threading doesn't address the OP's issue of "too many threads": a process using green threads needs to spawn one thread per CPU if it plans to use the CPU's full power. On say, a 16-core system like the OP's, you're back to 16 threads per process -- on average, more than the OP was complaining about (15).

tcfhgj 5y ago

in an ideal world you don't use blocking system calls?

colanderman 5y ago
Yep, that's a great goal, and I/O-intesive applications do this, managing concurrent workflows with custom user-mode schedulers. But now you have more than one application on the machine? Or even, code from multiple vendors which all need to issue I/O concurrently? How do you integrate their work with your custom scheduler? You need some centralized, system-wide scheduler everyone agrees on. That's what threads are.
- tcfhgj 5y ago
  Does it make sense to ask for a system wide scheduler that manages tasks (tasks in terms of the state machine behind async await) instead of threads? Currently every app that uses tasks has it's own executor with a corresponding thread pool, which may be more efficiently used if it was shared by using a single async executor across all processes. of course, it requires to agree on one executor which doesn't sound realistic, but would it make sense in general?
  - colanderman 5y ago
    This is essentially how an Erlang system works. There's a single executor with one "CPU" thread per CPU, and a bunch of "I/O" threads, the sum of which are shared among all Erlang applications executing under the runtime. Erlang can get away with this because it's interpreted. But also, this is essentially what threads are... the main advantage of threads being, they coordinate raw assembly code, instead of something interpreted or JITted. OS schedulers are pretty well optimized by this point. The major overheads relate to maintaining security boundaries (e.g. swapping out the TLBs on context switch) and the cost of starting up a thread. Just having threads sitting around doesn't really cost anything.
    - toast0 5y ago
      > Erlang can get away with this because it's interpreted. Erlang gets away with this because of its programming model; it doesn't matter if it's interpreted or JITed or if you managed to compile it ahead of time. The relevant parts of the programming model are that any function call can yield, and because it's a functional language, there's no looping without function calls, so you can't stay busy forever without yielding. The value of green thread over native threads is the difference in numbers. I haven't tried, but I imagine most OSes would fall over if you did a native thread per connection and you had a million connections; some OSes would probably hit a thread limit before then. Of course, if you've got a million connections, you're probably starting many per second, so startup cost is important (although maybe you could do a pool if it's cheaper to clean up an otherwise dying thread than kill it and start a new one) Edit: read the article; that was a weird rant; I'm sure there are better things to rant about on an idle windows machine (like how much disk I/O they do... especially on spinning drives with Windows 10)
  - Findecanor 5y ago
    Microsoft's research OS Midori supposedly did everything with async/await and got down to about one thread per processor (or at least one thread per process and processor). In Midori all programs were written in a special dialect of C#. Safety would have been ensured by the language, but all programs it actually ran were probably also written by very competent programmers (which doesn't happen in real life ...). If I understand things correctly, the language would have imposed an early example of "Structured Concurrency" (although that term hadn't been invented yet) <http://joeduffyblog.com/2015/11/19/asynchronous-everything/ http://joeduffyblog.com/2015/11/19/asynchronous-everything/>
astrange 5y ago
There's some fine reasons to use blocking I/O. Often you don't have anything to do until it comes back, so it's the most natural way to write it. For a very short system call, it's faster to make it blocking because you can resume the thread instantly without the overhead of however you send/handle an async reply. It avoids space leaks, because a blocking call will… block if it can't make progress, so you can't send an infinite amount of async calls. And it means backtraces in gdb work.

jayd16 5y ago

It's ideal in the sense that you don't need to block to wait and you can avoid context switches. But obviously that means abandoning some common patterns.

colanderman 5y ago
"Common patterns" including, running more than one application on a system. I've worked on several thread-per-core systems, including embedded systems in which each thread owns a core and does not make system calls or context switch. You can do that, and it's ideal performance-wise, but it does not apply to multi-process systems -- how else are they to coordinate workflows, than through kernel-visible concurrency constructs: i.e., threads? In language implementations with built-in user-mode schedulers like Go and Erlang, you get closer to this ideal for, as they will spin up a thread per core for CPU-intensive workloads. But even there, blocking system calls are still the norm (at least with Linux, which is only very recently moving toward a generic async system call system, with io_uring), requiring many more threads per core to saturate I/O workloads. And that still doesn't address the multi-process coordination problem.
- jayd16 5y ago
  All perfectly valid but I think you're letting perfect be the enemy of good here. Even if your app can be preempted, green threading is still useful.
  - colanderman 5y ago
    I don't disagree. But green threading doesn't address the OP's issue of "too many threads": a process using green threads needs to spawn one thread per CPU if it plans to use the CPU's full power. On say, a 16-core system like the OP's, you're back to 16 threads per process -- on average, more than the OP was complaining about (15).