3 ms·
Somewhat controversial take: the current threads implementation is usually already performant enough for most use cases. The actual reason why we don't use them
by nasretdinov 2y ago
Somewhat controversial take: the current threads implementation is usually already performant enough for most use cases. The actual reason why we don't use them to handle more than a few thousand concurrent operations is that, at least in Linux, threads are scheduled and treated very similarly to processes. E.g. if a single process with 3000 threads gets bottlenecked on some syscall, etc, your system load average will become 3000, and it will essentially lead to no other processes being able to run well on the same machine.
Another issue with threads performance is that they are visible to most system tools like `ps`, and thus having too mamy threads starts to affect operations _outside_ the kernel, e.g. many monitoring tools, etc.
So that's the main reason why user-space scheduling became so popular: it hides the "threads" from the system, allowing for processes to be scheduled more fairly (preventing stuff like reaching LA 3000 when writing to 3000 parallel connections), and not affecting performance of the system infrastructure around the kernel.
BTW the threads stacks, as well as everything else in Linux are allocated lazily, so if you only use like 4Kb of stack in the thread it wouldn't lead to RSS of full 8M. It will contribute to VMEM, but not RSS
- viraptor 2y ago> your system load average will become 3000, and it will essentially lead to no other processes being able to run well on the same machine. That doesn't sound right. I mean, there are many schedulers available, but I was under the impression that most have a separate blocked queue. (Or more specifically, anything blocked will not be in the runnable pool - those will be dequeued) I.e. anything waiting on a syscall will be mostly ignited. (Please correct me if I'm misunderstanding the usual behaviour here)
- nasretdinov 2y agoYeah I think you are right, it depends on the syscalls, etc, in question. When that happened I didn't check which exact syscalls were being used
- mike_hearn 2y agoLinux ps doesn't show threads (by default). That was true decades ago before NPTL but hasn't been the case for a long time. The reason not to use kernel threads for everything is RAM utilization.