3 ms·

Kernel threading does scale to the number of threads applications need these days. What can be slow are context switches, but they aren't slow in absolute term

Kernel threading does scale to the number of threads applications need these days. What can be slow are context switches, but they aren't slow in absolute terms. The vast majority of applications, including Web servers, are perfectly fine with 1:1 threading.

dozzie 9y ago
It doesn't really scale. Kernel level threads are quite expensive in the terms of memory. Erlang's processes take up few hundred bytes. It's rare to have dozen thousands of kernel-level threads running on Linux, while it's quite common for Erlang servers to have that many processes.
- api 9y ago
  Anyone know if this is better in Alpine Linux with musl libc and smaller stacks?
  - dozzie 9y ago
    It's not really up to a distribution to lessen this cost. Large part of it is what task descriptor in the kernel takes. Also, if nothing else, you'll run out of PID numbers, as they're usually still 16 bits, even today, though there was a kernel compilation option to change that, from what I remember.
    - bluetech 9y ago
      It can be changed at runtime. From proc(5), system wide limits: /proc/sys/kernel/pid_max > PID_MAX_LIMIT, approximately 4 million /proc/sys/kernel/threads-max > FUTEX_TID_MASK (0x3fffffff), [approximately 1 billion] For per-process limit, increase RLIMIT_NPROC.
    - pcwalton 9y ago
      > Large part of it is what task descriptor in the kernel takes. 4K for kernel stacks, only 2K with future work. That's really not much space at all if you're doing anything interesting with those threads. > Also, if nothing else, you'll run out of PID numbers, as they're usually still 16 bits, even today Not for a very long time.
pgwhalen 9y ago
I wrote a server application that runs with about a million goroutines and performs quite well. It's not a webserver, but it responds to "requests" within hundreds of micros. Surely this would not be possible with OS threads.
- deleted 9y ago
  [deleted]

dozzie 9y ago

It doesn't really scale. Kernel level threads are quite expensive in the terms of memory. Erlang's processes take up few hundred bytes. It's rare to have dozen thousands of kernel-level threads running on Linux, while it's quite common for Erlang servers to have that many processes.

api 9y ago
Anyone know if this is better in Alpine Linux with musl libc and smaller stacks?
- dozzie 9y ago
  It's not really up to a distribution to lessen this cost. Large part of it is what task descriptor in the kernel takes. Also, if nothing else, you'll run out of PID numbers, as they're usually still 16 bits, even today, though there was a kernel compilation option to change that, from what I remember.
  - bluetech 9y ago
    It can be changed at runtime. From proc(5), system wide limits: /proc/sys/kernel/pid_max > PID_MAX_LIMIT, approximately 4 million /proc/sys/kernel/threads-max > FUTEX_TID_MASK (0x3fffffff), [approximately 1 billion] For per-process limit, increase RLIMIT_NPROC.
  - pcwalton 9y ago
    > Large part of it is what task descriptor in the kernel takes. 4K for kernel stacks, only 2K with future work. That's really not much space at all if you're doing anything interesting with those threads. > Also, if nothing else, you'll run out of PID numbers, as they're usually still 16 bits, even today Not for a very long time.

pgwhalen 9y ago

I wrote a server application that runs with about a million goroutines and performs quite well. It's not a webserver, but it responds to "requests" within hundreds of micros. Surely this would not be possible with OS threads.

deleted 9y ago
[deleted]