4 ms·

Great reminder. Even if you pre-create a thread (thread pool), when the task is small enough (less than 1,000 cycles), it is less expensive to do it in place (

Great reminder. Even if you pre-create a thread (thread pool), when the task is small enough (less than 1,000 cycles), it is less expensive to do it in place (for example, with fibers), because of the cost of context switching.

mac01021 7y ago
Why is the cost of switching threads so much higher than the cost of switching fibers?
- fwsgonzo 7y ago
  Because threads are traditionally created and scheduled by the OS, so it inevitably involves a costly context switch both first into the kernel and then back again into the next thread, if one is ready. Userspace threads are more light-weight, but probably still worse than just using fibers and co-routines. Depends on your needs, I suppose.
- gpderetta 7y ago
  Switching threads require entering the kernel which costs from a few hundreds to thousands of clock cycles (and it got worse from all the spectre/meltdown mitigations). A fiber switch can be done in less than 10 clock cycles.
iforgotpassword 7y ago
Agree. A few years ago I noticed a C program we used in production spawned a new thread for each incoming connection. Since the vast majority of these just served two small requests (think two HTTP gets) I tried adding a very simple thread pool that would keep up to four idle threads around. To make a thread wait for work I used an eventfd (Linux). I tried a linked list and an array for the idle threads. I tried protecting the get/return code with a mutex and spin lock, and then made it lock free with C11s atomics. Two days later I still couldn't get this to be faster than just spawning a new thread every time, so I gave up this experiment. It seems at least the Linux folks optimized the crap out of clone() over the last years.
- cesarb 7y ago
  > It seems at least the Linux folks optimized the crap out of clone() over the last years. The most essential Linux benchmark is compiling the Linux kernel (since it's something the Linux kernel developers do all the time, so they really feel the impact). The clone() system call is used both to create new threads and to create new processes, and the Linux kernel compilation uses a large amount of short-lived processes (each C file is a new C compiler process). It's only natural that clone() is heavily optimized, together with the filesystem caches (each new C compiler process reads the source code files from scratch).
- proverbialbunny 7y ago
  I imagine this is why coroutines and the like are often used as threads within a thread pool.
- frankzinger 7y ago
  Thread spawning should only become a problem for very high concurrent client counts (ie, large N). How many concurrent clients did this program have? When I ran benchmarks to compare a thread-per-client model to a single-threaded, event-based one, the single-threaded throughput was around 2 to 3 times higher for as few as 1000 clients.
  - iforgotpassword 7y ago
    Not many, slightly above 800 on average iirc. It wasn't even a bottleneck or anything, I was just curious how much impact it would make. That's also why I didn't bother trying to change it to event based with single thread or one thread per CPU core, as that would've been way more work than two afternoons so just not justified.

mac01021 7y ago

Why is the cost of switching threads so much higher than the cost of switching fibers?

fwsgonzo 7y ago
Because threads are traditionally created and scheduled by the OS, so it inevitably involves a costly context switch both first into the kernel and then back again into the next thread, if one is ready. Userspace threads are more light-weight, but probably still worse than just using fibers and co-routines. Depends on your needs, I suppose.
gpderetta 7y ago
Switching threads require entering the kernel which costs from a few hundreds to thousands of clock cycles (and it got worse from all the spectre/meltdown mitigations). A fiber switch can be done in less than 10 clock cycles.

iforgotpassword 7y ago

Agree. A few years ago I noticed a C program we used in production spawned a new thread for each incoming connection. Since the vast majority of these just served two small requests (think two HTTP gets) I tried adding a very simple thread pool that would keep up to four idle threads around. To make a thread wait for work I used an eventfd (Linux). I tried a linked list and an array for the idle threads. I tried protecting the get/return code with a mutex and spin lock, and then made it lock free with C11s atomics. Two days later I still couldn't get this to be faster than just spawning a new thread every time, so I gave up this experiment. It seems at least the Linux folks optimized the crap out of clone() over the last years.

cesarb 7y ago
> It seems at least the Linux folks optimized the crap out of clone() over the last years. The most essential Linux benchmark is compiling the Linux kernel (since it's something the Linux kernel developers do all the time, so they really feel the impact). The clone() system call is used both to create new threads and to create new processes, and the Linux kernel compilation uses a large amount of short-lived processes (each C file is a new C compiler process). It's only natural that clone() is heavily optimized, together with the filesystem caches (each new C compiler process reads the source code files from scratch).
proverbialbunny 7y ago
I imagine this is why coroutines and the like are often used as threads within a thread pool.
frankzinger 7y ago
Thread spawning should only become a problem for very high concurrent client counts (ie, large N). How many concurrent clients did this program have? When I ran benchmarks to compare a thread-per-client model to a single-threaded, event-based one, the single-threaded throughput was around 2 to 3 times higher for as few as 1000 clients.
- iforgotpassword 7y ago
  Not many, slightly above 800 on average iirc. It wasn't even a bottleneck or anything, I was just curious how much impact it would make. That's also why I didn't bother trying to change it to event based with single thread or one thread per CPU core, as that would've been way more work than two afternoons so just not justified.