3 ms·

> I like the thread per core design because kernel context switches are expensive You can do a millions to tens of millions kernel entries/exits per second per

by ori_b 3y ago

> I like the thread per core design because kernel context switches are expensive You can do a millions to tens of millions kernel entries/exits per second per core. They're on the same order of magnitude as a single digit number of full cache misses. So, expensive, but a lot less expensive than many people assume. Edit: for some (outdated, 2018) measurements, https://eli.thegreenplace.net/2018/measuring-context-switching-and-memory-overheads-for-linux-threads/ https://eli.thegreenplace.net/2018/measuring-context-switchi...

cmrdporcupine 3y ago
And all our assumptions get blown up with every new hardware generation, too. Hardware engineers are building systems for the applications of today. And the kernel developers are optimizing as well. It's all a moving target. Dogma has no place here.
kldx 3y ago
That's an interesting view and one I agree with. Surely if context switch = expensive is common knowledge for a long time in computing, the kernel people and CPU companies would have tried optimizing it to the fullest, no?
- insanitybit 3y ago
  Syscalls have gotten quite a lot slower after Specter/Meltdown mitigations. Though I suspect that if you're writing a web service that wants TPC you should just disable these - I've told that to Scylla Cloud before, they have no reason to keep them enabled and plenty of reasons to disable them. The other thing is 'io_uring', which is trying to come at the problem by removing context switching altogether by providing a cheap primitive for communicating with the kernel.
  - kldx 3y ago
    Sadly io_uring has its share of vulnerabilities as well
    - insanitybit 3y ago
      Of course, I'm very aware of that fact. But it is designed to minimize system calls exactly because they can add significant overhead.
amluto 3y ago
You’re replying to a comment about context switches, not syscalls. Linux on x86 cannot do tens of millions of context switches per second per core. (I expect that, at high clock speed, carefully tuned, under ideal and possibly unrealistic conditions, you might get 2M/s — it’s been a while since I benchmarked this.) But this is silly — shared nothing will have no substantial difference in the rate of context switches (or of syscalls) compared to a thread-per-core-shared-everything architecture. The architecture that actually loses is many-threads-per-core if it ends up in a small-batch mode in which it context switches once per request or so.
- deleted 3y ago
  [deleted]
TexasMick 3y ago
I have seen context switches be a killer in some embedded contexts. Especially on FPGA soft core applications where you often are dealing with heavy I/O, if you just naively spin up threads for I/O then you will suffer heavily.

by ori_b 3y ago

cmrdporcupine 3y ago

And all our assumptions get blown up with every new hardware generation, too. Hardware engineers are building systems for the applications of today. And the kernel developers are optimizing as well. It's all a moving target. Dogma has no place here.

kldx 3y ago

That's an interesting view and one I agree with. Surely if context switch = expensive is common knowledge for a long time in computing, the kernel people and CPU companies would have tried optimizing it to the fullest, no?

insanitybit 3y ago
Syscalls have gotten quite a lot slower after Specter/Meltdown mitigations. Though I suspect that if you're writing a web service that wants TPC you should just disable these - I've told that to Scylla Cloud before, they have no reason to keep them enabled and plenty of reasons to disable them. The other thing is 'io_uring', which is trying to come at the problem by removing context switching altogether by providing a cheap primitive for communicating with the kernel.
- kldx 3y ago
  Sadly io_uring has its share of vulnerabilities as well
  - insanitybit 3y ago
    Of course, I'm very aware of that fact. But it is designed to minimize system calls exactly because they can add significant overhead.

amluto 3y ago

You’re replying to a comment about context switches, not syscalls. Linux on x86 cannot do tens of millions of context switches per second per core. (I expect that, at high clock speed, carefully tuned, under ideal and possibly unrealistic conditions, you might get 2M/s — it’s been a while since I benchmarked this.) But this is silly — shared nothing will have no substantial difference in the rate of context switches (or of syscalls) compared to a thread-per-core-shared-everything architecture. The architecture that actually loses is many-threads-per-core if it ends up in a small-batch mode in which it context switches once per request or so.

deleted 3y ago
[deleted]

TexasMick 3y ago

I have seen context switches be a killer in some embedded contexts. Especially on FPGA soft core applications where you often are dealing with heavy I/O, if you just naively spin up threads for I/O then you will suffer heavily.