3 ms·

Question for the peanut gallery: In a -nix OS system, let's say single-threaded process is running on core 0. It makes a syscall. Does core 0 have to context-sw

by kortex 5y ago

Question for the peanut gallery: In a -nix OS system, let's say single-threaded process is running on core 0. It makes a syscall. Does core 0 have to context-switch in order to handle the syscall, or could the kernel code run on core 1? This would allow a process to run longer without context switching and cache dumping, which would improve performance (as well as potentially security). Corollary question: could one make a kernel-specific core and would there be benefit to it? Handle all the I/O, interrupts and such, all with a dedicated, cloistered root-level cache.

LightG 5y ago
"A peanut gallery was, in the days of vaudeville, a nickname for the cheapest and ostensibly rowdiest seats in the theater, the occupants of which were often known to heckle the performers.[1] The least expensive snack served at the theatre would often be peanuts, which the patrons would sometimes throw at the performers on stage to convey their disapproval. Phrases such as "no comments from the peanut gallery" or "quiet in the peanut gallery" are extensions of the name.[1]" https://en.wikipedia.org/wiki/Peanut_gallery https://en.wikipedia.org/wiki/Peanut_gallery
- mousepilot 5y ago
  Posts like these are the gems that I come to HN for, as much as I've used the phrase, I never considered the source of it. I'd imagine I'd be in the cheap seats as well. Thanks!
benlivengood 5y ago
Cross-core interrupts and syscalls are slow. The caches for any memory to be transferred have to be flushed on the calling core and its APIC would have to interrupt the destination core to start executing the syscall, which means entering kernel mode on the calling core to get access to the APIC in the first place. If the goal is low latency then staying on a single core is almost always better. To effectively use multiple cores requires explicit synchronization such as io_uring which uses a lock-free ring buffer to transfer opcodes and results using shared buffers visible from userspace and the kernel. io_uring has an option to dedicate a kernel thread to servicing a particular ring buffer, and this can also be limited/expanded to a set of cores. I have zero experience with io_uring in practice and so I don't know what a good tradeoff is between servicing a ring buffer from multiple or single cores. The entries are lightweight and so cache coherency probably isn't too expensive and so for a high CPU workload that also needs high throughout allowing other cores to service IO probably makes sense. I think newer x86_64 chips also allow assigning interrupts from specific hardware to specific cores to effectively run kernel drivers mostly on a subset of cores, or to spread it to all cores under heavy I/O.
kccqzy 5y ago
If the system call is being handled by core 1, what does core 0 do? It must somehow wait for the core 1 system call to return. I'm not sure the waste created by waiting is outweighed by cache benefits.
- anticensor 5y ago
  Simple: It gets rescheduled to another process.

by kortex 5y ago

LightG 5y ago

"A peanut gallery was, in the days of vaudeville, a nickname for the cheapest and ostensibly rowdiest seats in the theater, the occupants of which were often known to heckle the performers.[1] The least expensive snack served at the theatre would often be peanuts, which the patrons would sometimes throw at the performers on stage to convey their disapproval. Phrases such as "no comments from the peanut gallery" or "quiet in the peanut gallery" are extensions of the name.[1]" https://en.wikipedia.org/wiki/Peanut_gallery https://en.wikipedia.org/wiki/Peanut_gallery

mousepilot 5y ago
Posts like these are the gems that I come to HN for, as much as I've used the phrase, I never considered the source of it. I'd imagine I'd be in the cheap seats as well. Thanks!

benlivengood 5y ago

Cross-core interrupts and syscalls are slow. The caches for any memory to be transferred have to be flushed on the calling core and its APIC would have to interrupt the destination core to start executing the syscall, which means entering kernel mode on the calling core to get access to the APIC in the first place. If the goal is low latency then staying on a single core is almost always better. To effectively use multiple cores requires explicit synchronization such as io_uring which uses a lock-free ring buffer to transfer opcodes and results using shared buffers visible from userspace and the kernel. io_uring has an option to dedicate a kernel thread to servicing a particular ring buffer, and this can also be limited/expanded to a set of cores. I have zero experience with io_uring in practice and so I don't know what a good tradeoff is between servicing a ring buffer from multiple or single cores. The entries are lightweight and so cache coherency probably isn't too expensive and so for a high CPU workload that also needs high throughout allowing other cores to service IO probably makes sense. I think newer x86_64 chips also allow assigning interrupts from specific hardware to specific cores to effectively run kernel drivers mostly on a subset of cores, or to spread it to all cores under heavy I/O.

kccqzy 5y ago

If the system call is being handled by core 1, what does core 0 do? It must somehow wait for the core 1 system call to return. I'm not sure the waste created by waiting is outweighed by cache benefits.

anticensor 5y ago
Simple: It gets rescheduled to another process.