4 ms·
Cross-core interrupts and syscalls are slow. The caches for any memory to be transferred have to be flushed on the calling core and its APIC would have to inte
by benlivengood 5y ago
Cross-core interrupts and syscalls are slow. The caches for any memory to be transferred have to be flushed on the calling core and its APIC would have to interrupt the destination core to start executing the syscall, which means entering kernel mode on the calling core to get access to the APIC in the first place.
If the goal is low latency then staying on a single core is almost always better. To effectively use multiple cores requires explicit synchronization such as io_uring which uses a lock-free ring buffer to transfer opcodes and results using shared buffers visible from userspace and the kernel. io_uring has an option to dedicate a kernel thread to servicing a particular ring buffer, and this can also be limited/expanded to a set of cores. I have zero experience with io_uring in practice and so I don't know what a good tradeoff is between servicing a ring buffer from multiple or single cores. The entries are lightweight and so cache coherency probably isn't too expensive and so for a high CPU workload that also needs high throughout allowing other cores to service IO probably makes sense.
I think newer x86_64 chips also allow assigning interrupts from specific hardware to specific cores to effectively run kernel drivers mostly on a subset of cores, or to spread it to all cores under heavy I/O.