5 ms·

I haven't seen a benchmark of huge numbers of native threads vs. a userland scheduler, but I have a hard time imagining that a userland scheduler will beat the

by ansible 10y ago

I haven't seen a benchmark of huge numbers of native threads vs. a userland scheduler, but I have a hard time imagining that a userland scheduler will beat the kernel's scheduler. The kernel scheduler has a much more global picture of the system compared to userland. Doesn't using kernel threads imply lots of context switches? Doesn't that tend to be expensive in terms of time on modern architectures?

pcwalton 10y ago
> Doesn't using kernel threads imply lots of context switches? No, it's actually fewer context switches. That's because each I/O completion event goes straight from the kernel to the code that was waiting on it (1 context switch), not from the kernel to the userland dispatcher to the code that was waiting on it (2 context switches).
- anonymousDan 10y ago
  What if you have async system calls (e.g. FlexSC)?
- wbl 10y ago
  This is not quite right. The userland dispatcher to run next thread is not a context switch: the TLB doesn't need to get flushed for instance. Furthermore many userland threads can be woken at once, and don't incur context switches when they suspend, prompting the next runnable thread to run. If you're willing to put in a lot of compiler work userland thread switching is a function call, literally. I don't 1:1 threads with small stack are going to compete with that.
- naasking 10y ago
  The real cost of a true "context switch" is the transition from user level to kernel level, which takes thousands of cycles. This cost isn't incurred on a userland context switch, so those costs aren't comparable.
  - kijiki 10y ago
    Thousands of cycles for a SYSCALL/SYSRET on a reasonably modern Intel/AMD CPU? I think you should try to measure that.
    - DblPlusUngood 10y ago
      "Thousands of cycles" is definitely not true on modern CPUs. SYSENTER/SYSEXIT takes 150 cycles on my Xeon X3460.
    - naasking 10y ago
      My knowledge of current timings are admittedly a little out of date since my microkernel days are behind me. Even the 150 cycles another poster suggested is still a significant cost over a userland context switch.
    - derefr 10y ago
      Given the kind of "cloud"-backed startups most of HN are working on, I think the more practical measurement would be the overhead of a ring 0/3 separation in a VM, plus hypercalls, plus a ring 0/3 separation in the hypervisor (for paravirtualized syscalls); averaged against a ring 0/3 separation in a VM, plus SR-IOV virtualization (for HVM-backed syscalls.) Yeah, this doesn't apply for context-switches that happen because of plain old pre-emption, but it does happen if the context-switch is because e.g. a network packet wants to arrive at another process in your VM than the one that's currently running. I would imagine that there's a reason bare-metal IaaS providers have a business model. :)

by ansible 10y ago

pcwalton 10y ago

> Doesn't using kernel threads imply lots of context switches? No, it's actually fewer context switches. That's because each I/O completion event goes straight from the kernel to the code that was waiting on it (1 context switch), not from the kernel to the userland dispatcher to the code that was waiting on it (2 context switches).

anonymousDan 10y ago
What if you have async system calls (e.g. FlexSC)?
wbl 10y ago
This is not quite right. The userland dispatcher to run next thread is not a context switch: the TLB doesn't need to get flushed for instance. Furthermore many userland threads can be woken at once, and don't incur context switches when they suspend, prompting the next runnable thread to run. If you're willing to put in a lot of compiler work userland thread switching is a function call, literally. I don't 1:1 threads with small stack are going to compete with that.
naasking 10y ago
The real cost of a true "context switch" is the transition from user level to kernel level, which takes thousands of cycles. This cost isn't incurred on a userland context switch, so those costs aren't comparable.
- kijiki 10y ago
  Thousands of cycles for a SYSCALL/SYSRET on a reasonably modern Intel/AMD CPU? I think you should try to measure that.
  - DblPlusUngood 10y ago
    "Thousands of cycles" is definitely not true on modern CPUs. SYSENTER/SYSEXIT takes 150 cycles on my Xeon X3460.
  - naasking 10y ago
    My knowledge of current timings are admittedly a little out of date since my microkernel days are behind me. Even the 150 cycles another poster suggested is still a significant cost over a userland context switch.
  - derefr 10y ago
    Given the kind of "cloud"-backed startups most of HN are working on, I think the more practical measurement would be the overhead of a ring 0/3 separation in a VM, plus hypercalls, plus a ring 0/3 separation in the hypervisor (for paravirtualized syscalls); averaged against a ring 0/3 separation in a VM, plus SR-IOV virtualization (for HVM-backed syscalls.) Yeah, this doesn't apply for context-switches that happen because of plain old pre-emption, but it does happen if the context-switch is because e.g. a network packet wants to arrive at another process in your VM than the one that's currently running. I would imagine that there's a reason bare-metal IaaS providers have a business model. :)