4 ms·

> If the CPU made those an opcode instead of a wad of kernel code, the switch could be reduced to a clock cycle (ns?) Uh, how do you think the kernel does it?

> If the CPU made those an opcode instead of a wad of kernel code, the switch could be reduced to a clock cycle (ns?) Uh, how do you think the kernel does it? Of course there's a CPU instruction for it - at least in modern ISAs like AArch64, but I can only assume it must have been tacked onto x86 too somehow or another. One reason that it's implemented in the kernel is to interact well with the scheduler: it's all well and good for a user space process to spinlock (with some kind of enter-low-power-state instruction), but it's going to be spinning until the timer comes around and kicks it back to the kernel anyways unless the other side of the lock just happens to be scheduled on a different core simultaneously. Even worse, if a thread is holding a spinlock and it gets unscheduled, now everyone else who wants to acquire that lock will waste their timeslices spinning away for nothing. The only way to avoid this would be for threads to be able to tell the OS "don't unschedule me" (much like the kernel masks interrupts during critical sections), but that would allow any such process to not only DOS the rest of the system, it could allow any such process to hang the system entirely. That's not to say it can't be done properly, but the easy and perhaps even correct way to do it is OS-managed semaphores. > not 2 or 4 hyperthreads but maybe 1000. So every thread in the system could be 'hot-staged' And of course this is what hyperthreads are, but where exactly do you plan on putting 1MB of architectural state while you're not using it? You're lucky if you have an L2 that big, but let's think... moving architectural state to the L2, that sounds a lot like... yes, correct! OS threads! If we want to rescue your suggestion back into the realm of feasibility, what you're saying is we should implement the OS (with features like scheduler-aware semaphores and thousands of ready threads) in hardware to make things like moving 1MB of architectural state to and from an L2 cheaper (and that we should have larger L2s so it's not incredibly wasteful to use 1MB on architectural state of non-executing threads). There's a reason we implement OSes in software, though, even with all the associated costs.

JoeAltmaier 10y ago
...and yet OSs do thread switches on the order of usec or even msec. So I repeat, there's much room for improvement.
- luckydude 10y ago
  Um, you need to think more deeply about this. Yes, you can switch a "thread" (says the guy who wrote a user level thread package) very, very quickly but that's meaningless. Unless your application is one where all it does is context switch and does no other work. "Context" is more than just the registers. You could make an instruction that did the context switch in zero time and a real context switch will take much more than zero time. Because there is more context than just the registers.
  - JoeAltmaier 10y ago
    Yes all that is very, very clear. But adding 10,000ns to the switch time is going to torpedo any chance of a quick turnaround on a queue wait or IO completion. That's a huge timewad compared to a procedural solution that doesn't use multiple threads or locked queues. Thus the initial comment that using threads adds unpredictable issues that are hard to diagnose. And the suggestion that removing this delay will definitely reduce latency of event handling at all levels. Including, returning to a thread 10,000ns quicker can help with cache utility - there's that much less cache washout occurring 'while you were away'. Tightening this up can allow, for the first time in any architecture, truly transparent multi-thread implementations of ordinary tasks without penalty. Imagine each thread executes a handful of instructions and then signals the other thread to continue. No cache issues (its only been nanoseconds). Sure, 'nobody does that'. But that's because, threads are so godawful expensive now. We'd be in a whole different headspace if that changed.
    - luckydude 10y ago
      You seem to be living in an imaginary world where main memory is much, much closer to the CPU than it is in real life. 10,000ns is 174 cache misses on my machine. That's not a lot of room when you are popping something off a queue or finishing an IO.
      - JoeAltmaier 10y ago
        The presumption of large cache misses is where our intuitions diverge. Algorithms where threads operated on the same data, for instance, would not actually have many cache misses or even zero cache misses. btw Lets take it down a notch, and back off on the personal aspersions? ok?

JoeAltmaier 10y ago

...and yet OSs do thread switches on the order of usec or even msec. So I repeat, there's much room for improvement.

luckydude 10y ago
Um, you need to think more deeply about this. Yes, you can switch a "thread" (says the guy who wrote a user level thread package) very, very quickly but that's meaningless. Unless your application is one where all it does is context switch and does no other work. "Context" is more than just the registers. You could make an instruction that did the context switch in zero time and a real context switch will take much more than zero time. Because there is more context than just the registers.
- JoeAltmaier 10y ago
  Yes all that is very, very clear. But adding 10,000ns to the switch time is going to torpedo any chance of a quick turnaround on a queue wait or IO completion. That's a huge timewad compared to a procedural solution that doesn't use multiple threads or locked queues. Thus the initial comment that using threads adds unpredictable issues that are hard to diagnose. And the suggestion that removing this delay will definitely reduce latency of event handling at all levels. Including, returning to a thread 10,000ns quicker can help with cache utility - there's that much less cache washout occurring 'while you were away'. Tightening this up can allow, for the first time in any architecture, truly transparent multi-thread implementations of ordinary tasks without penalty. Imagine each thread executes a handful of instructions and then signals the other thread to continue. No cache issues (its only been nanoseconds). Sure, 'nobody does that'. But that's because, threads are so godawful expensive now. We'd be in a whole different headspace if that changed.
  - luckydude 10y ago
    You seem to be living in an imaginary world where main memory is much, much closer to the CPU than it is in real life. 10,000ns is 174 cache misses on my machine. That's not a lot of room when you are popping something off a queue or finishing an IO.
    - JoeAltmaier 10y ago
      The presumption of large cache misses is where our intuitions diverge. Algorithms where threads operated on the same data, for instance, would not actually have many cache misses or even zero cache misses. btw Lets take it down a notch, and back off on the personal aspersions? ok?