5 ms·
On the order of a millisecond. Which seems to me, way too slow in this day and age. The context switch has been a hard wad that chokes algorithms in unexpected
by JoeAltmaier 10y ago
On the order of a millisecond. Which seems to me, way too slow in this day and age. The context switch has been a hard wad that chokes algorithms in unexpected ways, that are hard to diagnose.
This can be fixed with a different approach to CPU design? Most threads block on a semaphore-thingy or a timer or wait for io completion. If the CPU made those an opcode instead of a wad of kernel code, the switch could be reduced to a clock cycle (ns?)
How could this work? Maybe a global semaphore register where you Wait on a mask. Each bit could be mapped to one of those wait-condition. Another opcode would Set the bit.
To scale, the CPU would have to have not 2 or 4 hyperthreads but maybe 1000. So every thread in the system could be 'hot-staged' in a Wait, ready to execute, or executing.
There would still be cache and tlb issues, but they would be at the same level as any library call. They wouldn't include yards of kernel code messing with stacks and register loads and so on.
- luckydude 10y agoHey, same Joe from Sun? If so, howdy. On my i7-5930K I'm seeing 7-10 usecs, quite a bit better than a millisecond. Linus cares about this stuff, I'd be very surprised if he didn't measure it regularly.
- JoeAltmaier 10y agoProbably thinking of my brother, Rich? usecs are a thousand times better than a ms of course. And a ns is a thousand times better than that!
- j_s 10y agoCue the comment from the 10+ year old vaporware Mill architecture's team... :)
- belovedeagle 10y ago> If the CPU made those an opcode instead of a wad of kernel code, the switch could be reduced to a clock cycle (ns?) Uh, how do you think the kernel does it? Of course there's a CPU instruction for it - at least in modern ISAs like AArch64, but I can only assume it must have been tacked onto x86 too somehow or another. One reason that it's implemented in the kernel is to interact well with the scheduler: it's all well and good for a user space process to spinlock (with some kind of enter-low-power-state instruction), but it's going to be spinning until the timer comes around and kicks it back to the kernel anyways unless the other side of the lock just happens to be scheduled on a different core simultaneously. Even worse, if a thread is holding a spinlock and it gets unscheduled, now everyone else who wants to acquire that lock will waste their timeslices spinning away for nothing. The only way to avoid this would be for threads to be able to tell the OS "don't unschedule me" (much like the kernel masks interrupts during critical sections), but that would allow any such process to not only DOS the rest of the system, it could allow any such process to hang the system entirely. That's not to say it can't be done properly, but the easy and perhaps even correct way to do it is OS-managed semaphores. > not 2 or 4 hyperthreads but maybe 1000. So every thread in the system could be 'hot-staged' And of course this is what hyperthreads are, but where exactly do you plan on putting 1MB of architectural state while you're not using it? You're lucky if you have an L2 that big, but let's think... moving architectural state to the L2, that sounds a lot like... yes, correct! OS threads! If we want to rescue your suggestion back into the realm of feasibility, what you're saying is we should implement the OS (with features like scheduler-aware semaphores and thousands of ready threads) in hardware to make things like moving 1MB of architectural state to and from an L2 cheaper (and that we should have larger L2s so it's not incredibly wasteful to use 1MB on architectural state of non-executing threads). There's a reason we implement OSes in software, though, even with all the associated costs.
- JoeAltmaier 10y ago...and yet OSs do thread switches on the order of usec or even msec. So I repeat, there's much room for improvement.
- luckydude 10y agoUm, you need to think more deeply about this. Yes, you can switch a "thread" (says the guy who wrote a user level thread package) very, very quickly but that's meaningless. Unless your application is one where all it does is context switch and does no other work. "Context" is more than just the registers. You could make an instruction that did the context switch in zero time and a real context switch will take much more than zero time. Because there is more context than just the registers.
- JoeAltmaier 10y agoYes all that is very, very clear. But adding 10,000ns to the switch time is going to torpedo any chance of a quick turnaround on a queue wait or IO completion. That's a huge timewad compared to a procedural solution that doesn't use multiple threads or locked queues. Thus the initial comment that using threads adds unpredictable issues that are hard to diagnose. And the suggestion that removing this delay will definitely reduce latency of event handling at all levels. Including, returning to a thread 10,000ns quicker can help with cache utility - there's that much less cache washout occurring 'while you were away'. Tightening this up can allow, for the first time in any architecture, truly transparent multi-thread implementations of ordinary tasks without penalty. Imagine each thread executes a handful of instructions and then signals the other thread to continue. No cache issues (its only been nanoseconds). Sure, 'nobody does that'. But that's because, threads are so godawful expensive now. We'd be in a whole different headspace if that changed.
- luckydude 10y agoYou seem to be living in an imaginary world where main memory is much, much closer to the CPU than it is in real life. 10,000ns is 174 cache misses on my machine. That's not a lot of room when you are popping something off a queue or finishing an IO.
- luckydude 10y ago> If the CPU made those an opcode instead of a wad of kernel code, the switch could be reduced to a clock cycle (ns?) Not for any real world workload. It's worth reading section 6.6 of this: http://mcvoy.com/lm/bitmover/lmbench/lmbench-usenix.pdf http://mcvoy.com/lm/bitmover/lmbench/lmbench-usenix.pdf but I'll try and restate it here. So what's a context switch? Lots of people think it is putting one process to sleep and waking up another process. Which is sort of correct. Let's look at what that means. "Sleeping" a process means I have to take whatever hardware context there is and store it away (int/float registers are the basics). "Waking" a process is the opposite, go find the stored state, restore it, and return (sort of like longjmp()) to the awoken process. But that is the minimal context switch, that's what you would have if you had a yield_to(pid_t pid) system call and you had a tight loop where pid A yields to pid B which yields to pid A, etc. Do that a million times, measure the total time, divide by a million. That's not real world at all. In the real world your context is more than your registers, it is also your cache foot print. Pid A wasn't just yielding, it was doing some work. When it comes back, it has to either get lucky (with the right kind of cache that can hold more than one process's cache lines, look up physically vs virtually indexed caches, Hennessey & Patterson cover this nicely) or it has to reload state from L3 or main memory. That's what the lmbench paper goes into a lot of detail about this. People write benchmarks and don't think about the fact that there is more to life than benchmarks. lmbench's context switch benchmarks are done as a function ctx(int N_processes, int working_set). Most people would benchmark ctx(2, 0) but that's nonsense unless you care about the silly yield/yield numbers. I dunno about you but when I'm debugging a perf problem, in ~30 years of doing that I have yet to see a perf problem around yield/yield (well, sort of. Oracle's distributed lock manager was as close as I've seen but even it had a larger context than zero bytes more than the register store/restore). The interesting questions are: - are your caches set up for multiple processes so you still have state when you switch back? - how many different processes can you have in the cache[s] at once? - how big is your OS path to sleep/wakeup, that uses up space in the instruction cache? The lmbench paper talks about all this stuff, complete with crappy graphs written in troff/pic by yours truly that let you visualize when things will fit and when things will thrash.
- JoeAltmaier 10y agoOk it'll never be 1 clock cycle. But it sure can be better than milliseconds!