4 ms·
There is a section about pipeline stalls caused by switching stacks, but that is misleading. Switching stacks per se is not a huge penalty, the cpu [1] might ne
by gpderetta 7y ago
There is a section about pipeline stalls caused by switching stacks, but that is misleading. Switching stacks per se is not a huge penalty, the cpu [1] might need to add synchronization opcodes to synchronize the stack engine view of the stack pointer with the actual stack pointer, but the performance cost is minimal and often free.
A big penalty is due to (failed) branch prediction: a coroutine switch is essentially an indirect branch so the predictor might need help (for example using a different branch instruction address for each coroutine type); also call instruction used to call into the coroutine switch function is not paired with a ret instruction, which messes with the specialized call predictor. Changing the final jmp instruction in the switch function to push add; ret actually makes thing worse; the best solution is to inline the switch function in the caller via inline assembler (this also helps with the previous issue and, if the compiler provides the functionality, it allows only saving the registers that are actually in use)
edit: last time I did a synthetic benchmark of my one of my own coroutine implementations, the switch performance was only constrained by the number of taken branches the CPU could issue (one every other cycle): https://github.com/gpderetta/delimited/blob/master/benchmark.cc https://github.com/gpderetta/delimited/blob/master/benchmark...
[1] I'm talking about the typical x86 cpu.
- byuu 7y agoYou could well be right. In my own tests, the ESP swap is what tanked performance. A more limited stackless test did not have nearly the same overhead. And indeed, the largest speedup was from moving to set ESP as soon as possible, get the return address into EAX, and finally jump to it. CALL+RET with late ESP set behaved poorly, leading me to believe it was due to the CPU stalling on the ESP change. I did also try inlining this, but it was much less portable and actually performed slower in my emulator. I won't claim that will be the case for everyone, of course. YMMV.
- fwsgonzo 7y agoInteresting that you saw that. I guess it could be real, but sounds weird too.
- avdicius 7y agoWow. I used my own stack switch routine for several years now and didn't realize it could be improved until I read this thread. Thank you very much. This is my old naive version: https://github.com/ademakov/MainMemory/blob/master/src/base/arch/x86-64/cstack-switch.S https://github.com/ademakov/MainMemory/blob/master/src/base/... This is what I have now: https://github.com/ademakov/MainMemory/blob/cstack-switch-revamp/src/base/arch/x86-64/cstack.h https://github.com/ademakov/MainMemory/blob/cstack-switch-re...
- gpderetta 7y agoYou are missing x/y/zmm regs in the clobber list.
- avdicius 7y agoYes, that's true. However in my app this has never caused any problems. Apparently even if the compiler auto-vectorizes anything it does this only with scratch regs or in leaf procedures. I don't have any fp or simd code of myself.