3 ms·

TL,DR; Leverage CONFIG_DYNAMIC_FTRACE. This is an OS only fix resulting in effectively zero overhead for chips other than Skylake. https://lkml.org/lkml/2018/

TL,DR; Leverage CONFIG_DYNAMIC_FTRACE. This is an OS only fix resulting in effectively zero overhead for chips other than Skylake. https://lkml.org/lkml/2018/1/23/25 https://lkml.org/lkml/2018/1/23/25 There's another possible method to avoid deep stacks on Skylake, without compiler support: - Use the existing mcount based function tracing live patching machinery (CONFIG_FUNCTION_TRACER=y) to install a _very_ fast and simple stack depth tracking tracer which would issue a retpoline when stack depth crosses boundaries of ~16 entries. The overhead of that would _still_ very likely be much cheaper than a hundreds (thousands) of cycle expensive MSR write at every kernel entry (syscall entry, IRQ entry, etc.). Pretty damn cool if it works.

gtirloni 9y ago
I couldn't find much about this 16 entries stack depth issue. Does anyone have more information?
- bonzini 9y ago
  It's explained in the thread. Basically, the RET instruction has its own tiny branch predictor, the return address stack (also known as RSB) which is only 16 or 32 entries deep. If your call stack is deeper than that, sooner or later the stack becomes empty. In this case, old processors simply does not predict the jump and stalls. But Skylake instead uses the generic indirect branch predictor, since after all RET is an indirect branch! Except the idea behind retpolines was to avoid the generic branch predictor, and now every single RET can use it! So, if you don't want to use IBRS you need to find a different way to protect the kernel from someone forcing undesired speculative execution after the RET.
  - gtirloni 9y ago
    RSB! Thank you, now I know what to focus on.
lousken 9y ago
as a skylake user I hope I'll be able to turn those protections off
walterbell 9y ago
What do we want for post-Kaby Lake systems? Should Intel return future hardware to pre-Skylake behavior? Is AMD similar to Intel pre-Skylake, for the purpose of Spectre mitigation? Could Intel and OEMs restart sales of Broadwell hardware, until they can reverse the Skylake/KabyLake/KabyLakeRefresh changes (in Cannon Lake? Ice Lake)?
cthalupa 9y ago
I think there's some misunderstanding in this discussion thread on what Ingo is working on. This isn't a general fix or "pure-software alternative to IBRS" - it is specific to handling the 16 stack depth ret being vulnerable on skylake+ processors, as it falls back to an indirect branch prediction. The zero overhead for non-skylake+ chips isn't that it is a zero-overhead general IBRS solution, it's that it does not incur additional overhead on non-skylake+ chips as it is not doing any work to mitigate the issue on non-skylake chips. This is because all of the major distros already compile their kernels with the CONFIG_FUNCTION_TRACER=y option set, so no additional work is being done unless the conditions appear and it is on a skylake+ chip.

gtirloni 9y ago

I couldn't find much about this 16 entries stack depth issue. Does anyone have more information?

bonzini 9y ago
It's explained in the thread. Basically, the RET instruction has its own tiny branch predictor, the return address stack (also known as RSB) which is only 16 or 32 entries deep. If your call stack is deeper than that, sooner or later the stack becomes empty. In this case, old processors simply does not predict the jump and stalls. But Skylake instead uses the generic indirect branch predictor, since after all RET is an indirect branch! Except the idea behind retpolines was to avoid the generic branch predictor, and now every single RET can use it! So, if you don't want to use IBRS you need to find a different way to protect the kernel from someone forcing undesired speculative execution after the RET.
- gtirloni 9y ago
  RSB! Thank you, now I know what to focus on.

lousken 9y ago

as a skylake user I hope I'll be able to turn those protections off

walterbell 9y ago

What do we want for post-Kaby Lake systems? Should Intel return future hardware to pre-Skylake behavior? Is AMD similar to Intel pre-Skylake, for the purpose of Spectre mitigation? Could Intel and OEMs restart sales of Broadwell hardware, until they can reverse the Skylake/KabyLake/KabyLakeRefresh changes (in Cannon Lake? Ice Lake)?

cthalupa 9y ago

I think there's some misunderstanding in this discussion thread on what Ingo is working on. This isn't a general fix or "pure-software alternative to IBRS" - it is specific to handling the 16 stack depth ret being vulnerable on skylake+ processors, as it falls back to an indirect branch prediction. The zero overhead for non-skylake+ chips isn't that it is a zero-overhead general IBRS solution, it's that it does not incur additional overhead on non-skylake+ chips as it is not doing any work to mitigate the issue on non-skylake chips. This is because all of the major distros already compile their kernels with the CONFIG_FUNCTION_TRACER=y option set, so no additional work is being done unless the conditions appear and it is on a skylake+ chip.