3 ms·
Looking at the patches Linus is commenting on: as an Asm programmer, this is absolutely horrible --- basically every indirect call (one instruction) turns into
by userbinator 9y ago
Looking at the patches Linus is commenting on: as an Asm programmer, this is absolutely horrible --- basically every indirect call (one instruction) turns into a seven-instruction sequence that will, due to preventing speculation, result in massive slowdowns:
https://lkml.org/lkml/2018/1/3/770 https://lkml.org/lkml/2018/1/3/770
Unlike the KPTI patches, which only affect things on each system call, this happens on every indirect call and probably bloats the code considerably too. I can see why Linus is not happy.
Edit: To give a bit more background, predicting indirect calls and speculating into them is absolutely critical to getting good performance from OOP-ish code which tends to use them a lot (virtual functions in C++, function pointers in C). The Linux kernel is (thankfully?) not very OOP-ish, but it does rely on indirect calls (function pointers) extensively.
- revelation 9y agoI missed that, actually. But you are right, it's not just the crucial bits of inline assembly they are patching here, they expect your kernel to be built with their special version of GCC (yet unreleased) that mutilates every single freaking indirect jmp/call into what could cost more than a virtual function call.. We use a new compiler option -mindirect-branch=thunk-extern
- userbinator 9y agoI missed that it was indirect jumps too, which means switch statements also get affected. That's even worse.
- revelation 9y agoHere is an example with the very common programming patterns that result in indirect jumps/calls: https://godbolt.org/g/LPycpY https://godbolt.org/g/LPycpY Every single call rax and jump rax that you see on the right will be replaced with the following: push rax jmp .indirect_thunk And indirect_thunk does an lfence and a call to another function that gets rax from the stack and jumps there in a very roundabout way that interacts terribly with CPU prediction. I can see why they somehow didn't get around to benchmarking this yet, because they are not going to like the results. And it doesn't end with the kernel. Every JIT is going to need the treatment.
- mikeash 9y agoThat has to be crazy slow. They probably haven’t published benchmarks yet because they’re still waiting for it to finish running!
- rst 9y agoInteracting terribly with CPU branch prediction is the whole point -- Spectre attacks work by messing with the branch predictor, to misdirect speculative execution onto paths that reveal information useful to the attacker. So, the point of this whole crazy dance is to effectively disable the branch predictor.
- revelation 9y agoOf course. But it needs spelling out, because branch prediction is essential to good performance and why todays CPUs can be much faster despite not reaching higher clocks. It's much more than just the cost of a few more instructions.
- mike_hearn 9y agoApparently the perf impact isn't so bad outside of microbenchmarks actually. 10% for real world programs ... as long as you use profile guided optimisation to devirt as many calls as possible! https://support.google.com/faqs/answer/7625886 https://support.google.com/faqs/answer/7625886 http://webcache.googleusercontent.com/search?q=cache%3Ahttps%3A%2F%2Freviews.llvm.org%2FD41723&oq=cache%3Ahttps%3A%2F%2Freviews.llvm.org%2FD41723&aqs=chrome..69i57j69i58.950j0j4&sourceid=chrome&ie=UTF-8 http://webcache.googleusercontent.com/search?q=cache%3Ahttps... (some websites with info on this are DoSd and down right now, including the Arm website and LLVM code review site so switched to a cache link) Pretty painful for C++ but fixable with build changes. JVM users and other JIT-compiled languages will probably be fixable without any serious impact, as HotSpot and other advanced runtimes devirtualise calls (convert indirect to direct) as much as they can anyway, meaning most calls won't be indirect when finished and once the code has warmed up.
- smitherfield 9y agoThe above example compiled with optimizations and with `indirect_jump` altered so the compiler actually emits a jump table instead of an array lookup: https://godbolt.org/g/MPs43x https://godbolt.org/g/MPs43x
- JdeBP 9y agoClang has a new code generation option, too. * https://reviews.llvm.org/D41723 https://reviews.llvm.org/D41723 * https://news.ycombinator.com/item?id=16070050 https://news.ycombinator.com/item?id=16070050 * https://news.ycombinator.com/item?id=16070356 https://news.ycombinator.com/item?id=16070356
- jcranmer 9y agoIsn't that patch for the bug that's present on basically every CPU, Intel, AMD, ARM, etc., though? I thought the Intel-specific bug was fixed by "blank out the TLB every sysret".
- amluto 9y agoDifferent bug, AFAICT.
- keldaris 9y agoI'm frankly shocked Linus was anywhere near as restrained as he was upon seeing this monstrosity. Nothing of this sort should ever make it into the kernel without explicit and loudly announced opt-out switches. If anything close to this does make it in, even as an opt-in, this will easily be the biggest performance regression in recent (perhaps all-time?) kernel history.
- emeraldd 9y agoHis restraint and the fact that this stuff is even being considered tells me this is a really really bad bug. I doubt he would be so cordial in any other situation.
- redshirt 9y agoNothing like this should have ever made it into hardware. It's lazy implementation (or performance at the expense of security).
- keldaris 9y agoPerformance at the expense of security is a compromise made daily in every aspect of IT. The only way to make systems perfectly secure is to make them unusable. Now, you can certainly argue that this particular tradeoff wasn't worth it, and in hindsight it's certainly a questionable one. However, do note that side channel attacks have been known for decades (since out-of-order CPUs became widespread in the 90s) and until this point have rarely presented a significant threat to most systems.
- emeraldd 9y agoI'd be more inclined to think this was someone who didn't understand the implications implementing a performance improvement. This bug is clearly deep into unk-unk (i.e. well past the known areas of concern) territory and took a great deal of insight to the fundamentals of speculative execution to discover.