14 ms·
Intel disables Hardware Lock Elision on all current CPUs
- wahern 7y agoThe title "Intel disables hardware lock elision on all current CPUs" seems too broad. Intel is disabling the backward ISA compatible implicit HLE capability--I can't remember exactly how it worked without looking it up, but IIRC it was a hack that leveraged existing cache coherency and ISA semantics to permit optimized spin-lock implementations that without feature detection still worked correctly on older chips. Explicit TSX instructions are now merely opt-out, where previously they couldn't be disabled by the kernel. The posted patch doesn't seem to disable it but simply adds some of the bits needed to do so. EDIT: Okay, I guess it is disabling Hardware Lock Elision (HLE), proper. But "hardware lock elision", lower case, implies all TSX-based lock elision code will stop being optimized. AFAIU RSX (explicit TSX) is usually only used for lock elision, anyhow; it's more of a microcode and speculation hack than a generic transactional memory feature, so the dividends quickly become elusive for anything beyond lock elision.
- my123 7y ago> The other TSX sub-feature, Hardware Lock Elision (HLE), is unconditionally disabled by the new microcode but still enumerated as present by CPUID(EAX=7).EBX{bit4}, unless disabled by IA32_TSX_CTRL_MSR[1] - TSX_CTRL_CPUID_CLEAR. Disabling HLE makes the code still works because of the backward ISA compatibility... but you still lose the performance benefits of HLE. Also, disabling it but still keeping it shown in CPUID...
- saagarjha 7y agoSo if I'm understanding this correctly, xacquire/xrelease now unconditionally fall back to being interpreted as the (useless) repne/repe prefixes, even on chips that "support" HLE?
- my123 7y agoYes...
- the-dude 7y agoIs anything know about how much performance will be lost?
- NicoJuicy 7y agohttps://twitter.com/damageboy/status/1194751035136450560?s=19 https://twitter.com/damageboy/status/1194751035136450560?s=1... Up to 20% for Array.Sort
- asveikau 7y agoWhy would Array.Sort have locking in need of elision? [I'm reminded of the old Java "Vector" or "StringBuffer" that put a lock around every operation "because thread safety is good" which had high cost for single threaded usage. But this is likely not what this is about.]
- makomk 7y agoWell, judging from the Phoronix benchmarks, performance is worse than just outright disabling TSX in its entirety: https://www.phoronix.com/scan.php?page=article&item=zombieload-v2-taa&num=1 https://www.phoronix.com/scan.php?page=article&item=zombielo... (Though they show basically no performance benefit from TSX, so...)
- YayamiOmate 7y agoI concur. I wondered whether their patch hit AMDs as well. Unlikely, but not entirely impossible, at least to me after hearing news about Linus "commenting" on their practices of creating problems for everyone.
- my123 7y agoAs far as I know, HLE is not implemented on AMD processors. However, part of the fix for JCC (https://www.intel.com/content/dam/support/us/en/documents/processors/mitigations-jump-conditional-code-erratum.pdf https://www.intel.com/content/dam/support/us/en/documents/pr... ) to recover the performance loss will make other CPUs slower if separate executables aren't used for unaffected CPUs... which they won't. > The MCU prevents jump instructions from being cached in the Decoded ICache when the jump instructions cross a 32-byte boundary or when they end on a 32-byte boundary. In this context, Jump Instructionsinclude all jumptypes: conditional jump (Jcc), macro-fusedop-Jcc(where opis one ofcmp, test, add, sub, and, inc, ordec), direct unconditional jump, indirect jump, direct/indirect call, and return The workaround to lower perf loss of having jumps uncached when they are on a 32-byte boundary involves adding quite some nops... (or padding with meaningless prefixes)
- saagarjha 7y agoLooks like the impact is pretty bad in certain cases: https://twitter.com/damageboy/status/1194751035136450560 https://twitter.com/damageboy/status/1194751035136450560
- my123 7y agoYeah, edited the title to uppercase Hardware Lock Elision now.
- temac 7y agoGood to avoid any misunderstanding, even if I don't think there should have been some: hardware lock elision is hardware lock elision, and TSX is not limited to hardware lock elision. So Intel disabled hardware lock elision, really, and the locks are not longer elided... Now maybe some people also like to elide some locks thanks to explicit RTM, but that's up to them, and that's not even completely hardware in this case (you have to write your own algo to do that and provide an explicit soft fallback), and of course Intel is not going to analyse algorithms to determine if there is logically some lock elision in a transnational section or not, and to only break out of speculation on the blocks of code presenting some...
- jakeogh 7y agoIs https://github.com/speed47/spectre-meltdown-checker https://github.com/speed47/spectre-meltdown-checker the best tool? Impressive chunk of sh.
- izietto 7y agoWhat do you have against that script? It's a long script but it's pretty clean and readable
- jakeogh 7y agoOh, nothing! I think it's beyond excellent. I ran shellcheck to see if it was bash or sh and it didnt make a peep:). My Q is just asking if there is another similar quality tool. I see how you could interperit it the other way, thanks for asking.
- izietto 7y agoOh I see now, I misinterpreted the sense of the question :)
- lecarore 7y agoHe meant "impressive chunk of bash" (.sh) not "of shit"
- baybal2 7y agoI think speculative execution is in principle incompatible with untrusted code execution. Even if CPU makers will place memory protection in front of speculative execution, and not behind as it is now, any untrusted code/bytecode can still pwn the process running it, and there is no way to work that around as such.
- ben-schaaf 7y agoIf the only thing in the process is the untrusted code (+ interpreter) then there's nothing to pwn.
- baybal2 7y agoIndeed, and that's the reason why a lot of current "sandboxing" efforts are rather misguided. There is no reason to filter syscalls from some kind of bytecode with an interpreter run with full privileges if you simply run all of that unprivileged and you already have all syscall hardened and ACLed. But for as long as there is a remotest possibility of a process being able to get around MMU, there is no reason to do that either.
- leni536 7y agoThe problem is not the speculative execution. It's the observable side effects.
- monocasa 7y agoWith current CPUs, yes. Adding either a speculation barrier instruction, or even better, more fine grain ways than a process to describe memory protections (like segmentation) would go a long way.
- d1zzy 7y agoIt's more like how having JavaScript enabled in a browser is "in principle incompatible with untrusted execution". That is to say, yes, you are technically right, but at the same time the benefits are great so we are still pursuing that approach while at the same time fighting an arms race of finding and closing any security issues this design introduces.
- Yuioup 7y ago... in the Linux kernel.
- mehrdadn 7y ago... in the microcode updates. Which any of your BIOS, UEFI, or OS may and likely will apply.
- maddyboo 7y agoIf you're on Linux, you may need to install a package to get µcode updates. The Arch Wiki has a particularly informative article on the subject: https://wiki.archlinux.org/index.php/Microcode https://wiki.archlinux.org/index.php/Microcode
- jeltz 7y agoIt is installed by default in most distributions, e.g. Debian.
- bonzini 7y agoNope. The link is to a Linux kernel commit message, but it simply describes what the latest microcode update does.
- Yuioup 7y agoOk you can all stop downvoting me now.
- wizzwizz4 7y agoIf you haven't already, would you mind reading about HN's approach to comments (https://news.ycombinator.com/newswelcome.html https://news.ycombinator.com/newswelcome.html) and site guidelines (https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html)?
- ddtaylor 7y agoDoes this have any affect on AMD systems?
- dannyw 7y agoAMD not affected with the latest wave of speculative execution bugs.
- floatboth 7y agoAMD does not support any similar features at all.
- _-___________-_ 7y agoWhile the link is to a Linux commit message, the described change (disabling HLE) is implemented in an Intel microcode update, which of course will not have any effect on an AMD processor.
- mehrdadn 7y agoUh, so there's absolutely no way to avoid disabling HLE (unless you block microcode updates altogether)? Can people who paid extra for a computer whose CPU has HLE get a refund?
- age_bronze 7y agoYou can avoid it. They didn't disable it, but implemented MSR which disables it. Don't configure this MSR to disable HLE and you're good.
- deleted 7y ago[deleted]
- mehrdadn 7y agoBut it says "Hardware Lock Elision (HLE) is unconditionally disabled by the new microcode"?
- topbanana 7y agoHow do microcode changes actually work? My mental model of a CPU is hard baked logic paths.
- deleted 7y ago[deleted]
- edude03 7y agoI recently learned about how microcode works from this video actually https://youtu.be/Zg1NdPKoosU https://youtu.be/Zg1NdPKoosU. It might not be the state of the art anymore but is probably a good basis
- gmueckl 7y agoMy personal "Aha!" moment about this was in front of an old Zuse Z5 in a museum with a block diagram of the CPU hanging next to it. That diagram showed which bits toggled which functional unit within the CPU. The comtrol panel on the operator desk had the same bit labels on the data input buttons. This early machine had no modern instruction decoder at all. Each bit in an instruction word was used directly. This becomes wasteful when the number of functional units grows and not all bit combinations are reasonable. So the next reasonable thing to do is to introduce a lookup table from shorter coded instructions to the actual control bit patterns. The CPU in this video adds an internal step counter that allows splitting an instruction over multiple cycles with a different control word for each cycle. Modern CPUs are probably conceptually similar, but much more complex. Optimizations like pipelining and multiple ALUs make the whole decoding and checking process more complex and more dynamic.
- CriticalCathed 7y ago>Think of microcode like a firmware for your computer’s CPU. Microcode translates the instructions the CPU receives into the physical, circuit-level operations that happen inside the CPU. In other words, an updated microcode can send different instructions to the circuits inside the CPU. This can prevent certain Spectre attacks by changing the way the CPU functions. Microcode updates can also fix bugs and other errors, without requiring complete replacement of CPU hardware.
- comex 7y agoShame. When TSX was announced in 2012, I started looking forward to a day, years down the line, when it would be ubiquitous enough that I could write code that depended on it. (At least on Macs, which don't run AMD processors.) The first processors supporting it shipped in 2013... but then in 2014 Intel abruptly disabled it in all then-shipping processors via microcode update, due to an erratum. Eventually they started shipping processors without the bug, and 4-5 years passed, a decent chunk of the time needed for ubiquity. But now the clock's reset to zero once again. Though, apparently Intel always disabled TSX on i3 processors for product segmentation, so maybe universal support was never in the cards... I wish more CPUs would support at least a limited version of this. Basically, I want the equivalent of atomic<shared_ptr<T>>, but lock-free. That requires reading a pointer value from memory and then incrementing a reference count stored at that pointer, as a single atomic operation. I'm pretty sure that's doable with TSX, although I haven't actually tried it.
- mmastrac 7y agoDitto - and it seems like every time TSX was "re-released", it ended up being disabled later on for some random microcode issue. Transactional memory is hard!
- roblabla 7y agoFWIW, my understanding is that XBEGIN/XEND/XABORT are still available on the affected CPUs. Only XACQUIRE/XRELEASE are disabled. So the clock isn't really reset. I, too, wish TSX was more ubiquitous. I'm working on a kernel, and was hoping to use TSX to greatly simplify the logic of safely reading user memory from kernelspace - catching various exception cases without going through the real exception handler. Turns out, none of the machines I have at my disposal have TSX - not my Desktop PC nor my server machine. So, RIP that plan, I guess. What you want is absolutely doable with TSX. The overhead might be significant though. I wouldn't be surprised if locking a mutex was faster.
- jdsully 7y agoYou can’t simplify the logic because TSX transactions may abort for any reason and make no promises of forward progress. You must implement a fallback codepath. If you meant having an optimized codepath then that is doable. But given your writing a kernel there may be microarchitectural hazards that trigger excessive aborts.
- zatertip 7y agohttps://ieeexplore.ieee.org/document/6877452 https://ieeexplore.ieee.org/document/6877452 describes a speedup of 41% when using TSX under a specific HPC workload. https://www.phoronix.com/scan.php?page=news_item&px=RPCS3-Intel-TSX-Support https://www.phoronix.com/scan.php?page=news_item&px=RPCS3-In... describes a 40% speedup for an emulation workload with TSX. Do we know what the slowdown will be for current gen CPUs for various workloads?
- bonzini 7y agoAs far as I know the only software that really wants it is SAP HANA. The slowdown should be small, that 40% speedup on emulation is specifically on load-locked/store-conditional instructions, and the linked blog post says "non-TSX CPUs such as Ryzen [had] a noticeable improvement in performance, although not to the same extent".
- jstanley 7y agoHow do the microcode upgrades get delivered? Do people have to manually install them, or do Intel have some way to force a microcode update over the Internet?
- jakeogh 7y agoIt's distribution specific, but generally it gets loaded at every boot. It's not permanent. https://wiki.gentoo.org/wiki/Microcode https://wiki.gentoo.org/wiki/Microcode
- maddyboo 7y agoFor Arch Linux, see: https://wiki.archlinux.org/index.php/Microcode https://wiki.archlinux.org/index.php/Microcode
- microcolonel 7y agoOn Windows it's handled in Windows Update, and maybe there's a way to disable loading the new microcode. On Linux, it's explicitly provided to the kernel by whatever userspace you have. On Arch, for example, it's in a separate package called intel-ucode. Some board firmware loads updated microcode when it's updated. It has to be loaded at each boot by software in order to change.
- Darkstack 7y agoOn windows you can disable / enable some of the mitigation via the registry. https://support.microsoft.com/en-us/help/4072698/windows-server-speculative-execution-side-channel-vulnerabilities-prot https://support.microsoft.com/en-us/help/4072698/windows-ser...
- raxxorrax 7y agoOn many newer CPUs, the memory for the update is volatile, so it gets rewritten at every power-on. That could be done by your motherboard or the kernel of the OS you are using.
- d1zzy 7y agoMultiple ways: - through BIOS updates (so the motherboard OEM has to release a BIOS update for that model that includes newer microcode and you'd have to download/apply the update) - through the OS: both Linux and Windows can and do load ucode at boot, depending on the version/configuration/etc
- Forge36 7y agoIs this related to: https://news.ycombinator.com/item?id=21534232 https://news.ycombinator.com/item?id=21534232 Or do we have two bugs being fixed at the same time?
- saagarjha 7y agoIt’s a somewhat unrelated bug that also had a workaround released for it recently.
- shifto 7y agoIs there a place where I can find how much performance my Haswell CPU has lost due to all of these 'fixes'?
- vectorEQ 7y agoit depends on the kind of applications / loads you are performing, so really it's not possible to put a static value to that. best is just to test it with some benchmarks yourself to be sure how it impacts you. some people report losses of like 20%, others 1 or 2%...
- jotm 7y agoHaswell doesn't have this feature
- diffeomorphism 7y agoFor example from March 2019: https://www.phoronix.com/scan.php?page=article&item=spec-melt-8way&num=1 https://www.phoronix.com/scan.php?page=article&item=spec-mel... I think there are some further in-depth benchmarks with multiple generations of Intel and AMD, but can't find the link.
- deleted 7y ago[deleted]
- saati 7y agoThe Haswell TSX was found to be unsound and disabled not long after it came out.
- krzyk 7y agoIs there a way to NOT get that "fix"? I don't care for security, I prefer performance. A magic kernel switch to not load new microcode? Or a switch in my distro (Debian)? Something similar to `mitigations=off` in current kernels.
- kd3 7y agoWhat percentage of functionality has Intel already disabled on their CPUs? 25%? Soon they'll have to disable the entire CPU. It's fucking hilarious.
- age_bronze 7y agoThey do not disable it. They only implement MSR which can allow you to disable it. If you want to stay with updated microcode and with HLE, just don't configure it as disabled.
- jdsully 7y agoYes they have. The only conditionally disabled feature is xbegin/xend/xabort. From the article: > The other TSX sub-feature, Hardware Lock Elision (HLE), is unconditionally disabled by the new microcode...
- jSully24 7y agoEach time I’ve seen this story today as I scrolled down the list my first though has been “Why and how did intel disable Larry Ellison on CPUs? And why didn’t they do it years ago?” Maybe it’s just been a long week...
- Too 7y agoRead the whole thing and though it was an article, reached the Signed-off-by at the end and realized it was a commit message. This is how a good commit message should look like! Telling what, why and how something was fixed, instead of just "fixed X".