9 ms·
This is super cool. This exploit will be one of the canonical examples that just running something in a VM does not mean it's safe. We've always known about VM
by 0xbadcafebee 3y ago
This is super cool. This exploit will be one of the canonical examples that just running something in a VM does not mean it's safe. We've always known about VM breakout, but this is a no-breakout massive exploit that is simple to execute and gives big payoffs.
Remember: just because this one bug gets fixed in microcode doesn't mean there's not another one of these waiting to be discovered. Many (most?) 0-days are known about by black-hats-for-hire well before they're made public.
CPU vulnerabilities found in the past few years:
https://en.wikipedia.org/wiki/Meltdown_(security_vulnerability)
https://en.wikipedia.org/wiki/Spectre_(security_vulnerability)
https://aepicleak.com/
https://en.wikipedia.org/wiki/Software_Guard_Extensions#SGAxe
https://en.wikipedia.org/wiki/Software_Guard_Extensions#LVI
https://en.wikipedia.org/wiki/Software_Guard_Extensions#Plundervolt
https://en.wikipedia.org/wiki/Software_Guard_Extensions#MicroScope_replay_attack
https://en.wikipedia.org/wiki/Software_Guard_Extensions#Enclave_attack
https://en.wikipedia.org/wiki/Software_Guard_Extensions#Prime+Probe_attack
https://www.vusec.net/projects/crosstalk/
https://en.wikipedia.org/wiki/Hertzbleed
https://www.securityweek.com/amd-processors-expose-sensitive-data-new-squip-attack/
- cmrdporcupine 3y agoIn the end, I'm thinking most of these are related to branch prediction? It strikes me that it's either that branch prediction is so inherently complex enough it's always going to be vulnerable to this and/or it just so defies the way most of us intuitively think about code paths / instruction execution that it's hard to conceive of the edge cases until too late? At what point does the complexity of CPU architectures become so difficult to reason about that we just accept the performance penalty of keeping it simpler?
- c7DJTLrn 3y agoWe demanded more performance and we got what we demanded. I doubt manufacturers are going to walk back on branch prediction no matter how flawed it is. They'll add some more mitigations and features which will be broken-on-arrival.
- myself248 3y agoI didn't demand more performance. My 2008-era AthlonX2 would still be relevant if web browsers hadn't gotten so bloated. I still use it for real desktop applications, i.e. everything that isn't in Electron.
- lmz 3y agoYour Athlonx2 already had branch prediction. https://chipsandcheese.com/2022/07/28/amds-athlon-64-getting-the-basics-right/ https://chipsandcheese.com/2022/07/28/amds-athlon-64-getting...
- Tuna-Fish 3y ago> At what point does the complexity of CPU architectures become so difficult to reason about that we just accept the performance penalty of keeping it simpler? Never for branch prediction. It just gets you too much performance. If it becomes too much of a problem, the solution is greater isolation of workloads.
- hedgehog 3y agoIn certain cases isolation and simplicity overlap, I suspect for example that the dangers of SMT implementation complexity are part of why Apple didn't implement it for their respective CPUs. Likely we'll see this elsewhere too, for example Amazon may not ever push to have SMT in their Graviton chips (the early generations are off the shelf cores from ARM where they didn't have a readily available choice).
- asdfasgasdgasdg 3y agoI could be mistaken, but I don't think Zenbleed has anything to do with SMT, based on my reading of the document. There is a mention of hyperthreads sharing the same physical registers, but you can spy on anything happening on the same physical core, because the register file is shared across the whole core. It even says so in the document: Note that it is not sufficient to disable SMT. Apple's chips don't have this vulnerability, but it's not because they don't have SMT. They just didn't write this particular defect into their CPU implementation.
- hedgehog 3y agoCorrect, I was responding to parent writing "At what point does the complexity of CPU architectures become so difficult to reason about that we just accept the performance penalty of keeping it simpler?" I think we may be seeing an industry-wide shift away from SMT because the performance penalty is small and the complexity cost is high, if so that fits parent's speculation about the trend. In a narrow sense Zenbleed isn't related to SMT but OP's question seems perfectly relevant to me. I come from a security background and on average more complicated == less secure because engineering resources are finite and it's just harder and more work to make complicated things correct.
- paulmd 3y agoMore generally, most of them are related to speculative execution, where branch mis-prediction is a common gadget to induce speculative mis-execution. Speculation is hard, it's sort of akin to the idea of introducing multithreading into a program, you are explicitly choosing to tilt at the windmill of pure technical correctness because in a highly concurrent application every error will occur fairly routinely. Speculation is great too, in combination with out-of-order execution it's a multithreading-like boon to overall performance, because now you can resolve several chunks of code in parallel instead of one at a time. It's just also a minefield of correctness issues, but the alternative would be losing something like the equivalent of 10 years of performance gains (going back to like ARM A53 performance). The recent thing is that "observably correct" needs to include timings. If you can just guess at what the data might be, and the program runs faster if you're correct, that's basically the same thing as reading the data by another means. It's a timing oracle attack. (in this case AMD just fucked up though, there's no timing attack, this is just implemented wrong and this instruction can speculate against changes that haven't propagated to other parts of the pipeline yet) The cache is the other problem, modern processors are built with every tenant sharing this single big L3 cache and it turns out that it also needs to be proof against timing attacks for data present in the cache too.
- loeg 3y agoSpeculative execution, not branch prediction.
- 0cf8612b2e1e 3y agoIf you pin the VM to a different core/CPU, would that do anything to mitigate? Or are the OS affinity guarantees not that strong?
- saagarjha 3y agoIn this case, it would avoid the exploit, because it requires a shared register file.
- rcxdude 3y ago>At what point does the complexity of CPU architectures become so difficult to reason about that we just accept the performance penalty of keeping it simpler? Basically never for anything that's at all CPU-bound, that growth in complexity is really the only thing that's been powering single-threaded CPU performance improvements since Dennard scaling stopped in about 2006 (and by that time they were already plenty complex: by the late 90s and early 2000's x86 CPUs were firmly superscalar, out-of-order, branch-predicting and speculative executing devices). If your workload can be made fast without needing that stuff (i.e. no branches and easily parallelised), you're probably using a GPU instead nowadays.
- kiririn 3y agoYou can rent one of the Atom Kimsufi boxes (N2800) to experience first hand a cpu with no speculative execution. The performance is dire, but at least it hasn’t gotten worse over the years - they are immune to just about everything
- vetrom 3y agoTheres VLIW/'preprediction'/some other technical name I forget for infrastructures which instead ask you to explicitly schedule instruction/data/branch prediction. If I remember, the two biggest examples I can think of were IA64 and Alpha. I wanna think HP-PA did the same but I'm not clear on that one. For various reasons, all these infras eventually lost out in the market due to market pressure (and cost/watt/IPC, I guess).
- c7DJTLrn 3y agoRunning untrusted code whether in a sandbox, container, or VM, has not been safe since at least Rowhammer, maybe before. I believe a lot of these exploits are down to software and hardware people not talking. Software people make assumptions about the isolation guarantees, hardware people don't speak up when said assumptions are made.
- saagarjha 3y agoThat is not true in this case. It's just a CPU bug; not even a side channel.
- c7DJTLrn 3y agoThe statement about isolation guarantees is in general, doesn't relate to the OP.
- insanitybit 3y agoHardware people are the ones making those promises, so I don't think that's right at all. And Rowhammer is a way overstated vulnerability - there are all sorts of practical issues with it, especially if you're on modern, patched hardware.
- zamadatix 3y agoIn the case of the VM won't registers be wiped when entering/exiting the VM?
- loeg 3y agoThe problem is the freed entries in the register file. A VM can, at least, use this bug to read registers from a non-VM thread running on the adjacent SMT/HT of a single physical core. I suspect a VM could also read registers from other processes scheduled on the same SMT/HT.
- astrange 3y agoAre people running multiple untrusted VMs without turning SMT off? Even letting them share caches seems like asking for trouble.
- loeg 3y agoSomeone, somewhere is, of course. I don't know if the hyperscalers do, or not.
- Astronaut3315 3y agoThis specific CVE still applies even if SMT is off, per the article.
- zamadatix 3y agoIn the context of this conversation, SMT on/off is relevant to what scope of the vulnerability has with VMs beyond the claim in the article that the issue is in some way present inside VMs.
- bbojan 3y agoThe fine article states that simply turning off SMT doesn't help with this particular exploit.
- 3y ago
- Bluecobra 3y agoYup! I worked at a few companies that would co-mingle Internet facing/DMZ VMs with internal VMs. When pointing this out and recommending we should airgap these VMs to it's own dedicated hypervisor it always fell on deaf ears. Jokes on them I guess.
- Kwpolska 3y agoI'm pretty sure AWS/Azure/GCP don’t assign separate boxes to every customer, and somehow they’re fine.
- yencabulator 3y agoYou can pay AWS a premium to make sure you're the only tenant on the physical machine. You can also split your own stuff into multiple tenants, and keep those separate too.
- nicolas_17 3y agoAt which point you don't really need the flexibility of AWS and you might as well get a Dedicated Server elsewhere?
- yencabulator 3y agoIt'll still let you do the elastic scaling stuff, billing for actual usage instead of racked hardware.
- segfaultbuserr 3y agoEric Brandwine (VP/DE @ AWS) said publicly in 2019 that EC2 had never scheduled different tenants on the same physical core at the same time, even before we learned about these kinds of side-channel attacks. https://www.youtube.com/watch?v=kQ4H6XO-iao&t=2485s https://www.youtube.com/watch?v=kQ4H6XO-iao&t=2485s
- yencabulator 3y ago
- phendrenad2 3y agoThe problem is, VMs aren't really "Virtual Machines" anymore. You're not parsing opcodes in a big switch statement, you're running instructions on the actual CPU, with a few hardware flags that the CPU says will guarantee no data or instruction overlap. It promises! But that's a hard promise to make in reality.
- msla 3y agoThis is because VM means two different things and has for a long time: IBM's VM was and is a hypervisor. It dates to the mid 1960s, in the form of CP-40, and it didn't run opcodes in software, but in hardware. https://en.wikipedia.org/wiki/IBM_CP-40 https://en.wikipedia.org/wiki/IBM_CP-40 p-code machines, which interpret bytecode, date back almost as far, such as the O-code machine for BCPL. https://en.wikipedia.org/wiki/BCPL https://en.wikipedia.org/wiki/BCPL Getting people to distinguish between these concepts is probably a lost cause.
- Joker_vD 3y agoLooking at the IBM's tech from the sixties is somehow weirdly depressing: it's unbelievable how much of the architectural stuff they've invented already by the 1970.
- nine_k 3y agoNot depressing, but inspiring. So many great architectural ideas can be made accessible to millions of consumers, not limited to a few thousand megacorps.
- meepmorp 3y agoI remember seeing VMware for the first time and thinking that the PC world had finally entered the 1970s.
- cduzz 3y agoClose, but not quite -- you can't nest the VMs the way you can on "big iron"
- trebligdivad 3y agoThe comparison to Meltdown/Spectre are a bit misleading though - they were a whole new form of attack based on timing where the CPU did exactly what it should have done; This zenbleed case is a good old fashioned bug though - data in a register that shouldn't be.
- stcredzero 3y agothis is a no-breakout massive exploit that is simple to execute and gives big payoffs Wouldn't we be able to avoid the "big payoffs" of no-breakout exploits if we had specialized hardware handle the secrets?
- api 3y agoI’m quite surprised there hasn’t been a cloud apocalypse yet where something just runs rampant through AWS or something.
- jacquesm 3y agoIt's still early days for the cloud. I'm pretty sure such a thing will happen sooner or later.
- deleted 3y ago[deleted]
- alecco 3y agoCouldn't VMs zero all registers when switching? It shouldn't be much more latency than a typical context switch. Also purge CPU cache to be safe.
- sgerenser 3y agoIt does zero registers when context switching, but only the "logical" registers, not the physical ones, as described in this comment: https://news.ycombinator.com/item?id=36855266 https://news.ycombinator.com/item?id=36855266