3 ms·

1. Commercially motivated attacks against mere mid-sized entities (100 M$-1 G$ in revenue) can empirically derive 10s of M$ from successful attacks. Adequate se

by Veserv 2y ago

1. Commercially motivated attacks against mere mid-sized entities (100 M$-1 G$ in revenue) can empirically derive 10s of M$ from successful attacks. Adequate security must make such attacks unprofitable. This constitutes a absolute, rock-bottom, minimum standard for effective enterprise security. 2. We have always known that such attacks would eventually become feasible to execute once the hackers matured. This was so obvious that government security standards/certifications such as the Rainbow Series codified in the 80s already considered such threats and placed thwarting them as the middle-levels of security. So, we have always known that "adequate" security against commercially motivated attackers has always demanded protection against skilled teams. 3. The commercial IT systems in regular use have never once, over 40 years, demonstrated the levels of security needed to protect against such commercially motivated attacks. These techniques and processes have failed continuously to achieve adequate security despite claiming to solve it every year for 40 years. At some point you need to stop listening. 4. There are systems that did achieve adequate security against commercially motivated attackers and even security against state actors in those 40 years. These standards were not pie-in-the-sky unreachable goals. They were "practical" if you actually cared about security. 5. KVM is in the commercial IT system category, being derived and developed by people who have never once deployed, developed, designed, or likely even seen a system known to have adequate security. Such systems DO NOT get the benefit of the doubt. There is no metric of evaluation, no means of evaluating if the theorized improvements achieve adequacy. In fact, you would be hard-pressed to find literally anybody who would stand up and say: "KVM is unhackable by any team with a budget of 10 M$" (budget including the average salary for the team members so you do not get a talented team doing it just to prove it can be done). Nobody will vouch for the system claiming it achieves even the bare-minimum requirements for adequate security I stated above. That it might be "better" than the Linux kernel is irrelevant; bad is also better than terrible, but it is still bad. At the end of the day they are not meaningfully different; they are all inadequate and unfit for purpose. So, you can disagree on two primary positions: 1. 10 M$ is too high of a standard for mid-size enterprise security. 2. Commercial IT systems, such as KVM, achieve the 10 M$ level. You could also argue that I am being reductive by defining security as the cost for a successful attack. But that is silly because it is the one metric that exactly aligns with the operational goals. Every other metric is a means of helping quantify the cost of a successful attack. To put it another way, if you had a magic genie that told you that number, you would not even bother with any other metrics; you have a direct line to what matters. In summary, it is none of the above. The betterness of KVM is irrelevant except if it achieves adequacy. There is no evidence of that and until there is, KVM is not categorically distinct from any other inadequate system. So, the original point still holds, the reason these "secure VM" techniques are not applied to make a "secure OS" is that there are no "secure VM" techniques to be found in that corner of the world.

kasey_junk 2y ago
I can disagree on one major point. There are talented teams that cost well over 10M annually tasked, effectively full time, with finding exploits in kvm. I’m still not sure I understand what that means for your argument but a kvm exploit, especially a jailbreak, would be one of the highest value exploits in the world.
- Veserv 2y ago
  And is the net result of those teams that they find, on average, fewer than 1 security vulnerability per year? That KVM has, on average, less than one security fix per year? To quote the KVM escape Google Project Zero published in 2021 [1]: "While we have not seen any in-the-wild exploits targeting hypervisors outside of competitions like Pwn2Own, these capabilities are clearly achievable for a well-financed adversary. I’ve spent around two months on this research, working as an individual with only remote access to an AMD system. Looking at the potential ROI on an exploit like this, it seems safe to assume that more people are working on similar issues right now and that vulnerabilities in KVM, Hyper-V, Xen or VMware will be exploited in-the-wild sooner or later." A single, albeit highly capable, individual found a critical vulnerability in 2 months of work. KVM was already mature and the foundation of AWS at that time and people were already saying that it was highly secure and that it must be highly secure since it would be such a high value target, so logically it must be secure since only an incompetent would poorly secure high value targets, thus reverse logic means it must be secure. Despite that, 2 person-months to find an escape. What can we conclude? They actually are incompetent at security because they did poorly secure high value targets, and that entire train of logic is just wishful thinking. Crowdstrike must have good deployment practices because it would be catastrophic if they, like, I dunno, mass pushed a broken patch and bricked millions of machines, and only an incompetent would use poor deployment practices on such a critical system, therefore they must have good deployment practices. Turns out, no, people are incompetent all the time. The criticality of systems is almost entirely divorced from those systems actually being treated critically unless you have good processes which is emphatically and empirically not the case in commercial IT software as a whole, let alone commercial IT software security. That quote further illustrates how, despite how easy such an attack was to develop, no in-the-wild exploits were observed. Therefore, the presence of absence of known vulnerabilities and "implicit 7-figure bounty"s is no indication that exploits are hard to develop. The entire notion of some sort of bizarre ambient, osmotic proof of security just wrong-headed. You need actual, direct audits, with no discovered exploits to establish concrete evidence for a level of security. If you put a team with a budget of 10 M$ on it and they find 10 vulnerabilities, you can be fairly confidence that the development processes can not weed out vulnerabilities that require 10 M$, or possibly even 1 M$ effort to identify. You need repeated competent teams to fail to find anything at a level of effort to establish any sense of a lower bound. Actually, now that I am looking at that post, it says: "Even though KVM’s kernel attack surface is significantly smaller than the one exposed by a default QEMU configuration or similar user space VMMs, a KVM vulnerability has advantages that make it very valuable for an attacker: ... Due to the somewhat poor security history of QEMU, new user space VMMs like crosvm or Firecracker are written in Rust, a memory safe language. Of course, there can still be non-memory safety vulnerabilities or problems due to incorrect or buggy usage of the KVM APIs, but using Rust effectively prevents the large majority of bugs that were discovered in C-based user space VMMs in the past. Finally, a pure KVM exploit can work against targets that use proprietary or heavily modified user space VMMs. While the big cloud providers do not go into much detail about their virtualization stacks publicly, it is safe to assume that they do not depend on an unmodified QEMU version for their production workloads. In contrast, KVM’s smaller code base makes heavy modifications unlikely (and KVM’s contributor list points at a strong tendency to upstream such modifications when they exist)." So, this post already post-dates the key technologies tptacek mentioned that supposedly made modern hypervisors so "secure" such as: "everything uses the same small KVM interface", "Maximalist C/C+ hypervisors have been replaced with lightweight virtualization, which codebases are generally written in memory-safe Rust". KVM, Rust for the VMM, despite that one person on the Google Project Zero team invalidated the security in 2 months. Goes to show how effective and secure it actually was after those vaunted improvements and how my prediction that it would be easily broken despite such changes was correct, where as tptacek got it wrong. [1] https://googleprojectzero.blogspot.com/2021/06/an-epyc-escape-case-study-of-kvm.html https://googleprojectzero.blogspot.com/2021/06/an-epyc-escap...

by Veserv 2y ago

kasey_junk 2y ago

I can disagree on one major point. There are talented teams that cost well over 10M annually tasked, effectively full time, with finding exploits in kvm. I’m still not sure I understand what that means for your argument but a kvm exploit, especially a jailbreak, would be one of the highest value exploits in the world.

Veserv 2y ago
And is the net result of those teams that they find, on average, fewer than 1 security vulnerability per year? That KVM has, on average, less than one security fix per year? To quote the KVM escape Google Project Zero published in 2021 [1]: "While we have not seen any in-the-wild exploits targeting hypervisors outside of competitions like Pwn2Own, these capabilities are clearly achievable for a well-financed adversary. I’ve spent around two months on this research, working as an individual with only remote access to an AMD system. Looking at the potential ROI on an exploit like this, it seems safe to assume that more people are working on similar issues right now and that vulnerabilities in KVM, Hyper-V, Xen or VMware will be exploited in-the-wild sooner or later." A single, albeit highly capable, individual found a critical vulnerability in 2 months of work. KVM was already mature and the foundation of AWS at that time and people were already saying that it was highly secure and that it must be highly secure since it would be such a high value target, so logically it must be secure since only an incompetent would poorly secure high value targets, thus reverse logic means it must be secure. Despite that, 2 person-months to find an escape. What can we conclude? They actually are incompetent at security because they did poorly secure high value targets, and that entire train of logic is just wishful thinking. Crowdstrike must have good deployment practices because it would be catastrophic if they, like, I dunno, mass pushed a broken patch and bricked millions of machines, and only an incompetent would use poor deployment practices on such a critical system, therefore they must have good deployment practices. Turns out, no, people are incompetent all the time. The criticality of systems is almost entirely divorced from those systems actually being treated critically unless you have good processes which is emphatically and empirically not the case in commercial IT software as a whole, let alone commercial IT software security. That quote further illustrates how, despite how easy such an attack was to develop, no in-the-wild exploits were observed. Therefore, the presence of absence of known vulnerabilities and "implicit 7-figure bounty"s is no indication that exploits are hard to develop. The entire notion of some sort of bizarre ambient, osmotic proof of security just wrong-headed. You need actual, direct audits, with no discovered exploits to establish concrete evidence for a level of security. If you put a team with a budget of 10 M$ on it and they find 10 vulnerabilities, you can be fairly confidence that the development processes can not weed out vulnerabilities that require 10 M$, or possibly even 1 M$ effort to identify. You need repeated competent teams to fail to find anything at a level of effort to establish any sense of a lower bound. Actually, now that I am looking at that post, it says: "Even though KVM’s kernel attack surface is significantly smaller than the one exposed by a default QEMU configuration or similar user space VMMs, a KVM vulnerability has advantages that make it very valuable for an attacker: ... Due to the somewhat poor security history of QEMU, new user space VMMs like crosvm or Firecracker are written in Rust, a memory safe language. Of course, there can still be non-memory safety vulnerabilities or problems due to incorrect or buggy usage of the KVM APIs, but using Rust effectively prevents the large majority of bugs that were discovered in C-based user space VMMs in the past. Finally, a pure KVM exploit can work against targets that use proprietary or heavily modified user space VMMs. While the big cloud providers do not go into much detail about their virtualization stacks publicly, it is safe to assume that they do not depend on an unmodified QEMU version for their production workloads. In contrast, KVM’s smaller code base makes heavy modifications unlikely (and KVM’s contributor list points at a strong tendency to upstream such modifications when they exist)." So, this post already post-dates the key technologies tptacek mentioned that supposedly made modern hypervisors so "secure" such as: "everything uses the same small KVM interface", "Maximalist C/C+ hypervisors have been replaced with lightweight virtualization, which codebases are generally written in memory-safe Rust". KVM, Rust for the VMM, despite that one person on the Google Project Zero team invalidated the security in 2 months. Goes to show how effective and secure it actually was after those vaunted improvements and how my prediction that it would be easily broken despite such changes was correct, where as tptacek got it wrong. [1] https://googleprojectzero.blogspot.com/2021/06/an-epyc-escape-case-study-of-kvm.html https://googleprojectzero.blogspot.com/2021/06/an-epyc-escap...