6 ms·

Not a great paper, hence why the "advisories" are so short. All they've done is show that some products meet their advertised threat model. Intel has a solution

Not a great paper, hence why the "advisories" are so short. All they've done is show that some products meet their advertised threat model. Intel has a solution: upgrade your CPU. AMD do not. Once again Intel are ahead when it comes to confidential computing. The story here is a little complex. Some years ago I flew out to Oregon and met the designers of SGX. It's a good design and it's to our industries shame that we haven't used it much, as tech like this can solve a lot of different security and privacy problems. SGX as originally designed was not attackable this way. This kind of RAM interposer attack was anticipated and the hardware was designed to block it by using memory integrity trees, in other words, memory was not only being encrypted by the CPU on the fly (cheap) but RAM was also being hashed into a kind of Merkle tree iirc which the CPU would check on access. So even if you knew the encryption key, you could not overwrite RAM or play games with it. It's often overlooked but encryption doesn't magically make storage immutable. An attacker can still overwrite encrypted data, delete parts, replay messages, redirect your write requests or otherwise mess with it. It takes other cryptographic techniques to block those kinds of activities, and "client SGX" had them (I'm not sure SEV ever did). This made sense because SGX design followed security best practices, namely, you should minimize the size of the trusted computing base. More code that's trusted = more potential for mistakes = more vulnerabilities. So SGX envisions apps having small trusted "enclaves", sort of like protected kernels, that untrusted code then uses. Cryptography ties the whole thing together. In a model like this an enclave doesn't need a large amount of RAM because the bulk of the app is running outside of the TCB. Unfortunately, at this point Intel discovered a sad and depressing but fundamental truth about the software industry: our tolerance for taking on additional complexity to increase security rounds to zero, and the enclave programming model is complex. The number of people who actually understand how to use enclaves as a design primitive can probably fit into a single large conference room. The number of apps that used them in the real world, in a way that actually met some kind of useful threat model, I'm pretty sure is actually near zero [1]. This isn't the fault of SGX! From a theoretical perspective, it is sound and the way it was meant to be used is sound. But actually exploiting it properly required more lift than the software industry could give. For example, to obtain the biggest benefits (SaaS you can use without trusting it) would have required some tactical changes to web browsers, changes to databases, changes to how such apps are designed and so on. Nobody tried to coordinate such changes and Intel, being a business, could not afford to wait for a few decades to see if anyone picked up the ball on that (their own software engineering efforts were good as far as they went but not ambitious enough to pull off the vision). Instead what happened is that potential customers said to them (and AMD): look, we want extra security, but we don't want to make any effort. We want to just run containers/VMs in the cloud and have them be magically secure. Intel looked at what they had and said OK, well, um, I guess we can maybe run bigger apps inside enclaves. Maybe even whole VMs. So they went away and did a redesign, but then they hit a fundamental physics problem: as you expand the amount of encrypted and protected RAM the Merkle tree protecting its integrity gets bigger and bigger. That means every cache miss has to recursively do a tree walk to ensure the data read from RAM is correct. And that kills performance. For small enclaves the tree is shallow and the costs aren't too bad. For big enclaves, well ... the performance rapidly becomes problematic, especially as the software inside expects to be running at full speed (as we are no longer designing with SGX in mind now but just throwing any old stuff into the protected space). So Intel released a new version gamely called "scalable SGX" which scaled by removing the memory integrity tree. As the point of that tree was to stop bus interposer attacks, they provided an updated threat model that excluded them. The tech is still useful and blocks some attacks (e.g. imagine a corrupted developer on a cloud hypervisor team). But it was no longer as strong as it once was. Knowing this, they set about creating yet another memory encryption tech called TME-MK which assigns each memory page its own unique encryption key. This prevented the kind of memory relocation attacks the "Battering RAM" interposer is doing. They also released a new tech that is sort of like SGX for whole virtual machines, formally giving up on the idea the software industry would ever actually try to minimize TCBs. Sad, but there we go. Clouds have trusted brands and people aren't bothered by occasional reports of global root exploits in Azure. It would take a step change event to get more serious about this stuff. [1] You might think Signal would count. Its use of SGX does help to reduce the threat from malicious or hacked cloud operators, but it doesn't protect against the operators of the Signal service themselves as they control the client.

michaelt 1y ago
> This made sense because SGX design followed security best practices, namely, you should minimize the size of the trusted computing base. More code that's trusted = more potential for mistakes = more vulnerabilities. So SGX envisions apps having small trusted "enclaves", sort of like protected kernels, that untrusted code then uses. Let's say I was Google building gmail. What would I put in the 'secure enclave' ? Obviously the most important thing is the e-mail bodies, that's the real goldmine. And of course the logins / user session management. The SSL stuff naturally, the certificate for mail.google.com is priceless. And clearly if an attacker could compromise the server's static javascript it'd be game over, security-wise. At that point, is there anything left outside the secure enclave?
- mike_hearn 1y ago
  Right - it's not obvious, is it? I've spent a lot of time on the question of how to build such architectures and in the past (with a small team) built a platform called Conclave that made enclave development much easier. You could write enclaves in Java or JavaScript and get high level APIs to help architect a secure system. I should try and find time to write up the results of the investigations I did back then as lots of research was done into this topic. There are different threat models. Minimizing redesign costs means just lift'n'shifting into encrypted VMs. You then run cron jobs that connect to them and verify their remote attestations from time to time. This is a very weak approach, but it can help keep out some kinds of nosy neighbours who found a hypervisor/cloud auth exploit like the recent Azure failure, it reduces the number of cloud employees who can hack your stuff, and has a lot of other good benefits. That's why Intel and AMD are focusing on this easier target now. It doesn't provide your users any privacy against the email service provider (ESP), and some cloud employees can still beat you in various ways so it's no protection against a conspiracy e.g. US Gov wants your bytes. But it's got some security value and is a good place to start. But let's now minimize the size of the trusted computing base (TCB) by doing it with small enclaves instead of VMs. There can be networks of cooperating enclaves, that's OK. Thinking about email for a second, the design space is huge and this is an HN comment not a design doc. I will simplify aggressively to save space so we're going retro: IMAP, S/MIME and LDAP. In other words, we start from the basic infrastructure of an end-to-end encrypted email system. Passkeys can be used to set up a key pair that's backed up for the user. We assume slightly upgraded email clients from trusted suppliers that aren't the ESP, like Apple, Microsoft etc. We can then use enclaves to restore features that users expect from hosted email like server-side search and spam filtering. This is just an example of how to design enclave architectures, I'm not trying to show solutions to every threat at once. We will compromise on auth; identity will be supplied by the ESP. If a user wishes to take the ESP's LDAP servers out of the TCB they can exchange S/MIME certificates out of band to verify they match, or upload them to a variety of providers that are then all checked, use certificate transparency logs, etc. (by "the user can" I mean their upgraded email client implements these workflows). So what can we move out of the TCB when using SGX instead of SEV or TDX? The OS, obviously. The database, message queuing. Primary database keys are plaintext and sorted (there are ways to fix this leak), values are encrypted under an enclave key protected with AES-GCM. The value contains the hashed key and column name so we can verify the untrusted world gave us the right value to match a key. There are schemes that extend this to block replay attacks, but they're too large to fit in this margin. What else? We can move out TCP, TLS, IMAP. In other words, enclaves will be treated as semi-pure functions that have encrypted attestable memory spaces and an ability to derive their own private keys. They won't be making system calls or running full blown containers. The software running inside the enclave is designed for enclaves. First problem: how can we do indexing or spam filtering if the email is encrypted under a key the server doesn't have? The answer is obvious: the user uploads their private key to the enclaves! The process is: do a remote attestation with e.g. the spam filtering enclave, as part of which you learn the hash and/or signing key of the code running inside it. You also get a certificate chain showing the enclave is code signed by a trusted security firm that's audited the enclave source code for vulns, ensured it doesn't leak mail and so on. The email client verifies the RA protocol and certificate chain, becoming convinced in the process that the memory space of the spam filtering engine is secure and it will obey the social contract. Having done that it then uploads the user's private key to the enclave in an encrypted message tunnelled through TLS (two layers of encryption), the enclave then re-encrypts it and requests the untrusted world to store it to disk in the database. Now when an email is delivered the delivery and queueing infrastructure (all outside the TCB) hands it off to an (untrusted) spam server which loads the enclave into RAM, gives it the S/MIME message, loads the user's encrypted private key from the database, gives it spam scores for visible metadata like sending domain, hands all that off to the enclave and requests a classification. The enclave decrypts the user's private key, decrypts the S/MIME message, runs some classification on it and returns the result. For indexing it works similarly except the enclave has to encrypt the posting lists. If you want to get really fancy you have to hide access patterns as well to avoid statistical inference of likely email contents by looking at how popular certain posting lists are and such; it gets complicated fast but encrypting whole VMs doesn't actually block such attacks so you have to bite the bullet anyway. Notice how nearly everything about this system runs outside enclaves, yet, the ESP still can't read your mail. You do need help from third parties - someone has to write your upgraded email client, someone has to audit the enclave code the ESP runs, someone has to check the ESP isn't advertising the wrong public key for your username. But this is all quite tractable.

michaelt 1y ago

> This made sense because SGX design followed security best practices, namely, you should minimize the size of the trusted computing base. More code that's trusted = more potential for mistakes = more vulnerabilities. So SGX envisions apps having small trusted "enclaves", sort of like protected kernels, that untrusted code then uses. Let's say I was Google building gmail. What would I put in the 'secure enclave' ? Obviously the most important thing is the e-mail bodies, that's the real goldmine. And of course the logins / user session management. The SSL stuff naturally, the certificate for mail.google.com is priceless. And clearly if an attacker could compromise the server's static javascript it'd be game over, security-wise. At that point, is there anything left outside the secure enclave?

mike_hearn 1y ago
Right - it's not obvious, is it? I've spent a lot of time on the question of how to build such architectures and in the past (with a small team) built a platform called Conclave that made enclave development much easier. You could write enclaves in Java or JavaScript and get high level APIs to help architect a secure system. I should try and find time to write up the results of the investigations I did back then as lots of research was done into this topic. There are different threat models. Minimizing redesign costs means just lift'n'shifting into encrypted VMs. You then run cron jobs that connect to them and verify their remote attestations from time to time. This is a very weak approach, but it can help keep out some kinds of nosy neighbours who found a hypervisor/cloud auth exploit like the recent Azure failure, it reduces the number of cloud employees who can hack your stuff, and has a lot of other good benefits. That's why Intel and AMD are focusing on this easier target now. It doesn't provide your users any privacy against the email service provider (ESP), and some cloud employees can still beat you in various ways so it's no protection against a conspiracy e.g. US Gov wants your bytes. But it's got some security value and is a good place to start. But let's now minimize the size of the trusted computing base (TCB) by doing it with small enclaves instead of VMs. There can be networks of cooperating enclaves, that's OK. Thinking about email for a second, the design space is huge and this is an HN comment not a design doc. I will simplify aggressively to save space so we're going retro: IMAP, S/MIME and LDAP. In other words, we start from the basic infrastructure of an end-to-end encrypted email system. Passkeys can be used to set up a key pair that's backed up for the user. We assume slightly upgraded email clients from trusted suppliers that aren't the ESP, like Apple, Microsoft etc. We can then use enclaves to restore features that users expect from hosted email like server-side search and spam filtering. This is just an example of how to design enclave architectures, I'm not trying to show solutions to every threat at once. We will compromise on auth; identity will be supplied by the ESP. If a user wishes to take the ESP's LDAP servers out of the TCB they can exchange S/MIME certificates out of band to verify they match, or upload them to a variety of providers that are then all checked, use certificate transparency logs, etc. (by "the user can" I mean their upgraded email client implements these workflows). So what can we move out of the TCB when using SGX instead of SEV or TDX? The OS, obviously. The database, message queuing. Primary database keys are plaintext and sorted (there are ways to fix this leak), values are encrypted under an enclave key protected with AES-GCM. The value contains the hashed key and column name so we can verify the untrusted world gave us the right value to match a key. There are schemes that extend this to block replay attacks, but they're too large to fit in this margin. What else? We can move out TCP, TLS, IMAP. In other words, enclaves will be treated as semi-pure functions that have encrypted attestable memory spaces and an ability to derive their own private keys. They won't be making system calls or running full blown containers. The software running inside the enclave is designed for enclaves. First problem: how can we do indexing or spam filtering if the email is encrypted under a key the server doesn't have? The answer is obvious: the user uploads their private key to the enclaves! The process is: do a remote attestation with e.g. the spam filtering enclave, as part of which you learn the hash and/or signing key of the code running inside it. You also get a certificate chain showing the enclave is code signed by a trusted security firm that's audited the enclave source code for vulns, ensured it doesn't leak mail and so on. The email client verifies the RA protocol and certificate chain, becoming convinced in the process that the memory space of the spam filtering engine is secure and it will obey the social contract. Having done that it then uploads the user's private key to the enclave in an encrypted message tunnelled through TLS (two layers of encryption), the enclave then re-encrypts it and requests the untrusted world to store it to disk in the database. Now when an email is delivered the delivery and queueing infrastructure (all outside the TCB) hands it off to an (untrusted) spam server which loads the enclave into RAM, gives it the S/MIME message, loads the user's encrypted private key from the database, gives it spam scores for visible metadata like sending domain, hands all that off to the enclave and requests a classification. The enclave decrypts the user's private key, decrypts the S/MIME message, runs some classification on it and returns the result. For indexing it works similarly except the enclave has to encrypt the posting lists. If you want to get really fancy you have to hide access patterns as well to avoid statistical inference of likely email contents by looking at how popular certain posting lists are and such; it gets complicated fast but encrypting whole VMs doesn't actually block such attacks so you have to bite the bullet anyway. Notice how nearly everything about this system runs outside enclaves, yet, the ESP still can't read your mail. You do need help from third parties - someone has to write your upgraded email client, someone has to audit the enclave code the ESP runs, someone has to check the ESP isn't advertising the wrong public key for your username. But this is all quite tractable.