7 ms·
HSMs are shit. In a previous role we used a major vendor's HSM to protect our private keys. VERY expensive kit, more expensive than the load balancers and serv
by sofaofthedamned 10y ago
HSMs are shit.
In a previous role we used a major vendor's HSM to protect our private keys. VERY expensive kit, more expensive than the load balancers and servers combined.
We needed to use Elliptic Curve keys for a particular customer - so it got even more expensive as we had to buy:
1. A license from the LB vendor to use the HSM
2. A licence from the HSM vendor to use EC with the LB.
... even though they trumpeted these announcements of how radically great they were together we found:
1. The integration didn't work, full stop.
2. The version of OpenSSL we had to use (supplied) was about 18 months out of date
3. The specially b0rked version of OpenSSL supplied didn't support EC via a HSM
Even better - when Heartbleed came out I had a patch from RedHat on day 1. The load balancer?
Nope - nothing on their website - I had to create a ticket which said 'we are aware of the issue', at which point the ticket was closed. I questioned this and was told they couldn't keep it open, I had to create a new ticket every few weeks to find out whether they'd actually deigned to assign a bug id to the issue.
The HSM vendor just said nothing, zero, until a new version of the firmware was silently released 4 months later.
The whole industry is shit. I'd rather have a farm of Yubikeys than one of those HSMs.
- nailer 10y agoWant to (please) name the vendor?
- sofaofthedamned 10y agoNope, they'll know who they are and i'm not dealing with these things anymore.
- akytt 10y agoWhy do you have your HSM where Heartbleed matters?
- mentat 10y agoThis exactly. You use a HSM so it doesn't matter if your server gets popped.
- deleted 10y ago[deleted]
- xyzzy123 10y agoI think one should be really careful making statements like this. In the case where an HSM is used for e.g. database encryption, one absolutely still cares. The attacker can politely ask the HSM to decrypt all the data. On the bright side, you didn't lose the key, and your HSM should at least securely log the fact that all your data just walked out the door. For SSL offload the situation is less dire, but it is totally realistic that heartbleed might cause issues for a system with HSM assisted offload (e.g. leaking plaintext) because the protocol processing happens in software. Any sane architecture should protect the actual private key though. In general, the right way to think about HSMs is that they protect your keys, not your data. If you care more about the data than the keys, you absolutely need to care about the security of your host.
- hinkley 10y agoI used HSMs for a code signing project, and once someone can borrow your key to sign malicious payloads, it really doesn't matter if they have your signing key or not. You still have to get a new key, and clean up the mess.
- mentat 10y agoAs far as I understand heartbleed, the core issue was that you could search memory for the private key and then do whatever you wanted with it. With an HSM, that wouldn't be an issue. However, it's true that you could search for other data as well and that data could be sensitive, just not quite as sensitive (depending on your industry and architecture, of course).
- xyzzy123 10y agoI agree with you -- it depends on your threat model. If state actors or universal mitm or coffee shop attacks are primary threats then loss of private key is a very bad outcome for SSL offload. And this is what SSL is meant to help woth. For many (more?) sites though, the issue was loss of session plaintext (e.g api creds, basic auth creds, pii or credit cards). For a lot of applications where just hitting the webserver remotely is a lot easier than MITM, this was actually the more severe problem.
- tristor 10y agoDon't forget broken HA features. HSMs are a great idea in theory that are ruined by incompetent companies making them. They soon become a gun your org holds to its own head with the outcome of a data loss incident or a hack because of their janky OpenSSL integrations.
- sofaofthedamned 10y agoThat was exactly my problem. Also, the consultant they sent to help us with it didn't know anything about Linux when they knew that was our platform. I'd love to get hold of one of my old units from that vendor and try to attack it. I assume it hasn't been done as they're so expensive. It's basically the ultimate in security through obscurity, as it's too expensive to get one you're prepared to break.
- tluyben2 10y agoWe have a bunch of HSMs and we had to sign a quazillion pages which at least deny us and anyone we hire or ask from trying to temper with the machines. Also most of them you cannot buy; they remain owned by the vendor.
- pmarreck 10y agoIs there any way to do what a HSM does, in software? (Maybe with OS support)
- cyphar 10y agoYes, but then you lose the whole benefit of a HSM -- that keys generated in the HSM are physically protected from being extracted. Aside from that, a HSM is just an implementation of regular crypto algorithms.
- tjohns 10y agoIf your definition of an HSM is just "hides my key material from my application server", then sure. This is basically the idea behind HashiCorp's Vault. However, you'd still be vulnerable to someone gaining root access on your Vault server. A true HSM uses physically secure memory and a dedicated cryptoprocessor to make it impossible for the secret to ever be leaked, even if the host OS is compromised. This isn't really something that's possible in software alone. (In fact, the Vault documentation even mentions that you might still want to use a dedicated HSM for security, since they implement a plugable mechanism for secret storage.) That said... you could use a TPM chip or ARM TrustZone to implement true TPM functionality. It's still a hardware based solution, but it's hardware that's much more affordable and possibly already inside your computer.
- helloiamaperson 10y agotake a look at my post from earlier: https://news.ycombinator.com/item?id=13031870 https://news.ycombinator.com/item?id=13031870 . I'd be interested to hear your (and others') thoughts.
- EvilTerran 10y agoNot really, no. AIUI, the whole point of having a hardware security module is that it only leaves a very narrow attack surface (ie, the protocol for communicating with the device) for malicious software to try to exfiltrate the sensitive data through, with no way to get around it - so if that interface is sufficiently hardened, you should be secure regardless of any weaknesses elsewhere in the system. In contrast, while you could think of, say, an ssh key agent as a sort of "software security module", such things are inherently vulnerable to low-level attacks that side-step the intended interface. Say, a kernel exploit that grabs the decrypted key straight from the agent's memory. So if you need to keep your secrets safe even if you unwittingly use them on a comprised OS, you can only pull that off with a hardware module.
- yuhong 10y agoOT, but I wonder how many of these LBs are/were TLS version or extension intolerant: http://www.bus.umich.edu/KresgePublic/Journals/Gartner/research/117000/117029/117029_2.html http://www.bus.umich.edu/KresgePublic/Journals/Gartner/resea...
- EvanAnderson 10y agoWe started looking into an HSM solution for a Customer and quickly found out that the entrenched "big" HSM manufacturers are all about lock-in, questionable documentation (and strong insistence that we use their very expensive service rather than attempt our own integration), severe granularity in licensing features, and _insane_ prices (on a 3-5 year product replacement 'treadmill'). We ended up using the Nitrokey HSM platform and have been very pleased with the results.
- pharaohgeek 10y agoThey're not (necessarily) shit. They're just appropriate for a VERY specific purpose. A lot of companies that use them aren't using them right, or are using them in a scenario in which they are overkill. My company (medium-sized financial firm) uses them to protect the keys for our internal PKI. For these types of high-security situations, an HSM is ideal (if configured and managed properly). But I've seen places that use them to protect the SSL keys for webservers. While capable of doing that, an HSM is really overkill for that purpose. And, often results in them being mismanaged in the long run because the wrong teams end up responsible for them. That, and it's very important to choose the right vendor. I've worked with pretty much all of them over the years, and some of them can be a nightmare to deal with. Some of the devices themselves are INCREDIBLY frustrating and should never have seen the light of day (much like some of their sales staff!)
- matthiasb 10y agointroduction: "we used a major vendor's HSM to protect our private keys" conclusion: "The whole industry is shit." You had a bad experience with one vendor, not with the industry.