3 ms·

Guidance on implementing cryptography as a developer

33 points by ementally 3y ago

User23 3y ago
Back in the day a former NSA employee told me that they almost never crack the encryption algorithm. Rather, they crack errors in the implementation.
- guiambros 3y ago
  Indeed. You can find tons of similar similar stories iin "Crypto", by Stephen Levy [1]. Great book if you enjoy computer history or interested in infosec. [1] https://www.amazon.com/Crypto-Rebels-Government-Privacy-Digital/dp/0140244328 https://www.amazon.com/Crypto-Rebels-Government-Privacy-Digi...
geph2021 3y ago
Avoid : OpenSSL Sometimes you're using OpenSSL and you don't realize it. For example, the python cryptography library relies on OpenSSL, although in such a case you're relying on someone else having properly used the OpenSSL library.
beloch 3y ago
The short, short version: Don't. The short version: Don't. Use somebody else's implementation if at all possible, because you're almost certainly going to F it up.
matheusmoreira 3y ago
> most languages provide access to old algorithms (e.g. MD5 and SHA1) Why is this a problem? Isn't it fine to use them for non-cryptogrphic purposes?
- wmf 3y ago
  It's a bad code smell. We discussed this the other day: "If you need security don't use MD5 (because it's no longer secure). If you don't need security, use something faster than MD5." https://news.ycombinator.com/item?id=38173061 https://news.ycombinator.com/item?id=38173061
  - quesera 3y ago
    It is not a bad code smell for a language to provide access to old algorithms. Sometimes existing implementations require you to work with algorithms that are not ideal, but still adequate, today. Languages and stdlibs should obviously provide that ability!
    - wmf 3y ago
      I agree that languages and libraries should maintain backwards compatibility. It's a bad code smell to use broken algorithms unless you absolutely have to.
  - matheusmoreira 3y ago
    Maybe some old software uses MD5 as a database primary key for files and rehashing everything would just be too painful.
tptacek 3y ago
A writer I used to read did an annual Christmas guide for kitchen gifts. It started out interestingly, or at least idiosyncratically (they were weirdly enamored with the Thermomix, a blender that also cooks food and is only really available in Europe). But as it went on, they'd invariably get to the point where they were writing paragraphs about individual spoons, and not interesting ones but the kinds you can get off the shelf at Williams Sonoma or whatever. It became clear that they just really liked writing about their kitchen. That's the vibe here. There's something to be said for an article that forthrightly declares "here are all my idiosyncratic opinions about implementing cryptography". But this one instead purports to be offering guidance to developers. That's a little troublesome, because reasons: * Some of these opinions are pretty outré, such as the encouragement to compose your own CTR-based encrypt-then-MAC out of XChaCha20 and a Blake2b KMAC because... key committing? It's fine to have this opinion and to argue it forcefully but if you're offering guidance, it should probably include the fact that in most situations, simply using a Seal/Unseal AEAD is a more-than-cromulent call. * Some of them seem factually off? I'm no advocate for RSA, but the idea that you should avoid RSA-KEM because it's "worse than using hybrid encryption" --- RSA-KEM is a construction for doing hybrid encryption! (RSA-KEM, roughly: fill the RSA modulus with a full-width random number, then HKDF it to generate a symmetric key). * Some of this gets into Applied Cryptography levels of trivia; for instance: it is probably not all that important to tell developers to avoid Streebog, Meowhash, and SHAcrypt ("nobody uses this, and I’ve never even seen it in a cryptographic library"). * A bunch of things have landed on this guy's shitlist for no reason other than not having seen them in many places? This includes SIV (this author really doesn't like SIV), Blake2s, Kangaroo12, any PAKE, or any PQC (note: this follows an earlier recommendation to use PQC). "They're all bad". * There's a bunch of random NSA FUD sprinkled across this; for instance, "SHA2 was designed behind closed doors at the NSA", as if any serious practitioner avoids SHA2, or "P-256 is probably the most popular curve, the seeds for these curves haven’t been explained, which is not a good look considering that Dual_EC_DRBG was a NIST standard despite containing an NSA backdoor" --- Dual EC being basically unrelated to the NIST P-curves, you might as well apply the same logic to SHA2. None of this is like, disqualifying, especially if you're doing a "this is what makes me tick" kind of article, but I think the title it has right now is missing a word before "developer". Colin Percival kicked off this genre of articles with his "Cryptographic Right Answers", which I recall as essentially being a subtextual argument against elliptic curve cryptography. I found his recommendations a little odd (what set me off was probably "Use RSAES-OAEP with SHA256 as the hash function, MGF1+SHA256 as the mask generation function") and set out to write a "Crypographic Right Answers" that fit the zeitgeist as I understood it --- ChaPoly, Nacl/Sodium, Curve25519 and Ed25519. I updated that article at Latacora to be much more prescriptive, and have kind of regretted it; it had become a step back towards what Colin had done, a docket of fiddly decisions we would have made if you contracted us to design something, but not really a reflection of any kind of accepted best practices in our field. We have come full circle with stuff like this, which describes the best practices of exactly one developer in the world, the author. Again, fair enough. But maybe we can just kill this species of post off now.
candiddevmike 3y ago
If you're going to write things that touch crypto, make sure you test your stuff with inputs and outputs from other (known good) implementations. A lot of times the std lib provides primitives that give you plenty of rope to hang yourself with.
briHass 3y ago
The problem with pedantry over the actual crypto algorithms is that it almost never covers serious flaws in protecting the key material. This list only dedicates a few sentences that recommend clearing the memory used for keys/secrets, but noting GC languages provide no guarantees. For something like modern .NET, worrying about algos is bad advice. Use the library standard Data Protection APIs (in ASP.NET, but available for console apps also) which handle things like protecting/clearing key material in memory, key rotation, deriving context-specific sub keys from master keys, and even storing secrets in a manner that makes them far less likely to end up dumped to logs or floating around in memory. The algos (AES, HMACSHA2, PBKDF2) aren't the new hotness, but that tradeoff doesn't practically matter for normal attacks. Far more important is the simple API surface of Encrypt, Decrypt, and Protect/Verify password with all footguns removed. Persistence of keys is also abstracted and can be a simple as a file protected with a cert up to using a HSM with a change to one line of code.
- tptacek 3y ago
  I don't think this is an especially great critique of this list, because one of the most idiosyncratic things about it is that it expressly advocates constructions like AES+HMAC-SHA2 over misuse-resistant AEADs. (I mean, it recommends XChaCha and Blake2b KMAC, but the actual code to put those things together is basically the same as AES and HMAC-SHA2).
  - wolf550e 3y ago
    It is idiosyncratic but it's not a bad recommendation, right? AES+HMAC-SHA2 would be AES-CTR and you would need to handle the nonce yourself, and that's not great. XChaCha is extended nonce, which should be safer. But there is no standard for extended nonce AES. Filippo Valsorda wrote that he would like XAES-256-GCM/11 [1] XChaCha looks like a good recommendation. Because lacking key commitment can lead to problems, a scheme that also gives you key commitment is better [2]? So that's not crazy. What does Google Tink do by default / recommend? What does Sophie Schmieg or whoever recommend? 1 - https://words.filippo.io/dispatches/xaes-256-gcm-11/ https://words.filippo.io/dispatches/xaes-256-gcm-11/ 2 - https://eprint.iacr.org/2020/1153.pdf https://eprint.iacr.org/2020/1153.pdf

33 points by ementally 3y ago

Back in the day a former NSA employee told me that they almost never crack the encryption algorithm. Rather, they crack errors in the implementation.

guiambros 3y ago
Indeed. You can find tons of similar similar stories iin "Crypto", by Stephen Levy [1]. Great book if you enjoy computer history or interested in infosec. [1] https://www.amazon.com/Crypto-Rebels-Government-Privacy-Digital/dp/0140244328 https://www.amazon.com/Crypto-Rebels-Government-Privacy-Digi...

geph2021 3y ago

Avoid : OpenSSL Sometimes you're using OpenSSL and you don't realize it. For example, the python cryptography library relies on OpenSSL, although in such a case you're relying on someone else having properly used the OpenSSL library.

beloch 3y ago

The short, short version: Don't. The short version: Don't. Use somebody else's implementation if at all possible, because you're almost certainly going to F it up.

matheusmoreira 3y ago

> most languages provide access to old algorithms (e.g. MD5 and SHA1) Why is this a problem? Isn't it fine to use them for non-cryptogrphic purposes?

wmf 3y ago
It's a bad code smell. We discussed this the other day: "If you need security don't use MD5 (because it's no longer secure). If you don't need security, use something faster than MD5." https://news.ycombinator.com/item?id=38173061 https://news.ycombinator.com/item?id=38173061
- quesera 3y ago
  It is not a bad code smell for a language to provide access to old algorithms. Sometimes existing implementations require you to work with algorithms that are not ideal, but still adequate, today. Languages and stdlibs should obviously provide that ability!
  - wmf 3y ago
    I agree that languages and libraries should maintain backwards compatibility. It's a bad code smell to use broken algorithms unless you absolutely have to.
- matheusmoreira 3y ago
  Maybe some old software uses MD5 as a database primary key for files and rehashing everything would just be too painful.

tptacek 3y ago

A writer I used to read did an annual Christmas guide for kitchen gifts. It started out interestingly, or at least idiosyncratically (they were weirdly enamored with the Thermomix, a blender that also cooks food and is only really available in Europe). But as it went on, they'd invariably get to the point where they were writing paragraphs about individual spoons, and not interesting ones but the kinds you can get off the shelf at Williams Sonoma or whatever. It became clear that they just really liked writing about their kitchen. That's the vibe here. There's something to be said for an article that forthrightly declares "here are all my idiosyncratic opinions about implementing cryptography". But this one instead purports to be offering guidance to developers. That's a little troublesome, because reasons: * Some of these opinions are pretty outré, such as the encouragement to compose your own CTR-based encrypt-then-MAC out of XChaCha20 and a Blake2b KMAC because... key committing? It's fine to have this opinion and to argue it forcefully but if you're offering guidance, it should probably include the fact that in most situations, simply using a Seal/Unseal AEAD is a more-than-cromulent call. * Some of them seem factually off? I'm no advocate for RSA, but the idea that you should avoid RSA-KEM because it's "worse than using hybrid encryption" --- RSA-KEM is a construction for doing hybrid encryption! (RSA-KEM, roughly: fill the RSA modulus with a full-width random number, then HKDF it to generate a symmetric key). * Some of this gets into Applied Cryptography levels of trivia; for instance: it is probably not all that important to tell developers to avoid Streebog, Meowhash, and SHAcrypt ("nobody uses this, and I’ve never even seen it in a cryptographic library"). * A bunch of things have landed on this guy's shitlist for no reason other than not having seen them in many places? This includes SIV (this author really doesn't like SIV), Blake2s, Kangaroo12, any PAKE, or any PQC (note: this follows an earlier recommendation to use PQC). "They're all bad". * There's a bunch of random NSA FUD sprinkled across this; for instance, "SHA2 was designed behind closed doors at the NSA", as if any serious practitioner avoids SHA2, or "P-256 is probably the most popular curve, the seeds for these curves haven’t been explained, which is not a good look considering that Dual_EC_DRBG was a NIST standard despite containing an NSA backdoor" --- Dual EC being basically unrelated to the NIST P-curves, you might as well apply the same logic to SHA2. None of this is like, disqualifying, especially if you're doing a "this is what makes me tick" kind of article, but I think the title it has right now is missing a word before "developer". Colin Percival kicked off this genre of articles with his "Cryptographic Right Answers", which I recall as essentially being a subtextual argument against elliptic curve cryptography. I found his recommendations a little odd (what set me off was probably "Use RSAES-OAEP with SHA256 as the hash function, MGF1+SHA256 as the mask generation function") and set out to write a "Crypographic Right Answers" that fit the zeitgeist as I understood it --- ChaPoly, Nacl/Sodium, Curve25519 and Ed25519. I updated that article at Latacora to be much more prescriptive, and have kind of regretted it; it had become a step back towards what Colin had done, a docket of fiddly decisions we would have made if you contracted us to design something, but not really a reflection of any kind of accepted best practices in our field. We have come full circle with stuff like this, which describes the best practices of exactly one developer in the world, the author. Again, fair enough. But maybe we can just kill this species of post off now.

candiddevmike 3y ago

If you're going to write things that touch crypto, make sure you test your stuff with inputs and outputs from other (known good) implementations. A lot of times the std lib provides primitives that give you plenty of rope to hang yourself with.

briHass 3y ago

The problem with pedantry over the actual crypto algorithms is that it almost never covers serious flaws in protecting the key material. This list only dedicates a few sentences that recommend clearing the memory used for keys/secrets, but noting GC languages provide no guarantees. For something like modern .NET, worrying about algos is bad advice. Use the library standard Data Protection APIs (in ASP.NET, but available for console apps also) which handle things like protecting/clearing key material in memory, key rotation, deriving context-specific sub keys from master keys, and even storing secrets in a manner that makes them far less likely to end up dumped to logs or floating around in memory. The algos (AES, HMACSHA2, PBKDF2) aren't the new hotness, but that tradeoff doesn't practically matter for normal attacks. Far more important is the simple API surface of Encrypt, Decrypt, and Protect/Verify password with all footguns removed. Persistence of keys is also abstracted and can be a simple as a file protected with a cert up to using a HSM with a change to one line of code.

tptacek 3y ago
I don't think this is an especially great critique of this list, because one of the most idiosyncratic things about it is that it expressly advocates constructions like AES+HMAC-SHA2 over misuse-resistant AEADs. (I mean, it recommends XChaCha and Blake2b KMAC, but the actual code to put those things together is basically the same as AES and HMAC-SHA2).
- wolf550e 3y ago
  It is idiosyncratic but it's not a bad recommendation, right? AES+HMAC-SHA2 would be AES-CTR and you would need to handle the nonce yourself, and that's not great. XChaCha is extended nonce, which should be safer. But there is no standard for extended nonce AES. Filippo Valsorda wrote that he would like XAES-256-GCM/11 [1] XChaCha looks like a good recommendation. Because lacking key commitment can lead to problems, a scheme that also gives you key commitment is better [2]? So that's not crazy. What does Google Tink do by default / recommend? What does Sophie Schmieg or whoever recommend? 1 - https://words.filippo.io/dispatches/xaes-256-gcm-11/ https://words.filippo.io/dispatches/xaes-256-gcm-11/ 2 - https://eprint.iacr.org/2020/1153.pdf https://eprint.iacr.org/2020/1153.pdf