4 ms·

The reason why cryptography is so bad. Is this elitist culture in cryptography. Alot of algorithms are described incomprehensively. Let me give you an exampl

by max_ 2y ago

The reason why cryptography is so bad. Is this elitist culture in cryptography. Alot of algorithms are described incomprehensively. Let me give you an example. You might get a well documented specification for implementation of ECDSA. But it will lack something very 2 important concepts. 1. You should only use it with "safe curves" 2. It's does provide you with a list of "Safe curves" Same goes for Shamir Secret Sharing. You can know how to implement the algorithm. But there is so much extra "insider knowledge" or "tribal knowledge" that isn't obvious to non-cryptographers, that you need, for it to be secure. Such knowledge is often (if not always) not documented. It's in obscure forums, papers and blog posts like this one. There is no comprehensive encyclopedia for cryptography algorithms. Developers write bad encryption code because cryptographers have bad documentation. I would blame the cryptographers, not the developers.

tptacek 2y ago
I don't understand what "blaming" the cryptographers gets you. Your system is either secure or it isn't. If you ship something insecure, that's on you.
- AlotOfReading 2y ago
  There's definitely degrees of security that I'm having to constantly remind our security folks of when they aren't forced to deal with the costs of having expensive threat models.
  - xvector 2y ago
    If you're having to constantly remind your security engineers about "degrees of security," they may be trying to tell you that your threat model is unsustainably/nonsensically/dangerously lax.
    - AlotOfReading 2y ago
      The part I've been struggling with is getting them to put any threat model in writing that's not quote, "everything".
- dns_snek 2y ago
  "Blame" isn't the most productive way to frame the conversation, but as GP mentioned, there aren't many resources for developers to learn about cryptography. We're in desperate need of comprehensive resources that can (at a minimum) help us answer: 1. Has this problem been solved using a proven cryptographic protocol? Which guarantees does this protocol offer? What are all the important considerations? 2. What's the next closest thing? 3. Can we safely modify the protocol in any way to accommodate our use case, and what are some common pitfalls? Some people will ignore all that advice and do dumb things anyway, but if we want systems to be more secure, it's on cryptographers to make cryptography more approachable and on developers to actually listen when they do.
cowsaymoo 2y ago
What a coincidence! I was just browsing the Shamir's Secret Sharing Wikipedia page 30 seconds ago. There is a python implementation on it and I was worried the same exact thing as you before opening HN. So maybe we could start with that one. Is that code implementation sufficiently secure and well documented? https://en.wikipedia.org/wiki/Shamir's_secret_sharing?wprov=sfti1#Python_code https://en.wikipedia.org/wiki/Shamir's_secret_sharing?wprov=...
- some_furry 2y ago
  https://zkdocs.com https://zkdocs.com has a whole chapter on Shamir's Secret Sharing. (Ironically, this stuff is documented, cryptographers just aren't good at marketing.) https://www.zkdocs.com/docs/zkdocs/protocol-primitives/shamir/ https://www.zkdocs.com/docs/zkdocs/protocol-primitives/shami...
  - unscaled 2y ago
    I think this chapter just proves GP's point, rather than refuting it. It's much cleaner than the Wikipedia article and it mentions the zero share problem, but it doesn't mention any other concern like constant time implementations, cache side channel attacks or the leading zero coefficient that I pointed to at the other post. It's a good guide to learn about Secret Sharing, but it doesn't give you even 1% of what you need to know implement it safely by yourself.
    - some_furry 2y ago
      > or the leading zero coefficient that I pointed to at the other post. That isn't actually a problem, although I've seen a lot of people that think it is. The problem is framed as "if you have a leading zero coefficient, it's equivalent to a threshold of t-1", but you'd need a zero leading coefficient for every polynomial. With a 32 byte secret and GF(2^8), you expect at least 1 leading zero coefficient in 11% of random secrets, but a threshold reduction from t to t-1 only occurs with 2^-256 probability (that is, every leading coefficient has to be 0). You might think you can detect this condition, but SSS is kind of like a one-time pad if you don't have sufficient shares. > it doesn't mention any other concern like constant time implementations, cache side channel attacks These are table stakes for secure cryptography. ZKDocs is a guide to algorithms, not implementations.
alfiedotwtf 2y ago
It's not insider knowledge, it's just the collective experience people have built up over time after discovering issues with classes of ciphers. It would be nice to have a standard checklist that everyone could refer to (think OWASP Top Ten) while reviewing a cryptosystem.
- ikiris 2y ago
  The checklist is really simple: did you use a verified product or did you try to roll your own and fail?
- maqp 2y ago
  Cryptography isn't like that, it doesn't boil down to 10 simple steps. If you want ten, it's happening at higher level 1. Never invent your own primitives. Only professional cryptographers do that in algorithm competitions like AES, SHA-3, or PHC. 2. For interoperability, use BoringSSL, LibreSSL etc. Hire developer who knows how to do this. Or, hire a professional applied cryptographer to build TLS from low level primitives if you have a reason it's needed. 3. For closed ecosystems, use opinionated, misuse-resistant, high level libraries like libsodium, or libraries binding to it https://libsodium.gitbook.io/doc/bindings_for_other_languages https://libsodium.gitbook.io/doc/bindings_for_other_language... 4. Use kernel CSPRNG to generate all secrets. 5. Have the implementation audited by professionals. E.g. Kudelski Security or NCC Group. 6. Have a bug bounty program that compensates well, and that allows disclosure of the attack. 7. If you're in a market where the competition has open source clients, you should have an open source client too, so that anyone can verify your cryptography is correctly implemented. 8. Keep anything that's WIP, under "research prototype, do not use in production" banner. That's not a cone of shame. If Dan Boneh can do it (https://crypto.stanford.edu/balloon/ https://crypto.stanford.edu/balloon/), so can you. 9. Realize that 90% of the work in cryptography is key management. Use OS keyring when possible. 10. Deploy end-to-end encryption everywhere you can. Data is a toxic asset. https://www.schneier.com/blog/archives/2016/03/data_is_a_toxic.html https://www.schneier.com/blog/archives/2016/03/data_is_a_tox... So yeah, sorry, no "Use XChaCha20-Poly1305 or AES-GCM only", or "Use X25519/ed25519" or "remember to hash the X25519 shared secret". That's too in-depth for a ten point list. At that level, it's more of a top-ten book list.
maqp 2y ago
>Is this elitist culture in cryptography. It's elitist in the same way quantum physics, string theory and molecular biology are. A hard science with lot's of moving parts not everyone can manage. >But there is so much extra "insider knowledge" or "tribal knowledge" that isn't obvious to non-cryptographers, that you need, for it to be secure. Misuse-resistant cryptography is an open research problem. Claiming it's "tribal knowledge" makes it seem nefarious. Academics are not always good teachers, and collecting all the lessons from all papers you read into a book, is time away from doing more research that advances your career. >Such knowledge is often (if not always) not documented. It's in obscure forums, papers and blog posts like this one. You can pay for the papers, and you can pay for the school to learn to understand those 'obscure' papers. Or you can hire someone who has done that. If you're a developer expected to implement algorithms of theoretical physics, you're not complaining about the papers being obscure, you're hiring a physicist with double-major in computer science. There's way too much frustration towards cryptographers, and way too little frustration towards bad hiring practices, and incorrectly assigned work.

by max_ 2y ago

tptacek 2y ago

I don't understand what "blaming" the cryptographers gets you. Your system is either secure or it isn't. If you ship something insecure, that's on you.

AlotOfReading 2y ago
There's definitely degrees of security that I'm having to constantly remind our security folks of when they aren't forced to deal with the costs of having expensive threat models.
- xvector 2y ago
  If you're having to constantly remind your security engineers about "degrees of security," they may be trying to tell you that your threat model is unsustainably/nonsensically/dangerously lax.
  - AlotOfReading 2y ago
    The part I've been struggling with is getting them to put any threat model in writing that's not quote, "everything".
dns_snek 2y ago
"Blame" isn't the most productive way to frame the conversation, but as GP mentioned, there aren't many resources for developers to learn about cryptography. We're in desperate need of comprehensive resources that can (at a minimum) help us answer: 1. Has this problem been solved using a proven cryptographic protocol? Which guarantees does this protocol offer? What are all the important considerations? 2. What's the next closest thing? 3. Can we safely modify the protocol in any way to accommodate our use case, and what are some common pitfalls? Some people will ignore all that advice and do dumb things anyway, but if we want systems to be more secure, it's on cryptographers to make cryptography more approachable and on developers to actually listen when they do.

cowsaymoo 2y ago

What a coincidence! I was just browsing the Shamir's Secret Sharing Wikipedia page 30 seconds ago. There is a python implementation on it and I was worried the same exact thing as you before opening HN. So maybe we could start with that one. Is that code implementation sufficiently secure and well documented? https://en.wikipedia.org/wiki/Shamir's_secret_sharing?wprov=sfti1#Python_code https://en.wikipedia.org/wiki/Shamir's_secret_sharing?wprov=...

some_furry 2y ago
https://zkdocs.com https://zkdocs.com has a whole chapter on Shamir's Secret Sharing. (Ironically, this stuff is documented, cryptographers just aren't good at marketing.) https://www.zkdocs.com/docs/zkdocs/protocol-primitives/shamir/ https://www.zkdocs.com/docs/zkdocs/protocol-primitives/shami...
- unscaled 2y ago
  I think this chapter just proves GP's point, rather than refuting it. It's much cleaner than the Wikipedia article and it mentions the zero share problem, but it doesn't mention any other concern like constant time implementations, cache side channel attacks or the leading zero coefficient that I pointed to at the other post. It's a good guide to learn about Secret Sharing, but it doesn't give you even 1% of what you need to know implement it safely by yourself.
  - some_furry 2y ago
    > or the leading zero coefficient that I pointed to at the other post. That isn't actually a problem, although I've seen a lot of people that think it is. The problem is framed as "if you have a leading zero coefficient, it's equivalent to a threshold of t-1", but you'd need a zero leading coefficient for every polynomial. With a 32 byte secret and GF(2^8), you expect at least 1 leading zero coefficient in 11% of random secrets, but a threshold reduction from t to t-1 only occurs with 2^-256 probability (that is, every leading coefficient has to be 0). You might think you can detect this condition, but SSS is kind of like a one-time pad if you don't have sufficient shares. > it doesn't mention any other concern like constant time implementations, cache side channel attacks These are table stakes for secure cryptography. ZKDocs is a guide to algorithms, not implementations.

alfiedotwtf 2y ago

It's not insider knowledge, it's just the collective experience people have built up over time after discovering issues with classes of ciphers. It would be nice to have a standard checklist that everyone could refer to (think OWASP Top Ten) while reviewing a cryptosystem.

ikiris 2y ago
The checklist is really simple: did you use a verified product or did you try to roll your own and fail?
maqp 2y ago
Cryptography isn't like that, it doesn't boil down to 10 simple steps. If you want ten, it's happening at higher level 1. Never invent your own primitives. Only professional cryptographers do that in algorithm competitions like AES, SHA-3, or PHC. 2. For interoperability, use BoringSSL, LibreSSL etc. Hire developer who knows how to do this. Or, hire a professional applied cryptographer to build TLS from low level primitives if you have a reason it's needed. 3. For closed ecosystems, use opinionated, misuse-resistant, high level libraries like libsodium, or libraries binding to it https://libsodium.gitbook.io/doc/bindings_for_other_languages https://libsodium.gitbook.io/doc/bindings_for_other_language... 4. Use kernel CSPRNG to generate all secrets. 5. Have the implementation audited by professionals. E.g. Kudelski Security or NCC Group. 6. Have a bug bounty program that compensates well, and that allows disclosure of the attack. 7. If you're in a market where the competition has open source clients, you should have an open source client too, so that anyone can verify your cryptography is correctly implemented. 8. Keep anything that's WIP, under "research prototype, do not use in production" banner. That's not a cone of shame. If Dan Boneh can do it (https://crypto.stanford.edu/balloon/ https://crypto.stanford.edu/balloon/), so can you. 9. Realize that 90% of the work in cryptography is key management. Use OS keyring when possible. 10. Deploy end-to-end encryption everywhere you can. Data is a toxic asset. https://www.schneier.com/blog/archives/2016/03/data_is_a_toxic.html https://www.schneier.com/blog/archives/2016/03/data_is_a_tox... So yeah, sorry, no "Use XChaCha20-Poly1305 or AES-GCM only", or "Use X25519/ed25519" or "remember to hash the X25519 shared secret". That's too in-depth for a ten point list. At that level, it's more of a top-ten book list.

maqp 2y ago

>Is this elitist culture in cryptography. It's elitist in the same way quantum physics, string theory and molecular biology are. A hard science with lot's of moving parts not everyone can manage. >But there is so much extra "insider knowledge" or "tribal knowledge" that isn't obvious to non-cryptographers, that you need, for it to be secure. Misuse-resistant cryptography is an open research problem. Claiming it's "tribal knowledge" makes it seem nefarious. Academics are not always good teachers, and collecting all the lessons from all papers you read into a book, is time away from doing more research that advances your career. >Such knowledge is often (if not always) not documented. It's in obscure forums, papers and blog posts like this one. You can pay for the papers, and you can pay for the school to learn to understand those 'obscure' papers. Or you can hire someone who has done that. If you're a developer expected to implement algorithms of theoretical physics, you're not complaining about the papers being obscure, you're hiring a physicist with double-major in computer science. There's way too much frustration towards cryptographers, and way too little frustration towards bad hiring practices, and incorrectly assigned work.