5 ms·

What should you do if you can't avoid writing crypto code? I found myself in this situation when I tried to find a bcrypt implementation for Common Lisp. There

What should you do if you can't avoid writing crypto code? I found myself in this situation when I tried to find a bcrypt implementation for Common Lisp. There wasn't one. Folks in #lisp suggested I adapt the blowfish implementation in Ironclad, since 'bcrypt is just blowfish anyway'. I ended up writing a Lisp wrapper around one of the C implementations, a process documented at my blog (http://www.letsyouandhimfight.com/2010/07/14/cl-bcrypt-a-first-attempt/ http://www.letsyouandhimfight.com/2010/07/14/cl-bcrypt-a-fir...), but it's unsatisfactory for a couple of reasons: 1) Both the current C implementations are designed to be integrated into libc. The Openwall implementation does have the code factored out into its own file, but there is no support structure for building a shared library. (Python's bcrypt bundles a modified version of the Openwall C source directly with it, for example.) Common Lisp's FFI is intended for working with installed shared libraries 2) There appears to be a bias in the Lisp community towards pure-Lisp implementations, for (hopefully obvious) reasons, so an implementation as hacky as what I came up with is unlikely to see much use. If I do go back to trying to write a webapp in Common Lisp, I think I will find myself having to reimplement bcrypt in Common Lisp. First, I'll have to find a sufficiently portable method of getting cryptographically secure random numbers; as of the writing of that blog post, there wasn't one that I could find anyone recommending. The more difficult part will be to convert the C code into Lisp code without missing any places where operations on the C types don't precisely correspond to the same operations on the Lisp types (due to, say, overflow). I'm worried I might get something wrong, but I can't just use the crypto code written by wiser folks than I, because, at least in the Common Lisp community, that code doesn't seem to exist.

tptacek 16y ago
First, don't obsess too much about your password digest. I know this is head-explodey considering the source, but all I'm trying to do by ranting about it is to get people to stop using SHA1 (or SHA256 or Whirlpool or whatever) hashes. The risk to your users for doing that bit wrong is not very high. Second, my advice about how to do crypto security is very simple: * Use PGP for data at rest. * Use TLS for data in motion. Do not trust your own judgement (say, by using OTR because it "feels" like most of what you need, or trusting that you'll use Keyczar safely) on anything else without a formal external review. In practice, you will almost never need anything more than TLS or PGP.
- inklesspen 16y ago
  Okay, but in the more general case, where something literally does not exist for a particular platform/language and my choices are "write it myself" or "don't use that platform/language", is there any way to feel confident that a choice to write it myself will not be a hideously wrong decision?
  - khafra 16y ago
    I'm not Thomas Ptacek, but I think his answer would be something to the effect of "to be really confident, get tens of thousands of dollars worth of code review before shipping." You may be prominent enough in the CL/hypothetical crypto-less platform community to get most of that for free, but that's the value of the validation needed.
- alexgartrell 16y ago
  Any tips as far as a favorably licensed (BSD would be nice, public domain is probably asking too much) TLS library that doesn't suck?
weichi 16y ago
A comment and a question: 1. It should be fairly trivial to test that your implementation is giving exactly the same output as the c libs (once you have chosen a particular random number that feeds into the algorithm). It seems like the trickiest part of testing will be ensuring that you are using the same character set everywhere. 2. Why is it important to have a "cryptographically strong" PRNG? Doesn't this just turn into a salt? Does a salt generator really need to be cryptographically strong? Someone please correct me if I am being naive here.
- inklesspen 16y ago
  1. I worry I might have a bug that returns the proper output for some inputs, but improper output for other inputs. 2. Cryptographically strong random numbers isn't strictly required for a bcrypt salt, I guess. But if I'm building something which I plan to share with other people, I'd rather err on the side of too strong.
- wladimir 16y ago
  In cryptography you should always use a "cryptographically strong" PRNG, even (especially) if in doubt. There have been to many mistakes with lousy random number generators undermining what would otherwise have been a strong security mechanism.
  - weichi 16y ago
    But we're talking about generating a salt here. As I understand it, the reason you use a salt is to make it much harder to brute-force a dictionary of passwords ahead of time. I don't see how the use of a cryptographically strong PRNG is going to provide any additional security here. What am I missing?
    - wladimir 16y ago
      I don't know. But if I were to design such a system I'd use a cryptographically secure PRNG just to be sure. It's no extra work and has no drawbacks.
pjscott 16y ago
If you're trying to make something in pure Lisp, your odds of failure are less if you take an existing hashing algorithm (e.g. SHA-256) and just iterate it a bunch of times. Ironclad has SHA-256, so this is really easy: (defun slow-hash (password salt &key (iterations 10000)) "Produces a 256-bit hashed value of password and salt, slowly. Uses a tweakable number of iterations, which should not be less than 1000, and which defaults to 10000." (let ((hash (ironclad:make-digest :sha256))) ;; First, hash the salt and password (ironclad:update-digest hash (ironclad:ascii-string-to-byte-array salt)) (ironclad:update-digest hash (ironclad:ascii-string-to-byte-array password)) ;; Repeatedly hash the hash, to slow things down (dotimes (x iterations) (ironclad:update-digest hash (ironclad:produce-digest hash))) (ironclad:produce-digest hash)))
- koenigdavidmj 16y ago
  Even after one round, repeating that knocks down your plaintext-space to 256 bit strings (or however long the output is of the hash function that you are using). Tom/Colin, is this actually a problem?
  - inklesspen 16y ago
    That's the case for every hash function. If it's a good hash function, the outputs will be evenly distributed across the entire 256 bit space, which comes out to 2^256 possible outputs. I don't think this is a problem; bcrypt has a much smaller output space.

tptacek 16y ago

First, don't obsess too much about your password digest. I know this is head-explodey considering the source, but all I'm trying to do by ranting about it is to get people to stop using SHA1 (or SHA256 or Whirlpool or whatever) hashes. The risk to your users for doing that bit wrong is not very high. Second, my advice about how to do crypto security is very simple: * Use PGP for data at rest. * Use TLS for data in motion. Do not trust your own judgement (say, by using OTR because it "feels" like most of what you need, or trusting that you'll use Keyczar safely) on anything else without a formal external review. In practice, you will almost never need anything more than TLS or PGP.

inklesspen 16y ago
Okay, but in the more general case, where something literally does not exist for a particular platform/language and my choices are "write it myself" or "don't use that platform/language", is there any way to feel confident that a choice to write it myself will not be a hideously wrong decision?
- khafra 16y ago
  I'm not Thomas Ptacek, but I think his answer would be something to the effect of "to be really confident, get tens of thousands of dollars worth of code review before shipping." You may be prominent enough in the CL/hypothetical crypto-less platform community to get most of that for free, but that's the value of the validation needed.
alexgartrell 16y ago
Any tips as far as a favorably licensed (BSD would be nice, public domain is probably asking too much) TLS library that doesn't suck?

weichi 16y ago

A comment and a question: 1. It should be fairly trivial to test that your implementation is giving exactly the same output as the c libs (once you have chosen a particular random number that feeds into the algorithm). It seems like the trickiest part of testing will be ensuring that you are using the same character set everywhere. 2. Why is it important to have a "cryptographically strong" PRNG? Doesn't this just turn into a salt? Does a salt generator really need to be cryptographically strong? Someone please correct me if I am being naive here.

inklesspen 16y ago
1. I worry I might have a bug that returns the proper output for some inputs, but improper output for other inputs. 2. Cryptographically strong random numbers isn't strictly required for a bcrypt salt, I guess. But if I'm building something which I plan to share with other people, I'd rather err on the side of too strong.
wladimir 16y ago
In cryptography you should always use a "cryptographically strong" PRNG, even (especially) if in doubt. There have been to many mistakes with lousy random number generators undermining what would otherwise have been a strong security mechanism.
- weichi 16y ago
  But we're talking about generating a salt here. As I understand it, the reason you use a salt is to make it much harder to brute-force a dictionary of passwords ahead of time. I don't see how the use of a cryptographically strong PRNG is going to provide any additional security here. What am I missing?
  - wladimir 16y ago
    I don't know. But if I were to design such a system I'd use a cryptographically secure PRNG just to be sure. It's no extra work and has no drawbacks.

pjscott 16y ago

If you're trying to make something in pure Lisp, your odds of failure are less if you take an existing hashing algorithm (e.g. SHA-256) and just iterate it a bunch of times. Ironclad has SHA-256, so this is really easy: (defun slow-hash (password salt &key (iterations 10000)) "Produces a 256-bit hashed value of password and salt, slowly. Uses a tweakable number of iterations, which should not be less than 1000, and which defaults to 10000." (let ((hash (ironclad:make-digest :sha256))) ;; First, hash the salt and password (ironclad:update-digest hash (ironclad:ascii-string-to-byte-array salt)) (ironclad:update-digest hash (ironclad:ascii-string-to-byte-array password)) ;; Repeatedly hash the hash, to slow things down (dotimes (x iterations) (ironclad:update-digest hash (ironclad:produce-digest hash))) (ironclad:produce-digest hash)))

koenigdavidmj 16y ago
Even after one round, repeating that knocks down your plaintext-space to 256 bit strings (or however long the output is of the hash function that you are using). Tom/Colin, is this actually a problem?
- inklesspen 16y ago
  That's the case for every hash function. If it's a good hash function, the outputs will be evenly distributed across the entire 256 bit space, which comes out to 2^256 possible outputs. I don't think this is a problem; bcrypt has a much smaller output space.