5 ms·

I'm the py-bcrypt author. py-bcrypt is a thin wrapper around the original reference implementation of bcrypt from the authors of the paper describing it (Provos

by djmdjm 15y ago

I'm the py-bcrypt author. py-bcrypt is a thin wrapper around the original reference implementation of bcrypt from the authors of the paper describing it (Provos and Mazieres). The discrepancy in hash length is totally harmless (adding something like 2^-186 additional likelihood of collision) and was present in the reference implementation. I don't know exactly why the hash is slightly truncated, but I guess David or Niels thought that 60 character hashes were a more manageable length. The author of this assinine blog post only contacted me a few days ago and obviously couldn't be bothered to wait for a response before proceeding to imply malicious intent for what is clearly a trivial difference between academic paper and practical implementation. I hope he retracts it.

lisper 15y ago
For the record, I did not mean to imply that py-bcrypt was malicious. All I intended was that I think it's prudent to understand discrepancies like these because they could some day be an indication of something malicious. I apologize for the misunderstanding.
- tptacek 15y ago
  From your post: OK, so maybe someone accidentally introduced an off-by-one error into the python wrapper. Except that the problem is not in the python wrapper. You can find bcrypt test vectors on the web, and they are all 60-byte strings. ... and: That is a very big discrepancy between the actual behavior of the code and the description given in the literature. It's vastly too big a discrepancy to be explainable by a simple inadvertent bug. .... and: Now, some people might say I'm being excessively paranoid, but I don't think so. The higher the stakes in the internet security game get, the more incentive there is for attackers to try all kinds of sneaky and nefarious tricks to introduce weaknesses into people's defenses, and one of the easiest ways to do that is to publish some plausible-looking open-source security code that actually has a hidden weakness built in to it [!!!] and hope that nobody notices. Give me a break. And lest the tl;dr's think I'm cherry picking: this stuff is the bulk of the post. Ed: fixed ems.
  - TeMPOraL 15y ago
    Technical question: you want to emphasize the em-ed text or the plain text? I'm not sure, because it seems to me that the text you left unformatted is more relevant in this context that em-ed one. Edit: thx :).
  - lisper 15y ago
    I can see how someone could read that as implying that I thought the author of py-bcrypt was being malicious. All I can say is that wasn't my intent. My intent was that I didn't think it was being overly paranoid to try to understand what was going on, whatever it was.
  - true_religion 15y ago
    It's funny how we're looking at the same text and arriving at opposite meanings. Let me highlight some things from your quotes. 1. "Except that the problem is not in the python wrapper" - This directly says that the Python author is not being malicious---how can he be since he isn't at fault for the error? The essayist continues to explain why the Python author is not at fault. 2. "You can find bcrypt test vectors on the web, and they are all 60-byte strings." - This bug is wide-spread. 3. "That is a very big discrepancy between the actual behavior of the code and the description given in the literature. It's vastly too big a discrepancy to be explainable by a simple inadvertent bug." - The bug is too large to be an inadverdent bug introduced independently by every author. 4. "Now, some people might say I'm being excessively paranoid, but I don't think so. <snip>" Here the author is only defending his rationale for looking for bugs in an open source project. He isn't implying that he has now found one that shows the authors are malicious.
daeken 15y ago
> The discrepancy in hash length is totally harmless (adding something like 2^-186 additional likelihood of collision) and was present in the reference implementation. Assuming that this was not intentional -- and considering that the paper clearly supports this -- then it is not totally harmless. The additional likelihood of collision is increased by ~4.16% (8 / 192) which could be a Big Deal (TM) given the right circumstances, although it's still infinitesimally small. But what worries me more is that you say "this assinine [sic] blog post". The author of the post found what could be an issue, even if it's not in your code, and you write it off completely. That is simply not an appropriate way to handle issues with cryptographic software.
- tptacek 15y ago
  It was an asinine blog post, Cody. It strongly implied that the author of py-bcrypt intentionally sabotaged it. And, I disagree with your security analysis. Truncating an otherwise secure 192 bit hash to 185 bits is not dangerous. You are talking about percentages of quantities denominated in atoms-in-the-planet-core.
  - daeken 15y ago
    I agree that implying the creator of py-bcrypt was malicious is asinine (although that was apparently not his intention), but dropping 8 bits off the hash could be dangerous. When you're talking about an attack against a perfect hash function, it would mean nothing due to the large size, but in conjunction with theoretical flaws in bcrypt it could be a big deal. All that said! Bcrypt is still secure by all means and better off than damn near anything else out there. Edit: note, such theoretical flaws in bcrypt have not even been proposed. Didn't want to make this seem worse than it is.
    - tptacek 15y ago
      The sad thing is, the finding isn't asinine. It's truly interesting (though practically irrelevant, except that it appears to have rendered the ref implementation of bcrypt incompatible with Mazieres' paper). But instead of spending time talking about the interesting finding, this blogger clubbed their own reputation to death on it. I'd like to presume they were just spectacularly bored, but to steal some of the author's own words, that's "vastly too big a discrepancy to be explainable by a simple inadvertent bug".
marshray 15y ago
I guess David or Niels thought that 60 character hashes were a more manageable length. That's a pretty weird design decision to use a 128 bit salt and then chop bits off the actual hash value to make the result a "more manageable" length. The 128 bit salt wastes 4 bits in the base64 encoding. The 31 character base64 discards 8 of the 192 bit output bits (31/4*24 = 186). If they'd just used "only" a 126 bit salt they could base64 encode it in 21 chars with no wasted space. That would allow them to store the full 192 bits in 32 chars with no wasted space. So they threw away 8 hash output bits in order to save 2 salt bits.
RyanMcGreal 15y ago
First of all, thank you for py-bcrypt. Thanks to your hard work and generosity, the tiny sliver of the internet that I manage is a more secure place. Now for the request (sigh, you knew it was coming): at my day job, I have to develop on Windows and it's not feasible to install Visual Studio 2008 Express on my web server. Is there any other practical way to get py-bcrypt running in that environment?
- sp332 15y ago
  Maybe you could install just the VS 2008 runtime libraries? http://www.microsoft.com/downloads/en/details.aspx?familyid=a5c84275-3b97-4ab7-a40d-3802b2af5fc2&displaylang=en http://www.microsoft.com/downloads/en/details.aspx?familyid=...
  - RyanMcGreal 15y ago
    Thanks for the suggestion. I tried this, but the install script still returns the same error: "Unable to find vcvarsall.bat".
    - sp332 15y ago
      Hm, that's actually a compiler error. If you can't get the VS compiler installed easily I think http://blog.eddsn.com/2010/05/unable-to-find-vcvarsall-bat/ http://blog.eddsn.com/2010/05/unable-to-find-vcvarsall-bat/ is your best bet.
- djmdjm 15y ago
  I'd like to help, but I have no idea how to build Python modules for Windows any more. I once figured it out by following some guide that was linked to from python.org, but that broke when more recent versions of Python required more recent compilers. If you can create a build environment on your workstation, IIRC there is a bdist_win (or similar) setup.py target that will build a .msi package that you can load on your webserver. That's about the limit of my Windows Python knowledge right there :|

by djmdjm 15y ago

lisper 15y ago

For the record, I did not mean to imply that py-bcrypt was malicious. All I intended was that I think it's prudent to understand discrepancies like these because they could some day be an indication of something malicious. I apologize for the misunderstanding.

tptacek 15y ago
From your post: OK, so maybe someone accidentally introduced an off-by-one error into the python wrapper. Except that the problem is not in the python wrapper. You can find bcrypt test vectors on the web, and they are all 60-byte strings. ... and: That is a very big discrepancy between the actual behavior of the code and the description given in the literature. It's vastly too big a discrepancy to be explainable by a simple inadvertent bug. .... and: Now, some people might say I'm being excessively paranoid, but I don't think so. The higher the stakes in the internet security game get, the more incentive there is for attackers to try all kinds of sneaky and nefarious tricks to introduce weaknesses into people's defenses, and one of the easiest ways to do that is to publish some plausible-looking open-source security code that actually has a hidden weakness built in to it [!!!] and hope that nobody notices. Give me a break. And lest the tl;dr's think I'm cherry picking: this stuff is the bulk of the post. Ed: fixed ems.
- TeMPOraL 15y ago
  Technical question: you want to emphasize the em-ed text or the plain text? I'm not sure, because it seems to me that the text you left unformatted is more relevant in this context that em-ed one. Edit: thx :).
- lisper 15y ago
  I can see how someone could read that as implying that I thought the author of py-bcrypt was being malicious. All I can say is that wasn't my intent. My intent was that I didn't think it was being overly paranoid to try to understand what was going on, whatever it was.
- true_religion 15y ago
  It's funny how we're looking at the same text and arriving at opposite meanings. Let me highlight some things from your quotes. 1. "Except that the problem is not in the python wrapper" - This directly says that the Python author is not being malicious---how can he be since he isn't at fault for the error? The essayist continues to explain why the Python author is not at fault. 2. "You can find bcrypt test vectors on the web, and they are all 60-byte strings." - This bug is wide-spread. 3. "That is a very big discrepancy between the actual behavior of the code and the description given in the literature. It's vastly too big a discrepancy to be explainable by a simple inadvertent bug." - The bug is too large to be an inadverdent bug introduced independently by every author. 4. "Now, some people might say I'm being excessively paranoid, but I don't think so. <snip>" Here the author is only defending his rationale for looking for bugs in an open source project. He isn't implying that he has now found one that shows the authors are malicious.

daeken 15y ago

> The discrepancy in hash length is totally harmless (adding something like 2^-186 additional likelihood of collision) and was present in the reference implementation. Assuming that this was not intentional -- and considering that the paper clearly supports this -- then it is not totally harmless. The additional likelihood of collision is increased by ~4.16% (8 / 192) which could be a Big Deal (TM) given the right circumstances, although it's still infinitesimally small. But what worries me more is that you say "this assinine [sic] blog post". The author of the post found what could be an issue, even if it's not in your code, and you write it off completely. That is simply not an appropriate way to handle issues with cryptographic software.

tptacek 15y ago
It was an asinine blog post, Cody. It strongly implied that the author of py-bcrypt intentionally sabotaged it. And, I disagree with your security analysis. Truncating an otherwise secure 192 bit hash to 185 bits is not dangerous. You are talking about percentages of quantities denominated in atoms-in-the-planet-core.
- daeken 15y ago
  I agree that implying the creator of py-bcrypt was malicious is asinine (although that was apparently not his intention), but dropping 8 bits off the hash could be dangerous. When you're talking about an attack against a perfect hash function, it would mean nothing due to the large size, but in conjunction with theoretical flaws in bcrypt it could be a big deal. All that said! Bcrypt is still secure by all means and better off than damn near anything else out there. Edit: note, such theoretical flaws in bcrypt have not even been proposed. Didn't want to make this seem worse than it is.
  - tptacek 15y ago
    The sad thing is, the finding isn't asinine. It's truly interesting (though practically irrelevant, except that it appears to have rendered the ref implementation of bcrypt incompatible with Mazieres' paper). But instead of spending time talking about the interesting finding, this blogger clubbed their own reputation to death on it. I'd like to presume they were just spectacularly bored, but to steal some of the author's own words, that's "vastly too big a discrepancy to be explainable by a simple inadvertent bug".

marshray 15y ago

I guess David or Niels thought that 60 character hashes were a more manageable length. That's a pretty weird design decision to use a 128 bit salt and then chop bits off the actual hash value to make the result a "more manageable" length. The 128 bit salt wastes 4 bits in the base64 encoding. The 31 character base64 discards 8 of the 192 bit output bits (31/4*24 = 186). If they'd just used "only" a 126 bit salt they could base64 encode it in 21 chars with no wasted space. That would allow them to store the full 192 bits in 32 chars with no wasted space. So they threw away 8 hash output bits in order to save 2 salt bits.

RyanMcGreal 15y ago

First of all, thank you for py-bcrypt. Thanks to your hard work and generosity, the tiny sliver of the internet that I manage is a more secure place. Now for the request (sigh, you knew it was coming): at my day job, I have to develop on Windows and it's not feasible to install Visual Studio 2008 Express on my web server. Is there any other practical way to get py-bcrypt running in that environment?

sp332 15y ago
Maybe you could install just the VS 2008 runtime libraries? http://www.microsoft.com/downloads/en/details.aspx?familyid=a5c84275-3b97-4ab7-a40d-3802b2af5fc2&displaylang=en http://www.microsoft.com/downloads/en/details.aspx?familyid=...
- RyanMcGreal 15y ago
  Thanks for the suggestion. I tried this, but the install script still returns the same error: "Unable to find vcvarsall.bat".
  - sp332 15y ago
    Hm, that's actually a compiler error. If you can't get the VS compiler installed easily I think http://blog.eddsn.com/2010/05/unable-to-find-vcvarsall-bat/ http://blog.eddsn.com/2010/05/unable-to-find-vcvarsall-bat/ is your best bet.
djmdjm 15y ago
I'd like to help, but I have no idea how to build Python modules for Windows any more. I once figured it out by following some guide that was linked to from python.org, but that broke when more recent versions of Python required more recent compilers. If you can create a build environment on your workstation, IIRC there is a bdist_win (or similar) setup.py target that will build a .msi package that you can load on your webserver. That's about the limit of my Windows Python knowledge right there :|