3 ms·

I see what you mean now. And your intuition on cycles was spot-on. And to answer your question (I'm not a crypto expert, so take with a pinch of salt): With S

I see what you mean now. And your intuition on cycles was spot-on. And to answer your question (I'm not a crypto expert, so take with a pinch of salt): With SHA-1, that probability is generally accepted to be 2^(-160). 1000 is roughly 2^10, so the final probability is about 2^(-150). I guess that's still improbable enough... But I think you do make a very valid and interesting point.

tptacek 17y ago
The loop has nothing to do with the probability of a collision. If your key comes out of /usr/share/dict/words, you have a 2^18 search space, not a 2^160 space. If you have a 2^18 search space, you better hope your constant factors are very high. Hence the loop.
- paulgb 17y ago
  Yeah, I get why the loop is there. But I still think it increases the probability of a collision by nearly 1000x over the probability of a collision without the loop. Maybe a better way to explain my concern is to consider hashing something once versus twice, rather than 1000 times. Let h be a hash function and p be the probability P(h(x) = h(y)) for randomly chosen x and y. Say you have unique inputs a and b. Obviously, P(h(a) = h(b)) = p. Now consider P(h(h(a)) = h(h(b))). h(h(a)) = h(h(b)) if: h(a) = h(b) [probability p] OR h(a) != h(b) [probability 1-p], and h(h(a)) = h(h(b)) [probability p]. So the total probability when we repeat the hash is p + (1-p) * p Since p is small, the probability of a collision has nearly doubled just by applying the hash function again. [note: Higher up in this thread I mistakenly implied that the probability increased linearly. With a low number of iterations and realistic values for p it should be close to linear, but not exact.] [edit:formatting]
  - tptacek 17y ago
    H(m) -> SHA1(SHA1(m)) is not the same hash function as H(m) -> SHA1(m).
    - paulgb 17y ago
      Right, but if SHA1(m) = SHA1(n), then SHA1(SHA1(m)) = SHA1(SHA1(n)) also. The converse is not true. There will be cases where SHA1(SHA1(m)) = SHA1(SHA1(n)), but SHA1(m) != SHA1(n). (Or does SHA1 somehow guarantee that this will not happen?) So it seems to follow that the chances that H(a) = H(b) when H(m) -> SHA1(SHA1(m)) must be higher than when H(m) -> SHA1(m). Am I missing something?
      - tptacek 17y ago
        I think you're going to need to better explain the logic behind graf 2 before I can answer you. But on the off chance that we can end this thread gracefully, I'll point out that SHA256(SHA(256(m), m) is SHAd256(m), and considered by Ferguson to address security concerns in straight SHA256.
        paulgb 17y ago
        It boils down to this: If SHA1(n) = SHA1(m), we know SHA1(SHA1(n)) = SHA1(SHA1(m)). (this is a consequence of SHA1 being deterministic) Also, even if SHA1(n) != SHA1(m), there is still a chance that SHA1(SHA1(n)) = SHA1(SHA1(m)). (this is a consequence of collisions: SHA1(a) may equal SHA1(b) even if a != b.) So the probability that SHA1(SHA1(n)) = SHA1(SHA1(m)) must be more than the probability that SHA1(n) = SHA1(m).
        tptacek 17y ago
        You realize that despite the fact that I still don't follow the logic here, SHA collisions have no meaningful impact on the security of PBKDF, right?
        paulgb 17y ago
        Actually I didn't, but that more or less answers my original question. Now I realize why you (or rather, your character) said that it didn't matter that md5 is broken in this case. In general though, I'm still convinced that putting a hash function in a loop will increase the probability of a collision. Or maybe I misunderstood your point about the loop? I assume by putting a loop around the hash you mean something like this: key = SHA1(password) 1000 times do key = SHA1(key) return key

tptacek 17y ago

The loop has nothing to do with the probability of a collision. If your key comes out of /usr/share/dict/words, you have a 2^18 search space, not a 2^160 space. If you have a 2^18 search space, you better hope your constant factors are very high. Hence the loop.

paulgb 17y ago
Yeah, I get why the loop is there. But I still think it increases the probability of a collision by nearly 1000x over the probability of a collision without the loop. Maybe a better way to explain my concern is to consider hashing something once versus twice, rather than 1000 times. Let h be a hash function and p be the probability P(h(x) = h(y)) for randomly chosen x and y. Say you have unique inputs a and b. Obviously, P(h(a) = h(b)) = p. Now consider P(h(h(a)) = h(h(b))). h(h(a)) = h(h(b)) if: h(a) = h(b) [probability p] OR h(a) != h(b) [probability 1-p], and h(h(a)) = h(h(b)) [probability p]. So the total probability when we repeat the hash is p + (1-p) * p Since p is small, the probability of a collision has nearly doubled just by applying the hash function again. [note: Higher up in this thread I mistakenly implied that the probability increased linearly. With a low number of iterations and realistic values for p it should be close to linear, but not exact.] [edit:formatting]
- tptacek 17y ago
  H(m) -> SHA1(SHA1(m)) is not the same hash function as H(m) -> SHA1(m).
  - paulgb 17y ago
    Right, but if SHA1(m) = SHA1(n), then SHA1(SHA1(m)) = SHA1(SHA1(n)) also. The converse is not true. There will be cases where SHA1(SHA1(m)) = SHA1(SHA1(n)), but SHA1(m) != SHA1(n). (Or does SHA1 somehow guarantee that this will not happen?) So it seems to follow that the chances that H(a) = H(b) when H(m) -> SHA1(SHA1(m)) must be higher than when H(m) -> SHA1(m). Am I missing something?
    - tptacek 17y ago
      I think you're going to need to better explain the logic behind graf 2 before I can answer you. But on the off chance that we can end this thread gracefully, I'll point out that SHA256(SHA(256(m), m) is SHAd256(m), and considered by Ferguson to address security concerns in straight SHA256.
      - paulgb 17y ago
        It boils down to this: If SHA1(n) = SHA1(m), we know SHA1(SHA1(n)) = SHA1(SHA1(m)). (this is a consequence of SHA1 being deterministic) Also, even if SHA1(n) != SHA1(m), there is still a chance that SHA1(SHA1(n)) = SHA1(SHA1(m)). (this is a consequence of collisions: SHA1(a) may equal SHA1(b) even if a != b.) So the probability that SHA1(SHA1(n)) = SHA1(SHA1(m)) must be more than the probability that SHA1(n) = SHA1(m).
        tptacek 17y ago
        You realize that despite the fact that I still don't follow the logic here, SHA collisions have no meaningful impact on the security of PBKDF, right?
        paulgb 17y ago
        Actually I didn't, but that more or less answers my original question. Now I realize why you (or rather, your character) said that it didn't matter that md5 is broken in this case. In general though, I'm still convinced that putting a hash function in a loop will increase the probability of a collision. Or maybe I misunderstood your point about the loop? I assume by putting a loop around the hash you mean something like this: key = SHA1(password) 1000 times do key = SHA1(key) return key