3 ms·

> can we finally stop using MD5 While this is no reason not to use something else, I would also use newer & safer hash functions by default, MD5 is still perfe

> can we finally stop using MD5 While this is no reason not to use something else, I would also use newer & safer hash functions by default, MD5 is still perfectly fine for protecting against accidental corruption in storage or transit. Just don't use it if your protection model needs to consider deliberate changes. If you want to be extra paranoid, IIRC SHA256 outputs the full state so some types of length extension attack are still possible (though massively less _practical_ than against MD5) so perhaps use SHA384 or SHA224 (which are essentially truncated SHA512/SHA256 respectively), or SHA3 (none of its variants have the whole-state property), as your default choice.

charcircuit 5y ago
>MD5 is still perfectly fine for protecting against accidental corruption in storage or transit So would some form of CRC. It's possible to create colliding files with MD5 so you lose the protection of knowing if a file you are given hash has a near 100% probability of being the correct file. The length extension problems with SHA256 are not relevant to this use case. Edit: Since there is no practical second preimage attack on MD5 this it's less of a big deal in this use case, but I think it's simpler to just avoid relying on MD5 to prove that 2 flies are likely the same in general.
- dspillett 5y ago
  > So would some form of CRC. Less tooling directly supports them though. The Windows Explorer extension I use does, but others just support MD5 and maybe some SHA variants, similarly for download management browser extensions. And for this use people are just familiar with MD5. You'd be surprised how many in tech fields who don't touch encryption much, are relatively unaware of changes from the last decade or two. Even long-time command-line dwellers might have to lookup how to check a CRC despite cksum being part of the same package as md5sum/sha1sum/et.al. and installed by default practically everywhere unix-a-like just because they've always used stronger hashes for that. > The length extension problems with SHA256 are not relevant to this use case. No, but “by default” I meant in the general case. Being in the habit of using a choice that is better generally and no worse specifically, is safer than using good enough in one place and hoping you remember to vary when there are different concerns. My memory is somewhat random, so I prefer to err on the side of caution when trying to develop habits! > but I think it's simpler to just avoid relying on MD5 to prove that 2 flies are likely the same in general Agreed, and I do. Though I'd not go as far as to chastise those using md5 where it is still sufficient.
oconnor663 5y ago
> IIRC SHA256 outputs the full state so some types of length extension attack are still possible (though massively less _practical_ than against MD5) I'm not aware of any differences between the length extension attack on MD5 and the one on SHA-256. As far as I know, it works the same way on both. Here's an assignment I did on the SHA-256 version if you want a deep dive: https://github.com/oconnor663/applied_crypto_2021_fall/tree/main/sha256 https://github.com/oconnor663/applied_crypto_2021_fall/tree/... It's possible you're thinking about finding collisions? It's true that this is much easier for MD5 than for SHA-1. > MD5 is still perfectly fine for protecting against accidental corruption I take your point, but I think it's problematic that MD5 was designed to have security properties that we now know it doesn't have. Even if we don't need those properties, that creates some practical issues. A small issue: Those security properties come with a performance cost. If we don't need them, we're using the wrong tool. A big issue: Many people learned about MD5 back when it was still considered secure, and not everyone has updated their mental model. There are plenty of old (or new but badly mistaken) articles that recommend using MD5 for security, which still show up in search results. We're going to have to keep fighting this long tail of old advice for the foreseeable future, and the best way to do that is to document "MD5 BAD" as widely and clearly as possible. A caveat like "in cases where you need collision resistance" is useful and interesting for cryptographers but counterproductive as a public message. The message needs to be "MD5 BAD". We can put a link to the Wikipedia article on second-preimage resistance in the footnotes. > perhaps use SHA384 or SHA224 Note that SHA-224 only cuts 32 bits off the state, so you can still length-extend it if you're allowed a few billion guesses. SHA-384 is better (and SHA-512/256 is best, albeit the world's most confusingly named hash function), but I think the more important takeaway from all this is that the constellation of SHA-2 standards is too complicated and full of too many pitfalls. This is a good argument in favor of a modern design like SHA-3, which doesn't tempt us with unsafe choices. EDIT: I feel a little guilty that the largest subthread in this post about a beautiful new book is about the hash function in the fine print >.<
- dspillett 5y ago
  > I feel a little guilty that the largest subthread in this post about a beautiful new book is about the hash function in the fine print At least it is a polite exchange of thoughts. Far worse happens all the time.

charcircuit 5y ago

>MD5 is still perfectly fine for protecting against accidental corruption in storage or transit So would some form of CRC. It's possible to create colliding files with MD5 so you lose the protection of knowing if a file you are given hash has a near 100% probability of being the correct file. The length extension problems with SHA256 are not relevant to this use case. Edit: Since there is no practical second preimage attack on MD5 this it's less of a big deal in this use case, but I think it's simpler to just avoid relying on MD5 to prove that 2 flies are likely the same in general.

dspillett 5y ago
> So would some form of CRC. Less tooling directly supports them though. The Windows Explorer extension I use does, but others just support MD5 and maybe some SHA variants, similarly for download management browser extensions. And for this use people are just familiar with MD5. You'd be surprised how many in tech fields who don't touch encryption much, are relatively unaware of changes from the last decade or two. Even long-time command-line dwellers might have to lookup how to check a CRC despite cksum being part of the same package as md5sum/sha1sum/et.al. and installed by default practically everywhere unix-a-like just because they've always used stronger hashes for that. > The length extension problems with SHA256 are not relevant to this use case. No, but “by default” I meant in the general case. Being in the habit of using a choice that is better generally and no worse specifically, is safer than using good enough in one place and hoping you remember to vary when there are different concerns. My memory is somewhat random, so I prefer to err on the side of caution when trying to develop habits! > but I think it's simpler to just avoid relying on MD5 to prove that 2 flies are likely the same in general Agreed, and I do. Though I'd not go as far as to chastise those using md5 where it is still sufficient.

oconnor663 5y ago

> IIRC SHA256 outputs the full state so some types of length extension attack are still possible (though massively less _practical_ than against MD5) I'm not aware of any differences between the length extension attack on MD5 and the one on SHA-256. As far as I know, it works the same way on both. Here's an assignment I did on the SHA-256 version if you want a deep dive: https://github.com/oconnor663/applied_crypto_2021_fall/tree/main/sha256 https://github.com/oconnor663/applied_crypto_2021_fall/tree/... It's possible you're thinking about finding collisions? It's true that this is much easier for MD5 than for SHA-1. > MD5 is still perfectly fine for protecting against accidental corruption I take your point, but I think it's problematic that MD5 was designed to have security properties that we now know it doesn't have. Even if we don't need those properties, that creates some practical issues. A small issue: Those security properties come with a performance cost. If we don't need them, we're using the wrong tool. A big issue: Many people learned about MD5 back when it was still considered secure, and not everyone has updated their mental model. There are plenty of old (or new but badly mistaken) articles that recommend using MD5 for security, which still show up in search results. We're going to have to keep fighting this long tail of old advice for the foreseeable future, and the best way to do that is to document "MD5 BAD" as widely and clearly as possible. A caveat like "in cases where you need collision resistance" is useful and interesting for cryptographers but counterproductive as a public message. The message needs to be "MD5 BAD". We can put a link to the Wikipedia article on second-preimage resistance in the footnotes. > perhaps use SHA384 or SHA224 Note that SHA-224 only cuts 32 bits off the state, so you can still length-extend it if you're allowed a few billion guesses. SHA-384 is better (and SHA-512/256 is best, albeit the world's most confusingly named hash function), but I think the more important takeaway from all this is that the constellation of SHA-2 standards is too complicated and full of too many pitfalls. This is a good argument in favor of a modern design like SHA-3, which doesn't tempt us with unsafe choices. EDIT: I feel a little guilty that the largest subthread in this post about a beautiful new book is about the hash function in the fine print >.<

dspillett 5y ago
> I feel a little guilty that the largest subthread in this post about a beautiful new book is about the hash function in the fine print At least it is a polite exchange of thoughts. Far worse happens all the time.