4 ms·
Using md5 for this seems like an odd choice. Sha1 is a better choice even for non-cryptographic use cases, it's quite a bit faster than md5. Even better would
by chlorion 3y ago
Using md5 for this seems like an odd choice.
Sha1 is a better choice even for non-cryptographic use cases, it's quite a bit faster than md5. Even better would be something like xxhash!
According to a quick bash script I wrote to benchmark the popular hash functions, md5 comes out last compared to sha1, sha256, sha512, and blake2, and by a decent margin!
A good rule of thumb is to never use md5 at all. Not even for non-cryptographic use cases. It's not only broken, but also very slow!
- williamzeng0 3y agoThat sounds great, I'm going to see how Sweep does on this issue: https://github.com/sweepai/sweep/issues/3333 https://github.com/sweepai/sweep/issues/3333
- chlorion 3y agoI think python objects have a __hash__ method available on them as well that can be used for hashing. That should be even much faster than sha1, but for this use case I'm not sure how much it really matters. Would be interesting to benchmark!