5 ms·

A detailed analysis of Meow Hash: https://peter.website/meow-hash-cryptanalysis https://peter.website/meow-hash-cryptanalysis It's not the highest of quality h

A detailed analysis of Meow Hash: https://peter.website/meow-hash-cryptanalysis https://peter.website/meow-hash-cryptanalysis It's not the highest of quality hash functions (see the SMHasher benchmarks), but it is fast. A great alternative is XXH3 (https://cyan4973.github.io/xxHash/ https://cyan4973.github.io/xxHash/), which has seen far more usage in practice.

ilitirit 5y ago
I can vouch for xxHash (it does what it says on the can), but I'm really curious to hear from who have experience with meow hash.
thomasahle 5y ago
XXH3 has issues too. There are certain inputs, independent of the seed, on which it collides much more often than it should. On the other hand a hash like UMash guarantees low collisions on any input: https://engineering.backtrace.io/2020-08-24-umash-fast-enough-almost-universal-fingerprinting/ https://engineering.backtrace.io/2020-08-24-umash-fast-enoug...
jdcarter 5y ago
I'm using XXHash3 for verifying data integrity of large (10+MB) blobs. Very fast and appears to work very well in my testing--it's never missed any bit error I've thrown at it. Aside: when storing hashes, be sure to store the hash type as well so that you can change it later if needed, e.g. "xxh3-[hash value]". RFC-6920 also has things to say about storing hash types and values, although I haven't seen its format in common use.
- njt 5y ago
  > be sure to store the hash type as well so that you can change it later if needed Thanks for sharing this, I'd been doing this on my own for my own stuff (eg. foo.txt-xxh32-ea79e094), but it's good to know someone else has thought it through. I ran into the problem once where someone had named some files foo-fb490c or something similar without any annotation, and when there was a problem, it took a file to figure out they were using truncated sha256 hashes.
DeathArrow 5y ago
Useless analysys since the author says it's not a cryptographic hash but useful as a fast hash for change detection. "we wanted a fast, non-cryptographic hash for use in change detection and deduplication" >A great alternative is XXH3 Meow Hash is twice as fast.
- ncann 5y ago
  Saying it's twice as fast is rather misleading? They can both hash as fast as RAM speed allows anyway. And if it's something in cache I doubt one is significantly better than the other.
- jrochkind1 5y ago
  What determines whether a hash is "cryptographic"? What would make it suitable for change-detection but not be "cryptographic"? Is the claim here that it would not be suitable for detecting "malicious" changes, but is still suitable for detecting "natural" changes?
  - gleenn 5y ago
    A couple features are that is hard for an attacker to find a collision, and it is hard to reverse the original data from the hash. Both can cause serious security problems but aren't necessary in something like this where most image changes are "natural" and you gain speed by weakening those constraints.
- LeoPanthera 5y ago
  It's not useless analysis, because even for non-cryptographic hashes you want the likelihood of any arbitrary hash to be roughly equal. A hash function which "prefers" certain outputs has a far higher probability of collision.
- iratewizard 5y ago
  Meow hash is also written by people who initially thought SHA-1 was acceptable for large scale change hashing.
- IncRnd 5y ago

ilitirit 5y ago

I can vouch for xxHash (it does what it says on the can), but I'm really curious to hear from who have experience with meow hash.

thomasahle 5y ago

XXH3 has issues too. There are certain inputs, independent of the seed, on which it collides much more often than it should. On the other hand a hash like UMash guarantees low collisions on any input: https://engineering.backtrace.io/2020-08-24-umash-fast-enough-almost-universal-fingerprinting/ https://engineering.backtrace.io/2020-08-24-umash-fast-enoug...

jdcarter 5y ago

I'm using XXHash3 for verifying data integrity of large (10+MB) blobs. Very fast and appears to work very well in my testing--it's never missed any bit error I've thrown at it. Aside: when storing hashes, be sure to store the hash type as well so that you can change it later if needed, e.g. "xxh3-[hash value]". RFC-6920 also has things to say about storing hash types and values, although I haven't seen its format in common use.

njt 5y ago
> be sure to store the hash type as well so that you can change it later if needed Thanks for sharing this, I'd been doing this on my own for my own stuff (eg. foo.txt-xxh32-ea79e094), but it's good to know someone else has thought it through. I ran into the problem once where someone had named some files foo-fb490c or something similar without any annotation, and when there was a problem, it took a file to figure out they were using truncated sha256 hashes.

DeathArrow 5y ago

Useless analysys since the author says it's not a cryptographic hash but useful as a fast hash for change detection. "we wanted a fast, non-cryptographic hash for use in change detection and deduplication" >A great alternative is XXH3 Meow Hash is twice as fast.

ncann 5y ago
Saying it's twice as fast is rather misleading? They can both hash as fast as RAM speed allows anyway. And if it's something in cache I doubt one is significantly better than the other.
jrochkind1 5y ago
What determines whether a hash is "cryptographic"? What would make it suitable for change-detection but not be "cryptographic"? Is the claim here that it would not be suitable for detecting "malicious" changes, but is still suitable for detecting "natural" changes?
- gleenn 5y ago
  A couple features are that is hard for an attacker to find a collision, and it is hard to reverse the original data from the hash. Both can cause serious security problems but aren't necessary in something like this where most image changes are "natural" and you gain speed by weakening those constraints.
LeoPanthera 5y ago
It's not useless analysis, because even for non-cryptographic hashes you want the likelihood of any arbitrary hash to be roughly equal. A hash function which "prefers" certain outputs has a far higher probability of collision.
iratewizard 5y ago
Meow hash is also written by people who initially thought SHA-1 was acceptable for large scale change hashing.
IncRnd 5y ago