6 ms·

I added xxh3 to my fork of smhasher: https://github.com/injinj/smhasher https://github.com/injinj/smhasher I find it to be the fastest hash in my stable of has

• unknownknowns 8y ago
  Thank you! Currently using XXH64 to good success for small keys, and xxh3 looks fantastic as the next upgrade.
• BeeOnRope 8y ago
  For each size shown on the x-axis, every hashing call using exactly that size? It would be interesting to also have a test where an average size was chosen, but where the actual sizes varied randomly around that average - because that's a common case in real world code, and the results can be very different since branch prediction won't be perfect any more as it is with the fixed size tests. Also, latency usually means feeding the output of the function back into the input of the next call. Throughput is what you get when you don't do that but you still need to consume the result, so it's not optimized away. You can use something like DoNotOptimize here: https://github.com/google/benchmark/blob/master/include/benchmark/benchmark.h#L309 https://github.com/google/benchmark/blob/master/include/benc...
  • injinj 8y ago
    Yes, it is the same size. I agree, a complex test like that in the original article is better (the Throughput versus Latency section). The graphs from the "latency" test are doing this: for (;;) { h = hash( val, seed ); seed += h; } The original smhasher did this: for (;;) { hash( val, seed ); }
    • BeeOnRope 8y ago
      Right, so that is a latency test because each result feeds back into the next call (through seed). A typical throughput test would be like: total = 0; for (;;) { total += hash( val, seed ); } Note that the hash value is consumed, but doesn't feed into the next result so multiple hashes can run in parallel. If that wasn't the case, it is lucky the loop wasn't compiled away entirely (probably because the hash functions and the test loop are in different compilation units and LTO is not turned on).
      • injinj 8y ago
        I just re-ran this with a total += hash(). It adds about 2 - 4 cycles to the throughput test (the non-latency graph).
    • Cyan4973 8y ago
      Actually, making the next hash dependent on previous hash through the `seed` doesn't work for benchmarking latency. Several hashes only use the seed at the very end of calculation. Among them, CityHash, FarmHash, MeowHash, probably a few more. This makes it possible for them to start calculating next hash before the end of the previous one, so it's no longer "latency" for them, and the test condition becomes uneven. In reality, in a latency scenario, the hash function is waiting for the input. So it's the input which must depends on previous hash result. This way, all algorithms must wait for previous result, no more dependency on how a specific algorithm handle the seed.
      • injinj 8y ago
        Good point. I created a graph with these attributes: 0 total += hash (XXH364) 1 seed += hash (XXH364Seed) 2 seed += hash, val += hash (XXH364SH) https://gist.github.com/injinj/138543ccc6a23ceb1fcdc05f4628858e https://gist.github.com/injinj/138543ccc6a23ceb1fcdc05f46288... There are weird bumps at some key sizes with City and XXH when the key is updated with the previous hash. My guess is there is some underlying cache line latency added when the key is written to. smhasher should really have a test for good seeding. As I understand it, bad City seeding was the reason people started using SipHash.
• tmd83 8y ago
  What I really loved was the different benchmark ideas e.g. variable input size and hashing as part of a bigger algorithm.