3 ms·
When I started reading such papers this also confused me. Yes, it's about randomness and probability of collisions. Adding two characters might be OK or not, i
by thomasmg 3y ago
When I started reading such papers this also confused me.
Yes, it's about randomness and probability of collisions. Adding two characters might be OK or not, it depends on the hash function and the use case. For example, the Java "String.hashCode" method return "abc-1": 92597638, "abc-2": 92597639; the hash code of the "-2" is (almost) always one greater than the "-1" variant. So the two are not independent. Anyway 32-bit hash functions might not be good and you need 64 bit. Then you should use something like MurmurHash.
But you don't need to compute MurmurHash twice. It is probably enough to compute one MurmurHash, and then hash this hash code using an integer hash function [1]: h1=integerHash(hash + 1), h2=integerHash(hash + 2). This is almost twice as fast for string data.
For certain use cases, for example for Bloom filters, you need more hash "functions". For Bloom filters, it is even simpler: you can split a 64-bit hash code into a 32-bit start, and a 32-bit offset. And then use: h(x)=start+x*offset. That means h(3)=h(2)+offset and so on. It is really fast.
https://stackoverflow.com/questions/664014/what-integer-hash-function-are-good-that-accepts-an-integer-hash-key/12996028#12996028 https://stackoverflow.com/questions/664014/what-integer-hash...
- dalke 3y agoAlong the same lines, what about using the first `n` outputs from a random number generator? (With "it depends on the PRNG" replacing your "it depends on the hash function".) I come from cheminformatics, where we've used a form of superimposed coding very similar to a Bloom filter. In one incarnation, developed in the 1980s, enumerate all linear structures of up to length 7, determine a unique order based on atom and bond characteristics, convert that into a PRNG seed, and generate (typically) 4 values, each used to set a bit in a bitstring of size ~1024 bits. I write similar because (1) in some cases the number of bits is not fixed, eg, it may vary depending on the structure, and (2) it uses a PRNG instead of `n` different independent hash functions. I'm therefore curious if PRNGs have been examined in a Bloom filter context, or for MinHash, which also has `n` different hash functions. Then I can drop (2) from my list of differences.