4 ms·

That's crazy, I was just diving into the nilsimsa LSH utility (http://ixazon.dynip.com/~cmeclax/nilsimsa.html http://ixazon.dynip.com/~cmeclax/nilsimsa.html) co

• paulgb 12y ago
  This is the first time I've heard of nilsimsa. Looking through the docs to find the algorithm, here's how it is described, but I'm still not clear on what a "character separation" is in this context or which 3 characters are chosen: > Nilsimsa uses eight sets of character separations (character, character, etc.) and takes the three characters and performs a computation on them: (((tran[((a)+(n))&255]^tran[(b)]*((n)+(n)+1))+tran[(c)^tran[n]])&255), where a, b, and c are the characters, n is 0-7 indicating which separation, and tran is a permutation of [0-255]. The result is a byte, and nilsimsa throws all these bytes from all eight separations into one histogram and encodes the histogram. I've found that in practice, shingling text and then minhashing it is scary-good at finding similar documents.