4 ms·
Yeah I'm unclear on why this even requires a clever algorithm. I'd think that given the 1.4 million precomputed hashes, a simple/naive Python function (<10 line
by throwaway287391 5y ago
Yeah I'm unclear on why this even requires a clever algorithm. I'd think that given the 1.4 million precomputed hashes, a simple/naive Python function (<10 lines) that builds a dict mapping hash -> (list of images with that hash) could surface all the collisions in a few seconds. (It's a cool article though! I'm glad someone tested this.)
Edit: I'm procrastinating so I tried it. It's 8 lines including IO/parsing and runs in 2.8 seconds on my laptop:
import collections
hash_to_filenames = collections.defaultdict(list)
with open('hashes.txt') as f:
for line in f.readlines():
filename, hash = line.strip().split()
hash_to_filenames[hash].append(filename)
dupes = {h: fs for h, fs in hash_to_filenames.items() if len(fs) > 1}
print(f'Done. Found {len(dupes)} dupes.')
(hashes.txt is from the zip in the github repo, and it finds 8865 dupes which looks almost right from the article text (8272 + 595 = 8867).)