Deduplication is a hard problem: what constitutes a "dupe" is somewhat arguable. My first approach just used metadata, but that doesn't aggregate the files str

Deduplication is a hard problem: what constitutes a "dupe" is somewhat arguable. My first approach just used metadata, but that doesn't aggregate the files stripped of their original metadata, like what you get from a Google Takeout, so I had to add image hashing as well. I actually generate three mean hashes in L*a*b color space for PhotoStructure (many hashes ignore color). I've also found that metadata needs to be normalized, including captured-at time, and even exposure metadata. It's a lot of whack-a-mole, especially add new cameras and image formats are released every year. I described more about what I've written for PhotoStructure (which does deduplication for both videos and images) here: https://photostructure.com/faq/what-do-you-mean-by-deduplicate/ https://photostructure.com/faq/what-do-you-mean-by-deduplica... -- it might help you avoid some of the pitfalls I've had to overcome.