If you are finding that many matches that early, it might help to do multiple passes. On the first pass, identify up to X number of duplicates. Once you have X

If you are finding that many matches that early, it might help to do multiple passes. On the first pass, identify up to X number of duplicates. Once you have X number of duplicates or reach the end of the file, iterate over the CSV and merge those rows. Then start again identifying a new set of duplicates, then apply those updates, over and over, until the file has no more duplicates