Do you know what percent of records you expect to be unique? Also, what do you want to do with the unique records? That might effect what initial processing m

Do you know what percent of records you expect to be unique? Also, what do you want to do with the unique records? That might effect what initial processing method is best for your goal.