I have this weird issue where about a third of people I meet for the first time swear they know me from somewhere, and it's somewhere specific that I know I've

I have this weird issue where about a third of people I meet for the first time swear they know me from somewhere, and it's somewhere specific that I know I've never been. My dad and brother have the same issue, and we strongly resemble each other, so I think I just have a congenitally familiar face. I have no idea if feelings would automatically transfer to me from people with amnesia, but they certainly do for people without it, even though I don't remind them of anyone they know well enough to name.