I agree people shouldn't judge, and I'm not defending it, but I think the fear they have is this: if you look like a certain "type", there's a decent chance you

I agree people shouldn't judge, and I'm not defending it, but I think the fear they have is this: if you look like a certain "type", there's a decent chance your behaviour is predictable, and thus, safe (or at least manageable). But if you're just completely different people have no model to predict how you'll act. When people are confronted with that, some people react with hostility, some with avoidance, some by freezing up -- and some rare souls who have the confidence to have an open mind.