People don't treat others all alike. We naturally interact with people of different genders, ages, and in different contexts differently, and continually pick u

People don't treat others all alike. We naturally interact with people of different genders, ages, and in different contexts differently, and continually pick up social cues (such as how loud or forward someone is) and change our behaviour with regard to the situation. When people are unfamiliar with the signals they perceive from others, this usually causes them anxiety. Clearly, with your experience of interacting with disabled people, you've become accustomed to it; you know to speak with a quieter voice for instance. Most people do not have much such experience, and feel uncomfortable because of it.