To me it seems that this connotation is very deeply embedded in Western culture (but not only Western culture, compare for example yin and yang) and doesn't act

To me it seems that this connotation is very deeply embedded in Western culture (but not only Western culture, compare for example yin and yang) and doesn't actually have much to do with race, and more with day and night, light and darkness and the fact that humans are not a nocturnal species and darkness can be dangerous to us. According to https://en.wikipedia.org/wiki/Black-and-white_dualism https://en.wikipedia.org/wiki/Black-and-white_dualism this is traceable to the Ancient Near East and listed in the Pythagorean Table of Opposites for example, so it goes much further back than the Atlantic slave trade. I do not think that it would be possible to rip this metaphor out of our thinking. By the way, a really great linguistic book about how much of our thinking is in systematic metaphors in general, is "Metaphors We Live By", by George Lakoff and Mark Johnson.