I think people tend not to see it as a big thing (or even not notice it at all) because colour being wildly inconsistent is the normal case in the real world,

I think people tend not to see it as a big thing (or even not notice it at all) because colour being wildly inconsistent is the normal case in the real world, due to changing lighting conditions, and our brains adjust for it automatically.