The assumption being made here is that the labels are fixed categories, immutable and the same from person to person; and yet we know that's not always the case

The assumption being made here is that the labels are fixed categories, immutable and the same from person to person; and yet we know that's not always the case. It seems to me that we have the capability of deriving labels from experience, but don't (especially in the case of the colour-blind) all have the same experiences. I know there has been a fair amount of research on colour names in cross-cultural contexts and that for the most part there is a pretty clear similarity between the colour names employed in different cultures, but that not all cultures distinguish between blue and green for instance.