Because hue is scientifically defined and objective. That lets us sidestep all the linguistic stuff so we have something meaningfully perceptual to ground the c

Because hue is scientifically defined and objective. That lets us sidestep all the linguistic stuff so we have something meaningfully perceptual to ground the conversation. And white, gray and black have no hue at all. They're not the same color -- they have no color at all. The hue is indeterminate. (Even if we call them "colors" colloquially.)