There's a paper which does something similar including an analysis of different statistical models on top of a CIELAB color space. Estimating Color-Concept Ass

There's a paper which does something similar including an analysis of different statistical models on top of a CIELAB color space. Estimating Color-Concept Associations from Image Statistics. Ragini Rathore et al., IEEE VIS InfoVis 2019. https://arxiv.org/pdf/1908.00220.pdf https://arxiv.org/pdf/1908.00220.pdf