In fact I did a bit of research after I started working on this and it turns out Adobe did exactly what you're describing: https://research.adobe.com/publicatio

In fact I did a bit of research after I started working on this and it turns out Adobe did exactly what you're describing: https://research.adobe.com/publication/playful-palette-an-interactive-parametric-color-mixer-for-artists/ https://research.adobe.com/publication/playful-palette-an-in... (the "learn more" button links to a detailed pdf with pictures of the interface)