Try turning down the color saturation. For me ~ 25% of original saturation is sufficient for using UIs that use color information. I had the same annoyance as

Try turning down the color saturation. For me ~ 25% of original saturation is sufficient for using UIs that use color information. I had the same annoyance as you, switching back and forth wasn't pleasant, and I'd forget. I think part of the reason is that switching between 0% <-> 100% saturation doesn't give room to adapt. It feels like a bandwidth of information is missing. But, if the colorspace is barely noticeable, your brain will fill in the colors for you with the information it gets... and the high-saturation default color-set would start to appear abrading and unnatural.