In theory, you could put the filters on for just a fraction of the time to measure the colors, and spend most of the time collecting all the light. I'm not sure

In theory, you could put the filters on for just a fraction of the time to measure the colors, and spend most of the time collecting all the light. I'm not sure how to do that and stay low power. You could have separate smaller color sensing pixels, using only a fraction of the surface area. ETA: There are also transparent solar panels which only absorb UV. If that were tuned for visible colors, you could have three transparent layers over an ordinary panel, and maybe not lose anything.