You could solve the averaging problem by applying the L1/L2 trained net to exponential color gradients rather than linear ones. By doing so you are telling the

You could solve the averaging problem by applying the L1/L2 trained net to exponential color gradients rather than linear ones. By doing so you are telling the NN that humans care about exponential changes in color because that's how our eyes work. E.g. we notice if you double the amount of green but probably not if you just increment it. I think there are rather simple solutions to these types of problems. But sometimes everything looks like a nail.