I think my last link describes the problem accurately from the ground up. TL;DR the raw pixel values are typically not linearly proportional to the luminescenc

I think my last link describes the problem accurately from the ground up. TL;DR the raw pixel values are typically not linearly proportional to the luminescence of the color channels. But a cone cell in the eye actually averages the luminescence of different areas if you look at a fine structured image from far away, so if you want to emulate the perception of the eye then you have to average luminescence values instead of raw pixel values. The conversion is well defined by the colorspaces in use. Most pictures on the Internet uses the sRGB colorspace, which is not linear. Color perception and colorspaces are actually not straightforward in the sense that naive algorithms often fail. But the basics are not that hard.