No it wouldn't. An actually red object would have the exact same red-cone spatial pattern no matter how far away it is, because the pattern is coming from insid

No it wouldn't. An actually red object would have the exact same red-cone spatial pattern no matter how far away it is, because the pattern is coming from inside the retina not from the object itself. An actually weirdly striped black and white object with the exact same pattern as the retina would behave like you said.