What things should and should not be does not affect what they can be. The point is that the image data can be encoded. It's not "extra," it's the actual image

What things should and should not be does not affect what they can be. The point is that the image data can be encoded. It's not "extra," it's the actual image itself being used to relay additional information covertly. How do you differentiate between which pixels are "good" and which ones are "bad"? There is no way to know. To process such a file, you DO need an interpreter that follows the "rules" of the encoding, but the interpreter can be small, concealed, and varied, resulting in a cat and mouse game if you are trying to block such implementations.