True, but how often do you care about the general case? Most images are representations of real objects, or symbols or diagrams designed for human comprehension

True, but how often do you care about the general case? Most images are representations of real objects, or symbols or diagrams designed for human comprehension. Ignoring cases that in practice are very unlikely to occur lets you optimize for the more common cases. Techniques like resizing in a sigmoidal color space (see http://www.imagemagick.org/Usage/resize/#resize_sigmoidal http://www.imagemagick.org/Usage/resize/#resize_sigmoidal ) don't have any rigorous mathematical basis, but they're tuned subjectively to have good results in common cases.