How can the scaling algorithm distinguish between "checkerboard representing high spatial frequency detail" and "checkerboard representing intermediate color"?

How can the scaling algorithm distinguish between "checkerboard representing high spatial frequency detail" and "checkerboard representing intermediate color"? There are games that use both. The only scaling technique I know that can do this with acceptable accuracy (in real time!) is "human brain scaling".