I think it's in part because differences in gradient are quite difficult to distinguish by eye, and the difference in the higher order curves is in just that (a

I think it's in part because differences in gradient are quite difficult to distinguish by eye, and the difference in the higher order curves is in just that (and scaling things to be visible means the difference in curvature gets smaller as you go to higher orders). Even nearly-straight lines can be quite difficult to tell from actually-straight without a nearby reference.