The way I think of it is that there is some hypothetical continuous diffusion process leading from the original noise sample to the final output. We could get

The way I think of it is that there is some hypothetical continuous diffusion process leading from the original noise sample to the final output. We could get a very good estimate of this trajectory if we took a very large number of very small steps, but that's inefficient and unnecessary. What we want to do is take a small number of steps, and still get a good output. The sampler tells us how we should space out our steps so that we can get the most useful information out of them. If you imagine the diffusion trajectory as a curve line in 2D space, you can afford to take a big step in regions where the line is flat. But if you took a big step in a curved region, you'd move tangent to the curve and end up stepping off the trajectory (thus getting a worse image - or at least one with less fidelity to the underlying continuous trajectory).