I think it's that annoying because there is a mismatch between your intention (scrolling the page incrementally) and its reaction (moving to the next "slide").

I think it's that annoying because there is a mismatch between your intention (scrolling the page incrementally) and its reaction (moving to the next "slide"). It acts out of your control.