It's not just about competition riding, but simply the terrain and trails have changed. Modern MTBs allow you ride down steeper and rougher terrain with confide

It's not just about competition riding, but simply the terrain and trails have changed. Modern MTBs allow you ride down steeper and rougher terrain with confidence and open up the possibility of riding more trails. All the commenters here have been talking about how new geometry has changed the game. But unfortunately, you cannot find "modern" geometry on older MTBs built before ~2013 unless the frame was an odd custom. Even the king of retrogrouches Grant Petersen of Rivendell has been tweaking his frames to offer longer chainstays for off-road riding. On the otherhand, road bikes haven't changed their frontend geometry since the 80s (72º~73º HTA, ~60mm of trail). For the most part roads haven't changed, but wider tires help with the lack of investment in infrastructure.