Not that irrational. Lots of filesystems slow down with tons of files on the same folder. And your window manager has to draw all their icons and potentially

Not that irrational. Lots of filesystems slow down with tons of files on the same folder. And your window manager has to draw all their icons and potentially render (and hopefully cache) previews when you see your Desktop, which is also costly. I've seen OSs take their sweet time (4-10 seconds) to render all icons on a cluttered Desktop.