A simple, fundamental reason that hasn't shifted in 20 years and in some ways has only grown worse as our collective hard drives have grown increasingly larger

A simple, fundamental reason that hasn't shifted in 20 years and in some ways has only grown worse as our collective hard drives have grown increasingly larger versus RAM size: File I/O. Compilers eat a lot of files and produce a number of more files, sometimes with several passes of intermediate files along the way. You could have the most efficient possible compiler and it still has to read N files and output X binary objects back to disk. Those will always be bottlenecks that are sometimes extremely visible in wall clock time (even on the fastest SSDs sometimes, once you account also for OS paging and caching and background processing like anti-virus tools and other security audits).