Caching is nice, but if it takes an hour when the cache is cold then it's not usable even if a directory with millions of entries posed no problems for any tool

Caching is nice, but if it takes an hour when the cache is cold then it's not usable even if a directory with millions of entries posed no problems for any tools. Now for most projects that's not going to happen, but for something like the Linux kernel it just might. And there might be many caches to warm, and much to cache. For a large enough project and a computer that one might not think of as small that might just fail to work at all. This is why as Git grows up it's using Bloom filters to optimize things like git blame and git log on one file rather than adopting the fully relational model of Fossil. (Please read into this all the chagrin I'm feeling about that, because I'm quite the fan of Fossil's relational model.) PS, the single biggest problem with million-entry directories is the propensity of tools like ls(1) to want to sort the darned things, so one has to remember to use `ls -f`, or to at least use the C locale to get memset() collations instead of much much slower Unicode collations. Another problem is that the POSIX stat(2) family of functions combine reading metadata that could come from just the directory (e.g., a file's inode number) (and which contents has already been read) with reading metadata that requires [possibly much] extra I/O to get, so if you're doing `ls -l` on a million-entry directory you might as well go on vacation (but make sure to send the output to some file, cause your scrollback buffer just won't do).