2 ms·
The reasons for doing so vary. Organisation and performance are the two largest considerations. Old-style Linux filesystems (ext2fs) used a serial list rather
by dredmorbius 6y ago
The reasons for doing so vary. Organisation and performance are the two largest considerations.
Old-style Linux filesystems (ext2fs) used a serial list rather than btree or hash for insertions and deletions. I discovered this in the late 1990s when working on a web dump of about 125,000 files. Rather than split the directory, I created a reiserfs filesystem (the most available hash-based option at the time. Insertions dropped from several seconds per instance to thousandths of a second, possibly less (I don't have the data any more, this by recollection). Ext3, ext4, btrfs, and other alternatives have resolved this concern.
Other concerns are globbing, max shell argument length, possibly hazardous filenames (dashes, switches, etc.), file-scanning for filetype highlighting, the risks of unintended interactions, and simply preferable organisation (date, source, topic/context).
In practice, most filesystems should handle tens of thousands to millions of files per directory. But setting caps of a thousand or so may make life more tolerable if there's any direct human interaction. Far fewer for GUI tools.
Squid's 'cache_dir' tuning parameter is an example of hashing practices "in the wild". It's based not on total objects per directory but on the level 1 & 2 directory tree sizes, 16 and 256 directories respectively (8,192 total directories), typically for a few GB worth of content.
http://www.squid-cache.org/Doc/config/cache_dir/ http://www.squid-cache.org/Doc/config/cache_dir/