3 ms·
> mkdir -p "0/2/7/8/c" That's at most 16 entries per directory which seems unnecessarily low. Taking 2 chars/level gives 96 entries per directory which seems b
by masklinn 11y ago
> mkdir -p "0/2/7/8/c"
That's at most 16 entries per directory which seems unnecessarily low. Taking 2 chars/level gives 96 entries per directory which seems better, that's what Git uses (3 is 4096 which may be a tad high). Yields a flatter but not too broad hierarchy.
- tyingq 11y agoThat's a reasonable optimization, but wouldn't help much with runtime, since you aren't traversing any of it, but rather opening with full paths. I suppose it might make backups, or some sort of monitoring more efficient.
- masklinn 11y ago> That's a reasonable optimization, but wouldn't help much with runtime, since you aren't traversing any of it Depends what data you're storing (or not storing) I guess. If you're storing all metadata anyway, you may even discard the original file name entirely and have the full path be composed of hash sections. But you may want to traverse the tree (because you're not storing much/any) data, and split things out because there's a number of APIs which don't deal well with humongous directories.
- tyingq 11y agoRight, but open("a/b/c/d/e/f/0/1/2/3") isn't slower than open("ab/cd/ef/01/23"). Flatter only matters if you are traversing the hierarchy, which wouldn't be a normal use case for this.
- loeg 11y agoSure it is. You have to do twice as many directory lookups, even if they're cached.
- tyingq 11y agoThe open() system call isn't doing recursive "directory lookups" (like readdir()) per se. It does have to traverse a linked list (dentries) to map the path+filename into an inode. But 6 deep into a linked list vs 3 deep is, I suspect, hardly measurable in terms of elapsed time. It's all hashed, and cached.
- lazyant 11y agoI must be stupid; with 2 hex chars (dir structure like 0a/7b/...) wouldn't we have 16*16 = 256 (edit: can't do math) entries per directory, not 96?
- masklinn 11y ago256 actually, but I have no idea where that 96 comes from, it was probably the alcohol typing (or I was already thinking 4096 and I wrote part of that instead of the correct number)
- tyingq 11y agoWorth mentioning that's "directory entries per directory". The "data files per directory" isn't driven by the intermediate path...just the total number of "full paths".