2 ms·
This is good to know. The other week, I ended up breaking a flat directory with approaching that number of files in it, into a hierarchy of subdirectories. Fo
by neilv 5y ago
This is good to know. The other week, I ended up breaking a flat directory with approaching that number of files in it, into a hierarchy of subdirectories.
Fortunately, the filenames were timestamp-based (down to the nanosecond), so could be be solved in one line of `sed`, and the hierarchy made tons of sense.
Normally, especially knowing older filesystem limits, I would've done such a filesystem hierarchy from the start (even if something wasn't as inherently hierarchical as timestamp strings, like if it used hash strings).
But this wealth of files (originally a secondary QA/debugging check, now a potentially valuable training data set) was due to unexpected success of an MVP launch, and it held up perfectly in production. I only cleaned it up when saving that data offline the other week, so it would be convenient&correct to use with tools that might not scale as well as that Linux filesystem and GNU tools did.