3 ms·
Whether a large number of files becomes sluggish really depends on your file system. In any case, a common technique is to break large numbers of files into sub
by michaelmior 11y ago
Whether a large number of files becomes sluggish really depends on your file system. In any case, a common technique is to break large numbers of files into subfolders which usually does reasonably well at solving this problem.
As for updating, flock[0] solves this issue on operating systems which support it.
[0] http://linux.die.net/man/2/flock http://linux.die.net/man/2/flock
- jamiesonbecker 11y agoReiserFS (v3) is a great small filesystem that's fantastic at lots of small files (and also copes great with power outage events, like on my laptop for the last 15 years). I've had tons of issues with ext3/4 (running out of extents, slow performance on lots of small files), btrfs (running out of metadata space when I still have hundreds of GB left?!), xfs (great on everything except lots of tiny files or power loss). It even supported reliable shrinking and growing on LVM. It's too bad no one is supporting it anymore, since the founder is in prison and the only other people who seem to be able to support it seem to be focused on a Reiser4 pipe dream instead of supporting great, reliable technology that had most of the bugs worked out a long time ago.
- sqrt17 11y agoUsenet news and maildir are cases where current operating systems already have to cope with that kind of load, so it's definitely possible. The question is, can this be useful without becoming a partial and bug-ridden reimplementation of a NoSQL database (just because we have NoSQL databases that fit the bill and carry less maintenance costs wrt a spit-and-glue solution).