4 ms·
Don't modern filesystems handle this nicely - eg btrfs
by ape4 4y ago
Don't modern filesystems handle this nicely - eg btrfs
- londons_explore 4y agoEven if the filesystem handles it nicely, there are lots of applications which don't. So many applications think it's fine to iterate through every file in a directory for various reasons. If there are 10 million files in a directory, that is effectively a linear scan over a huge database table - which is generally advised against...
- pixl97 4y agoHeh, in Windows I've seen some applications do worse, they iterate the same file multiple times! Developer never noticed it because they look at 10 test files and never profiled their behavior, then you stick it in the customer environment and they face it at a huge directory and things explode.
- klooney 4y agoAlso `ls` in a big filesystem used to spend all its time allocating tiny chunks of memory- the breakage is pervasive.
- mprovost 4y agoReiserFS handles this really well but I don't know if you can call it modern anymore. I can't think of any other filesystem that followed its design choices though.
- space_ghost 4y agoReiserFS was my first thought when I read the headline. Fantastic performance on folders with obscene file counts.
- pixl97 4y agoMuch better especially on SSD, but you still run into issues. At the end of the day the filesystem is only part of the issue, quite often it's stupid application interaction where the designer of the application thinks they'll only ever see a few hundred or maybe a thousand files and you present the app with a million files.