4 ms·

Someone will explain this better but, as I understand it, having a huge number of files in a single directory becomes inefficient at a certain number; organisin

• leephillips 12y ago
  Not only inefficient; some linux commands fail when they're used on more than a few million files at a time - there is a maximum number of arguments that they can handle. EDIT: bzbarsky's explanation below is more accurate.
  • bzbarsky 12y ago
    Typically the issue is not the commands themselves but the shell. Trying to do "something *" on the shell command line will expand out the glob, and if the resulting string is too long (e.g. you fail to malloc() it!) the shell will do something ranging from crashing to not running the command and giving a useful error message.
    • larrys 12y ago
      Not claiming to be an expert here but I typically do this when confronted with a large number of files. Instead of: command * for i in [someregex]* do command $i done I know I could also do command [someregex]* but like the comfort of having each item echo back to the terminal so I know the progress.
      • andreasvc 12y ago
        That still relies on the shell expanding a glob of millions of files. Another method is to use 'find' and 'xargs' to avoid specifying the files as arguments explicitly.