4 ms·

> It was not the spinning up of the disk because I first did ls in the parent folder which was immediate That doesn't tell you anything; the parent's dentries

> It was not the spinning up of the disk because I first did ls in the parent folder which was immediate That doesn't tell you anything; the parent's dentries could have been cached days ago and still present, meaning it didn't actually access the disk or cause it to be spun up (if it wasn't) at all. When doing any kind of repeatable measurement or experimentation on disks you will want to drop the page cache every time first: # echo 3 >/proc/sys/vm/drop_caches

amelius 2y ago
Well, I just booted the computer ;) But you are right, dropping the cache is probably a better way. I have a folder with 5500 subfolders. Doing "ls -U" in that folder (after dropping the page cache like above) takes 50 seconds (!) And the dir entries appear all at once, i.e. not in a streaming way. Its parent folder only contains 6 subfolders. Doing a cache drop followed by "ls -U" gives immediate results. How to investigate this further? (Using an Ubuntu 18.04 system)
- rcxdude 2y ago
  strace can tell you what system calls it's doing, what the results are, and how long they're taking, which may help narrow it down.
  - amelius 2y ago
    Thanks. Interestingly, strace speeds up the operation. What took 50s after a cache drop now becomes immediate with "strace -f ls -U". Dropping the cache and doing "time ls -U" gives: real 0m51,204s user 0m0,116s sys 0m0,718s Update: never mind, it appears to be something in my shell. Switching to tcsh completely eradicated the problem.
    - MarceColl 2y ago
      It's probably that ls in the other shell is the builtin instead of the binary, when strace runs ls -U it does run the binary and not the shell builtin. tcsh must also delegate to the binary instead of a builtin, or their builtin is faster.
      - amelius 2y ago
        Yeah, I haven't figured it out yet, but when I do /bin/ls, then indeed the problem does not show. Probably a case of Bash trying to be smarter than it needs to be.

amelius 2y ago

Well, I just booted the computer ;) But you are right, dropping the cache is probably a better way. I have a folder with 5500 subfolders. Doing "ls -U" in that folder (after dropping the page cache like above) takes 50 seconds (!) And the dir entries appear all at once, i.e. not in a streaming way. Its parent folder only contains 6 subfolders. Doing a cache drop followed by "ls -U" gives immediate results. How to investigate this further? (Using an Ubuntu 18.04 system)

rcxdude 2y ago
strace can tell you what system calls it's doing, what the results are, and how long they're taking, which may help narrow it down.
- amelius 2y ago
  Thanks. Interestingly, strace speeds up the operation. What took 50s after a cache drop now becomes immediate with "strace -f ls -U". Dropping the cache and doing "time ls -U" gives: real 0m51,204s user 0m0,116s sys 0m0,718s Update: never mind, it appears to be something in my shell. Switching to tcsh completely eradicated the problem.
  - MarceColl 2y ago
    It's probably that ls in the other shell is the builtin instead of the binary, when strace runs ls -U it does run the binary and not the shell builtin. tcsh must also delegate to the binary instead of a builtin, or their builtin is faster.
    - amelius 2y ago
      Yeah, I haven't figured it out yet, but when I do /bin/ls, then indeed the problem does not show. Probably a case of Bash trying to be smarter than it needs to be.