4 ms·
Actually, you're right. I just tested recursively grepping a remote directory with a thousand 16KiB files. The speed is completely catastrophic: $ time gre
by paskozdilar 5y ago
Actually, you're right. I just tested recursively grepping a remote directory with a thousand 16KiB files. The speed is completely catastrophic:
$ time grep -r foo test
...
real 7m36,154s
user 0m0,110s
sys 0m0,249s
The average download speed from remote host is ~2MiB/s, so it isn't a network issue - a thousand 16KiB files sum up to 16MiB so it shouldn't take more than ten seconds. Seven minutes is terrible.
I don't understand why is this happening. Is this an inherent limitation of remote filesystems or is sshfs implementation just sucky?
EDIT: parallelization seems to help - running 100 concurrent reads at a time uses up to ~1MiB bandwidth. I suppose ordinary unix tools don't play well with remote filesystems because they do stuff sequentially.
That being said, if JetBrains is struggling with SSHFS, then they really need to fix their file browsing code.
- mekster 5y agoI think it's because it's not simply a file download but instead keeps on asking for file content or meta data making a roundtrip on each files to cause excessive latencies against many files. All the tools are basically designed to work on local files and they tend to work bad interacting directly on mounted file systems.