3 ms·
The times seem sublinear, 10k files is less than 10x 1k files. I remember getting in to a situation during the ext2 and spinning rust days where production dir
by tln 1y ago
The times seem sublinear, 10k files is less than 10x 1k files.
I remember getting in to a situation during the ext2 and spinning rust days where production directories had 500k files. ls processes were slow enough to overload everything. ls -F saved me there.
And filesystems got a lot better at lots of files. What filesystem was used here?
It's interesting how well busybox fares, it's written for size not speed iirc?
- SkiFire13 1y ago> The times seem sublinear, 10k files is less than 10x 1k files Two points are not enough to say it's sublinear. It might very well be some constant factor that becomes less and less important the bigger the linear factor becomes. Or in other words 10000n+C < 10000(n+C)
- tln 1y agoThe article has data points for n=10,100,1000,10000. Taking (n=10,000 - n=10)/(n=1,000 - n=10) would eliminate the constant factor and we'd expect about 10.09x higher times for a linear algorithm. But for lsr, it's 9.34. The other tools have factors close to 10.09 or higher. Since ls has to sort it's output (unless -F is specified) I'd not be too surprised with a little superlinearity. https://docs.google.com/spreadsheets/d/1EAYua3B3UeTGBtAejPw2PfVK__Fd_x2PVe0xW20cFAU/edit?gid=119533988#gid=119533988 https://docs.google.com/spreadsheets/d/1EAYua3B3UeTGBtAejPw2...
- otterley 1y agoExt2 never got better with large directories even with SSDs (this includes up to ext4). The benchmarks don’t include the filesystem type, which is actually extremely important when it comes to the performance of reading directories.