10 ms·
Yes they do. What APIs does Linux offer that allows you to list a directories contents alphabetically starting at a specific filename in constant time? You have
by orf 3y ago
Yes they do. What APIs does Linux offer that allows you to list a directories contents alphabetically starting at a specific filename in constant time? You have to iterate the directory contents.
You can maybe use “d_off” with readdir in some way, but that’s specific to the filesystem. There’s no portable way to do this with POSIX.
Regardless of if you can do it with a single directory, you can’t do it for all files recursively under a given prefix. You can’t just ignore directories, or say that “for this list request, ‘-‘ is my directory separator”.
The use of b-trees in file systems is completely beside the point.
- adrian_b 3y agoThe POSIX API is indeed even older, so it is not helpful. But as you say, there are filesystem-specific methods or operating-system specific methods to reach the true performance of the filesystem. It is likely that for maximum performance one would have to write custom directory search functions using directly the Linux syscalls, instead of using the standard libc functions, but I would rather do that instead of paying for S3 or something like it.
- orf 3y agoYes. You could also just use a SQLite table with two columns (path, contents), then just query that. Or do any number of other things. The question isn’t if it’s possible, because of course it is, the question is if it’s portable and well supported with the POSIX interface. Because if it’s not, then…
- anamexis 3y ago> The question isn’t if it’s possible, because of course it is, the question is if it’s portable and well supported with the POSIX interface. Because if it’s not, then… Where did this goalpost come from? S3 is not portable or POSIX compliant.
- orf 3y agoFrom the article we're commenting on, which is comparing the interface of S3 to the POSIX interface. Not any given filesystem + platform specific interface.
- anamexis 3y agoThe article does not mention POSIX, or anything about listing files, at all.
- zaphar 3y agoThe article starts out by making a comparison between the posix api filesystem calls and S3's api. The context is very much a comparison between those two api surface areas.
- orf 3y agoIt mistakenly mentions UNIX whilst referencing the POSIX filesystem API, and I literally quoted where it talks about listing in my original comment.
- justincormack 3y agoThere are no specific syscalls that you can use for this. The libc functions and the syscalls are extremely similar.