2 ms·

Related: https://mastodon.social/@pervognsen/110739397974530013 https://mastodon.social/@pervognsen/110739397974530013 But this tests perf for only finding one

by rurban 3y ago

Related: https://mastodon.social/@pervognsen/110739397974530013 https://mastodon.social/@pervognsen/110739397974530013 But this tests perf for only finding one file. bfs is optimized for finding many files at once, as it defers closing the dirhandle, and uses io_uring for parallel IO callbacks.

tavianator 3y ago
`bfs` doesn't actually use io_uring yet, but it is planned. I'm not sure I'd say it's specifically optimized for finding multiple files at once either, I try to make it fast for many different use cases. There's two benchmarks in the blog post and a few more that I run regularly, e.g. https://github.com/tavianator/bfs/pull/107 https://github.com/tavianator/bfs/pull/107
austin-cheney 3y ago
I am performing a similar file system tree navigation asynchronously in Node.js which is just a shallow API over the C Linux FS APIs. I can see you are using opendir and closedir functions? What is the benefit from using the opendir function[1] when readdir[2] can be called on a location directly? Is the benefit that opendir returns a file descriptor for use in opening a stream to gather directory object descriptors? [1] https://man7.org/linux/man-pages/man3/opendir.3.html https://man7.org/linux/man-pages/man3/opendir.3.html [2] https://man7.org/linux/man-pages/man3/readdir.3.html https://man7.org/linux/man-pages/man3/readdir.3.html Your project is probably more mature but if you want an alternate approach to examine here is I have been doing it: https://github.com/prettydiff/share-file-systems/blob/master/lib/terminal/commands/library/directory.ts https://github.com/prettydiff/share-file-systems/blob/master... I considering changing my use of readdir to use the withFileTypes option so that it returns a list of directory entries (objects of artifact name and type) instead of a list of conditions to discern types like I am doing on lines 382-432.
- lmz 3y ago
  If you read the manpages you linked to you'll see that in the C API, readdir cannot be called with a path but needs a directory handle returned by opendir. Node.js is probably doing this in the background when you call readdir.

by rurban 3y ago

tavianator 3y ago

`bfs` doesn't actually use io_uring yet, but it is planned. I'm not sure I'd say it's specifically optimized for finding multiple files at once either, I try to make it fast for many different use cases. There's two benchmarks in the blog post and a few more that I run regularly, e.g. https://github.com/tavianator/bfs/pull/107 https://github.com/tavianator/bfs/pull/107

austin-cheney 3y ago

I am performing a similar file system tree navigation asynchronously in Node.js which is just a shallow API over the C Linux FS APIs. I can see you are using opendir and closedir functions? What is the benefit from using the opendir function[1] when readdir[2] can be called on a location directly? Is the benefit that opendir returns a file descriptor for use in opening a stream to gather directory object descriptors? [1] https://man7.org/linux/man-pages/man3/opendir.3.html https://man7.org/linux/man-pages/man3/opendir.3.html [2] https://man7.org/linux/man-pages/man3/readdir.3.html https://man7.org/linux/man-pages/man3/readdir.3.html Your project is probably more mature but if you want an alternate approach to examine here is I have been doing it: https://github.com/prettydiff/share-file-systems/blob/master/lib/terminal/commands/library/directory.ts https://github.com/prettydiff/share-file-systems/blob/master... I considering changing my use of readdir to use the withFileTypes option so that it returns a list of directory entries (objects of artifact name and type) instead of a list of conditions to discern types like I am doing on lines 382-432.

lmz 3y ago
If you read the manpages you linked to you'll see that in the C API, readdir cannot be called with a path but needs a directory handle returned by opendir. Node.js is probably doing this in the background when you call readdir.