4 ms·
> This is likely in part because operating systems themselves also lack these facilities. This is a surprising statement. My impression was that OS file operati
by robertlagrant 6y ago
> This is likely in part because operating systems themselves also lack these facilities.
This is a surprising statement. My impression was that OS file operations are generally async, with sync built on top.
- jeffbee 6y agoDepends on the operating system. This is one reason why it is folly to try to abstract away the differences between operating systems. Ordinary Linux file i/o for example cannot be made non-blocking and ignores O_NONBLOCK flag. See open(2).
- iameli 6y agoHm. Confused by Node.js async fs operations in this context. Does it just not actually work asynchronously behind the scenes?
- dmm 6y agoI haven't looked at Node's implementation in a few years but local i/o and was done with a thread pool to make the sync operations async. Lookup the libuv threadpool for more details.
- satori99 6y agoNode.js uses libuv for async file operations -- And libuv has its own internal thread pool to make file ops appear fully async to a node program. From the libuv docs [0]: Unlike network I/O, there are no platform-specific file I/O primitives libuv could rely on, so the current approach is to run blocking file I/O operations in a thread pool. For a thorough explanation of the cross-platform file I/O landscape, checkout this post [1]. [0] http://docs.libuv.org/en/v1.x/design.html#file-i-o http://docs.libuv.org/en/v1.x/design.html#file-i-o [1] https://blog.libtorrent.org/2012/10/asynchronous-disk-io/ https://blog.libtorrent.org/2012/10/asynchronous-disk-io/
- deleted 6y ago[deleted]
- merb 6y ago> Ordinary Linux file i/o for example cannot be made non-blocking well reading and writing can, opening can't. there was also a good conversation about that on HN: https://news.ycombinator.com/item?id=17654550 https://news.ycombinator.com/item?id=17654550
- simcop2387 6y agoThis has actually changed with io_uring in the latest sets of linux kernels. It's all still very new and not well supported in most standard libraries or languages yet though. https://lwn.net/Articles/810414/ https://lwn.net/Articles/810414/ For cases like that discussion about opening lots of files over NFS this should allow it to be done completely async. The actual time to do it won't be better on a per file basis but you'd be able to parallelize them and do other work while waiting on the opens to complete.
- merb 6y agowell yeah io_uring would fix this, but as far as I know there is no support for opening files with io_uring, yet? maybe I'm wrong here tough, but the initial interface was without.
- wtallis 6y agoSupport for openat2() and close() was merged during the 5.6-rc period. So it's only been in stable kernels for about 5 months.