4 ms·

The example of async I/O in Node.js is with a filesystem operation. How does that work? AFAIK Linux, unlike windows, doesn't have a proper async API for filesy

The example of async I/O in Node.js is with a filesystem operation. How does that work? AFAIK Linux, unlike windows, doesn't have a proper async API for filesystem I/O. POSIX AIO is implemented using threads in libc and Linux AIO (io_submit, etc) only works with O_DIRECT (no kernel caching) and there are other issues depending on underlying filesystem driver. Is there any solution?

biokoda 9y ago
Yes a user space thread pool which is what node.js does on top of libuv.
steveklabnik 9y ago
https://nikhilm.github.io/uvbook/filesystem.html https://nikhilm.github.io/uvbook/filesystem.html > The libuv filesystem operations are different from socket operations. Socket operations use the non-blocking operations provided by the operating system. Filesystem operations use blocking functions internally, but invoke these functions in a thread pool and notify watchers registered with the event loop when application interaction is required.
- bogomipz 9y ago
  I'm not familiar with libuv but is there a dedicated "watcher" thread in a thread pool that the kernel wakes up when when the disk driver has completed that I/O that it was sleeping on? If so is there a dedicated "watcher" thread for each non-blocking I/O request handled by libuv?
  - steveklabnik 9y ago
    I’m not familiar at that level.
  - dullgiulio 9y ago
    There is no watcher: threads get parked by the kernel as "not runnable" while they perform the I/O operation. When the operation is finished, then they become runnable again so the kernel scheduler runs them--that is, it runs the code that comes after the I/O operation. This code (from libuv) notifies the requester that the operation is complete.
    - bogomipz 9y ago
      Thanks. So is the libuv package implemented as two distinct components then consisting of: 1) The main event loop thread. 2) a thread pool of "workers" which make the actual I/O request and sleep while waiting for the I/O. Then when a "worker" thread is set to runnable again by the kernel(because it's I/O request has been completed)notifies the main event loop thread via a signal? Is that correct?
      - dullgiulio 9y ago
        That's correct, except the notification back is probably via some queue (think of futures or Go channels) rather than signals.
        bogomipz 9y ago
        I see that makes sense. Cheers.

biokoda 9y ago

Yes a user space thread pool which is what node.js does on top of libuv.

steveklabnik 9y ago

https://nikhilm.github.io/uvbook/filesystem.html https://nikhilm.github.io/uvbook/filesystem.html > The libuv filesystem operations are different from socket operations. Socket operations use the non-blocking operations provided by the operating system. Filesystem operations use blocking functions internally, but invoke these functions in a thread pool and notify watchers registered with the event loop when application interaction is required.

bogomipz 9y ago
I'm not familiar with libuv but is there a dedicated "watcher" thread in a thread pool that the kernel wakes up when when the disk driver has completed that I/O that it was sleeping on? If so is there a dedicated "watcher" thread for each non-blocking I/O request handled by libuv?
- steveklabnik 9y ago
  I’m not familiar at that level.
- dullgiulio 9y ago
  There is no watcher: threads get parked by the kernel as "not runnable" while they perform the I/O operation. When the operation is finished, then they become runnable again so the kernel scheduler runs them--that is, it runs the code that comes after the I/O operation. This code (from libuv) notifies the requester that the operation is complete.
  - bogomipz 9y ago
    Thanks. So is the libuv package implemented as two distinct components then consisting of: 1) The main event loop thread. 2) a thread pool of "workers" which make the actual I/O request and sleep while waiting for the I/O. Then when a "worker" thread is set to runnable again by the kernel(because it's I/O request has been completed)notifies the main event loop thread via a signal? Is that correct?
    - dullgiulio 9y ago
      That's correct, except the notification back is probably via some queue (think of futures or Go channels) rather than signals.
      - bogomipz 9y ago
        I see that makes sense. Cheers.