4 ms·

In really new linux there is a preadv2 sysscall that was merged. preadv2 supports a flag RWF_NOWAIT to tell the kernel to not block if a read requires uncached

In really new linux there is a preadv2 sysscall that was merged. preadv2 supports a flag RWF_NOWAIT to tell the kernel to not block if a read requires uncached data. This lets you play around with policy for blocking reads in user space. Examples: try to make progress on partially read data & queue up the rest on another thread; try performing reads from your network thread and if unavailable queue up the blocking read onto another disk thread & serve a different request. There's no RWF_NOWAIT support for write in pwritev2. But it's technically possible to implement it and if your write will cause a writeout to disk return EWOULDBLOCK. LWN description: https://lwn.net/Articles/612483/ https://lwn.net/Articles/612483/ LWN summary of my talk: https://lwn.net/Articles/636967/ https://lwn.net/Articles/636967/ Final patch set by Christoph: https://lwn.net/Articles/731700/ https://lwn.net/Articles/731700/ Disclaimer: I'm the author of the preadv2/pwritev2 syscalls. And the original author of the support for RWF_NOWAIT, which Christoph Hellwig took over. So feel free to consider this to be self congratulatory. Correction: It looks like there's a patch set floating around for adding support for RWF_NOWAIT in pwritev2 as well. https://patchwork.kernel.org/patch/9787271/ https://patchwork.kernel.org/patch/9787271/ (comment from Christoph)

colanderman 9y ago
> try performing reads from your network thread and if unavailable How does this differ from recv(..., MSG_DONTWAIT)?
- tmzt 9y ago
  I think he means perform a disk read in response to a network request, and choose to respond to a different queued/pooled request if the data is not available to complete it. I would assume the intent is similar but applied to a different type/source of IO.
- mtanski 9y ago
  This was my original model when I came up with the idea; in fact, my v1 hack used recv and MSG_DONTWAIT. For a number of a reasons the kernel community did not want to overload that interface and that flag. Besides some technical reasons, the big reason was an "impedance mismatch" of the recv API which was working with stream data. In that context you're depending on another party send data. Also, you can wait on this to happen using another API (select et. al) so the buffer will be filled not by your actions. On the other hand preadv2(..., RWF_NOWAIT) is not going to trigger any more read in and theres no wait to wait on the data. Although the previous statement is not a 100% true... preadv2 may or may not trigger readahead (if it's enabled). Here's the whole thread about it... if you're interested in the history of how this came to be: https://lkml.org/lkml/2014/7/24/787 https://lkml.org/lkml/2014/7/24/787
thekozmo 9y ago
preadv sounds real nice. It isn't relevant to the mmap previous discussion. It's certainly makes cached aio work loads better
hawski 9y ago
Is this a way to have non-blocking reads from file system? For regular files O_NONBLOCK does nothing, so one has to use thread pools. For plain files served from local disk it's usually not worth it. But files can be served by sshfs, nfs or something more exotic made with FUSE. If I understand this correctly one can use RWF_NOWAIT to do what one would naively expect from O_NONBLOCK. Am I correct?

colanderman 9y ago

> try performing reads from your network thread and if unavailable How does this differ from recv(..., MSG_DONTWAIT)?

tmzt 9y ago
I think he means perform a disk read in response to a network request, and choose to respond to a different queued/pooled request if the data is not available to complete it. I would assume the intent is similar but applied to a different type/source of IO.
mtanski 9y ago
This was my original model when I came up with the idea; in fact, my v1 hack used recv and MSG_DONTWAIT. For a number of a reasons the kernel community did not want to overload that interface and that flag. Besides some technical reasons, the big reason was an "impedance mismatch" of the recv API which was working with stream data. In that context you're depending on another party send data. Also, you can wait on this to happen using another API (select et. al) so the buffer will be filled not by your actions. On the other hand preadv2(..., RWF_NOWAIT) is not going to trigger any more read in and theres no wait to wait on the data. Although the previous statement is not a 100% true... preadv2 may or may not trigger readahead (if it's enabled). Here's the whole thread about it... if you're interested in the history of how this came to be: https://lkml.org/lkml/2014/7/24/787 https://lkml.org/lkml/2014/7/24/787

thekozmo 9y ago

preadv sounds real nice. It isn't relevant to the mmap previous discussion. It's certainly makes cached aio work loads better

hawski 9y ago

Is this a way to have non-blocking reads from file system? For regular files O_NONBLOCK does nothing, so one has to use thread pools. For plain files served from local disk it's usually not worth it. But files can be served by sshfs, nfs or something more exotic made with FUSE. If I understand this correctly one can use RWF_NOWAIT to do what one would naively expect from O_NONBLOCK. Am I correct?