3 ms·

1. make a non-blocking read on the data 2. while it is being paged into memory, <DO OTHER STUFF> 3. now your data is in memory and you can update it (writes a

by smegel 10y ago

1. make a non-blocking read on the data 2. while it is being paged into memory, <DO OTHER STUFF> 3. now your data is in memory and you can update it (writes are async by default even on Linux, as the write just goes into memory and will get synced out by the kernels page sync mechanism but you can override that by setting the O_SYNC flag).

Dylan16807 10y ago
Triggering a non-blocking read and continuing execution requires either writing different code or an extremely elaborate page fault feature that can dynamically rewrite your execution flow behind your back. Writing different code is not the question I asked. That's not an OS feature. Do the OSes you like have the latter ability, or are you blowing smoke?
- deleted 10y ago
  [deleted]
- smegel 10y ago
  > Triggering a non-blocking read and continuing execution requires either writing different code Well you are asking about doing something non-blocking. That means you must be in some kind of event-loop scenario, otherwise you would be happy with synchronous operations. And yes, you would need to add a read event on the file, and then trigger the socket.send() once the data is in memory. I never said it was free. But at least it is viable.
  - Dylan16807 10y ago
    Your response to paging being blocking was to blame Linux. Every OS has paging be blocking. Linux has ways to do non-blocking IO. There is no reason to blame Linux.
- to3m 10y ago
  Windows can do this - its "non-blocking" read calls are better described as "asynchronous", and take ownership of the buffer until the job is cancelled or it runs to completion. So if the buffer is paged out, that's fine; the program can carry on and the buffer can get paged in when the system needs it (perhaps on an otherwise idle CPU - so it needn't necessarily take any time from the perspective of the calling thread). The POSIX semantics on the other hand are simply that the read won't block due to lack of input, so it does as much as it can straight away and then returns. If there's data, but the buffer is paged out, the non-blocking read call has to take more time, because the problem is a paged-out buffer and not lack of input. (The Windows equivalents of man page section 1 are so awful that most POSIX fans just run a mile and install cygwin. More fool them; once you get to man page section 2, it's a lot better. MAXIMUM_WAIT_OBJECTS is lame, though.)
mfukar 10y ago
Your step 2 is magical, in the sense that OTHER STUFF wants THEIR data in memory as well.
lgunsch 10y ago
This assumes you are doing read and write system calls, not following pointers in memory. In the Redis article, it is talking about in-memory data structures. The page fault happens from following a pointer to another location in memory, for example, a linked list, or a skip list. There is no read call to replace with an async read call. Your code evaluates the pointer to a page that has been swapped out, a page fault occurs, and the OS has to swap it back in for you to be able to read that memory. You could potentially create a new signal, for page faults (maybe there is one I've never heard of), but that would still not let you continue executing from the previous location.

by smegel 10y ago

Dylan16807 10y ago

Triggering a non-blocking read and continuing execution requires either writing different code or an extremely elaborate page fault feature that can dynamically rewrite your execution flow behind your back. Writing different code is not the question I asked. That's not an OS feature. Do the OSes you like have the latter ability, or are you blowing smoke?

deleted 10y ago
[deleted]
smegel 10y ago
> Triggering a non-blocking read and continuing execution requires either writing different code Well you are asking about doing something non-blocking. That means you must be in some kind of event-loop scenario, otherwise you would be happy with synchronous operations. And yes, you would need to add a read event on the file, and then trigger the socket.send() once the data is in memory. I never said it was free. But at least it is viable.
- Dylan16807 10y ago
  Your response to paging being blocking was to blame Linux. Every OS has paging be blocking. Linux has ways to do non-blocking IO. There is no reason to blame Linux.
to3m 10y ago
Windows can do this - its "non-blocking" read calls are better described as "asynchronous", and take ownership of the buffer until the job is cancelled or it runs to completion. So if the buffer is paged out, that's fine; the program can carry on and the buffer can get paged in when the system needs it (perhaps on an otherwise idle CPU - so it needn't necessarily take any time from the perspective of the calling thread). The POSIX semantics on the other hand are simply that the read won't block due to lack of input, so it does as much as it can straight away and then returns. If there's data, but the buffer is paged out, the non-blocking read call has to take more time, because the problem is a paged-out buffer and not lack of input. (The Windows equivalents of man page section 1 are so awful that most POSIX fans just run a mile and install cygwin. More fool them; once you get to man page section 2, it's a lot better. MAXIMUM_WAIT_OBJECTS is lame, though.)

mfukar 10y ago

Your step 2 is magical, in the sense that OTHER STUFF wants THEIR data in memory as well.

lgunsch 10y ago

This assumes you are doing read and write system calls, not following pointers in memory. In the Redis article, it is talking about in-memory data structures. The page fault happens from following a pointer to another location in memory, for example, a linked list, or a skip list. There is no read call to replace with an async read call. Your code evaluates the pointer to a page that has been swapped out, a page fault occurs, and the OS has to swap it back in for you to be able to read that memory. You could potentially create a new signal, for page faults (maybe there is one I've never heard of), but that would still not let you continue executing from the previous location.