3 ms·

Which kernels actually do this? Also, if I read(2), it's supposed to be putting stuff in my buffer (obtained with malloc or whatever). It would seem very comp

Which kernels actually do this? Also, if I read(2), it's supposed to be putting stuff in my buffer (obtained with malloc or whatever). It would seem very complicated to change the mapping for addresses in this buffer. And if I write to the buffer, it must be copy-on-write (in fact, it has to copy the page if I dereference any pointer into it, because most hardware doesn't distinguish between read-only and read-write at the instruction level. It just seems like if the kernel really did this level of mapping tricks, that MPI implementations could do something better without this patch. http://lkml.org/lkml/2010/9/14/468 http://lkml.org/lkml/2010/9/14/468

luckydude 16y ago
IRIX did page flipping on the way out to users (read(2)) and copy-on-write page borrowing on the way in from users (write(2)). That's for networking. For file I/O, if you used O_DIRECT, then the data was dma-ed to/from your pages directly, never went through the kernel's file cache at all. All the file performance came from getting many disk DMA engines running in parallel.
- caf 16y ago
  The "copy-on-write borrowing" on write() is where the problem lies. To turn the writable pages into COWs you have to update the process's page tables to remove write permission, which requires a TLB flush. These kinds of games are a net loss, at least on modern hardware. (And then the app likely reuses the buffer for the next `read()` anyway, requiring either replacing the page or faulting in a fresh one and doing the C in COW).
  - luckydude 16y ago
    Huh. Like to see some numbers on that. Not arguing, just a little surprised. So you have data that shows bcopy() to be faster than the copy-on-write stuff? As for the buffers, known problem, all the apps that used this sort of thing cycled through N buffers for exactly that reason. I think N=4 was way more than enough.

luckydude 16y ago

IRIX did page flipping on the way out to users (read(2)) and copy-on-write page borrowing on the way in from users (write(2)). That's for networking. For file I/O, if you used O_DIRECT, then the data was dma-ed to/from your pages directly, never went through the kernel's file cache at all. All the file performance came from getting many disk DMA engines running in parallel.

caf 16y ago
The "copy-on-write borrowing" on write() is where the problem lies. To turn the writable pages into COWs you have to update the process's page tables to remove write permission, which requires a TLB flush. These kinds of games are a net loss, at least on modern hardware. (And then the app likely reuses the buffer for the next `read()` anyway, requiring either replacing the page or faulting in a fresh one and doing the C in COW).
- luckydude 16y ago
  Huh. Like to see some numbers on that. Not arguing, just a little surprised. So you have data that shows bcopy() to be faster than the copy-on-write stuff? As for the buffers, known problem, all the apps that used this sort of thing cycled through N buffers for exactly that reason. I think N=4 was way more than enough.