3 ms·

This is a terrific post. I've looked elsewhere for specific information on how Linux deals with posix_fadvise, and haven't found this clarity before.

cbsmith 14y ago
The short answer is "poorly". If Linus complains that Linux apps that are buffer cache sensitive end up using O_DIRECT (which in a lot of ways is worse) simply because fadvise() and similar functions have never been done properly.
- paulsutter 14y ago
  This post explains some ofthe subtleties that in the past had made me question whether fadvise() worked at all. Admittedly, I still find unbuffered io to be simplest and most predictable. But Linus' basic arguments against unbuffered io have been reasonable, and I feel more resolved about the matter understanding that fadvise() can be made to work. How would you improve fadvise()? And what problems have you found with O_DIRECT?
  - cbsmith 14y ago
    The problem with O_DIRECT is it pretty much puts each app in the business of doing its own buffer cache, which bypasses the ability of the kernel to look at the system holistically and make decisions about how to buffer data. Compound that with fairly inconsistent contracts around the interface and it's semi-synchronous behaviour... ick. As to improve fadvise()? I'd like to see FADV_SEQUENTIAL (or perhaps a variant) not just double the read ahead buffer, but also dump pages immediately after they've been read unless there is another FD open somewhere else (you can keep calling fadvise with FADV_DONTNEED, but that's lame on several levels). I'd like to see semantics that make it clear to the kernel that data you are writing to a file (particularly if it is in append mode) likely won't be read for a very long time, so it can minimize polluting the buffer cache with freshly written data. I'd like to see fadvise() calls that specify a portion of a file only effect the portion of the file. What'd be REALLY nice would be a way to express "buffer part X and Y of the file, but if you are under pressure, dump Y before you dump X".

cbsmith 14y ago

The short answer is "poorly". If Linus complains that Linux apps that are buffer cache sensitive end up using O_DIRECT (which in a lot of ways is worse) simply because fadvise() and similar functions have never been done properly.

paulsutter 14y ago
This post explains some ofthe subtleties that in the past had made me question whether fadvise() worked at all. Admittedly, I still find unbuffered io to be simplest and most predictable. But Linus' basic arguments against unbuffered io have been reasonable, and I feel more resolved about the matter understanding that fadvise() can be made to work. How would you improve fadvise()? And what problems have you found with O_DIRECT?
- cbsmith 14y ago
  The problem with O_DIRECT is it pretty much puts each app in the business of doing its own buffer cache, which bypasses the ability of the kernel to look at the system holistically and make decisions about how to buffer data. Compound that with fairly inconsistent contracts around the interface and it's semi-synchronous behaviour... ick. As to improve fadvise()? I'd like to see FADV_SEQUENTIAL (or perhaps a variant) not just double the read ahead buffer, but also dump pages immediately after they've been read unless there is another FD open somewhere else (you can keep calling fadvise with FADV_DONTNEED, but that's lame on several levels). I'd like to see semantics that make it clear to the kernel that data you are writing to a file (particularly if it is in append mode) likely won't be read for a very long time, so it can minimize polluting the buffer cache with freshly written data. I'd like to see fadvise() calls that specify a portion of a file only effect the portion of the file. What'd be REALLY nice would be a way to express "buffer part X and Y of the file, but if you are under pressure, dump Y before you dump X".