3 ms·
No need to make it an option for each text util. We use an O_DIRECT version of the (z)cat command as the start of pipelines with big files.
by kieckerjan 6y ago
No need to make it an option for each text util. We use an O_DIRECT version of the (z)cat command as the start of pipelines with big files.
- specialist 6y agoNice tip, thanks. Related: The experimental async aware (io_uring optimized) Glommio API may eventually supplant POSIX I/O. https://github.com/DataDog/glommio https://github.com/DataDog/glommio "Modern storage is plenty fast. It is the APIs that are bad." https://itnext.io/modern-storage-is-plenty-fast-it-is-the-apis-that-are-bad-6a68319fbc1a https://itnext.io/modern-storage-is-plenty-fast-it-is-the-ap...
- tankenmate 6y agoexcept that the fastest (and also parallel) versions of grep tend to use memory mapping. for grep it tends to be more linear (except the parallel case), but for others like sort it might not be the case that linear access to the file is always best.
- guenthert 6y agoI'm not aware of an O_DIRECT version of cat. The (older) GNU cat (GNU coreutils 0.83) I just looked at doesn't seem to support it. There is however GNU dd which supports direct i/o ('direct' flag) for ages.