3 ms·
> The reason for this is that our I/O is unbuffered, and we were reading one byte at a time in order to read lines. This changed on December 24, by adding read
by legulere 11y ago
> The reason for this is that our I/O is unbuffered, and we were reading one byte at a time in order to read lines. This changed on December 24, by adding read buffering for reading lines, vastly improving the performance of rred.
Why not use memory mapped files and let the operating system deal with caching in a way that is most efficient for that specific case?
- ars 11y agoI believe mmap is not fully portable without extra work, so they avoid it - debian runs on a LOT of different systems.
- legulere 11y agoIt was developed inside the original BSD and is part of POSIX. So it's supported on all unixes today. The only place it isn't supported is Windows.
- icefox 11y agoIt is everywhere even Windows. There is a little bit of weirdness on some older versions of Windows CE and some undefined behavior with edge conditions on some OS's if I recall, but you can find the capability everywhere. I added the map API to QFile back in the day and went through checking different OS's for compatibility. A quick google search here is a link to the windows method: https://github.com/radekp/qt/blob/master/src/corelib/io/qfsfileengine_win.cpp#L1948 https://github.com/radekp/qt/blob/master/src/corelib/io/qfsf... unix version: https://github.com/radekp/qt/blob/master/src/corelib/io/qfsfileengine_unix.cpp#L1238 https://github.com/radekp/qt/blob/master/src/corelib/io/qfsf...
- julian-klode 11y agoNot all our files are uncompressed, so directly using mmap does not make sense for us, because we also need to be able to use files compressed with gzip, bzip2, xz (and soonish probably lz4).