3 ms·
mmap() anyone? Seriously, this should be the top of the list and default solution when you need to process large file that does not fit in memory. The excepti
by lmilcin 7y ago
mmap() anyone?
Seriously, this should be the top of the list and default solution when you need to process large file that does not fit in memory.
The exceptions would be when you can read the file once (then stream might be easier but not cheaper), when you don't want to rely on FS cage (you need your own) or when you need easy interoperability between different OS-es.
- jdmoreira 7y agoI came to the comments after reading the article just to find the person suggesting mmap and upvote them :)
- neop1x 7y agoExactly! Me too :D
- lmilcin 7y agoIt is quite saddening for me that so few developers try to take advantage of the tools available. It is really magical thing to be able to run just a single function and have your entire terabyte file suddenly present itself as addressable continuous memory space ripe for random access. Regular random access file I/O seems silly to me, like trying to work with the file through a key hole. Have you ever seen how ships are built inside a bottle? That's exactly the picture I have in my mind...