3 ms·
Thanks for putting this out there! There are some projects out there using memory mapped files to do fast CSV parsing. Could be a nice way to speed up the memo
by elcritch 9y ago
Thanks for putting this out there!
There are some projects out there using memory mapped files to do fast CSV parsing. Could be a nice way to speed up the memory loading and scroll it in real time. Can't find the link to the library I saw it used in, but it might be an interesting venue to consider. Another library that does it seems to be astropy fast ascii IO module [1].
[1]: http://docs.astropy.org/en/stable/io/ascii/fast_ascii_io.html http://docs.astropy.org/en/stable/io/ascii/fast_ascii_io.htm...
- camkego 9y agoTry benchmarking OS read() calls vs. either sequential or random reads using memory mapping, whenever I do this OS read() calls end up being quite a bit faster.