4 ms·
Textual dumps of SQL databases also fall in this category. I remember having downloaded a huge sql dump. I couldn't open it in most editors because they all buf
by terminalcommand 9y ago
Textual dumps of SQL databases also fall in this category. I remember having downloaded a huge sql dump. I couldn't open it in most editors because they all buffered into RAM. The file was about 13G in size and did not fit into the RAM of any machine I own. But in this case using ropes wouldn't have helped either, I guess.
Text editors should also take huge files into account and provide sequential reading from the disk to memory. Even on Emacs, I couldn't work with the file. I ended up fixing it and importing it to Postgresql. And then I spent hours on indexing the necessary fields :).
- MetaDark 9y agoI believe you can use the vlf package (https://github.com/m00natic/vlfi https://github.com/m00natic/vlfi) for dealing with large files in Emacs. I haven't used it myself, so I am not sure how stable it is.
- terminalcommand 9y agoThanks for the pointer. I guess I could also have written a small program or script to go through the file. Regular fopen and fread should work. However the vlf package does exactly what I looked for. Only prerequisite I can see is that one must compile Emacs with bignum support on 32bit systems to read files greater than 512M.