3 ms·

> A truly efficient implementation would never need to load the entire original file in RAM. It would just need to remember the binary offset of each line in a

by jmull 5y ago

> A truly efficient implementation would never need to load the entire original file in RAM. It would just need to remember the binary offset of each line in a way that combines random access and reasonably fast insertion/deletion (e.g. K-fold trees). To know the offset of each line, you'll need to load the entire file from disk. So you're talking about loading it, processing it, and throwing it out again. You should qualify calling that "efficient" since you will need to re-load portions of the file, as needed. Also, an OS's virtual memory systems may work well for you with minor tuning, so it's work looking into that before you spend a lot of time essentially writing your own. I think unless you have a good reason not to, your best bet for a text editor on a modern OS is to read the entire file into memory and leave it there. (But don't make multiple copies of it or use two bytes to store each character as the current version of NotepadNext is apparently doing.)

jmmv 5y ago
> To know the offset of each line, you'll need to load the entire file from disk. No. You will need to read the file, not load it all at once in memory. And for a sequential scan, you need very little memory at any one time. And if you are doing that, you can start displaying the file as soon as you have scanned enough to fill the view. The rest could happen in the background.
- jmull 5y ago
  I didn’t say you had to keep it all in memory to process the line ending. Of course you don’t. You need only two characters of memory (assuming you’re handling windows line breaks).
nickduggets 5y ago
With a 64-bit address space, you're probably better off just mapping the entire file into memory, not specifically loading it all. Let the OS swap in and out the pieces you need as you access it (and build your index of line offsets or whatever other structures you need).
- jmull 5y ago
  The big advantage to mapping is that you only read from disk the portions of the file you need, and only when you need them. Disk access is monumentally slow, so this can be a big win: if you don’t need to access all of a large file, or don’t need to access it all at once, mapping saves you very expensive operations or spreads them out over time. The problem with that for a text editor is that probably the first thing you want to do is process the line endings. So you’re just going to access the entire file, right away, as fast as possible anyway. So mapping doesn’t really give an advantage in this case.

by jmull 5y ago

jmmv 5y ago

> To know the offset of each line, you'll need to load the entire file from disk. No. You will need to read the file, not load it all at once in memory. And for a sequential scan, you need very little memory at any one time. And if you are doing that, you can start displaying the file as soon as you have scanned enough to fill the view. The rest could happen in the background.

jmull 5y ago
I didn’t say you had to keep it all in memory to process the line ending. Of course you don’t. You need only two characters of memory (assuming you’re handling windows line breaks).

nickduggets 5y ago

With a 64-bit address space, you're probably better off just mapping the entire file into memory, not specifically loading it all. Let the OS swap in and out the pieces you need as you access it (and build your index of line offsets or whatever other structures you need).

jmull 5y ago
The big advantage to mapping is that you only read from disk the portions of the file you need, and only when you need them. Disk access is monumentally slow, so this can be a big win: if you don’t need to access all of a large file, or don’t need to access it all at once, mapping saves you very expensive operations or spreads them out over time. The problem with that for a text editor is that probably the first thing you want to do is process the line endings. So you’re just going to access the entire file, right away, as fast as possible anyway. So mapping doesn’t really give an advantage in this case.