4 ms·

If you're interested in visual editors and if you're looking for something perhaps more accessible (and I can't honestly say how much it really is, I have not l

If you're interested in visual editors and if you're looking for something perhaps more accessible (and I can't honestly say how much it really is, I have not looked at it) then consider taking a look at 'less', the pager. Less does almost everything an "editor" does, at least visually, except change text. It pages through text, forward and backward, line by line, it handles lines that are too long, tab expansion, it searches, even has an extended command set. (Can less do syntax coloring?) It also handles files too big for memory. These are all editor problems. Mind its solutions may not be optimized for an editor, but it's certainly smaller. Today, modern machines "suffer" from "too much" performance which actually frees you from not having to worry so much about the actual backing store, especially early on. Do you really intend to be editing a 2GB file? Honestly, how big is an average text file? And how many billion cycles per second does a modern CPU handle? Sucking the entire file in to RAM, and just pushing stuff around with block moves will take you very far on a modern machine. Not that you should not look at the other data structures (there are many), but you don't have to start there, depending on where your interest lies. Also consider hunting down the book "Software Tools". There's two editions, the original and "Software Tools in Pascal". It's by Kernighan and Plauger. They go through in detail and write a version of the 'ed' line editor. And if you really want to work on an editor, the CP/M world would love a new one. There, it's all about efficiency.

nicoburns 5y ago
> Do you really intend to be editing a 2GB file? I don't know about most people, but I deal with such files (usually JSON) on a weekly basis. It's surely one of the things that makes implementing text editors tricky.
- hutzlibu 5y ago
  Most people won't, they will just get annoyed, if they accidently open a video and the editor hangs up for some time, trying to process it. And I regulary work with JSON files the size of some MB, which already puts some editors into a real struggle. Out of curiosity, why do you have so big files? (A whole DB as a json file?) And what editor do you use for them?
  - nicoburns 5y ago
    > A whole DB as a json file More like a whole database table as a JSON file - I split the whole database export into tables using `jq` (an export from Firebase Realtime Database). I generally use sublime text to open them, it's quite happy with anything that will fit in RAM. For very large file it just shows a load bar while it loads it. It was vital for exploring the shape of our data during a data migration to Postgres as our data was not very regular in shape and firebase's querying capabilities are pretty limited.
- whartung 5y ago
  Honest question, do you just open the files, or do you change them and write them back out? I, too, have dealt with huge files. My primary large file use case for vim is as an interactive grep, opening files, then just performing iterative filters reducing the file to something "manageable". If the file is too big, I might resort to raw streaming filters to temp files before going all vim on it (even vim has issues with huge files -- huge files are issues all their own). But it's more a matter of pragmatism as to how far one wants to take a pet project like this. There's a curve of diminishing return depending on what someone wants from such a thing. A single RAM buffer is a bad idea, to be sure, but raw horsepower of modern systems and CPUs cover up a lot of bad ideas. Simply the idea that you COULD mmap or even malloc a 2G buffer to suck the file in, and it would WORK, is enough to make ones teeth itch. I remember long a ago an article from someone encountering an early workstation with 1G of RAM, and the worlds it opened up. We have come far from those days. If the OP is interested in buffer mechanics, then starting with an API backed by a simple buffer goes a long way. If they're interested in screen painting, window management, etc, then the RAM buffer may be all they need. Otherwise, they can work on replacing their back end with all of the assorted structures folks have mentioned, see what they like best, as they all have tradeoffs.
  - nicoburns 5y ago
    I don't often need to edit the files (I also mostly use for searching and my most common editing action would be pretty printing which I could well manage externally if need be). However, I very commonly accidentally edit the files due to a stray keypress, so if edits are not disabled entirely then not locking up or crashing if edits do occur would definitely be a requirement. Occasionally it useful to be able to edit, and it's nice to have at least an editor that can do this.
- cmg 5y ago
  Just this morning I needed to do a global find/replace for a string in a 3GB .sql file. I was in VS Code anyway so tried that - but the app choked on it. Ended up going with sed and it worked perfectly.

nicoburns 5y ago

> Do you really intend to be editing a 2GB file? I don't know about most people, but I deal with such files (usually JSON) on a weekly basis. It's surely one of the things that makes implementing text editors tricky.

hutzlibu 5y ago
Most people won't, they will just get annoyed, if they accidently open a video and the editor hangs up for some time, trying to process it. And I regulary work with JSON files the size of some MB, which already puts some editors into a real struggle. Out of curiosity, why do you have so big files? (A whole DB as a json file?) And what editor do you use for them?
- nicoburns 5y ago
  > A whole DB as a json file More like a whole database table as a JSON file - I split the whole database export into tables using `jq` (an export from Firebase Realtime Database). I generally use sublime text to open them, it's quite happy with anything that will fit in RAM. For very large file it just shows a load bar while it loads it. It was vital for exploring the shape of our data during a data migration to Postgres as our data was not very regular in shape and firebase's querying capabilities are pretty limited.
whartung 5y ago
Honest question, do you just open the files, or do you change them and write them back out? I, too, have dealt with huge files. My primary large file use case for vim is as an interactive grep, opening files, then just performing iterative filters reducing the file to something "manageable". If the file is too big, I might resort to raw streaming filters to temp files before going all vim on it (even vim has issues with huge files -- huge files are issues all their own). But it's more a matter of pragmatism as to how far one wants to take a pet project like this. There's a curve of diminishing return depending on what someone wants from such a thing. A single RAM buffer is a bad idea, to be sure, but raw horsepower of modern systems and CPUs cover up a lot of bad ideas. Simply the idea that you COULD mmap or even malloc a 2G buffer to suck the file in, and it would WORK, is enough to make ones teeth itch. I remember long a ago an article from someone encountering an early workstation with 1G of RAM, and the worlds it opened up. We have come far from those days. If the OP is interested in buffer mechanics, then starting with an API backed by a simple buffer goes a long way. If they're interested in screen painting, window management, etc, then the RAM buffer may be all they need. Otherwise, they can work on replacing their back end with all of the assorted structures folks have mentioned, see what they like best, as they all have tradeoffs.
- nicoburns 5y ago
  I don't often need to edit the files (I also mostly use for searching and my most common editing action would be pretty printing which I could well manage externally if need be). However, I very commonly accidentally edit the files due to a stray keypress, so if edits are not disabled entirely then not locking up or crashing if edits do occur would definitely be a requirement. Occasionally it useful to be able to edit, and it's nice to have at least an editor that can do this.
cmg 5y ago
Just this morning I needed to do a global find/replace for a string in a 3GB .sql file. I was in VS Code anyway so tried that - but the app choked on it. Ended up going with sed and it worked perfectly.